FIT-VTO

KO | EN

gitlite — search

#javascript #hacktoberfest #react #android #machine-learning #deep-learning #nodejs #macos #java #awesome #pytorch #security #openai #open-source

(★ 21)

[SIGGRAPH 2026] Official implementation for "FIT: A Large-Scale Dataset for Fit-Aware Virtual Try-On"

FIT-VTO Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center">FIT: A Large-Scale Dataset for Fit-Aware Virtual Try-On</h1>

Yuanhao Wang*&nbsp;&nbsp;Johanna Karras*&nbsp;&nbsp;Yingwei Li&nbsp;&nbsp;Ira Kemelmacher-Shlizerman
 
 * Equal Contribution, listed in no particular order
 
 University of Washington &nbsp;·&nbsp; Google Research
 
 ACM Transactions on Graphics (SIGGRAPH 2026)

---

## Abstract

Given a person and a garment image, virtual try-on (VTO) aims to synthesize a realistic image of the person wearing the garment while preserving their original pose and identity. Although recent VTO methods excel at visualizing garment appearance, they largely overlook a crucial aspect of the try-on experience: the accuracy of garment fit — for example, depicting how an extra-large shirt looks on an extra-small person. A key obstacle is the absence of datasets that provide precise garment and body size information, particularly for "ill-fit" cases, where garments are significantly too large or too small. Consequently, current VTO methods default to generating well-fitted results regardless of the garment or person size.

In this paper, we take the first steps towards solving this open problem. We introduce **FIT** (**F**it-**I**nclusive **T**ry-on), a large-scale VTO dataset comprising over **1M try-on image triplets** accompanied by precise body and garment measurements. We overcome the challenges of data collection via a scalable synthetic strategy: (1) We programmatically generate 3D garments using GarmentCode and drape them via physics simulation to capture realistic garment fit. (2) We employ a novel re-texturing framework to transform synthetic renderings into photorealistic images while strictly preserving geometry. (3) We introduce person identity preservation into our re-texturing model to generate paired person images (same person, different garments) for supervised training. Finally, we leverage our FIT dataset to train a baseline fit-aware virtual try-on model. Our data and results set the new state-of-the-art for fit-aware virtual try-on, as well as offer a robust benchmark for future research. We will make all data and code publicly available.

---

## Dataset

A preview release of the FIT dataset is available on HuggingFace:

**[`Yuanhao-Harry-Wang/fitvto-100k`](https://huggingface.co/datasets/Yuanhao-Harry-Wang/fitvto-100k)** — 100K training samples · 5K evaluation benchmark · CC-BY-NC-ND-4.0

Each sample contains:

| Field | Description |
|:---:|:---|
| `target` | Photorealistic try-on image (768×1024) |
| `person` | Paired person image — same person, different garment (768×1024) |
| `cloth` | Layflat garment image (768×1024) |
| `body_height` / `body_bust` / `body_waist` / `body_hips` | Person body measurements (cm) |
| `garment_bust` / `garment_length` / `garment_sleeve_length` | Garment measurements (cm) |

> **Preview scale:** 100K training · 5K eval · Full dataset (1.13M triplets · 168 body shapes · 528 poses · 158K+ garment designs) coming soon.

---

## Data Generation and Preparation

This repository contains the full pipeline used to generate the FIT dataset. Follow the steps below in order.

### Step 1 — 3D Garment Simulation

Programmatically generates sewing patterns across a range of body and garment sizes, runs physics-based cross-draping simulation, and renders the results.

📄 &nbsp;See **[`garmentcode_generation/`](garmentcode_generation/README.md)** for installation, asset setup, and usage.

### Step 2 — Sim2Real Retexturing

Transforms synthetic GarmentCode renders into photorealistic try-on images via a two-stage FLUX.1-dev LoRA pipeline conditioned on composite normal maps.

📄 &nbsp;See **[`sim2real/`](sim2real/README.md)** for installation, model weights, and usage.

---

## Repository Structure

```
FIT-VTO/
├── garmentcode_generation/              # 3D garment simulation (GarmentCode-based)
│   ├── run_experiment.py                # Main entrypoint: pattern sampling + physics simulation
│   ├── pattern_sampler.py               # Cross-draping pattern generation
│   ├── batch_sim.py                     # GPU-accelerated physics simulation
│   ├── gen_masks.py                     # Post-simulation garment mask generation
│   ├── postprocess.py                   # Measurement recomputation, filtering, and output collection
│   ├── generate_design_templates.py     # Generate custom garment design template pools
│   └── README.md
│
└── sim2real/                 # Synthetic-to-photorealistic retexturing (FLUX-based)
    ├── run_example.py        # End-to-end Stage 1 + Stage 2 demo on example pairs
    └── README.md
```

---

## Citation

If you find this work useful, please cite:

```bibtex
@article{fitvto2026,
  author    = {Karras, Johanna and Wang, Yuanhao and Li, Yingwei and Kemelmacher-Shlizerman, Ira},
  title     = {FIT: A Large-Scale Dataset for Fit-Aware Virtual Try-On},
  journal   = {SIGGRAPH},
  year      = {2026},
}
```

---

## Acknowledgements

This work builds on [GarmentCode](https://github.com/maria-korosteleva/GarmentCode) for procedural garment simulation, [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) for image generation, and [OminiControl](https://github.com/Yuanshi9815/OminiControl) for multi-condition FLUX fine-tuning.

FIT-VTO

FIT-VTO Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

FIT-VTO

FIT-VTO Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README