KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #dsh #dsh-plugin #ai-agent #claude-code #codex #typescript #ai-agents #mcp #cli #rust #deepseek-harness

Cloth2Tex

★ 523 Open GitHub ↗

Cloth2Tex: A Customized Cloth Texture Generation Pipeline for 3D Virtual Try-On

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center">Cloth2Tex: A Customized Cloth Texture Generation Pipeline for 3D Virtual Try-On</h1>
 <h2 align="center">3DV 2024</h2>
 <div align="center">
 <img src="./imgs/teaser.png" alt="Logo" width="100%">
 </div>
 
 <div class="is-size-5 publication-authors">
 
 <a href="">Daiheng Gao</a>1,
 
 <a href="">Xu Chen</a>2,3,
 
 <a href="">Xindi Zhang</a>1,
 
 
 <a href="">Qi Wang</a>1,
 
 
 <a href="">Ke Sun</a>1,
 
 
 <a href="">Bang Zhang</a>1,
 
 
 <a href="">Liefeng Bo</a>1,
 
 
 <a href="">Qixing Huang</a>4,
 
 </div>
 <div class="is-size-5 publication-authors">
 1Alibaba XR Lab,
 2ETH Zurich, Department of Computer Science,
 3Max Planck Institute for Intelligent Systems,
 4The University of Texas at Austin
 </div>
 </div>

---
## 1. Installation

Our enviroment is *python3.8, pytorch1.13, cuda11.7*, you can change the following instructions that suitable for your settings.

```shell
    sudo apt-get update -y
    sudo apt-get install libgl1
    sudo apt-get install libboost-dev
```

- **pytorch3d** [code](https://pytorch3d.org/)
- **psbody-mesh** [code](https://github.com/MPI-IS/mesh)
- **Kaolin** [code](https://github.com/NVIDIAGameWorks/kaolin)
- **torch_geometric** [code](https://data.pyg.org/whl/)

```
pip install torch_geometric
pip install pyg_lib-0.3.0+pt113cu117-cp38-cp38-linux_x86_64.whl
pip install torch_cluster-1.6.1+pt113cu117-cp38-cp38-linux_x86_64.whl
pip install torch_scatter-2.1.1+pt113cu117-cp38-cp38-linux_x86_64.whl
pip install torch_sparse-0.6.15+pt113cu117-cp38-cp38-linux_x86_64.whl
```

---

## 2. Architecture

Cloth2Tex is composed of two phase: (1) **Coarse texture generation** and (2) **Fine texture completion**. Where Phase I is to determine the 3D garment shape and coarse texture. We do this by registering our parametric garment meshes onto catalog images using a neural mesh renderer. The pipeline’s Then Phase II is to recover fine textures from the coarse estimate of Phase I. We use image translation networks trained on large-scale data synthesized by pre-trained latent diffusion models.

We only made **Phase I** publicly available for now.

![](https://raw.githubusercontent.com/HumanAIGC/Cloth2Tex/main/imgs/method.png)

---

## 3. Inference

### Phase I (w/o automatic scaling mechanism)

``` shell
python phase1_inference.py --g 1_wy --s 1.2 --d "20231017_wy" --steps_one 501 --steps_two 1001
```

The optimized results are saved in `experiments/20231017_wy`, `x_texture_uv_1000.jpg` is the final UV texture.

Users can check it with **Blender**, remember you should only reserve one material, and remove other redundant materials for textured mesh.

![img](https://raw.githubusercontent.com/HumanAIGC/Cloth2Tex/main/imgs/phase1_demo.png)

#### (a) reference scale coefficient

The noteworthy thing here is that we are not make automatic scaling mechanism code publicly available, if you need it, you could self-implement it or manually adjust the `--s` (scale).

Default coefficient for test images:
``` python
per_scale_dict = {"1_wy": 1.1,
                  "2_Polo": 0.8, # default 0.8
                  "3_Tshirt": 0.9, # default 0.7
                  "4_shorts": 0.75, # # default 0.7
                  "5_trousers": 0.75,
                  "6_zipup": 1.1,
                  "7_windcoat": 0.65,
                  "9_jacket": 1.0,
                  "11_skirt": 1.0} 
```

#### (b) The landmark detector

We are not going to release the 2D landmark detector. If you need an accurate 2D landmarks in accordance with **Cloth2Tex**, you can annotate it manually or train a simple 2D cloth landmark detector with the same definition from **Cloth2Tex**.

### Phase II (Inpainting/Completion Network)

We are applying for the open-source of Phase II, we will update once approval procedure has finished.

---

## 4. Demo

### Real world 3D Try-On
![](https://raw.githubusercontent.com/HumanAIGC/Cloth2Tex/main/imgs/tryon1.png)
![](https://raw.githubusercontent.com/HumanAIGC/Cloth2Tex/main/imgs/tryon2.png)

Please check cloth2tex web page for animated visual results: [cloth2tex](https://tomguluson92.github.io/projects/cloth2tex/) or check our youtube video [youtube](https://www.youtube.com/watch?v=RFMNKe6supE).

---

## 5. Citation

```bibtex
@article{gao2023cloth2tex,
  title={Cloth2Tex: A Customized Cloth Texture Generation Pipeline for 3D Virtual Try-On},
  author={Gao, Daiheng and Chen, Xu and Zhang, Xindi and Wang, Qi and Sun, Ke and Zhang, Bang and Bo, Liefeng and Huang, Qixing},
  journal={arXiv preprint arXiv:2308.04288},
  year={2023}
}
```

Cloth2Tex

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README