AutoSDF

KO | EN

gitlite — search

#javascript #hacktoberfest #react #typescript #go #golang #deep-learning #docker #awesome-list #pytorch #openai #ios #mcp #open-source

(★ 255)

No description available.

File Explorer

Download Latest Version (.zip)

- baseline_lang_je_code.yaml
- bert2vq_shapeglot.yaml
- paths.py
- pvqvae_snet.yaml
- rand_tf_snet_code.yaml
- __init__.py
- base_dataloader.py
- base_dataset.py
- dataloader.py
- pix3d_dataset.py
- shapeglot_dataset.py
- snet_dataset.py
- chair_2598.jpg
- chair_2598_mask.png
- chair_model.obj
- extract_pvqvae_pix3d.sh
- extract_pvqvae_snet.sh
- train_bert2vq_shapeglot_lang.sh
- train_pvqvae_snet.sh
- train_rand_tf_snet_code.sh
- - __init__.py
  - bert2vq_net.py
  - losses.py
  - lpips.py
  - resnet.py
  - resnet2vq_net.py
- __init__.py
- base_model.py
- bert2vq_model.py
- pvqvae_model.py
- rand_tf_model.py
- resnet2vq_model.py
- __init__.py
- base_options.py
- test_options.py
- train_options.py
- - info-euclid-pix3d.json
- create_sdf_file_lst.py
- process_one_mesh.py
- __init__.py
- binvox_rw.py
- demo_util.py
- pix3d_util.py
- qual_util.py
- util.py
- util_3d.py
- visualizer.py
autosdf.yaml
demo-lang-conditional.ipynb
demo_shape_comp.ipynb
demo_single_view_recon.ipynb
extract_code.py
README.md
train.py

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation
[[`arXiv`](https://arxiv.org/abs/2203.09516)]
[[`Project Page`](https://yccyenchicheng.github.io/AutoSDF/)]
[[`BibTex`](#citation)]

Code release for the CVPR 2022 paper "AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation".

https://user-images.githubusercontent.com/27779063/159215086-6889da7c-07c6-4909-b51f-bc04364072cf.mp4

# Installation
Please install [`pytorch`](https://pytorch.org/) and [`pytorch3d`](https://github.com/facebookresearch/pytorch3d). Or you can setup the environment using `conda`:

```
conda env create -f autosdf.yaml
conda activate autosdf
```

However, the environments varies by each machine. We tested the code on `Ubuntu 20.04`, `cuda=11.3`, `python=3.8.11`, `pytorch=1.9.0`, `pytorch3d=0.5.0`.

# Demo
We provide a jupyter notebook for demo. First download the pretrained weights from [this link](https://drive.google.com/drive/folders/1n8W_8CfQ7uZDYNrv487sd0oyhRoNLfGo?usp=sharing), and put them under `saved_ckpt`. Then start the notebook server with
```
jupyter notebook
```
And run:
- `demo_shape_comp.ipynb` for shape completion
- `demo_single_view_recon.ipynb` for single-view reconstruction
- `demo-lang-conditional.ipynb` for language-guided generation

# Preparing the Data
1. [ShapeNet](https://www.shapenet.org)

First you need to download the `ShapeNetCore.v1` following the instruction of `https://www.shapenet.org/account/`. Put them under `data/ShapeNet`. Then unzip the downloaded zip file. We assume the path to the unzipped folder is `data/ShapeNet/ShapeNetCore.v1`. To extract SDF values, we followed the [preprocessing steps from DISN](https://github.com/laughtervv/DISN/blob/master/preprocessing/create_point_sdf_grid.py).

2. [Pix3D](https://github.com/xingyuansun/pix3d)

The Pix3D dataset can be downloaded here: https://github.com/xingyuansun/pix3d.

# Training
1. First train the `P-VQ-VAE` on `ShapeNet`:
```
./launchers/train_pvqvae_snet.sh
```

2. Then extract the code for each sample of ShapeNet (caching them for training the transformer):
```
./launchers/extract_pvqvae_snet.sh
```

3. Train the random-order-transformer to learn the shape prior:
```
./launchers/train_rand_tf_snet_code.sh
```

4. To train the image marginal on Pix3D, first extract the code for each training data of Pix3D
```
./launchers/extract_pvqvae_pix3d.sh
```

5. Train the image marginal on Pix3D
```
./launchers/train_resnet2vq_pix3d_img.sh
```

# Issues and FAQ

## 1. Regarding `mcubes` functions
We originally use the implementation of the marching cubes from this repo: https://github.com/JustusThies/PyMarchingCubes. However, some of the dependencies seems to be outdated and makes the installation troublesome. Currently the quick workaround is installing `mcubes` from https://github.com/pmneila/PyMCubes:
```
pip install PyMCubes
```
and replace all the lines `import marching_cubes as mcubes` in our code with `import mcubes`.

# <a name="citation"></a>Citing AutoSDF

If you find this code helpful, please consider citing:

```BibTeX
@inproceedings{autosdf2022,
  title={{AutoSDF}: Shape Priors for 3D Completion, Reconstruction and Generation},
  author={Mittal, Paritosh and Cheng, Yen-Chi and Singh, Maneesh and Tulsiani, Shubham},
  booktitle={CVPR},
  year={2022}
}
```

# Acknowledgement
This code borrowed heavily from [Cycle-GAN](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix), [VQ-GAN](https://github.com/CompVis/taming-transformers). Thanks for the efforts for making their code available!

AutoSDF

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

AutoSDF

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README