InterScene

KO | EN

gitlite — search

#javascript #python #typescript #android #rust #docker #java #nodejs #windows #cli #kubernetes #awesome-list #pytorch #open-source

(★ 124)

[3DV 2024] Official repo of "Synthesizing Physically Plausible Human Motions in 3D Scenes"

InterScene Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h2 align="center">Synthesizing Physically Plausible Human Motions in 3D Scenes</h2>
 
 <a href="https://liangpan99.github.io/">Liang Pan</a>
 · 
 <a href="https://scholar.google.com/citations?user=GStTsxAAAAAJ&hl=en&oi=ao">Jingbo Wang</a>
 ·
 <a href="http://www.buzhenhuang.com/">Buzhen Huang</a>
 ·
 <a href="https://budiu-39.github.io/">Junyu Zhang
 ·
 <a href="https://haofanwang.github.io/">Haofan Wang</a>
 ·
 <a href="https://tangxuvis.github.io/">Xu Tang</a>
 ·
 <a href="https://www.yangangwang.com/">Yangang Wang</a>
 
 Southeast University&emsp;Shanghai AI Laboratory&emsp;Xiaohongshu Inc.
 
 <h2 align="center">3DV 2024</h2>
 <img src='https://github.com/liangpan99/InterScene/blob/main/docs/assets/teaser.png'>

We propose InterScene, a novel method that generates physically plausible long-term motion sequences in 3D indoor scenes. Our approach enables physics-based characters to exhibit natural interaction-involved behaviors, such as sitting down (gray), getting up (blue), and walking while avoiding obstacles (pink).

## News
- **[2025-03-03]** Update camera-ready paper and website.
- **[2023-11-09]** Release code for training and evaluating the sit policy.
- **[2023-10-16]** Paper got accepted by 3DV 2024.

## Dependencies

### Environment

To create the environment, follow the following instructions:

1. We recommend to install all the requirements through Conda by
```
conda create -n rlgpu python=3.7
pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
pip install -r requirements.txt
```

2. Download IsaacGym Preview 4 from the [official site](https://developer.nvidia.com/isaac-gym) and install it via pip.

### Dataset

To prepare data for training/evaluating InterCon (sit & get-up policies), follow the following instructions:

1. Download SMPL-X v1.1 from the [official site](https://smpl-x.is.tue.mpg.de/). Put them in the `body_models/smplx` folder.

2. Download SAMP motion dataset from the [official site](https://samp.is.tue.mpg.de/). Put them in the `samp` folder. Please download Motion Clips (.pkl), which contains the SMPL-X parameters.

3. The file structure should look like this:

4. Run the following script to generate reference motion dataset:

```
python InterScene/data/dataset_samp_sit/generate_motion.py --samp_pkl_dir ./samp --smplx_dir ./body_models/smplx
```

5. Run the following script to generate 3D object dataset:

```
python InterScene/data/dataset_samp_sit/generate_obj.py
```

## Getting Started

### InterCon (sit & get-up policies)

```
## training sit policy
python InterScene/run.py --task HumanoidLocationSit --cfg_env InterScene/data/cfg/humanoid_location_sit.yaml --cfg_train InterScene/data/cfg/train/rlg/amp_task_location_sit.yaml --motion_file InterScene/data/dataset_samp_sit/dataset_samp_sit.yaml --num_envs 4096 --headless

## evaluating sit policy
python InterScene/run.py --task HumanoidLocationSit --cfg_env InterScene/data/cfg/humanoid_location_sit.yaml --cfg_train InterScene/data/cfg/train/rlg/amp_task_location_sit.yaml --motion_file InterScene/data/dataset_samp_sit/dataset_samp_sit.yaml --num_envs 4096 --headless --checkpoint InterScene/data/models/policy_sit.pth --test
```

## Citation

```bibtex
@inproceedings{pan2024synthesizing,
  title={Synthesizing physically plausible human motions in 3d scenes},
  author={Pan, Liang and Wang, Jingbo and Huang, Buzhen and Zhang, Junyu and Wang, Haofan and Tang, Xu and Wang, Yangang},
  booktitle={2024 International Conference on 3D Vision (3DV)},
  pages={1498--1507},
  year={2024},
  organization={IEEE}
}
```

## References
This repository is built on the top of the following amazing codebases: 
* Main code framework is from: [ASE](https://github.com/nv-tlabs/ASE)
* Some scripts are from: [Pacer](https://github.com/nv-tlabs/pacer/tree/main), [HuMoR](https://github.com/davrempe/humor)

Please follow the license of the above repositories for the usage of this project.

InterScene

InterScene Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

InterScene

InterScene Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README