KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #codex #typescript #ai-agents #claude-code #dsh #dsh-plugin #open-source #cli #deepseek-harness #macos #ai-agent

InterScene

★ 124 Open GitHub ↗

[3DV 2024] Official repo of "Synthesizing Physically Plausible Human Motions in 3D Scenes"

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h2 align="center">Synthesizing Physically Plausible Human Motions in 3D Scenes</h2>
 
 <a href="https://liangpan99.github.io/">Liang Pan</a>
 · 
 <a href="https://scholar.google.com/citations?user=GStTsxAAAAAJ&hl=en&oi=ao">Jingbo Wang</a>
 ·
 <a href="http://www.buzhenhuang.com/">Buzhen Huang</a>
 ·
 <a href="https://budiu-39.github.io/">Junyu Zhang
 ·
 <a href="https://haofanwang.github.io/">Haofan Wang</a>
 ·
 <a href="https://tangxuvis.github.io/">Xu Tang</a>
 ·
 <a href="https://www.yangangwang.com/">Yangang Wang</a>
 
 Southeast University&emsp;Shanghai AI Laboratory&emsp;Xiaohongshu Inc.
 
 <h2 align="center">3DV 2024</h2>
 <img src='https://github.com/liangpan99/InterScene/blob/main/docs/assets/teaser.png'>

We propose InterScene, a novel method that generates physically plausible long-term motion sequences in 3D indoor scenes. Our approach enables physics-based characters to exhibit natural interaction-involved behaviors, such as sitting down (gray), getting up (blue), and walking while avoiding obstacles (pink).

## News
- **[2025-03-03]** Update camera-ready paper and website.
- **[2023-11-09]** Release code for training and evaluating the sit policy.
- **[2023-10-16]** Paper got accepted by 3DV 2024.

## Dependencies

### Environment

To create the environment, follow the following instructions:

1. We recommend to install all the requirements through Conda by
```
conda create -n rlgpu python=3.7
pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
pip install -r requirements.txt
```

2. Download IsaacGym Preview 4 from the [official site](https://developer.nvidia.com/isaac-gym) and install it via pip.

### Dataset

To prepare data for training/evaluating InterCon (sit & get-up policies), follow the following instructions:

1. Download SMPL-X v1.1 from the [official site](https://smpl-x.is.tue.mpg.de/). Put them in the `body_models/smplx` folder.

2. Download SAMP motion dataset from the [official site](https://samp.is.tue.mpg.de/). Put them in the `samp` folder. Please download Motion Clips (.pkl), which contains the SMPL-X parameters.

3. The file structure should look like this:

4. Run the following script to generate reference motion dataset:

```
python InterScene/data/dataset_samp_sit/generate_motion.py --samp_pkl_dir ./samp --smplx_dir ./body_models/smplx
```

5. Run the following script to generate 3D object dataset:

```
python InterScene/data/dataset_samp_sit/generate_obj.py
```

## Getting Started

### InterCon (sit & get-up policies)

```
## training sit policy
python InterScene/run.py --task HumanoidLocationSit --cfg_env InterScene/data/cfg/humanoid_location_sit.yaml --cfg_train InterScene/data/cfg/train/rlg/amp_task_location_sit.yaml --motion_file InterScene/data/dataset_samp_sit/dataset_samp_sit.yaml --num_envs 4096 --headless

## evaluating sit policy
python InterScene/run.py --task HumanoidLocationSit --cfg_env InterScene/data/cfg/humanoid_location_sit.yaml --cfg_train InterScene/data/cfg/train/rlg/amp_task_location_sit.yaml --motion_file InterScene/data/dataset_samp_sit/dataset_samp_sit.yaml --num_envs 4096 --headless --checkpoint InterScene/data/models/policy_sit.pth --test
```

## Citation

```bibtex
@inproceedings{pan2024synthesizing,
  title={Synthesizing physically plausible human motions in 3d scenes},
  author={Pan, Liang and Wang, Jingbo and Huang, Buzhen and Zhang, Junyu and Wang, Haofan and Tang, Xu and Wang, Yangang},
  booktitle={2024 International Conference on 3D Vision (3DV)},
  pages={1498--1507},
  year={2024},
  organization={IEEE}
}
```

## References
This repository is built on the top of the following amazing codebases: 
* Main code framework is from: [ASE](https://github.com/nv-tlabs/ASE)
* Some scripts are from: [Pacer](https://github.com/nv-tlabs/pacer/tree/main), [HuMoR](https://github.com/davrempe/humor)

Please follow the license of the above repositories for the usage of this project.

InterScene

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README