InterScene - GitLite

KO | EN

gitlite — search

#javascript #hacktoberfest #ai #typescript #llm #golang #go #nodejs #awesome #cli #ios #mcp #openai #open-source

InterScene

(★ 124)

[3DV 2024] Official repo of "Synthesizing Physically Plausible Human Motions in 3D Scenes"

파일 탐색기

최종 버전 다운로드 (.zip)

- - pipeline.png
  - poster.jpg
  - teaser.png
- index.html
- - amp_agent.py
  - amp_datasets.py
  - amp_models.py
  - amp_network_builder.py
  - amp_players.py
  - common_agent.py
  - common_player.py
  - replay_buffer.py
- - __init__.py
- - __init__.py
  - build_humanoid_mesh.py
  - config.py
  - gym_util.py
  - logger.py
  - motion_lib.py
  - object_lib.py
  - parse_task.py
  - sp_animation.py
  - torch_utils.py
- __init__.py
- run.py
.gitignore
README.md
requirements.txt

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/liangpan99/InterScene

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd InterScene

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. 공식 설치 스크립트

쉬움 추천

사전 준비물

Python 3 pip 명령어를 쓰려면 Python이 필요합니다.

pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

✅ 설치 후 새 터미널을 열고, 프로그램의 버전 확인 명령(예: --version)으로 정상 설치됐는지 확인하세요.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

3. Python

쉬움

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

pip install -r requirements.txt

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<p align="center">

<h2 align="center">Synthesizing Physically Plausible Human Motions in 3D Scenes</h2>
  <p align="center">
    <a href="https://liangpan99.github.io/"><strong>Liang Pan</strong></a>
    ·  
    <a href="https://scholar.google.com/citations?user=GStTsxAAAAAJ&hl=en&oi=ao"><strong>Jingbo Wang</strong></a>
    ·
    <a href="http://www.buzhenhuang.com/"><strong>Buzhen Huang</strong></a>
    ·
    <a href="https://budiu-39.github.io/"><strong>Junyu Zhang</strong>
    ·
    <a href="https://haofanwang.github.io/"><strong>Haofan Wang</strong></a>
    ·
    <a href="https://tangxuvis.github.io/"><strong>Xu Tang</strong></a>
    ·
    <a href="https://www.yangangwang.com/"><strong>Yangang Wang</strong></a>
    <br>
    Southeast University&emsp;Shanghai AI Laboratory&emsp;Xiaohongshu Inc.
  </p>
  <h2 align="center">3DV 2024</h2>
  <img src='https://github.com/liangpan99/InterScene/blob/main/docs/assets/teaser.png'>
</p>

<p align="center">
    <a href="https://arxiv.org/abs/2308.09036">
      <img src='https://img.shields.io/badge/Paper-PDF-green?style=for-the-badge&logo=adobeacrobatreader&logoWidth=20&logoColor=white&labelColor=66cc00&color=94DD15' alt='Paper PDF'>
    </a>
    <a href='https://liangpan99.github.io/InterScene'>
      <img src='https://img.shields.io/badge/InterScene-Page-orange?style=for-the-badge&logo=Google%20chrome&logoColor=white&labelColor=D35400' alt='Project Page'></a>
  </p>

We propose InterScene, a novel method that generates physically plausible long-term motion sequences in 3D indoor scenes. Our approach enables physics-based characters to exhibit natural interaction-involved behaviors, such as sitting down (gray), getting up (blue), and walking while avoiding obstacles (pink).

## News
- **[2025-03-03]** Update camera-ready paper and website.
- **[2023-11-09]** Release code for training and evaluating the sit policy.
- **[2023-10-16]** Paper got accepted by 3DV 2024.

## Dependencies

### Environment

To create the environment, follow the following instructions:

1. We recommend to install all the requirements through Conda by
```
conda create -n rlgpu python=3.7
pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
pip install -r requirements.txt
```

2. Download IsaacGym Preview 4 from the [official site](https://developer.nvidia.com/isaac-gym) and install it via pip.

### Dataset

To prepare data for training/evaluating InterCon (sit & get-up policies), follow the following instructions:

1. Download SMPL-X v1.1 from the [official site](https://smpl-x.is.tue.mpg.de/). Put them in the `body_models/smplx` folder.

2. Download SAMP motion dataset from the [official site](https://samp.is.tue.mpg.de/). Put them in the `samp` folder. Please download Motion Clips (.pkl), which contains the SMPL-X parameters.

3. The file structure should look like this:

```
|-- InterScene
|-- body_models
    |-- smplx
        |-- SMPLX_FEMALE.npz
        |-- SMPLX_FEMALE.pkl
        |-- SMPLX_MALE.npz
        |-- ...
|-- samp
    |-- chair_mo_stageII.pkl
    |-- chair_mo001_stageII.pkl
    |-- chair_mo002_stageII.pkl
    |-- ...
```

4. Run the following script to generate reference motion dataset:

```
python InterScene/data/dataset_samp_sit/generate_motion.py --samp_pkl_dir ./samp --smplx_dir ./body_models/smplx
```

5. Run the following script to generate 3D object dataset:

```
python InterScene/data/dataset_samp_sit/generate_obj.py
```

## Getting Started

### InterCon (sit & get-up policies)

```
## training sit policy
python InterScene/run.py --task HumanoidLocationSit --cfg_env InterScene/data/cfg/humanoid_location_sit.yaml --cfg_train InterScene/data/cfg/train/rlg/amp_task_location_sit.yaml --motion_file InterScene/data/dataset_samp_sit/dataset_samp_sit.yaml --num_envs 4096 --headless

## evaluating sit policy
python InterScene/run.py --task HumanoidLocationSit --cfg_env InterScene/data/cfg/humanoid_location_sit.yaml --cfg_train InterScene/data/cfg/train/rlg/amp_task_location_sit.yaml --motion_file InterScene/data/dataset_samp_sit/dataset_samp_sit.yaml --num_envs 4096 --headless --checkpoint InterScene/data/models/policy_sit.pth --test
```

## Citation

```bibtex
@inproceedings{pan2024synthesizing,
  title={Synthesizing physically plausible human motions in 3d scenes},
  author={Pan, Liang and Wang, Jingbo and Huang, Buzhen and Zhang, Junyu and Wang, Haofan and Tang, Xu and Wang, Yangang},
  booktitle={2024 International Conference on 3D Vision (3DV)},
  pages={1498--1507},
  year={2024},
  organization={IEEE}
}
```

## References
This repository is built on the top of the following amazing codebases: 
* Main code framework is from: [ASE](https://github.com/nv-tlabs/ASE)
* Some scripts are from: [Pacer](https://github.com/nv-tlabs/pacer/tree/main), [HuMoR](https://github.com/davrempe/humor)

Please follow the license of the above repositories for the usage of this project.