Awesome-Human-Interaction-Motion-Generation

KO | EN

gitlite — search

#javascript #go #machine-learning #rust #nodejs #windows #awesome #cli #kubernetes #awesome-list #vue #security #mcp #openai

Awesome-Human-Interaction-Motion-Generation

(★ 313)

This repository collects papers on Human-Interaction-Motion-Generation applications. We will update new papers irregularly.

File Explorer

Download Latest Version (.zip)

- interaction_types.png
- timeline.png
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Awesome-Human-Interaction-Motion-Generation

[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/Naereen/StrapDown.js/graphs/commit-activity)
[![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome)
[![PR's Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat)](http://makeapullrequest.com)
## 🔥🔥🔥 [A Survey on Human Interaction Motion Generation](https://arxiv.org/abs/2503.12763)

**Human Interaction Motion Generation** is a rapidly growing field that aims to model how humans interact with others, objects, and their surroundings.
These interactions are crucial for applications in robotics, virtual reality, and animation.
While deep generative models and new datasets have propelled advancements, challenges remain in capturing the complexity of human dynamics in interactive settings.

![Timeline Fig](https://raw.githubusercontent.com/soraproducer/Awesome-Human-Interaction-Motion-Generation/main/figs/timeline.png)

This repository provides a structured collection of research papers and datasets related to human interaction motion generation.
By organizing the literature across human-human, human-object, and human-scene interactions, we hope to offer a valuable reference for researchers and practitioners in this field.

![Interaction Types Fig](https://raw.githubusercontent.com/soraproducer/Awesome-Human-Interaction-Motion-Generation/main/figs/interaction_types.png)

**If you are interested in this project, you can contribute to this repo by pulling requests 😊😊😊**

## 📢 News

🚀 **What's New in This Update**:
- [2025.09.08]: 🔄 Update 6 different HIMG works during 2024-2025, including: DuetGen, SceneMI, CooHOI, HOT3D, HUMOTO, CORE4D
- [2025.03.17]: 🎉 Our Arxiv Pre-print is now available at https://arxiv.org/abs/2503.12763
- [2025.03.16]: 🔥 We release our Human Interaction Motion Generation paper list repo!

## 📜 Citation

If you find our survey useful, please consider citing:

```
@misc{sui2025surveyhumaninteractionmotion,
      title={A Survey on Human Interaction Motion Generation}, 
      author={Kewei Sui and Anindita Ghosh and Inwoo Hwang and Jian Wang and Chuan Guo},
      year={2025},
      eprint={2503.12763},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2503.12763}, 
}
```

## :rainbow: Table of Contents
- [Awesome-Human-Interaction-Motion-Generation](#awesome-human-interaction-motion-generation)
 - [Human Interaction Motion Generation](#human-interaction-motion-generation)
 - [Human-Human Interaction Motion Generation](#human-human-interaction-motion-generation)
 - [Human-Object Interaction Motion Generation](#human-object-interaction-motion-generation)
 - [Human-Scene Interaction Motion Generation](#human-scene-interaction-motion-generation)
 - [Human-Mix Interaction Motion Generation](#human-mix-interaction-motion-generation)
 - [Human Interaction Datasets](#human-interaction-datasets)
 - [Human-Human Interaction Datasets](#human-human-interaction-datasets)
 - [Human-Object Interaction Datasets](#human-object-interaction-datasets)
 - [Human-Scene Interaction Datasets](#human-scene-interaction-datasets)
 - [Human-Mix Interaction Datasets](#human-mix-interaction-datasets)

## Human Interaction Motion Generation

### Human-Human Interaction Motion Generation

| Title | Venue | Year | Code | Project |
|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------:|:----:|:--------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| ![Star](https://img.shields.io/github/stars/anindita127/DuetGen.svg?style=social&label=Star) [**DuetGen: Music Driven Two-Person Dance Generation via Hierarchical Masked Modeling**](https://arxiv.org/pdf/2506.18680) | SIGGRAPH | 2025 | [Github](https://github.com/anindita127/DuetGen) | [Project](https://anindita127.github.io/DuetGen/) |
| ![Star](https://img.shields.io/github/stars/liangxuy/ReGenNet.svg?style=social&label=Star) [**ReGenNet: Towards Human Action-Reaction Synthesis**](https://arxiv.org/pdf/2403.11882) | CVPR | 2024 | [Github](https://github.com/liangxuy/ReGenNet) | [Project](https://liangxuy.github.io/ReGenNet/) |
| ![Star](https://img.shields.io/github/stars/pabloruizponce/in2IN.svg?style=social&label=Star) [**in2IN: Leveraging individual Information to Generate Human INteractions**](https://arxiv.org/pdf/2404.09988) | CVPRW | 2024 | [Github](https://github.com/pabloruizponce/in2IN) | [Project](https://pabloruizponce.github.io/in2IN/) |
| ![Star](https://img.shields.io/github/stars/anindita127/ReMoS.svg?style=social&label=Star) [**ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions**](https://arxiv.org/pdf/2311.17057) | ECCV | 2024 | [Github](https://github.com/anindita127/ReMoS) | [Project](https://vankouf.github.io/FreeMotion/) |
| ![Star](https://img.shields.io/github/stars/VankouF/FreeMotion-Codes.svg?style=social&label=Star) [**FreeMotion: A Unified Framework for Number-free Text-to-Motion Synthesis**](https://arxiv.org/pdf/2405.15763) | ECCV | 2024 | [Github](https://github.com/VankouF/FreeMotion-Codes) | [Project](https://vcai.mpi-inf.mpg.de/projects/remos/) |
| [**Towards Open Domain Text-Driven Synthesis of Multi-Person Motions**](https://arxiv.org/pdf/2405.18483) | ECCV | 2024 | Github | [Project](https://shanmy.github.io/Multi-Motion/) |
| ![Star](https://img.shields.io/github/stars/lisiyao21/Duolando.svg?style=social&label=Star) [**Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment**](https://arxiv.org/pdf/2403.18811) | ICLR | 2024 | [Github](https://github.com/lisiyao21/Duolando) | [Project](https://lisiyao21.github.io/projects/Duolando/) |
| ![Star](https://img.shields.io/github/stars/priorMDM/priorMDM.svg?style=social&label=Star) [**PriorMDM: Human Motion Diffusion as a Generative Prior**](https://arxiv.org/pdf/2303.01418) | ICLR | 2024 | [Github](https://github.com/priorMDM/priorMDM) | [Project](https://priormdm.github.io/priorMDM-page/) |
| ![Star](https://img.shields.io/github/stars/tr3e/InterGen.svg?style=social&label=Star) [**InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions**](https://arxiv.org/pdf/2304.05684) | IJCV | 2024 | [Github](https://github.com/tr3e/InterGen) | [Project](https://tr3e.github.io/intergen-page/) |
| ![Star](https://img.shields.io/github/stars/zhenzhiwang/intercontrol.svg?style=social&label=Star) [**InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint**](https://arxiv.org/pdf/2311.15864) | NeurIPS | 2024 | [Github](https://github.com/zhenzhiwang/intercontrol) | Project |
| ![Star](https://img.shields.io/github/stars/jutanke/social_diffusion.svg?style=social&label=Star) [**Social Diffusion: Long-term Multiple Human Motion Anticipation**](https://openaccess.thecvf.com/content/ICCV2023/papers/Tanke_Social_Diffusion_Long-term_Multiple_Human_Motion_Anticipation_ICCV_2023_paper.pdf) | ICCV | 2023 | [Github](https://github.com/jutanke/social_diffusion) | Project |
| ![Star](https://img.shields.io/github/stars/line/Human-Interaction-Generation.svg?style=social&label=Star) [**Role-Aware Interaction Generation from Textual Description**](https://openaccess.thecvf.com/content/ICCV2023/papers/Tanaka_Role-Aware_Interaction_Generation_from_Textual_Description_ICCV_2023_paper.pdf) | ICCV | 2023 | [Github](https://github.com/line/Human-Interaction-Generation) | Project |
| ![Star](https://img.shields.io/github/stars/Szy-Young/ActFormer.svg?style=social&label=Star) [**ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generation**](https://arxiv.org/pdf/2203.07706) | ICCV | 2023 | [Github](https://github.com/Szy-Young/ActFormer) | [Project](https://liangxuy.github.io/actformer/) |
| ![Star](https://img.shields.io/github/stars/CRISTAL-3DSAM/InterFormer.svg?style=social&label=Star) [**Interaction Transformer for Human Reaction Generation**](https://arxiv.org/pdf/2207.01685) | TMM | 2023 | [Github](https://github.com/CRISTAL-3DSAM/InterFormer) | Project |
| ![Star](https://img.shields.io/github/stars/evendrow/somoformer.svg?style=social&label=Star) [**SoMoFormer: Multi-Person Pose Forecasting with Transformers**](https://arxiv.org/pdf/2208.14023) | WACV | 2023 | [Github](https://github.com/evendrow/somoformer) | Project |
| ![Star](https://img.shields.io/github/stars/skelemoa/dsag.svg?style=social&label=Star) [**DSAG: A Scalable Deep Framework for Action-Conditioned Multi-Actor Full Body Motion Synthesis**](https://openaccess.thecvf.com/content/WACV2023/papers/Gupta_DSAG_A_Scalable_Deep_Framework_for_Action-Conditioned_Multi-Actor_Full_Body_WACV_2023_paper.pdf) | WACV | 2023 | [Github](https://github.com/skelemoa/dsag) | [Project](https://skeleton.iiit.ac.in/dsag) |
| [**GAN-based Reactive Motion Synthesis with Class-aware Discriminators for Human-human Interaction**](https://arxiv.org/pdf/2110.00380) | C&G | 2022 | Github | Project |
| ![Star](https://img.shields.io/github/stars/GUO-W/MultiMotion.svg?style=social&label=Star) [**Multi-Person Extreme Motion Prediction**](https://arxiv.org/pdf/2105.08825) | CVPR | 2022 | [Github](https://github.com/GUO-W/MultiMotion) | [Project](https://team.inria.fr/robotlearn/multi-person-extreme-motion-prediction/) |
| ![Star](https://img.shields.io/github/stars/Aman-Goel1/IMM.svg?style=social&label=Star) [**Interaction Mix and Match: Synthesizing Close Interaction using Conditional Hierarchical GAN with Multi-Hot Class Embedding**](https://arxiv.org/pdf/2208.00774) | SCA | 2022 | [Github](https://github.com/Aman-Goel1/IMM) | Project |
| ![Star](https://img.shields.io/github/stars/skelemoa/mugl.svg?style=social&label=Star) [**MUGL: Large Scale Multi Person Conditional Action Generation with Locomotion**](https://arxiv.org/pdf/2110.11460) | WACV | 2022 | [Github](https://github.com/skelemoa/mugl) | [Project](https://skeleton.iiit.ac.in/mugl) |
| [**TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild**](https://arxiv.org/pdf/2104.04029) | ICCV | 2021 | Github | [Project](https://somof.stanford.edu/) |
| ![Star](https://img.shields.io/github/stars/jiashunwang/MRT.svg?style=social&label=Star) [**Multi-Person 3D Motion Prediction with Multi-Range Transformers**](https://arxiv.org/pdf/2111.12073) | NeurIPS | 2021 | [Github](https://github.com/jiashunwang/MRT) | [Project](https://jiashunwang.github.io/MRT/) |
| [**A Multimodal Predictive Agent Model for Human Interaction Generation**](https://openaccess.thecvf.com/content_CVPRW_2020/papers/w66/Baruah_A_Multimodal_Predictive_Agent_Model_for_Human_Interaction_Generation_CVPRW_2020_paper.pdf) | CVPRW | 2020 | Github | Project |
| [**Socially and Contextually Aware Human Motion and Pose Forecasting**](https://arxiv.org/pdf/2007.06843) | RA-L | 2020 | Github | [Project](https://somof.stanford.edu/) |
| [**Statistics-based Motion Synthesis for Social Conversations**](https://www.cs.cmu.edu/~dyadic-conversation/supp/Yang2020.pdf) | SIGGRAPH | 2020 | Github | [Project](https://www.cs.cmu.edu/~dyadic-conversation/) |
| [**Cross-Conditioned Recurrent Networks for Long-Term Synthesis of Inter-Person Human Motion Interactions**](https://openaccess.thecvf.com/content_WACV_2020/papers/Kundu_Cross-Conditioned_Recurrent_Networks_for_Long-Term_Synthesis_of_Inter-Person_Human_Motion_WACV_2020_paper.pdf) | WACV | 2020 | Github | Project |
| ![Star](https://img.shields.io/github/stars/CMU-Perceptual-Computing-Lab/ssp.svg?style=social&label=Star) [**Towards Social Artificial Intelligence: Nonverbal Social Signal Prediction in A Triadic Interaction**](https://arxiv.org/pdf/1906.04158) | CVPR | 2019 | [Github](https://github.com/CMU-Perceptual-Computing-Lab/ssp) | [Project](http://domedb.perception.cs.cmu.edu/ssp.html) |
| [**To React or not to React: End-to-End Visual Pose Forecasting for Personalized Avatar during Dyadic Conversations**](https://arxiv.org/pdf/1910.02181) | ICMI | 2019 | Github | [Project](https://ai.meta.com/research/publications/to-react-or-not-to-react-end-to-end-visual-pose-forecasting-for-personalized-avatar-during-dyadic-conversations/) |
| [**Performance-Driven Dance Motion Control of a Virtual Partner Character**](https://web.ics.purdue.edu/~cmousas/papers/conf18-IEEEVR.pdf) | VR | 2018 | Github | Project |
| [**Action-Reaction: Forecasting the Dynamics of Human Interaction**](https://www.ri.cmu.edu/pub_files/2014/9/ECCV14_DAF.pdf) | ECCV | 2014 | Github | Project |

[🎯Back to Top](#head-content)

### Human-Object Interaction Motion Generation

| Title | Venue | Year | Code | Project |
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------:|:----:|:--------------------------------------------------------:|:-------------------------------------------------------------:|
| [**BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objects**](https://arxiv.org/pdf/2412.05066v1) | CVPR | 2025 | Github | [Project](https://vcai.mpi-inf.mpg.de/projects/bimart/) | 
|![Star](https://img.shields.io/github/stars/ptrvilya/tridi.svg?style=social&label=Star) [**TriDi: Trilateral Diffusion of 3D Humans, Objects, and Interactions**](https://arxiv.org/pdf/2412.06334) | ICCV | 2025 | [Github](https://github.com/ptrvilya/tridi?tab=readme-ov-file) | [Project](https://virtualhumans.mpi-inf.mpg.de/tridi/)|
| [**3D Whole-body Grasp Synthesis with Directional Controllability**](https://arxiv.org/pdf/2408.16770v1) | | 2024 | Github | Project |
| [**GraspDiffusion: Synthesizing Realistic Whole-body Hand-Object Interaction**](https://arxiv.org/pdf/2410.13911) | | 2024 | Github | Project |
| ![Star](https://img.shields.io/github/stars/zdchan/artigrasp.svg?style=social&label=Star) [**ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulation**](https://arxiv.org/pdf/2309.03891) | 3DV | 2024 | [Github](https://github.com/zdchan/artigrasp) | [Project](https://eth-ait.github.io/artigrasp/) |
| ![Star](https://img.shields.io/github/stars/otaheri/GRIP.svg?style=social&label=Star) [**GRIP: Generating Interaction Poses Using Spatial Cues and Latent Consistency**](https://arxiv.org/pdf/2308.11617) | 3DV | 2024 | [Github](https://github.com/otaheri/GRIP) | [Project](https://grip.is.tue.mpg.de/) |
| [**MACS: Mass Conditioned 3D Hand and Object Motion Synthesis**](https://arxiv.org/pdf/2312.14929) | 3DV | 2024 | Github | [Project](https://vcai.mpi-inf.mpg.de/projects/MACS/) |
| ![Star](https://img.shields.io/github/stars/eth-ait/phys-fullbody-grasp.svg?style=social&label=Star) [**Physically Plausible Full-Body Hand-Object Interaction Synthesis**](https://arxiv.org/pdf/2309.07907) | 3DV | 2024 | [Github](https://github.com/eth-ait/phys-fullbody-grasp) | [Project](https://eth-ait.github.io/phys-fullbody-grasp/) |
| [**CG-HOI: Contact-Guided 3D Human-Object Interaction Generation**](https://openaccess.thecvf.com/content/CVPR2024/papers/Diller_CG-HOI_Contact-Guided_3D_Human-Object_Interaction_Generation_CVPR_2024_paper.pdf) | CVPR | 2024 | Github | [Project](https://www.christian-diller.de/projects/cg-hoi/) |
| ![Star](https://img.shields.io/github/stars/JudyYe/ghop.svg?style=social&label=Star) [**G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis**](https://arxiv.org/pdf/2404.12383) | CVPR | 2024 | [Github](https://github.com/JudyYe/ghop) | [Project](https://judyye.github.io/ghop-www/) |
| ![Star](https://img.shields.io/github/stars/kzhou23/gears.svg?style=social&label=Star) [**GEARS: Local Geometry-aware Hand-object Interaction Synthesis**](https://openaccess.thecvf.com/content/CVPR2024/papers/Zhou_GEARS_Local_Geometry-aware_Hand-object_Interaction_Synthesis_CVPR_2024_paper.pdf) | CVPR | 2024 | [Github](https://github.com/kzhou23/gears) | [Project](https://virtualhumans.mpi-inf.mpg.de/gears/) |
| ![Star](https://img.shields.io/github/stars/Mq-Zhang1/HOIDiffusion.svg?style=social&label=Star) [**HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data**](https://arxiv.org/pdf/2403.12011) | CVPR | 2024 | [Github](https://github.com/Mq-Zhang1/HOIDiffusion) | [Project](https://mq-zhang1.github.io/HOIDiffusion/) |
| ![Star](https://img.shields.io/github/stars/AfterJourney00/IMHD-Dataset.svg?style=social&label=Star) [**I’M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactions**](https://openaccess.thecvf.com/content/CVPR2024/papers/Zhao_IM_HOI_Inertia-aware_Monocular_Capture_of_3D_Human-Object_Interactions_CVPR_2024_paper.pdf) | CVPR | 2024 | [Github](https://github.com/AfterJourney00/IMHD-Dataset) | [Project](https://afterjourney00.github.io/IM-HOI.github.io/) |
| ![Star](https://img.shields.io/github/stars/jyunlee/InterHandGen.svg?style=social&label=Star) [**InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusion**](https://arxiv.org/pdf/2403.17422) | CVPR | 2024 | [Github](https://github.com/jyunlee/InterHandGen) | [Project](https://jyunlee.github.io/projects/interhandgen/) |
| ![Star](https://img.shields.io/github/stars/nileshkulkarni/nifty.svg?style=social&label=Star) [**NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis**](https://openaccess.thecvf.com/content/CVPR2024/papers/Kulkarni_NIFTY_Neural_Object_Interaction_Fields_for_Guided_Human_Motion_Synthesis_CVPR_2024_paper.pdf) | CVPR | 2024 | [Github](https://github.com/nileshkulkarni/nifty) | [Project](https://nileshkulkarni.github.io/nifty/) |
| ![Star](https://img.shields.io/github/stars/JunukCha/Text2HOI.svg?style=social&label=Star) [**Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction**](https://arxiv.org/pdf/2404.00562v2) | CVPR | 2024 | [Github](https://github.com/JunukCha/Text2HOI) | [Project](https://junukcha.github.io/project/text2hoi/) |
| ![Star](https://img.shields.io/github/stars/lijiaman/chois_release.svg?style=social&label=Star) [**Controllable Human-Object Interaction Synthesis**](https://arxiv.org/pdf/2312.03913) | ECCV | 2024 | [Github](https://github.com/lijiaman/chois_release) | [Project](https://lijiaman.github.io/projects/chois/) |
| [**F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactions**](https://arxiv.org/pdf/2407.12435) | ECCV | 2024 | Github | [Project](https://f-hoi.github.io/) |
| ![Star](https://img.shields.io/github/stars/LvXinTao/HIMO_dataset.svg?style=social&label=Star) [**HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects**](https://arxiv.org/pdf/2407.12371) | ECCV | 2024 | [Github](https://github.com/LvXinTao/HIMO_dataset) | [Project](https://lvxintao.github.io/himo/) |
| ![Star](https://img.shields.io/github/stars/sisidai/InterFusion.svg?style=social&label=Star) [**InterFusion: Text-Driven Generation of 3D Human-Object Interaction**](https://arxiv.org/pdf/2403.15612) | ECCV | 2024 | [Github](https://github.com/sisidai/InterFusion) | [Project](https://sisidai.github.io/InterFusion/) |
| ![Star](https://img.shields.io/github/stars/Meowuu7/GeneOH-Diffusion.svg?style=social&label=Star) [**GeneOH Diffusion: Towards Generalizable Hand-Object Interaction Denoising via Denoising Diffusion**](https://arxiv.org/pdf/2402.14810) | ICLR | 2024 | [Github](https://github.com/Meowuu7/GeneOH-Diffusion) | [Project](https://meowuu7.github.io/GeneOH-Diffusion/) |
| [**InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction**](https://proceedings.neurips.cc/paper_files/paper/2024/file/5eca2e4fe7858cbbfef4e08573cfcb25-Paper-Conference.pdf) | NeurIPS | 2024 | Github | [Project](https://sirui-xu.github.io/InterDreamer/) |
| [**DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptions**](https://arxiv.org/pdf/2403.17827) | SIGGRAPH Asia | 2024 | Github | [Project](https://diffh2o.github.io/) |
| [**GraspDiff: Grasping Generation for Hand-Object Interaction with Multimodal Guided Diffusion**](https://ieeexplore.ieee.org/document/10689328) | TVCG | 2024 | Github | Project |
| [**Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations**](https://openaccess.thecvf.com/content/WACV2024/papers/Li_Task-Oriented_Human-Object_Interactions_Generation_With_Implicit_Neural_Representations_WACV_2024_paper.pdf) | WACV | 2024 | Github | Project |
| ![Star](https://img.shields.io/github/stars/neu-vi/HOI-Diff.svg?style=social&label=Star) [**HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Models**](https://arxiv.org/pdf/2312.06553) | | 2023 | [Github](https://github.com/neu-vi/HOI-Diff) | [Project](https://neu-vi.github.io/HOI-Diff/) |
| ![Star](https://img.shields.io/github/stars/cams-hoi/CAMS.svg?style=social&label=Star) [**CAMS: CAnonicalized Manipulation Spaces for Category-Level Functional Hand-Object Manipulation Synthesis**](https://openaccess.thecvf.com/content/CVPR2023/papers/Zheng_CAMS_CAnonicalized_Manipulation_Spaces_for_Category-Level_Functional_Hand-Object_Manipulation_Synthesis_CVPR_2023_paper.pdf) | CVPR | 2023 | [Github](https://github.com/cams-hoi/CAMS) | [Project](https://cams-hoi.github.io/) |
| ![Star](https://img.shields.io/github/stars/purvaten/FLEX.svg?style=social&label=Star) [**FLEX: Full-Body Grasping Without Full-Body Grasps**](https://arxiv.org/pdf/2211.11903) | CVPR | 2023 | [Github](https://github.com/purvaten/FLEX) | [Project](https://flex.cs.columbia.edu/) |
| ![Star](https://img.shields.io/github/stars/Juzezhang/NeuralDome.svg?style=social&label=Star) [**NeuralDome: A Neural Modeling Pipeline on Multi-View Human-Object Interactions**](https://openaccess.thecvf.com/content/CVPR2023/papers/Zhang_NeuralDome_A_Neural_Modeling_Pipeline_on_Multi-View_Human-Object_Interactions_CVPR_2023_paper.pdf) | CVPR | 2023 | [Github](https://github.com/Juzezhang/NeuralDome) | [Project](https://juzezhang.github.io/NeuralDome/) |
| ![Star](https://img.shields.io/github/stars/anindita127/IMoS.svg?style=social&label=Star) [**IMoS: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions**](https://arxiv.org/pdf/2212.07555) | Eurographics | 2023 | [Github](https://github.com/anindita127/IMoS) | [Project](https://vcai.mpi-inf.mpg.de/projects/IMoS/) |
| ![Star](https://img.shields.io/github/stars/Sirui-Xu/InterDiff.svg?style=social&label=Star) [**InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion**](https://arxiv.org/pdf/2308.16905) | ICCV | 2023 | [Github](https://github.com/Sirui-Xu/InterDiff) | [Project](https://sirui-xu.github.io/InterDiff/) |
| ![Star](https://img.shields.io/github/stars/Ericcsr/synthesize_pregrasp.svg?style=social&label=Star) [**Synthesizing Dexterous Nonprehensile Pregrasp for Ungraspable Objects**](https://arxiv.org/pdf/2305.04654) | SIGGRAPH | 2023 | [Github](https://github.com/Ericcsr/synthesize_pregrasp) | [Project](https://ericcsr.github.io/publication/pregrasp) |
| ![Star](https://img.shields.io/github/stars/lijiaman/omomo_release.svg?style=social&label=Star) [**Object Motion Guided Human Motion Synthesis**](https://arxiv.org/pdf/2309.16237) | SIGGRAPH Asia | 2023 | [Github](https://github.com/lijiaman/omomo_release) | [Project](https://lijiaman.github.io/projects/omomo/) |
| ![Star](https://img.shields.io/github/stars/christsa/dgrasp.svg?style=social&label=Star) [**D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions**](https://arxiv.org/pdf/2112.03028) | CVPR | 2022 | [Github](https://github.com/christsa/dgrasp) | [Project](https://eth-ait.github.io/d-grasp/) |
| ![Star](https://img.shields.io/github/stars/otaheri/GOAL.svg?style=social&label=Star) [**GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping**](https://arxiv.org/pdf/2112.11454) | CVPR | 2022 | [Github](https://github.com/otaheri/GOAL) | [Project](https://goal.is.tue.mpg.de/) |
| ![Star](https://img.shields.io/github/stars/JiahaoPlus/SAGA.svg?style=social&label=Star) [**SAGA: Stochastic Whole-Body Grasping With Contact**](https://arxiv.org/pdf/2112.10103) | ECCV | 2022 | [Github](https://github.com/JiahaoPlus/SAGA) | [Project](https://jiahaoplus.github.io/SAGA/saga.html) |
| ![Star](https://img.shields.io/github/stars/kzhou23/toch.svg?style=social&label=Star) [**TOCH: Spatio-Temporal Object-to-Hand Correspondence for Motion Refinement**](https://virtualhumans.mpi-inf.mpg.de/papers/zhou22toch/toch.pdf) | ECCV | 2022 | [Github](https://github.com/kzhou23/toch) | [Project](https://virtualhumans.mpi-inf.mpg.de/toch/) |
| ![Star](https://img.shields.io/github/stars/rishabhdabral/gravicap.svg?style=social&label=Star) [**Gravity-Aware Monocular 3D Human-Object Reconstruction**](https://arxiv.org/pdf/2108.08844) | ICCV | 2021 | [Github](https://github.com/rishabhdabral/gravicap) | [Project](https://4dqv.mpi-inf.mpg.de/GraviCap/) |
| ![Star](https://img.shields.io/github/stars/cghezhang/ManipNet.svg?style=social&label=Star) [**ManipNet: Neural Manipulation Synthesis with a Hand-Object Spatial Representation**](https://www.ipab.inf.ed.ac.uk/cgvu/zhang2021.pdf) | SIGGRAPH | 2021 | [Github](https://github.com/cghezhang/ManipNet) | Project |
| ![Star](https://img.shields.io/github/stars/enriccorona/GanHand.svg?style=social&label=Star) [**GanHand: Predicting Human Grasp Affordances in Multi-Object Scenes**](https://openaccess.thecvf.com/content_CVPR_2020/papers/Corona_GanHand_Predicting_Human_Grasp_Affordances_in_Multi-Object_Scenes_CVPR_2020_paper.pdf) | CVPR | 2020 | [Github](https://github.com/enriccorona/GanHand) | [Project](https://www.iri.upc.edu/people/ecorona/ganhand/) |

[🎯Back to Top](#head-content)

### Human-Scene Interaction Motion Generation

| Title | Venue | Year | Code | Project |
|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------:|:----:|:----------------------------------------------------------------------:|:-----------------------------------------------------------------------------:|
| ![Star](https://img.shields.io/github/stars/GuyTevet/CLoSD.svg?style=social&label=Star) [**CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control**](https://arxiv.org/pdf/2410.03441) | ICLR | 2025 | [Github](https://github.com/GuyTevet/CLoSD) | [Project](https://guytevet.github.io/CLoSD-page/) |
| ![Star](https://img.shields.io/github/stars/woo0818/SceneMI.svg?style=social&label=Star) [**SceneMI: Motion In-betweening for Modeling Human-Scene Interaction**](https://arxiv.org/pdf/2503.16289) | ICCV | 2025 | [Github](https://github.com/woo0818/SceneMI) | [Project](https://inwoohwang.me/SceneMI/) |
| [**GHOST: Grounded Human Motion Generation with Open Vocabulary Scene-and-Text Contexts**](https://arxiv.org/pdf/2405.18438) | WACV | 2025 | Github | Project |
| [**ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generation**](https://arxiv.org/abs/2412.18600) | - | 2024 | Github | [Project](https://awfuact.github.io/zerohsi/) |
| [**SIMS: Simulating Human-Scene Interactions with Real World Script Planning**](https://arxiv.org/pdf/2411.19921) | - | 2024 | Github | Project |
| ![Star](https://img.shields.io/github/stars/jingyugong/DiffusionImplicitPolicy.svg?style=social&label=Star) [**DIP: Diffusion Implicit Policy for Unpaired Scene-aware Motion Synthesis**](https://arxiv.org/abs/2410.03441) | - | 2024 | [Github](https://github.com/jingyugong/DiffusionImplicitPolicy) | [Project](https://jingyugong.github.io/DiffusionImplicitPolicy/) |
| [**GPT-Connect: Interaction between Text-Driven Human Motion Generator and 3D Scenes in a Training-free Manner**](https://arxiv.org/pdf/2403.14947) | - | 2024 | Github | Project |
| ![Star](https://img.shields.io/github/stars/4DVLab/LaserHuman.svg?style=social&label=Star) [**LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environment**](https://arxiv.org/pdf/2403.13307) | - | 2024 | [Github](https://github.com/4DVLab/LaserHuman) | Project |
| ![Star](https://img.shields.io/github/stars/liangpan99/InterScene.svg?style=social&label=Star) [**InterScene: Synthesizing Physically Plausible Human Motions in 3D Scenes**](https://arxiv.org/pdf/2308.09036) | 3DV | 2024 | [Github](https://github.com/liangpan99/InterScene) | [Project](https://liangpan99.github.io/InterScene/) |
| ![Star](https://img.shields.io/github/stars/nicolasugrinovic/purposer.svg?style=social&label=Star) [**Purposer: Putting Human Motion Generation in Context**](https://arxiv.org/pdf/2404.12942) | 3DV | 2024 | [Github](https://github.com/nicolasugrinovic/purposer) | [Project](https://www.iri.upc.edu/people/nugrinovic/purposer/) |
| ![Star](https://img.shields.io/github/stars/afford-motion/afford-motion.svg?style=social&label=Star) [**Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordance**](https://arxiv.org/pdf/2403.18036) | CVPR | 2024 | [Github](https://github.com/afford-motion/afford-motion) | [Project](https://afford-motion.github.io/) |
| ![Star](https://img.shields.io/github/stars/zju3dv/text_scene_motion.svg?style=social&label=Star) [**Generating Human Motion in 3D Scenes from Text Descriptions**](https://arxiv.org/pdf/2405.07784) | CVPR | 2024 | [Github](https://github.com/zju3dv/text_scene_motion) | [Project](https://zju3dv.github.io/text_scene_motion/) |
| [**GenZI: Zero-Shot 3D Human-Scene Interaction Generation**](https://arxiv.org/pdf/2311.17737) | CVPR | 2024 | Github | [Project](https://craigleili.github.io/projects/genzi/) |
| ![Star](https://img.shields.io/github/stars/kjle6/SIF3D-master.svg?style=social&label=Star) [**SIF3D: Multimodal Sense-Informed Forecasting of 3D Human Motions**](https://arxiv.org/pdf/2405.02911v1) | CVPR | 2024 | [Github](https://github.com/kjle6/SIF3D-master) | [Project](https://sites.google.com/view/cvpr2024sif3d) |
| ![Star](https://img.shields.io/github/stars/jn-tang/S2Fusion.svg?style=social&label=Star) [**A Unified Diffusion Framework for Scene-aware Human Motion Estimation from Sparse Signals**](https://arxiv.org/pdf/2404.04890) | CVPR | 2024 | [Github](https://github.com/jn-tang/S2Fusion) | Project |
| ![Star](https://img.shields.io/github/stars/jnnan/trumans_utils.svg?style=social&label=Star) [**TRUMANS: Scaling Up Dynamic Human-Scene Interaction Modeling**](https://arxiv.org/pdf/2403.08629) | CVPR | 2024 | [Github](https://github.com/jnnan/trumans_utils) | [Project](https://jnnan.github.io/trumans/) |
| ![Star](https://img.shields.io/github/stars/nv-tlabs/tesmo.svg?style=social&label=Star) [**TeSMo: Generating Human Interaction Motions in Scenes with Text Control**](https://arxiv.org/pdf/2404.10685) | ECCV | 2024 | [Github](https://github.com/nv-tlabs/tesmo) | [Project](https://research.nvidia.com/labs/toronto-ai/tesmo/) |
| ![Star](https://img.shields.io/github/stars/HaowenHou/Motion-Occupancy-Base.svg?style=social&label=Star) [**MOB: Revisit Human-Scene Interaction via Space Occupancy**](https://arxiv.org/pdf/2312.02700) | ECCV | 2024 | [Github](https://github.com/HaowenHou/Motion-Occupancy-Base) | [Project](https://foruck.github.io/occu-page/) |
| ![Star](https://img.shields.io/github/stars/xccyue/MutualDistance.svg?style=social&label=Star) [**MutualDistance: Scene-aware Human Motion Forecasting via Mutual Distance Prediction**](https://arxiv.org/pdf/2310.00615) | ECCV | 2024 | [Github](https://github.com/xccyue/MutualDistance) | Project |
| ![Star](https://img.shields.io/github/stars/OpenRobotLab/UniHSI.svg?style=social&label=Star) [**UniHSI: Unified Human-Scene Interaction via Prompted Chain-of-Contacts**](https://arxiv.org/pdf/2309.07918) | ICLR | 2024 | [Github](https://github.com/OpenRobotLab/UniHSI) | [Project](https://xizaoqu.github.io/unihsi/) |
| [**Multi-Condition Latent Diffusion Network for Scene-Aware Neural Human Motion Prediction**](https://arxiv.org/pdf/2405.18700) | IEEE TIP | 2024 | Github | Project |
| ![Star](https://img.shields.io/github/stars/mileret/lingo-release.svg?style=social&label=Star) [**LINGO: Autonomous Character-Scene Interaction Synthesis from Text Instruction**](https://arxiv.org/pdf/2410.03187) | SIGGRAPH Asia | 2024 | [Github](https://github.com/mileret/lingo-release) | [Project](https://lingomotions.com/) |
| [**Generating Continual Human Motion in Diverse 3D Scenes**](https://arxiv.org/pdf/2304.02061) | 3DV | 2023 | Github | [Project](https://virtualhumans.mpi-inf.mpg.de/origin_2/) |
| ![Star](https://img.shields.io/github/stars/Stanford-TML/circle_dataset.svg?style=social&label=Star) [**CIRCLE: Capture In Rich Contextual Environments**](https://arxiv.org/pdf/2303.17912) | CVPR | 2023 | [Github](https://github.com/Stanford-TML/circle_dataset) | [Project](https://stanford-tml.github.io/circle_dataset/) |
| ![Star](https://img.shields.io/github/stars/scenediffuser/Scene-Diffuser.svg?style=social&label=Star) [**SceneDiffuser: Diffusion-based Generation, Optimization, and Planning in 3D Scenes**](https://arxiv.org/abs/2410.03441) | CVPR | 2023 | [Github](https://github.com/scenediffuser/Scene-Diffuser) | [Project](https://scenediffuser.github.io/) |
| ![Star](https://img.shields.io/github/stars/zkf1997/DIMOS.svg?style=social&label=Star) [**DINOS: Synthesizing Diverse Human Motions in 3D Indoor Scenes**](https://arxiv.org/pdf/2305.12411) | ICCV | 2023 | [Github](https://github.com/zkf1997/DIMOS) | [Project](https://zkf1997.github.io/DIMOS/) |
| ![Star](https://img.shields.io/github/stars/jiyewise/LAMA.svg?style=social&label=Star) [**Locomotion-Action-Manipulation: Synthesizing Human-Scene Interactions in Complex 3D Environments**](https://arxiv.org/pdf/2301.02667) | ICCV | 2023 | [Github](https://github.com/jiyewise/LAMA/) | [Project](https://jiyewise.github.io/projects/LAMA/) |
| ![Star](https://img.shields.io/github/stars/HaibiaoXuan/Narrator.svg?style=social&label=Star) [**Narrator: Towards Natural Control of Human-Scene Interaction Generation via Relationship Reasoning**](https://arxiv.org/pdf/2303.09410) | ICCV | 2023 | [Github](https://github.com/HaibiaoXuan/Narrator) | [Project](https://haibiaoxuan.github.io/Narrator/) |
| [**MAMMOS: MApping Multiple human MOtion with Scene understanding and natural interactions**](https://openaccess.thecvf.com/content/ICCV2023W/CV4Metaverse/papers/Lim_MAMMOS_MApping_Multiple_Human_MOtion_with_Scene_Understanding_and_Natural_ICCVW_2023_paper.pdf) | ICCVW | 2023 | Github | Project |
| [**Synthesizing Physical Character-Scene Interactions**](https://arxiv.org/pdf/2302.00883) | SIGGRAPH | 2023 | Github | Project |
| [**PAAK: Placing Human Animations into 3D Scenes by Learning Interaction- and Geometry-Driven Keyframes**](https://arxiv.org/pdf/2209.06314) | WACV | 2023 | Github | [Project](https://gamma.umd.edu/researchdirections/animation-placement/paak/) |
| ![Star](https://img.shields.io/github/stars/yz-cnsdqz/GAMMA-release.svg?style=social&label=Star) [**GAMMA: The Wanderings of Odysseus in 3D Scenes**](https://arxiv.org/pdf/2112.09251) | CVPR | 2022 | [Github](https://github.com/yz-cnsdqz/GAMMA-release) | [Project](https://yz-cnsdqz.github.io/eigenmotion/GAMMA/) |
| [**Towards diverse and natural scene-aware 3d human motion synthesis**](https://arxiv.org/pdf/2205.13001) | CVPR | 2022 | Github | Project |
| ![Star](https://img.shields.io/github/stars/zkf1997/COINS.svg?style=social&label=Star) [**COINS: Compositional Human-Scene Interaction Synthesis with Semantic Control**](https://arxiv.org/pdf/2207.12824) | ECCV | 2022 | [Github](https://github.com/zkf1997/COINS) | [Project](https://zkf1997.github.io/COINS/index.html) |
| ![Star](https://img.shields.io/github/stars/Silverster98/HUMANISE.svg?style=social&label=Star) [**HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes**](https://arxiv.org/pdf/2210.09729) | NeurIPS | 2022 | [Github](https://github.com/Silverster98/HUMANISE) | [Project](https://silvester.wang/HUMANISE/) |
| ![Star](https://img.shields.io/github/stars/jiashunwang/Long-term-Motion-in-3D-Scenes.svg?style=social&label=Star) [**Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes**](https://arxiv.org/pdf/2012.05522) | CVPR | 2021 | [Github](https://github.com/jiashunwang/Long-term-Motion-in-3D-Scenes) | [Project](https://jiashunwang.github.io/Long-term-Motion-in-3D-Scenes/) |
| ![Star](https://img.shields.io/github/stars/mohamedhassanmus/SAMP.svg?style=social&label=Star) [**SAMP: Stochastic Scene-Aware Motion Prediction**](https://arxiv.org/abs/2410.03441) | ICCV | 2021 | [Github](https://github.com/mohamedhassanmus/SAMP) | [Project](https://samp.is.tue.mpg.de/) |
| ![Star](https://img.shields.io/github/stars/sanweiliti/PLACE.svg?style=social&label=Star) [**PLACE: Proximity Learning of Articulation and Contact in 3D Environments**](https://arxiv.org/pdf/2008.05570) | 3DV | 2020 | [Github](https://github.com/sanweiliti/PLACE) | [Project](https://sanweiliti.github.io/PLACE/PLACE.html) |
| ![Star](https://img.shields.io/github/stars/yz-cnsdqz/PSI-release.svg?style=social&label=Star) [**Generating 3D People in Scenes without People**](https://arxiv.org/pdf/1912.02923) | CVPR | 2020 | [Github](https://github.com/yz-cnsdqz/PSI-release) | Project |
| ![Star](https://img.shields.io/github/stars/wei-mao-2019/ContAwareMotionPred.svg?style=social&label=Star) [**Long-term Human Motion Prediction with Scene Context**](https://arxiv.org/pdf/2007.03672) | ECCV | 2020 | [Github](https://github.com/wei-mao-2019/ContAwareMotionPred) | [Project](https://zhec.github.io/hmp/) |
| [**Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments**](https://arxiv.org/pdf/1903.05690) | CVPR | 2019 | Github | [Project](https://sites.google.com/view/3d-affordance-cvpr19) |
| [**Neural state machine for character-scene interactions**](https://dl.acm.org/doi/pdf/10.1145/3355089.3356505) | SIGGRAPH Asia | 2019 | Github | Project |
| ![Star](https://img.shields.io/github/stars/msavva/pigraphs.svg?style=social&label=Star) [**PiGraphs: Learning Interaction Snapshots from Observations**](https://dl.acm.org/doi/pdf/10.1145/2897824.2925867) | SIGGRAPH | 2016 | [Github](https://github.com/msavva/pigraphs) | [Project](https://graphics.stanford.edu/projects/pigraphs/) |

[🎯Back to Top](#head-content)

### Human-Mix Interaction Motion Generation

[🎯Back to Top](#head-content)

## Human Interaction Datasets

### Human-Human Interaction Datasets

| Dataset | Title | Venue | Year | Code | Project |
|:---------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------:|:----:|:-----------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------:|
| Inter-X | ![Star](https://img.shields.io/github/stars/liangxuy/Inter-X.svg?style=social&label=Star) [**Inter-X: Towards Versatile Human-Human Interaction Analysis**](https://arxiv.org/pdf/2312.16051) | CVPR | 2024 | [Github](https://github.com/liangxuy/Inter-X) | [Project](https://liangxuy.github.io/inter-x/) |
| ReMoCap | ![Star](https://img.shields.io/github/stars/anindita127/ReMoS.svg?style=social&label=Star) [**ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions**](https://arxiv.org/pdf/2311.17057) | ECCV | 2024 | [Github](https://github.com/anindita127/ReMoS) | [Project](https://vcai.mpi-inf.mpg.de/projects/remos/) |
| LAION-Pose & WebVid-Motion | [**Towards Open Domain Text-Driven Synthesis of Multi-Person Motions**](https://arxiv.org/pdf/2405.18483) | ECCV | 2024 | Github | [Project](https://shanmy.github.io/Multi-Motion/) |
| DD100 | ![Star](https://img.shields.io/github/stars/lisiyao21/Duolando.svg?style=social&label=Star) [**Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment**](https://arxiv.org/pdf/2403.18811) | ICLR | 2024 | [Github](https://github.com/lisiyao21/Duolando) | [Project](https://lisiyao21.github.io/projects/Duolando/) |
| InterHuman | ![Star](https://img.shields.io/github/stars/tr3e/InterGen.svg?style=social&label=Star) [**InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions**](https://arxiv.org/pdf/2304.05684) | IJCV | 2024 | [Github](https://github.com/tr3e/InterGen) | [Project](https://tr3e.github.io/intergen-page/) |
| Hi4D | ![Star](https://img.shields.io/github/stars/yifeiyin04/Hi4D.svg?style=social&label=Star) [**Hi4D: 4D Instance Segmentation of Close Human Interaction**](https://arxiv.org/pdf/2303.15380) | CVPR | 2023 | [Github](https://github.com/yifeiyin04/Hi4D) | [Project](https://yifeiyin04.github.io/Hi4D/) |
| GTA Combat | ![Star](https://img.shields.io/github/stars/Szy-Young/ActFormer.svg?style=social&label=Star) [**ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generation**](https://arxiv.org/pdf/2203.07706) | ICCV | 2023 | [Github](https://github.com/Szy-Young/ActFormer) | [Project](https://liangxuy.github.io/actformer/) |
| ExPI | ![Star](https://img.shields.io/github/stars/GUO-W/MultiMotion.svg?style=social&label=Star) [**Multi-person extreme motion prediction**](https://arxiv.org/pdf/2105.08825) | CVPR | 2022 | [Github](https://github.com/GUO-W/MultiMotion) | [Project](https://team.inria.fr/robotlearn/multi-person-extreme-motion-prediction/) |
| MultiHuman | ![Star](https://img.shields.io/github/stars/DSaurus/DeepMultiCap.svg?style=social&label=Star) [**DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras**](https://arxiv.org/pdf/2105.00261) | ICCV | 2021 | [Github](https://github.com/DSaurus/DeepMultiCap) | [Project](https://liuyebin.com/dmc/dmc.html) |
| Chi3D | [**Three-Dimensional Reconstruction of Human Interactions**](https://openaccess.thecvf.com/content_CVPR_2020/papers/Fieraru_Three-Dimensional_Reconstruction_of_Human_Interactions_CVPR_2020_paper.pdf) | CVPR | 2020 | Github | [Project](https://ci3d.imar.ro/chi3d) |
| You2Me | ![Star](https://img.shields.io/github/stars/facebookresearch/you2me.svg?style=social&label=Star) [**You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions**](https://arxiv.org/pdf/1904.09882) | CVPR | 2020 | [Github](https://github.com/facebookresearch/you2me) | Project |
| 2C | ![Star](https://img.shields.io/github/stars/hubertshum/InHAC-Interaction-Based-Human-Activity-Comparison.svg?style=social&label=Star) [**Interaction-based Human Activity Comparison**](http://hubertshum.com/publications/tvcg2020interaction/files/tvcg2020interaction.pdf) | TVCG | 2020 | [Github](https://github.com/hubertshum/InHAC-Interaction-Based-Human-Activity-Comparison) | [Project](http://hubertshum.com/pbl_tvcg2020interaction.htm) |
| DuetDance | [**Cross-Conditioned Recurrent Networks for Long-Term Synthesis of Inter-Person Human Motion Interactions**](https://openaccess.thecvf.com/content_WACV_2020/papers/Kundu_Cross-Conditioned_Recurrent_Networks_for_Long-Term_Synthesis_of_Inter-Person_Human_Motion_WACV_2020_paper.pdf) | WACV | 2020 | Github | Project |
| Haggling | ![Star](https://img.shields.io/github/stars/CMU-Perceptual-Computing-Lab/ssp.svg?style=social&label=Star) [**Towards Social Artificial Intelligence: Nonverbal Social Signal Prediction in A Triadic Interaction**](https://arxiv.org/pdf/1906.04158) | CVPR | 2019 | [Github](https://github.com/CMU-Perceptual-Computing-Lab/ssp) | [Project](http://domedb.perception.cs.cmu.edu/ssp.html) |
| Talking With Hands 16.2M | ![Star](https://img.shields.io/github/stars/facebookresearch/TalkingWithHands32M.svg?style=social&label=Star) [**Talking With Hands 16.2M: A Large-Scale Dataset of Synchronized Body-Finger Motion and Audio for Conversational Motion Analysis and Synthesis**](https://openaccess.thecvf.com/content_ICCV_2019/papers/Lee_Talking_With_Hands_16.2M_A_Large-Scale_Dataset_of_Synchronized_Body-Finger_ICCV_2019_paper.pdf) | ICCV | 2019 | [Github](https://github.com/facebookresearch/TalkingWithHands32M) | Project |
| NTU RGB+D 120 | ![Star](https://img.shields.io/github/stars/shahroudy/NTURGB-D.svg?style=social&label=Star) [**NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding**](https://arxiv.org/pdf/1905.04757) | TPAMI | 2019 | [Github](https://github.com/shahroudy/NTURGB-D) | [Project](https://rose1.ntu.edu.sg/dataset/actionRecognition/) |
| MuPoTS-3D | [**Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB**](https://arxiv.org/pdf/1712.03453) | 3DV | 2018 | Github | [Project](https://vcai.mpi-inf.mpg.de/projects/SingleShotMultiPerson/) |
| 3DPW | [**Recovering Accurate 3D Human Pose in The Wild Using IMUs and a Moving Camera**](https://openaccess.thecvf.com/content_ECCV_2018/papers/Timo_von_Marcard_Recovering_Accurate_3D_ECCV_2018_paper.pdf) | ECCV | 2018 | Github | [Project](https://virtualhumans.mpi-inf.mpg.de/3DPW/) |
| JTA | ![Star](https://img.shields.io/github/stars/fabbrimatteo/JTA-Dataset.svg?style=social&label=Star) [**Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World**](https://arxiv.org/pdf/1803.08319) | ECCV | 2018 | [Github](https://github.com/fabbrimatteo/JTA-Dataset) | Project |
| CMU-Panoptic | [**Panoptic Studio: A Massively Multiview System for Social Interaction Capture**](https://arxiv.org/pdf/1612.03153) | TPAMI | 2017 | Github | [Project](http://domedb.perception.cs.cmu.edu/) |
| ShakeFive2 | [**Spatio-Temporal Detection of Fine-Grained Dyadic Human Interactions**](https://www.projects.science.uu.nl/shakefive/doc/VanGemerenEtAl_ECCV2016HBU.pdf) | HBU | 2016 | Github | [Project](https://www.projects.science.uu.nl/shakefive/) |
| MARCOnI | [**Efficient ConvNet-based Marker-less Motion Capture in General Scenes with a Low Number of Cameras**](https://openaccess.thecvf.com/content_cvpr_2015/papers/Elhayek_Efficient_ConvNet-Based_Marker-Less_2015_CVPR_paper.pdf) | CVPR | 2015 | Github | [Project](https://marconi.mpi-inf.mpg.de/) |
| K3HI | [**Efficient Interaction Recognition through Positive Action Representation**](https://onlinelibrary.wiley.com/doi/pdf/10.1155/2013/795360) | Math. Probl. Eng. | 2013 | Github | [Project](http://www.lmars.whu.edu.cn/prof_web/zhuxinyan/DataSetPublish/dataset.html) |
| SBU Kinect | ![Star](https://img.shields.io/github/stars/xrenaa/SBU_Kinect_dataset_process.svg?style=social&label=Star) [**Two-person Interaction Detection Using Body-Pose Features and Multiple Instance Learning**](http://www.tamaraberg.com/papers/kiwon_hau3d12.pdf) | CVPRW | 2012 | [Github](https://github.com/xrenaa/SBU_Kinect_dataset_process) | [Project](https://cove.thecvf.com/datasets/57) |
| MHHI | [**Markerless Motion Capture of Interacting Characters Using Multi-view Image Segmentation**](http://gall.cv-uni-bonn.de/download/jgall_multitrack_cvpr11.pdf) | CVPR | 2011 | Github | Project |
| UMPM | [**UMPM benchmark: a multi-person dataset with synchronized video and motion capture data for evaluation of articulated human motion and interaction**](https://www.projects.science.uu.nl/umpm/HICV0005.pdf) | ICCVW | 2011 | Github | [Project](https://www.projects.science.uu.nl/umpm/) |
| UT-Interaction | [**ICPR 2010 Contest on Semantic Description of Human Activities**](https://cvrc.ece.utexas.edu/SDHA2010/Human_Interaction.html) | ICPR | 2010 | Github | [Project](https://cvrc.ece.utexas.edu/SDHA2010/Human_Interaction.html) |
| CMU Graphics Lab MoCap | [**CMU Graphics Lab Motion Capture Database**](http://mocap.cs.cmu.edu/) | - | 2008 | Github | [Project](http://mocap.cs.cmu.edu/) |

[🎯Back to Top](#head-content)

### Human-Object Interaction Datasets

| Dataset | Title | Venue | Year | Code | Project |
|:------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------:|:----:|:----------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------:|
| HOT3D | ![Star](https://img.shields.io/github/stars/facebookresearch/hot3d.svg?style=social&label=Star) [**HOT3D: An Egocentric Dataset for 3D Hand and Object Tracking**](https://arxiv.org/pdf/2406.09598) | CVPR | 2025 | [Github](https://github.com/facebookresearch/hot3d/) | [Project](https://facebookresearch.github.io/hot3d/) |
| HUMOTO | [**HUMOTO: A 4D Dataset of Mocap Human Object Interactions**](https://arxiv.org/pdf/2504.10414) | ICCV | 2025 | [Github](https://adobe-research.github.io/humoto/) | [Project](https://jiaxin-lu.github.io/humoto/) |
| IMHD2 | ![Star](https://img.shields.io/github/stars/AfterJourney00/IMHD-Dataset.svg?style=social&label=Star) [**I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactions**](https://arxiv.org/pdf/2312.08869) | CVPR | 2024 | [Github](https://github.com/AfterJourney00/IMHD-Dataset) | [Project](https://afterjourney00.github.io/IM-HOI.github.io/) |
| Semantic-HOI | [**F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactions**](https://arxiv.org/pdf/2407.12435) | ECCV | 2024 | Github | [Project](https://f-hoi.github.io/) |
| HIMO | ![Star](https://img.shields.io/github/stars/LvXinTao/HIMO_dataset.svg?style=social&label=Star) [**HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects**](https://arxiv.org/pdf/2407.12371) | ECCV | 2024 | [Github](https://github.com/LvXinTao/HIMO_dataset) | [Project](https://lvxintao.github.io/himo/) |
| ARCTIC | ![Star](https://img.shields.io/github/stars/zc-alexfan/arctic.svg?style=social&label=Star) [**ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation**](https://arxiv.org/pdf/2204.13662) | CVPR | 2023 | [Github](https://github.com/zc-alexfan/arctic) | [Project](https://arctic.is.tue.mpg.de/) |
| HODome | ![Star](https://img.shields.io/github/stars/Juzezhang/NeuralDome_Toolbox.svg?style=social&label=Star) [**NeuralDome: A Neural Modeling Pipeline on Multi-View Human-Object Interactions**](https://openaccess.thecvf.com/content/CVPR2023/papers/Zhang_NeuralDome_A_Neural_Modeling_Pipeline_on_Multi-View_Human-Object_Interactions_CVPR_2023_paper.pdf) | CVPR | 2023 | [Github](https://github.com/Juzezhang/NeuralDome_Toolbox) | [Project](https://juzezhang.github.io/NeuralDome/) |
| CHAIRS | ![Star](https://img.shields.io/github/stars/jnnan/chairs.svg?style=social&label=Star) [**Full-Body Articulated Human-Object Interaction**](https://arxiv.org/pdf/2212.10621) | ICCV | 2023 | [Github](https://github.com/jnnan/chairs) | [Project](https://jnnan.github.io/chairs/) |
| OMOMO | ![Star](https://img.shields.io/github/stars/lijiaman/omomo_release.svg?style=social&label=Star) [**Object Motion Guided Human Motion Synthesis**](https://arxiv.org/pdf/2309.16237) | SIGGRAPH Asia | 2023 | [Github](https://github.com/lijiaman/omomo_release) | [Project](https://lijiaman.github.io/projects/omomo/) |
| BEHAVE | ![Star](https://img.shields.io/github/stars/xiexh20/behave-dataset.svg?style=social&label=Star) [**BEHAVE: Dataset and Method for Tracking Human Object Interactions**](https://virtualhumans.mpi-inf.mpg.de/papers/bhatnagar22behave/behave.pdf) | CVPR | 2022 | [Github](https://github.com/xiexh20/behave-dataset) | [Project](https://virtualhumans.mpi-inf.mpg.de/behave/) |
| HOI4D | ![Star](https://img.shields.io/github/stars/leolyliu/HOI4D-Instructions.svg?style=social&label=Star) [**HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction**](https://hoi4d.github.io/HOI4D_cvpr2022.pdf) | CVPR | 2022 | [Github](https://github.com/leolyliu/HOI4D-Instructions) | [Project](https://hoi4d.github.io/) |
| H2O-3D | ![Star](https://img.shields.io/github/stars/shreyashampali/kypt_transformer.svg?style=social&label=Star) [**Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimation**](https://arxiv.org/pdf/2104.14639) | CVPR | 2022 | [Github](https://github.com/shreyashampali/kypt_transformer) | [Project](https://www.tugraz.at/index.php?id=57823) |
| PROX-S | ![Star](https://img.shields.io/github/stars/zkf1997/COINS.svg?style=social&label=Star) [**Compositional Human-Scene Interaction Synthesis with Semantic Control**](https://arxiv.org/pdf/2207.12824) | ECCV | 2022 | [Github](https://github.com/zkf1997/COINS) | [Project](https://zkf1997.github.io/COINS/index.html) |
| COUCH | ![Star](https://img.shields.io/github/stars/xz6014/couch.svg?style=social&label=Star) [**COUCH: Towards Controllable Human-Chair Interactions**](https://virtualhumans.mpi-inf.mpg.de/couch/zhang2022couch.pdf) | ECCV | 2022 | [Github](https://github.com/xz6014/couch/) | [Project](https://virtualhumans.mpi-inf.mpg.de/couch/) |
| InterCap | ![Star](https://img.shields.io/github/stars/YinghaoHuang91/InterCap.svg?style=social&label=Star) [**InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interaction**](https://intercap.is.tue.mpg.de/media/upload/main.pdf) | GCPR | 2022 | [Github](https://github.com/YinghaoHuang91/InterCap/tree/master) | [Project](https://intercap.is.tue.mpg.de/) |
| DexYCB | ![Star](https://img.shields.io/github/stars/NVlabs/dex-ycb-toolkit.svg?style=social&label=Star) [**DexYCB: A Benchmark for Capturing Hand Grasping of Objects**](https://arxiv.org/pdf/2104.04631) | CVPR | 2021 | [Github](https://github.com/NVlabs/dex-ycb-toolkit) | [Project](https://dex-ycb.github.io/) |
| GRAVICAP | ![Star](https://img.shields.io/github/stars/rishabhdabral/gravicap.svg?style=social&label=Star) [**Gravity-Aware Monocular 3D Human-Object Reconstruction**](https://arxiv.org/pdf/2108.08844) | ICCV | 2021 | [Github](https://github.com/rishabhdabral/gravicap) | [Project](https://4dqv.mpi-inf.mpg.de/GraviCap/) |
| H2O | ![Star](https://img.shields.io/github/stars/taeinkwon/h2odataset.svg?style=social&label=Star) [**H2O: Two Hands Manipulating Objects for First Person Interaction Recognition**](https://arxiv.org/pdf/2104.11181) | ICCV | 2021 | [Github](https://github.com/taeinkwon/h2odataset) | [Project](https://taeinkwon.com/projects/h2o/) |
| HO-3D | ![Star](https://img.shields.io/github/stars/shreyashampali/ho3d.svg?style=social&label=Star) [**HOnnotate: A method for 3D Annotation of Hand and Object Poses**](https://arxiv.org/pdf/1907.01481) | CVPR | 2020 | [Github](https://github.com/shreyashampali/ho3d) | [Project](https://www.tugraz.at/institute/icg/research/team-lepetit/research-projects/hand-object-3d-pose-annotation/) |
| ContactPose | ![Star](https://img.shields.io/github/stars/facebookresearch/ContactPose.svg?style=social&label=Star) [**ContactPose: A Dataset of Grasps with Object Contact and Hand Pose**](https://arxiv.org/pdf/2007.09545) | ECCV | 2020 | [Github](https://github.com/facebookresearch/ContactPose) | [Project](https://contactpose.cc.gatech.edu/) |
| GRAB | ![Star](https://img.shields.io/github/stars/otaheri/GRAB.svg?style=social&label=Star) [**GRAB: A Dataset of Whole-Body Human Grasping of Objects**](https://arxiv.org/pdf/2008.11200) | ECCV | 2020 | [Github](https://github.com/otaheri/GRAB) | [Project](https://grab.is.tue.mpg.de/) |
| ContactDB | ![Star](https://img.shields.io/github/stars/samarth-robo/contactdb_prediction.svg?style=social&label=Star) [**ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging**](https://arxiv.org/pdf/1904.06830) | CVPR | 2019 | [Github](https://github.com/samarth-robo/contactdb_prediction) | [Project](https://contactdb.cc.gatech.edu/) |
| ObMan | ![Star](https://img.shields.io/github/stars/hassony2/obman.svg?style=social&label=Star) [**Learning joint reconstruction of hands and manipulated objects**](https://arxiv.org/pdf/1904.05767) | CVPR | 2019 | [Github](https://github.com/hassony2/obman) | [Project](https://hassony2.github.io/obman) |
| Dexter+Object | [**Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input**](https://handtracker.mpi-inf.mpg.de/projects/RealtimeHO/content/RealtimeHO_ECCV2016.pdf) | ECCV | 2016 | Github | [Project](https://handtracker.mpi-inf.mpg.de/projects/RealtimeHO/) |

[🎯Back to Top](#head-content)

### Human-Scene Interaction Datasets

| Dataset | Title | Venue | Year | Code | Project |
|:-----------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------:|:----:|:--------------------------------------------------------:|:---------------------------------------------------------------:|
| ParaHome | ![Star](https://img.shields.io/github/stars/snuvclab/ParaHome.svg?style=social&label=Star) [**ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactions**](https://arxiv.org/pdf/2401.10232) | CVPR | 2025 | [Github](https://github.com/snuvclab/ParaHome) | [Project](https://jlogkim.github.io/parahome/) |
| LaserHuman | ![Star](https://img.shields.io/github/stars/4DVLab/LaserHuman.svg?style=social&label=Star) [**LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environment**](https://arxiv.org/pdf/2403.13307) | - | 2024 | [Github](https://github.com/4DVLab/LaserHuman) | Project |
| iReplica | ![Star](https://img.shields.io/github/stars/vguzov/ireplica.svg?style=social&label=Star) [**Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motion**](https://arxiv.org/pdf/2205.02830) | 3DV | 2024 | [Github](https://github.com/vguzov/ireplica) | [Project](https://virtualhumans.mpi-inf.mpg.de/ireplica/) |
| TRUMANS | ![Star](https://img.shields.io/github/stars/jnnan/trumans_utils.svg?style=social&label=Star) [**TRUMANS: Scaling Up Dynamic Human-Scene Interaction Modeling**](https://arxiv.org/pdf/2403.08629) | CVPR | 2024 | [Github](https://github.com/jnnan/trumans_utils) | [Project](https://jnnan.github.io/trumans/) |
| LINGO | ![Star](https://img.shields.io/github/stars/mileret/lingo-release.svg?style=social&label=Star) [**LINGO: Autonomous Character-Scene Interaction Synthesis from Text Instruction**](https://arxiv.org/pdf/2410.03187) | SIGGRAPH Asia | 2024 | [Github](https://github.com/mileret/lingo-release) | [Project](https://lingomotions.com/) |
| CIRCLE | ![Star](https://img.shields.io/github/stars/Stanford-TML/circle_dataset.svg?style=social&label=Star) [**CIRCLE: Capture In Rich Contextual Environments**](https://arxiv.org/pdf/2303.17912) | CVPR | 2023 | [Github](https://github.com/Stanford-TML/circle_dataset) | [Project](https://stanford-tml.github.io/circle_dataset/) |
| RICH | ![Star](https://img.shields.io/github/stars/paulchhuang/rich_toolkit.svg?style=social&label=Star) [**Capturing and Inferring Dense Full-Body Human-Scene Contact**](https://arxiv.org/pdf/2206.09553) | CVPR | 2022 | [Github](https://github.com/paulchhuang/rich_toolkit) | [Project](https://rich.is.tue.mpg.de/) |
| GIMO | ![Star](https://img.shields.io/github/stars/y-zheng18/GIMO.svg?style=social&label=Star) [**GIMO: Gaze-Informed Human Motion Prediction in Context**](https://arxiv.org/pdf/2204.09443) | ECCV | 2022 | [Github](https://github.com/y-zheng18/GIMO) | [Project](https://geometry.stanford.edu/projects/gimo/) |
| EgoBody | ![Star](https://img.shields.io/github/stars/sanweiliti/EgoBody.svg?style=social&label=Star) [**EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devices**](https://arxiv.org/pdf/2112.07642) | ECCV | 2022 | [Github](https://github.com/sanweiliti/EgoBody) | [Project](https://sanweiliti.github.io/egobody/egobody.html) |
| HUMANISE | ![Star](https://img.shields.io/github/stars/Silverster98/HUMANISE.svg?style=social&label=Star) [**HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes**](https://arxiv.org/pdf/2210.09729) | NeurIPS | 2022 | [Github](https://github.com/Silverster98/HUMANISE) | [Project](https://silvester.wang/HUMANISE/) |
| HPS | ![Star](https://img.shields.io/github/stars/miraymen/hps.svg?style=social&label=Star) [**Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors**](https://arxiv.org/pdf/2103.17265) | CVPR | 2021 | [Github](https://github.com/miraymen/hps) | [Project](https://virtualhumans.mpi-inf.mpg.de/hps/) |
| SAMP | ![Star](https://img.shields.io/github/stars/mohamedhassanmus/SAMP.svg?style=social&label=Star) [**SAMP: Stochastic Scene-Aware Motion Prediction**](https://arxiv.org/abs/2410.03441) | ICCV | 2021 | [Github](https://github.com/mohamedhassanmus/SAMP) | [Project](https://samp.is.tue.mpg.de/) |
| GTA-IM | ![Star](https://img.shields.io/github/stars/ZheC/GTA-IM-Dataset.svg?style=social&label=Star) [**GTA-IM: Long-term Human Motion Prediction with Scene Context**](https://arxiv.org/pdf/2007.03672) | ECCV | 2020 | [Github](https://github.com/ZheC/GTA-IM-Dataset) | [Project](https://zhec.github.io/hmp/index.html) |
| PROX | ![Star](https://img.shields.io/github/stars/mohamedhassanmus/prox.svg?style=social&label=Star) [**PROX: Resolving 3D Human Pose Ambiguities with 3D Scene Constraints**](https://arxiv.org/pdf/1908.06963) | ICCV | 2019 | [Github](https://github.com/mohamedhassanmus/prox) | [Project](https://prox.is.tue.mpg.de/) |
| i3DB | ![Star](https://img.shields.io/github/stars/amonszpart/iMapper.svg?style=social&label=Star) [**iMapper: Interaction-guided Joint Scene and Human Motion Mapping from Monocular Videos**](https://arxiv.org/pdf/1806.07889) | SIGGRAPH | 2019 | [Github](https://github.com/amonszpart/iMapper) | [Project](https://geometry.cs.ucl.ac.uk/projects/2019/imapper/) |
| PiGraphs | ![Star](https://img.shields.io/github/stars/msavva/pigraphs.svg?style=social&label=Star) [**PiGraphs: Learning Interaction Snapshots from Observations**](https://dl.acm.org/doi/pdf/10.1145/2897824.2925867) | SIGGRAPH | 2016 | [Github](https://github.com/msavva/pigraphs) | [Project](https://graphics.stanford.edu/projects/pigraphs/) |

[🎯Back to Top](#head-content)

### Human-Mix Interaction Datasets

[🎯Back to Top](#head-content)

## Contributors 
Thanks to all the contributors!

Special thank to [JackFYL](https://github.com/JackYFL) for providing the repo template.

## Star History 
[![Star History Chart](https://api.star-history.com/svg?repos=soraproducer/Awesome-Human-Interaction-Motion-Generation&type=date&legend=top-left)](https://www.star-history.com/#soraproducer/Awesome-Human-Interaction-Motion-Generation&type=date&legend=top-left)

Awesome-Human-Interaction-Motion-Generation

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Awesome-Human-Interaction-Motion-Generation

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README