GRPose

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

★ 59 Open GitHub ↗

GRPose: Learning Graph Relations for Human Image Generation with Pose Priors (AAAI 2025)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# GRPose: Learning Graph Relations for Human Image Generation with Pose Priors (AAAI 2025)

By [Xiangchen Yin](), [Donglin Di](), [Lei Fan](), [Hao Li](), [Wei Chen](), [Xiaofei Gou](), [Yang Song](), [Xiao Sun](), [Xun Yang]()

## Abstract

Recent methods using diffusion models have made significant progress in human image generation with various additional controls such as pose priors. However, existing approaches still struggle to generate high-quality images with consistent pose alignment, resulting in unsatisfactory outputs. In this paper, we propose a framework delving into the graph relations of pose priors to provide control information for human image generation. The main idea is to establish a graph topological structure between the pose priors and latent representation of diffusion models to capture the intrinsic associations between different pose parts. A Progressive Graph Integrator (PGI) is designed to learn the spatial relationships of the pose priors with the graph structure, adopting a hierarchical strategy within an Adapter to gradually propagate information across different pose parts. A pose perception loss is further introduced based on a pretrained pose estimation network to minimize the pose differences. Extensive qualitative and quantitative experiments conducted on the Human-Art and LAION-Human datasets demonstrate that our model achieves superior performance, with a 9.98% increase in pose average precision compared to the latest benchmark model.

## Updates
- **`2024/12/25`**: Release the code of inference and evaluation.
- **`2024/12/15`**: Release the code of model and training.
- **`2024/12/10`**: Congratulations! Our **GRPose** has been accepted at AAAI2025.

## Environment

```
conda create -n grpose python=3.10
conda activate grpose
pip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu116
pip install -r requirements.txt
```

## Datasets and Pretrain-Models Preparation

You need to prepare the [Human-Art](https://idea-research.github.io/HumanArt/) and [LAION-Human](https://arxiv.org/pdf/2304.04269) datasets to train models, the data path is set in config files.

We provided pretrain model checkpoint here： [huggingface](https://huggingface.co/suxi123/GRPose/tree/main), then you can run this commond to get init grpose weights.

```python
python prepare_weights.py models/v1-5-pruned.ckpt configs/grpose/humanart.yaml models/init_grpose.ckpt
```

The pretrain model dir as follows:

## Train the models

Training shell file is provide:

```
sh train.sh
```

You can also directly run as follows:

```
python train.py --config configs/grpose/humanart.yaml --max_epochs 40 \
--control_ckpt models/init_grpose.ckpt --devices 8 --scale_lr false
```

The training weights will be saved in `experiments` dir.

## Evaluate the models

Due to the problem of the mmpose version, we provide an available mmpose code, which is in `utils/mmpose-0.x.zip`. We install mmpose to evaluate the pose alignment of the model.

```
cd utils && unzip mmpose-0.x.zip
cd mmpose-0.x 
pip install -e .
```

Evaluation shell file is provided:

```
sh infer.sh
```

You can run the single gpu evaluation

```
python eval_pose.py --config_model configs/grpose/humanart.yaml --ckpt experiments/grpose/grpose_HumanArt-8-L40S/epoch=000035.ckpt
```
The predict results will be saved in `outputs` and the quantitative results are saved in `metrics/pose.csv`. To evaluate FID, KID for the image quality 
```
python eval_quality.py 
```
It should be noted that KID is multiplied by 100 for Human-Art and 1000 for LAION-Human for readability in our paper.

## Citation

```
@inproceedings{yin_grpose, title={GRPose: Learning Graph Relations for Human Image Generation with Pose Priors}, volume={39}, DOI={10.1609/aaai.v39i9.33032}, number={9}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Yin, Xiangchen and Di, Donglin and Fan, Lei and Li, Hao and Chen, Wei and , Gouxiaofei and Song, Yang and Sun, Xiao and Yang, Xun}, year={2025}, pages={9526-9534} }
```

## Acknowledgments

The project is designed on [ControlNet](https://github.com/lllyasviel/ControlNet), [HumanSD](https://github.com/IDEA-Research/HumanSD), [Stable-Pose](https://github.com/ai-med/StablePose), [ViG](https://github.com/huawei-noah/Efficient-AI-Backbones) and [HumanArt](https://idea-research.github.io/HumanArt/), thanks to these works!

GRPose

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README