LayerAnimate

KO | EN

gitlite — search

#python #typescript #llm #machine-learning #rust #deep-learning #nodejs #java #windows #awesome-list #security #pytorch #mcp #open-source

(★ 195)

[ICCV 2025] LayerAnimate: Layer-specific Control for Animation

File Explorer

Download Latest Version (.zip)

- - demos.gif
- __init__.py
- model.py
- utils.py
- vae.py
- __init__.py
- LICENSE
- - dataset.py
- - autoencoder.py
  - condition.py
  - controlnet.py
  - layer_controlnet.py
  - unet.py
- - ae_dualref_modules.py
  - ae_modules.py
  - attention.py
  - attention_svd.py
- - pipeline_animation.py
- basics.py
- common.py
- utils.py
- animate_Layer.py
- app.py
- demo1.yaml
- demo2.yaml
- demo3.yaml
- demo4.yaml
- demo5.yaml
- infer_DiT.py
.gitignore
LICENSE.txt
README.md
requirements.txt

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# LayerAnimate: Layer-level Control for Animation

[Yuxue Yang](https://yuxueyang1204.github.io/)1,2, [Lue Fan](https://lue.fan/)2, [Zuzeng Lin](https://www.researchgate.net/scientific-contributions/Zuzeng-Lin-2192777418)3, [Feng Wang](https://happynear.wang/)4, [Zhaoxiang Zhang](https://zhaoxiangzhang.net)1,2†

1UCAS&emsp; 2CASIA&emsp; 3TJU&emsp; 4CreateAI&emsp; †Corresponding author

Official implementation of **LayerAnimate: Layer-level Control for Animation**, ICCV 2025

**Videos on the [project website](https://layeranimate.github.io) vividly introduces our work and presents qualitative results for an enhanced view experience.**

## Updates

- [25-08-22] Release the [Layer Curation Pipeline](https://github.com/YuxueYang1204/Layer-Curation-Pipeline), including the demo and comprehensive usage guidance.
- [25-06-26] Our work is accepted by ICCV 2025! 🎉
- [25-05-29] We have extended LayerAnimate to the DiT ([Wan2.1 1.3B](https://github.com/Wan-Video/Wan2.1)) variant, enabling the generation of 81 frames at 480 × 832 resolution. It performs surprisingly well in the [Real-World Domain](https://layeranimate.github.io/#real_world) shown in the project website.
- [25-03-31] Release the online demo on [Hugging Face](https://huggingface.co/spaces/IamCreateAI/LayerAnimate).
- [25-03-30] Release a gradio script [app.py](scripts/app.py) to run the demo locally. Please raise an issue if you encounter any problems.
- [25-03-22] Release the checkpoint and the inference script. **We update layer curation pipeline and support trajectory control for a flexible composition of various layer-level controls.**
- [25-01-15] Release the project page and the arXiv preprint.

## Layer curation pipeline

We have released [a comprehensive pipeline](https://github.com/YuxueYang1204/Layer-Curation-Pipeline) for extracting motion-based layers from video sequences. The layer curation pipeline automatically decomposes videos into different layers based on motion patterns, where you can control the number of extracted layers by adjusting the layer capacity parameter to obtain varying levels of motion granularity.

More details can be found in the [repo](https://github.com/YuxueYang1204/Layer-Curation-Pipeline).

| Input Videos | Layer Results |
|:--:|:--:|
| <video src="https://github.com/user-attachments/assets/d8e28bc4-541f-4b9f-bb95-379ebc83aa89"> | <video src="https://github.com/user-attachments/assets/464c8b53-71d6-4a68-80b4-e4b9b81bdf8a"> |
| <video src="https://github.com/user-attachments/assets/89360155-622c-40cd-8619-07c14935e3d7"> | <video src="https://github.com/user-attachments/assets/39304b8d-a794-43d0-a271-2682958b9e82"> |
| <video src="https://github.com/user-attachments/assets/da338346-5e6f-4cc8-8822-b76e715146e7"> | <video src="https://github.com/user-attachments/assets/b951f04c-3ce9-4eab-8aa9-a3eac625371e"> |
| <video src="https://github.com/user-attachments/assets/4507d1fa-136f-4298-a58f-90082ebcaf4c"> | <video src="https://github.com/user-attachments/assets/575d7c2e-92d3-440d-93ee-fbcaba55aac9"> |

## Installation

```bash
git clone git@github.com:IamCreateAI/LayerAnimate.git
conda create -n layeranimate python=3.10 -y
conda activate layeranimate
pip install -r requirements.txt
pip install wan@git+https://github.com/Wan-Video/Wan2.1  # If you want to use DiT variant.
```

## Models

| Models                   | Download Link                                                                                                                                           | Video Size        |
|--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------|
| UNet variant | [Huggingface](https://huggingface.co/Yuppie1204/LayerAnimate-Mix) 🤗  | 16 x 320 x 512  |
| DiT variant | [Huggingface](https://huggingface.co/Yuppie1204/LayerAnimate-DiT) 🤗  | 81 x 480 x 832  |

Download the pretrained weights and put them in `checkpoints/` directory as follows:

```bash
checkpoints/
├─ LayerAnimate-Mix (UNet variant)
└─ LayerAnimate-DiT
```

## Inference script

### UNet variant (Paper version)

Run the following command to generate a video from input images:

```bash
python scripts/animate_Layer.py --config scripts/demo1.yaml --savedir outputs/sample1

python scripts/animate_Layer.py --config scripts/demo2.yaml --savedir outputs/sample2

python scripts/animate_Layer.py --config scripts/demo3.yaml --savedir outputs/sample3

python scripts/animate_Layer.py --config scripts/demo4.yaml --savedir outputs/sample4

python scripts/animate_Layer.py --config scripts/demo5.yaml --savedir outputs/sample5
```

Note that the layer-level controls are prepared in `__assets__/demos`.

#### Run demo locally

You can run the demo locally by executing the following command:

```bash
python scripts/app.py --savedir outputs/gradio
```

Then, open the link in your browser to access the demo interface. The output video and the video with trajectory will be saved in the `outputs/gradio` directory.

### DiT variant (Wan2.1 1.3B)

Run the following command to generate a video from input images:

```bash
python scripts/infer_DiT.py --config __assets__/demos/realworld/config.yaml --savedir outputs/realworld
```

We take the `config.yaml` in `demos/realworld/` as an example. You can also modify the config file to suit your needs.

## Todo

- [x] Release the code and checkpoint of LayerAnimate.
- [x] Upload a gradio script to run the demo locally.
- [x] Create a online demo in the huggingface space.
- [x] DiT-based LayerAnimate.
- [x] Release layer curation pipeline.
- [ ] Training script for LayerAnimate.

## Acknowledgements

We sincerely thank the great work [ToonCrafter](https://doubiiu.github.io/projects/ToonCrafter/), [LVCD](https://luckyhzt.github.io/lvcd), [AniDoc](https://yihao-meng.github.io/AniDoc_demo/), and [Wan-Video](https://github.com/Wan-Video/Wan2.1) for their inspiring work and contributions to the AIGC community.

## Citation

Please consider citing our work as follows if it is helpful.
```bib
@article{yang2025layeranimate,
  author    = {Yang, Yuxue and Fan, Lue and Lin, Zuzeng and Wang, Feng and Zhang, Zhaoxiang},
  title     = {LayerAnimate: Layer-level Control for Animation},
  journal   = {arXiv preprint arXiv:2501.08295},
  year      = {2025},
}
```

LayerAnimate

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

LayerAnimate

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README