Stable-Video-Infinity

KO | EN

gitlite — search

#javascript #python #hacktoberfest #typescript #go #machine-learning #docker #macos #java #awesome #cli #awesome-list #ios #openai

Stable-Video-Infinity

(★ 2,551)

[ICLR 26 Oral] Stable Video Infinity: Infinite-Length Video Generation with Error Recycling

파일 탐색기

최종 버전 다운로드 (.zip)

- intro.png
- logo.png
- logo_white.png
- youtube1.png
- youtube2.png
- youtube3.png
- SVI-Shot-StreamingPrompt-1028.json
- SVI-Shot-StreamingPrompt.mp4
- test.jpg
- - __init__.py
  - model_config.py
  - model_config_talk.py
- - __init__.py
  - controlnet_unit.py
  - processors.py
- - __init__.py
  - simple_text_image.py
  - video.py
- - __init__.py
  - xdit_context_parallel.py
- - __init__.py
- - __init__.py
  - attention.py
  - cog_dit.py
  - cog_vae.py
  - downloader.py
  - flux_controlnet.py
  - flux_dit.py
  - flux_ipadapter.py
  - flux_text_encoder.py
  - flux_vae.py
  - hunyuan_dit.py
  - hunyuan_dit_text_encoder.py
  - hunyuan_video_dit.py
  - hunyuan_video_text_encoder.py
  - hunyuan_video_vae_decoder.py
  - hunyuan_video_vae_encoder.py
  - kolors_text_encoder.py
  - lora.py
  - model_manager.py
  - omnigen.py
  - sd3_dit.py
  - sd3_text_encoder.py
  - sd3_vae_decoder.py
  - sd3_vae_encoder.py
  - sd_controlnet.py
  - sd_ipadapter.py
  - sd_motion.py
  - sd_text_encoder.py
  - sd_unet.py
  - sd_vae_decoder.py
  - sd_vae_encoder.py
  - sdxl_controlnet.py
  - sdxl_ipadapter.py
  - sdxl_motion.py
  - sdxl_text_encoder.py
  - sdxl_unet.py
  - sdxl_vae_decoder.py
  - sdxl_vae_encoder.py
  - stepvideo_dit.py
  - stepvideo_text_encoder.py
  - stepvideo_vae.py
  - svd_image_encoder.py
  - svd_unet.py
  - svd_vae_decoder.py
  - svd_vae_encoder.py
  - tiler.py
  - utils.py
  - wan_video_dit.py
  - wan_video_dit_talk.py
  - wan_video_image_encoder.py
  - wan_video_text_encoder.py
  - wan_video_vae.py
- - __init__.py
  - base.py
  - cog_video.py
  - dancer.py
  - flux_image.py
  - hunyuan_image.py
  - hunyuan_video.py
  - omnigen_image.py
  - pipeline_runner.py
  - sd3_image.py
  - sd_image.py
  - sd_video.py
  - sdxl_image.py
  - sdxl_video.py
  - step_video.py
  - svd_video.py
  - svi_video.py
  - svi_video_dance.py
  - svi_video_talk.py
  - wan_video.py
- - __init__.py
  - base.py
  - FastBlend.py
  - PILEditor.py
  - RIFE.py
  - sequencial_processor.py
- - __init__.py
  - base_prompter.py
  - cog_prompter.py
  - flux_prompter.py
  - hunyuan_dit_prompter.py
  - hunyuan_video_prompter.py
  - kolors_prompter.py
  - omnigen_prompter.py
  - omost.py
  - prompt_refiners.py
  - sd3_prompter.py
  - sd_prompter.py
  - sdxl_prompter.py
  - stepvideo_prompter.py
  - wan_prompter.py
- - __init__.py
  - continuous_ode.py
  - ddim.py
  - flow_match.py
- - __init__.py
- - __init__.py
  - text_to_image.py
- - __init__.py
  - fm_solvers.py
  - fm_solvers_unipc.py
  - multitalk_utils.py
  - prompt_extend.py
  - qwen_vl_utils.py
  - utils.py
  - vace_processor.py
- - __init__.py
  - layers.py
- __init__.py
- causal.png
- DevLog.md
- FAQ.md
- talk.png
- wan22_preview.png
- __init__.py
- onnxdet.py
- onnxpose.py
- util.py
- wholebody.py
- - prepare_video_audio.py
  - prepare_video_pose.py
  - process_mixkit.py
- - svi_2.0.sh
  - svi_dance.sh
  - svi_film.sh
  - svi_shot.sh
  - svi_talk.sh
  - svi_tom.sh
- - svi_dance.sh
  - svi_film.sh
  - svi_shot.sh
  - svi_talk.sh
- - utils.py
- audio_process.py
- extract_lora.py
- image_process.py
- metadata_gen.py
- process.py
- project_utils.py
- run_align_pose.py
- text_utils.py
- video_process.py
.gitignore
gradio_demo.py
gradio_demo.sh
LICENSE
README.md
requirements.txt
setup.py
test_svi.py
test_svi_dance.py
test_svi_talk.py
train_svi.py
train_svi_dance.py
train_svi_talk.py

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1>Stable Video Infinity: Infinite-Length Video Generation with Error Recycling</h1>

[Wuyang Li](https://wymancv.github.io/wuyang.github.io/) · [Wentao Pan](https://scholar.google.com/citations?user=sHKkAToAAAAJ&hl=zh-CN) · [Po-Chien Luan](https://scholar.google.com/citations?user=Y2Oth4MAAAAJ&hl=zh-TW) · [Yang Gao](https://scholar.google.com/citations?user=rpT0Q6AAAAAJ&hl=en) · [Alexandre Alahi](https://scholar.google.com/citations?user=UIhXQ64AAAAJ&hl=en)

[VITA@EPFL](https://www.epfl.ch/labs/vita/)

Technical introduction (unofficial): [AI Papers Slop (English)](https://www.youtube.com/watch?v=vKPCqPsCfZg); [WechatApp (Chinese)](https://mp.weixin.qq.com/s?__biz=MzIwMTE1NjQxMQ==&mid=2247641601&idx=1&sn=e86ae40b54fda22eda2ebd818b38de73&chksm=978a0c69a14a79192b1ca81f257f093362add316acdcdff69c67ab5d186f8af7f8e84931632a&mpshare=1&srcid=1016e1aTWfR71TRJJHDFgMHf&sharer_shareinfo=273ee623f20eba9542ff4b8c3a0c35d1&sharer_shareinfo_first=559e5442227d44f61573005b4e12d83c&from=timeline&scene=2&subscene=2&clicktime=1761249340&enterid=1761249340&sessionid=0&ascene=45&fasttmpl_type=0&fasttmpl_fullversion=7965100-zh_CN-zip&fasttmpl_flag=0&realreporttime=1761249340647#rd)
</div>

<div align="center">
<table width="100%">
  <tr>
    <td align="center" width="33%">
      <a href="https://youtu.be/p71Wp1FuqTw">
        <img src="assets/youtube1.png" alt="Watch the video" width="100%">
      </a>
      <br>
      Quick Glance at the SVI Family
    </td>
    <td align="center" width="33%">
      <a href="https://www.youtube.com/watch?v=xEgVF3fAZ5o">
        <img src="assets/youtube2.png" alt="Watch the video" width="100%">
      </a>
      <br>
      8‑minute crazy Tom & Jerry video made with SVI‑Tom
    </td>
    <td align="center" width="33%">
      <a href="https://www.youtube.com/watch?v=a7Zx5e9ZjK4">
        <img src="assets/youtube3.png" alt="Watch the video" width="100%">
      </a>
      <br>
      14‑minute videos made with SVI‑2.0 (based on Wan 2.1) and SVI‑Talk.
    </td>
  </tr>
</table>
</div>

## 🚀 [27 May 2026] Update EverAnimate

- [EverAnimate](https://everanimate.github.io/homepage/): A long-range human animation method based on SVI 2.0 Pro. Have a try!

- [SVI `main branch`](https://github.com/vita-epfl/Stable-Video-Infinity#): Wan 2.1 base model (both SVI 1.0/2.0)

- [SVI `svi_wan22 branch`](https://github.com/vita-epfl/Stable-Video-Infinity/tree/svi_wan22): Wan 2.2 base model (both SVI 2.0/2.0 Pro)

## ✨ SVI 2.0 Pro ComfyUI Workflows and Videos Generated by Community Users

Thanks to many enthusiastic community users who keep creating and updating various SVI workflows, we now have a growing collection of different features and use cases. Please refer to the [pinned issue](https://github.com/vita-epfl/Stable-Video-Infinity/issues/51) for a summarized overview of these workflows. We will continuously update that issue to showcase more interesting and useful SVI workflows. When using them, please check out the pinned issue for updated important tips, e.g.,

- **Use different seeds for different clips, which is very important!**
- **Enhance prompts & reduce LightX2V usage & use more optimal resolution (480p) to relieve slow motion.**
- **Avoid using the wrong SVI 1.0 workflow in this repo.**

###  Community Deployment

SVI-2.0 Pro is now available on the Poe platform! You can access it through the Poe chat interface or integrate it via their API. Check it out here [link](https://poe.com/SVI-2.0-Pro). ❤️ Big thanks to [@empiriolabsai](https://empiriolabs.ai/) for their support!

### Some Community Workflow Tutorials

Really appreciate the attention from community Youtubers and Bilibili creators.

- ❤️ Big thanks to the amazing Youtuber [@AI Search](https://www.youtube.com/@theAIsearch) for his fantastic SVI tutorial [[Link]](https://www.youtube.com/watch?v=-3DVJu72VhE)!

- ❤️ Big thanks to the amazing Youtuber @[ComfyUI Workflow Blog](https://www.youtube.com/@ComfyUIworkflows) for making a tutorial about generating **40-second highly dynamic videos without any color degradation** [[Link]](https://www.youtube.com/watch?v=PJnTcVOqJCM&t=209s).
  
- ❤️ Big thanks to the amazing Bilibili creator [@AI Aiwood](https://space.bilibili.com/503934057?spm_id_from=333.788.upinfo.detail.click) for his three amazing SVI tutorials about long-shot videos ([[Link]](https://www.bilibili.com/video/BV1oevyB6Eyh/?spm_id_from=333.1387.homepage.video_card.click)), multi-shot videos ([[Link]](https://www.bilibili.com/video/BV1LjvpBCE1t/?spm_id_from=333.1387.homepage.video_card.click)), and video extension ([[Link]](https://www.bilibili.com/video/BV1DdvxBCExf/?spm_id_from=333.1387.homepage.video_card.click))!

- ❤️ Big thanks to the amazing Bilibili creators [@AI 与AI同行1996](https://www.bilibili.com/video/BV11yigBfE4H/?spm_id_from=333.337.search-card.all.click) for his 1-min stress test of SVI without color drift! [@AI绘视玩家](https://www.bilibili.com/video/BV1ggvWBqEb6/?spm_id_from=333.337.search-card.all.click&vd_source=04231a7d0b782d8fd0204e75f4f7dd34) for his stress test of storytelling long videos. [@三当家AI](https://www.bilibili.com/video/BV1BQveBEExr/?spm_id_from=333.1387.favlist.content.click) for the test of different Wan base model variants, and the videos from amazing Youtuber [@Jaevlon](https://www.youtube.com/@AIArtistryAtelier).

### Use Cases from the Community

Here are some beautiful videos generated by creative **community users (not us)** using SVI 2.0 Pro workflows! Please don’t hesitate to share your SVI creations with us!

**If your video quality differs significantly from the community example below (e.g., flickering or noticeable degradation), please double-check that you are using the workflow correctly. Besides, please turn on the sound of the following video for the best experience.**

<video src="https://github.com/user-attachments/assets/76444344-f033-4ecb-a987-2dd1973a84b6"
       controls
       muted
       width="100%">
</video>
<p align="center">Caption: Please turn on the sound at first! Video credit to community creator @PT. This is an unsolicited, non-paid promotional video with sound for SVI Pro 2.0 created independently by a community user (not affiliated with us). The video is first generated with SVI, then lip alignment is refined using InfiniteTalk@Meituan (PS: Big thanks to Longcat team!). The English voiceover says: “Many people ask what SVI Pro can do, it's about generating long videos without quality degradation. I love continuous camera moves and narration. Combined with amazing Wan 2.2, it’s simply an epic ride westward.”</p>

<p>Caption: Please turn on the sound at first! Big thanks to @ ̮  (̲͡-̲̅ .̲̅ ̲̅͡- ̲). Happy New Year!</p>
</div>

<table>
  <tr>
    <td>
      <video src="https://github.com/user-attachments/assets/8ece79f2-cd40-45ad-9f9d-3c835195137d"
             controls
             muted
             width="100%">
      </video>
      <p align="center">Big thanks to @PT.</p>
    </td>
    <td>
      <video src="https://github.com/user-attachments/assets/4f8b828a-cb6d-4287-bd55-c1585f8cfc19"
             controls
             muted
             width="100%">
      </video>
      <p align="center">Big thanks to @邂逅2004.</p>
    </td>
    <td>
      <video src="https://github.com/user-attachments/assets/46684a37-6f5f-4c84-b69a-a8e5e358dda1"
             controls
             muted
             width="100%">
      </video>
      <p align="center">Big thanks to <a href="https://github.com/RuneGjerde">@RuneGjerde</a>.</p>
    </td>
  </tr>
</table>

<table>
  <tr>
    <td>
      <video src="https://github.com/user-attachments/assets/c02d680e-d64e-42fd-905c-2031588a67b4"
             controls
             muted
             width="100%">
      </video>
      <p align="center">Big thanks to @XXX.</p>
    </td>
    <td>
      <video src="https://github.com/user-attachments/assets/ac31b884-b1b5-438e-a38a-b189b97ee606"
             controls
             muted
             width="100%">
      </video>
      <p align="center">Big thanks to <a href="https://github.com/Jaevlon">@Jaevlon</a>.</p>
    </td>
    <td>
      <video src="https://github.com/user-attachments/assets/e499966a-89b2-4f16-9ac7-d30da0d435a3"
             controls
             muted
             width="100%">
      </video>
      <p align="center">Big thanks to @高姿态的浅唱.</p>
    </td>
  </tr>
</table>

<table>
  <tr>
    <td>
      <video src="https://github.com/user-attachments/assets/e068db3e-a25f-4557-8462-2ca82c2881c0"
             controls
             muted
             width="100%">
      </video>
      <p align="center">Big thanks to @Aiwood.</p>
    </td>
    <td>
      <video src="https://github.com/user-attachments/assets/f522b325-2088-473e-b6e1-183ec0f2acfb"
             controls
             muted
             width="100%">
      </video>
       <p align="center">Big thanks to @Aiwood.</p>
    </td>
    <td>
      <video src="https://github.com/user-attachments/assets/f837b116-6d1c-473d-ae57-c13dfce70ba7"
             controls
             muted
             width="100%">
      </video>
      <p align="center">Big thanks to @PT.</p>
    </td>
  </tr>
</table>

<table>
  <tr>
    <td>
      <video src="https://github.com/user-attachments/assets/37b6992a-8b45-4798-b33f-38205f2b8f3d"
             controls
             muted
             width="100%">
      </video>
      <p align="center">Big thanks to @wallen.</p>
    </td>
    <td>
      <video src="https://github.com/user-attachments/assets/a2fd8e7f-480d-46a5-a9e0-a49a0aed51b8"
             controls
             muted
             width="100%">
      </video>
       <p align="center">Big thanks to <a href="https://github.com/RuneGjerde">@RuneGjerde</a>.</p>
    </td>
    <td>
      <video src="https://github.com/user-attachments/assets/c2880978-e48f-4faa-8aea-52ee01fbbfe2"
             controls
             muted
             width="100%">
      </video>
      <p align="center">Big thanks to @CUDA out of memory.</p>
    </td>
  </tr>
</table>

What is our next release? Wan 2.2 Animate SVI. We found that tuning with only 1k samples is sufficient to unlock infinite-length generation for Wan 2.2 Animate, and we are trying to scale up now. The performance is far better than our original SVI-Dance based on UniAnimate-DiT.

## ✨ Highlight

*Stable Video Infinity* (SVI) is able to generate ANY-length videos with high temporal consistency, plausible scene transitions, and controllable streaming storylines in ANY domain.

- **OpenSVI**: Everything is open-sourced: training & evaluation scripts, datasets, and more.
- **Infinite Length**: No inherent limit on video duration; generate arbitrarily long stories (see the 10‑minute “Tom and Jerry” demo).
- **Versatile**: Supports diverse in-the-wild generation tasks: multi-scene short films, single‑scene animations, skeleton-/audio-conditioned generation, cartoons, and more.
- **Efficient**: Only LoRA adapters are tuned, requiring very little training data: anyone can make their own SVI easily.

</div>

**📧 Contact**: [wuyang.li@epfl.ch](mailto:wuyang.li@epfl.ch)

## 😀 SVI 1.0 ComfyUI Workflow

### Official ComfyUI

We've recently discovered that some users have been incorrectly using SVI workflows. We apologize for any confusion. Please note that **SVI LoRA cannot directly use the original Wan 2.1 workflow** - it requires modified padding settings.

**Please use our official workflow**: `Stable-Video-Infinity/comfyui_workflow`, which supports independent prompts for each video clip. Big thanks to @RuneGjerde, @Kijai, and @Taiwan1912!

Due to the significant impact of quantization and step distillation on the SVI-Film workflow, we currently only open-source the SVI-Shot workflow. Using our official workflow will generate infinite-length videos without drifting and forgetting. Below is a 3-minute interactive video demo (distinct prompts for each 5-second video continuation):

https://github.com/user-attachments/assets/2498edf4-cdda-4728-b11f-ab5731cf6e20

</div>

### Some Important To-Checks
If you can’t wait for the official ComfyUI release, try the testing versions of the Shot and Film workflows first with commercial GPUs based on quantization and distilled LoRAs: [Here](https://github.com/kijai/ComfyUI-WanVideoWrapper/issues/1519#issuecomment-3447933556). The official one (more stable) might be updated soon. Due to model quantization, the video quality may be affected (Better to try more sampling steps than 4/8).

- Please ensure that every video clip uses a different seed.
- SVI-Film uses 5 motion frames (last 5 frames) for i2v, not 1.
- SVI-Tom shares the workflow with SVI-Film, but uses 1 motion frame.
- SVI-Shot uses 1 motion frame (last frame) and uses extra VACE-based padding (the given reference image).
- Use the boat and cat demos for 50s generation and compare them with the [reproduced ones](https://github.com/kijai/ComfyUI-WanVideoWrapper/issues/1519#issuecomment-3443540666) to verify correctness.
- SVI-Shot also supports using different text for clips. See [here](https://www.reddit.com/r/StableDiffusion/comments/1oh4q3w/wan21_svishot_lora_long_video_test_1min/). Thanks @Taiwan1912！

Thank you for playing with SVI!

## 🔥 News

- [01-17-2025] SVI-2.0 Pro is available on the Poe platform! see [link](https://poe.com/SVI-2.0-Pro). Thanks [@empiriolabsai](https://empiriolabs.ai/)!
- [12-26-2025] SVI-2.0 Pro released!
- [12-07-2025] SVI-2.0 WanVideoWrapper ComfyUI workflow (native ComfyUI workflow is under deployment)
- [12-04-2025] SVI-2.0 released, supporting both Wan 2.1 and Wan 2.2
- [10-31-2025] Official SVI-Shot ComfyUI workflow! 
- [10-23-2025] Preview of Wan 2.2-5B-SVI and some tips for custom SVI implementation: See [DevLog](docs/DevLog.md)!  
- [10-21-2025] The error-banking strategy is optimized, further improving the stability. See details in [DevLog](docs/DevLog.md)!  
- [10-13-2025] SVI is now fully open-sourced and online!

## ❓ Frequently Asked Questions

### Bidirectional or Causal (Self-Forcing)?

*Self-Forcing achieves **frame-by-frame causality**, whereas SVI, a hybrid version, operates with **clip-by-clip causality** and **bidirectional attention within each clip**.*

Targeting film and creative content production, our SVI design mirrors a director's workflow: (1) Directors repeatedly review clips in both forward and reverse directions to ensure quality, often calling "CUT" and "AGAIN" multiple times during the creative process. SVI maintains bidirectionality within each clip to emulate this process. (2) After that, directors seamlessly connect different clips along the temporal axis with causality (and some scene-transition animation), which aligns with SVI's clip-by-clip causality. The Self-Forcing series is better suited for scenarios prioritizing real-time interaction (e.g., gaming). In contrast, SVI focuses on story content creation, requiring higher standards for both content and visual quality. Intuitively, SVI's paradigm has unique advantages in end-to-end high-quality video content creation.

### Please Refer to [FAQ](docs/FAQ.md) for More Questions.

## 🔧 Environment Setup

We have tested the environment with A100 80G, cuda 12.0, and torch 2.8.0. This is our reproduced [environment](https://github.com/user-attachments/files/22899587/env.txt). The following script will automatically install the older version torch==2.5.0. We have also tested with the lower version: torch==2.4.1 and torch==2.5.0. Feel free to let me know if you meet issues.

```bash
conda create -n svi python=3.10 
conda activate svi

# For svi family
pip install -e .
pip install flash_attn==2.8.0.post2
# If you encounter issues with flash-attn installation, please refer to the details at https://github.com/vita-epfl/Stable-Video-Infinity/issues/3.

conda install -c conda-forge ffmpeg
conda install -c conda-forge librosa
conda install -c conda-forge libiconv
```

## 📦 Model Preparation

### Download Wan 2.1 I2V 14B

```bash
huggingface-cli download Wan-AI/Wan2.1-I2V-14B-480P --local-dir ./weights/Wan2.1-I2V-14B-480P
```

### Download SVI Family

| Model                           | Task                    | Input                      | Output           | Hugging Face Link                                                                                                                | Comments                                                                                                   |
| ------------------------------- | ----------------------- | -------------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| **SVI-2.0**              | Single-scene (supports some transitions) | Image + Text prompt stream        | Long video       | [🤗 Huggingface](https://huggingface.co/vita-video-gen/svi-model/resolve/main/version-2.0/SVI_Wan2.1-I2V-14B_lora_v2.0.safetensors?download=true) [🤖 Modelscope](https://www.modelscope.cn/models/epfl-vita/svi-model/file/view/master/version-2.0%2FSVI_Wan2.1-I2V-14B_lora_v2.0.safetensors?status=2)            | Generate consistent long video with 1 text prompt stream.                                 |                          
| **ALL SVI-1.0**                   | Infinite possibility    | Image + X                  | X video          | [🤗 Huggingface](https://huggingface.co/vita-video-gen/svi-model/tree/main/version-1.0) [🤖 Modelscope](https://www.modelscope.cn/models/epfl-vita/svi-model/tree/master/version-1.0)                                                  | Family bucket! I want to play with all!                                                                    |
| **SVI-Shot**              | Single-scene generation | Image + Text prompt        | Long video       | [🤗 Huggingface](https://huggingface.co/vita-video-gen/svi-model/resolve/main/version-1.0/svi-shot.safetensors?download=true)  [🤖 Modelscope](https://www.modelscope.cn/models/epfl-vita/svi-model/file/view/master/version-1.0%2Fsvi-shot.safetensors?status=2)           | Generate consistent long video with 1 text prompt. (This will never drift or forget in our 20 min test)                                 |
| **SVI-Film-Opt-10212025**  (Latest)            | Multi-scene generation  | Image + Text prompt stream | Film-style video | [🤗 Huggingface](https://huggingface.co/vita-video-gen/svi-model/resolve/main/version-1.0/svi-film-opt-10212025.safetensors) [🤖 Modelscope](https://www.modelscope.cn/models/epfl-vita/svi-model/file/view/master/version-1.0%2Fsvi-film-opt-10212025.safetensors?status=2)            | Generate creative long video with 1 text prompt stream (5 second per text).                                |
| **SVI-Film**              | Multi-scene generation  | Image + Text prompt stream | Film-style video | [🤗 Huggingface](https://huggingface.co/vita-video-gen/svi-model/resolve/main/version-1.0/svi-film.safetensors?download=true) [🤖 Modelscope](https://www.modelscope.cn/models/epfl-vita/svi-model/file/view/master/version-1.0%2Fsvi-film.safetensors?status=2)            | Generate creative long video with 1 text prompt stream (5 second per text).                                |
| **SVI-Film (Transition)** | Multi-scene generation  | Image + Text prompt stream | Film-style video | [🤗 Huggingface](https://huggingface.co/vita-video-gen/svi-model/resolve/main/version-1.0/svi-film-transitions.safetensors?download=true) [🤖 Modelscope](https://www.modelscope.cn/models/epfl-vita/svi-model/file/view/master/version-1.0%2Fsvi-film-transitions.safetensors?status=2) | Generate creative long video with 1 text prompt stream. (More scene transitions due to the training data)  |
| **SVI-Tom&Jerry**         | Cartoon animation       | Image                      | Cartoon video    | [🤗 Huggingface](https://huggingface.co/vita-video-gen/svi-model/resolve/main/version-1.0/svi-tom.safetensors?download=true) [🤖 Modelscope](https://www.modelscope.cn/models/epfl-vita/svi-model/file/view/master/version-1.0%2Fsvi-tom.safetensors?status=2)             | Generate creative long cartoon videos with 1 text prompt stream (This will never drift or forget in our 20 min test) |
| **SVI-Talk**              | Talking head            | Image + Audio              | Talking video    | [🤗 Huggingface](https://huggingface.co/vita-video-gen/svi-model/resolve/main/version-1.0/svi-talk.safetensors?download=true)  [🤖 Modelscope](https://www.modelscope.cn/models/epfl-vita/svi-model/file/view/master/version-1.0%2Fsvi-talk.safetensors?status=2)           | Generate long videos with audio-conditioned human speaking   (This will never drift or forget in our 10 min test)                                              |
| **SVI-Dance**             | Dancing animation       | Image + Skeleton           | Dance video      | [🤗 Huggingface](https://huggingface.co/vita-video-gen/svi-model/resolve/main/version-1.0/svi-dance.safetensors?download=true)  [🤖 Modelscope](https://www.modelscope.cn/models/epfl-vita/svi-model/file/view/master/version-1.0%2Fsvi-dance.safetensors?status=2)          | Generate long videos with skeleton-conditioned human dancing                                               |

Note: If you want to play with T2V, you can directly use SVI with an image generated by any T2I model!

### SVI-2.0

For this model, you can try the sample in [100-prompt-sample](data/toy_test/svi_2.0) with SVI-Shot inference script. It should generate results similar to the ones shown in our 14-min YouTube video.

```bash
# This uses the SVI-Shot inference script and workflow, supporting both 5 and 1 motion frames
huggingface-cli download vita-video-gen/svi-model version-2.0/SVI_Wan2.1-I2V-14B_lora_v2.0.safetensors --local-dir ./weights/Stable-Video-Infinity

```

### SVI-1.0
```bash
# login with your fine-grained token
huggingface-cli login

# Option 1: Download SVI Family bucket!
huggingface-cli download vita-video-gen/svi-model --local-dir ./weights/Stable-Video-Infinity --include "version-1.0/*"

# Option 2: Download individual models
# huggingface-cli download vita-video-gen/svi-model version-1.0/svi-shot.safetensors --local-dir ./weights/Stable-Video-Infinity
# huggingface-cli download vita-video-gen/svi-model version-1.0/svi-film-opt-10212025.safetensors  --local-dir ./weights/Stable-Video-Infinity
# huggingface-cli download vita-video-gen/svi-model version-1.0/svi-film.safetensors --local-dir ./weights/Stable-Video-Infinity
# huggingface-cli download vita-video-gen/svi-model version-1.0/svi-film-transitions.safetensors --local-dir ./weights/Stable-Video-Infinity
# huggingface-cli download vita-video-gen/svi-model version-1.0/svi-tom.safetensors --local-dir ./weights/Stable-Video-Infinity
# huggingface-cli download vita-video-gen/svi-model version-1.0/svi-talk.safetensors --local-dir ./weights/Stable-Video-Infinity
# huggingface-cli download vita-video-gen/svi-model version-1.0/svi-dance.safetensors --local-dir ./weights/Stable-Video-Infinity
```

### Download Multitalk Cross-Attention for SVI-Talk Training/Test

```bash
# Download audio encoder
huggingface-cli download TencentGameMate/chinese-wav2vec2-base --local-dir ./weights/chinese-wav2vec2-base 
huggingface-cli download TencentGameMate/chinese-wav2vec2-base model.safetensors --revision refs/pr/1 --local-dir ./weights/chinese-wav2vec2-base

# Download multitalk weight
huggingface-cli download MeiGen-AI/MeiGen-MultiTalk --local-dir ./weights/MeiGen-MultiTalk

# Link Multitalk
ln -s $PWD/weights/MeiGen-MultiTalk/multitalk.safetensors weights/Wan2.1-I2V-14B-480P/
```

### Download UniAnimate-DiT LoRA for SVI-Dance Training

```bash
huggingface-cli download ZheWang123/UniAnimate-DiT --local-dir ./weights/UniAnimate-DiT
```

### Check Model

After downloading all the models, your `weights/` directory structure should look like this:

```
weights/
├── Wan2.1-I2V-14B-480P/
│   ├── diffusion_pytorch_model-00001-of-00007.safetensors
│   ├── diffusion_pytorch_model-00002-of-00007.safetensors
│   ├── diffusion_pytorch_model-00003-of-00007.safetensors
│   ├── diffusion_pytorch_model-00004-of-00007.safetensors
│   ├── diffusion_pytorch_model-00005-of-00007.safetensors
│   ├── diffusion_pytorch_model-00006-of-00007.safetensors
│   ├── diffusion_pytorch_model-00007-of-00007.safetensors
│   ├── diffusion_pytorch_model.safetensors.index.json
│   ├── models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth
│   ├── models_t5_umt5-xxl-enc-bf16.pth
│   ├── Wan2.1_VAE.pth
│   ├── multitalk.safetensors (symlink)
│   └── README.md
├── Stable-Video-Infinity/
│   ├── version-2.0/
│   │   └── SVI_Wan2.1-I2V-14B_lora_v2.0.safetensors (Improved Wan 2.1 14B SVI )
│   └── version-1.0/
│       ├── svi-shot.safetensors
│       ├── svi-film.safetensors
│       ├── svi-film-transitions.safetensors
│       ├── svi-tom.safetensors
│       ├── svi-talk.safetensors
│       └── svi-dance.safetensors
├── chinese-wav2vec2-base/ (for SVI-Talk)
│   ├── config.json
│   ├── model.safetensors
│   ├── preprocessor_config.json
│   └── README.md
├── MeiGen-MultiTalk/ (for SVI-Talk)
│   ├── diffusion_pytorch_model.safetensors.index.json
│   ├── multitalk.safetensors
│   └── README.md
└── UniAnimate-DiT/ (for SVI-Dance)
    ├── dw-ll_ucoco_384.onnx
    ├── UniAnimate-Wan2.1-14B-Lora-12000.ckpt
    ├── yolox_l.onnx
    └── README.md
```

## 🎮 Play with Official SVI

### Inference Scripts

The following scripts will use data in `data/demo` for inference. You can also use custom data to inference by simply changing the data path.

```bash
# SVI-2.0
bash scripts/test/svi_2.0.sh

# SVI-Shot
bash scripts/test/svi_shot.sh

# SVI-Film
bash scripts/test/svi_film.sh

# SVI-Talk
bash scripts/test/svi_talk.sh

# SVI-Dance
bash scripts/test/svi_dance.sh

# SVI-Tom&Jerry
bash scripts/test/svi_tom.sh 
```

### Gradio Demo

Currently, gradio demo only supports SVI-Shot and SVI-Film.

```bash
bash gradio_demo.sh
```

## 🔥 Train Your Own SVI

We have prepared the toy training data `data/toy_train/`. You can simply follow the data format to train SVI with your custom data.
Please modify `--num_nodes` if you use more nodes for training. We have tested both 8 and 64 GPUs for training, where larger batch-size gave a better performance.

### SVI-Shot

```bash
# (Optionally) Use scripts/data_preprocess/process_mixkit.py from CausVid to pre-process data
# start training
bash scripts/train/svi_shot.sh 
```

### SVI-Film

```bash
# (Optionally) Use scripts/data_preprocess/process_mixkit.py from CausVid to pre-process data
# start training
bash scripts/train/svi_film.sh 
```

### SVI-Talk

```bash
# Preprocess the toy training data
python scripts/data_preprocess/prepare_video_audio.py

# Start training
bash scripts/train/svi_talk.sh 
```

### SVI-Dance

```bash
# Preprocess the toy training data
python scripts/data_preprocess/prepare_video_audio.py

# Start training
bash scripts/train/svi_dance.sh 
```

## 📝 Test Your Trained SVI

### Model Post-processing

```bash
# Change .pt files to .safetensors files
# zero_to_fp32.py will be automatically generated in your model dir, change $DIR_WITH_SAFETENSORS into your desired DIR
python zero_to_fp32.py . $DIR_WITH_SAFETENSORS --safe_serialization

# (Optionally) Extract and only save LoRA parameters to reduce disk space
python utils/extract_lora.py --checkpoint_dir $DIR_WITH_SAFETENSORS --output_dir $XXX
```

### Inference

Please modify the inference scripts in `./scripts/test/` accordingly by changing the inference samples and your new weight

## 🗃️ Datasets

You can also use our benchmark datasets made by our Automatic Prompt Stream Engine (see Appendix. A for more details), where you can find images and associated prompt streams according to specific storylines.

| Data                                               | Use  | HuggingFace Link                                                                                            | Comment                                                                                           |
| -------------------------------------------------- | ---- | ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| **Consistent Video Generation**              | Test | [🤗 Dataset](https://huggingface.co/datasets/vita-video-gen/svi-benchmark/tree/main/consisent_video_gen)       | Generate 1 long video using 1 text prompt                                                         |
| **Creative Video Generation**                | Test | [🤗 Dataset](https://huggingface.co/datasets/vita-video-gen/svi-benchmark/tree/main/creative_video_gen)        | Generate 1 long video using 1 text prompt stream according to storyline (1 prompt for 5 sec clip) |
| **Creative Video Generation (More prompts)** | Test | [🤗 Dataset](https://huggingface.co/datasets/vita-video-gen/svi-benchmark/tree/main/creative_video_gen_longer) | Generate 1 long video using 1 text prompt stream according to storyline (1 prompt for 5 sec clip) |

The following is the training data we used for SVI family.

| Data                                           | Use   | HuggingFace Link                                                                                     | Comment                                                 |
| ---------------------------------------------- | ----- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------- |
| **Customized Datasets**                  | Train | [🤗 Dataset](https://huggingface.co/datasets/vita-video-gen/svi-benchmark/tree/main/customized_dataset) | You can make your customized datasets using this format |
| **Consistent/Creative Video Generation** | Train | [🤗 Dataset](https://huggingface.co/datasets/LanguageBind/Open-Sora-Plan-v1.1.0/tree/main/all_mixkit)   | MixKit Dataset                                           |
| **Consistent/Creative Video Generation** | Train | [🤗 Dataset](https://huggingface.co/datasets/APRIL-AIGC/UltraVideo-Long)                                | UltraVideo Dataset                                      |
| **Human Talking**                        | Train | [🤗 Dataset](https://huggingface.co/fudan-generative-ai/hallo3)                                         | 5k subset from Hallo 3                                  |
| **Human Dancing**                        | Train | [🤗 Dataset](https://www.kaggle.com/datasets/yasaminjafarian/tiktokdataset)                             | TikTok                                                  |

```bash
huggingface-cli download --repo-type dataset vita-video-gen/svi-benchmark --local-dir ./data/svi-benchmark
```

## 📋 TODO List

- [X] Release everything about SVI 1.0  
- [X] SVI 2.0 for Wan 2.1 and Wan 2.1
- [ ] Wan 2.2 Animate SVI
- [ ] Customizable video generation

## 🙏 Acknowledgement

We greatly appreciate the tremendous effort for the following fantastic projects!

[1] [Wan: Open and Advanced Large-Scale Video Generative Models](https://arxiv.org/abs/2503.20314)  
[2] [UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformer](https://arxiv.org/abs/2504.11289)  
[3] [Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation](https://arxiv.org/abs/2505.22647)

## ❤️ Citation

If you find our work helpful for your research, please consider citing our paper. Thank you so much!

```bibtex
@article{li2025stable,
  title={Stable Video Infinity: Infinite-Length Video Generation with Error Recycling},
  author={Li, Wuyang and Pan, Wentao and Luan, Po-Chien and Gao, Yang and Alahi, Alexandre},
  journal={arXiv preprint arXiv:2510.09212},
  year={2025}
}
```

## 📌 Abstract

We propose **Stable Video Infinity (SVI)** that is able to generate infinite-length videos with high temporal consistency, plausible scene transitions, and controllable streaming storylines. While existing long-video methods attempt to *mitigate accumulated errors* via handcrafted anti-drifting (e.g., modified noise scheduler, frame anchoring), they remain limited to single-prompt extrapolation, producing homogeneous scenes with repetitive motions. We identify that the fundamental challenge extends beyond error accumulation to a critical discrepancy between the training assumption (seeing clean data) and the test-time autoregressive reality (conditioning on self-generated, error-prone outputs). To bridge this hypothesis gap, SVI incorporates **Error-Recycling Fine-Tuning**, a new type of efficient training that recycles the Diffusion Transformer (DiT)'s self-generated errors into supervisory prompts, thereby encouraging DiT to *actively identify and correct its own errors*. This is achieved by injecting, collecting, and banking errors through closed-loop recycling, autoregressively learning from error-injected feedback. Specifically, we (i) inject historical errors made by DiT to intervene on clean inputs, simulating error-accumulated trajectories in flow matching; (ii) efficiently approximate predictions with one-step bidirectional integration and calculate errors with residuals; (iii) dynamically bank errors into replay memory across discretized timesteps, which are resampled for new input. SVI is able to scale videos from seconds to infinite durations with no additional inference cost, while remaining compatible with diverse conditions (e.g., audio, skeleton, and text streams). We evaluate SVI on three benchmarks, including consistent, creative, and conditional settings, thoroughly verifying its versatility and state-of-the-art role.

Stable-Video-Infinity

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Stable-Video-Infinity

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README