WanFM

KO | EN

gitlite — search

#javascript #hacktoberfest #react #typescript #llm #go #machine-learning #linux #java #nodejs #windows #security #openai #open-source

(★ 18)

No description available.

WanFM Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Wan Frame Morphing (WanFM)
<p align="center">
<img height="250" alt="logo" src="https://github.com/user-attachments/assets/5c38e92c-727f-47e4-84b6-5df3f0121aa8" />
<p>
<p align="center">
Pengjun Fang, <a href="https://leehomyc.github.io/">Harry Yang</a>
</p>
In this repository, we present WanFM (frame morphing), which builds upon Wan 2.2 Image-to-Video (I2V) and introduces several key enhancements:

- **Last Frame Constraint**: Enforces precise alignment between the generated last frame and the target frame, ensuring consistent video endpoints.

- **Bidirectional Denoising with Time Reversal Fusion**: Performs denoising both forward (first-to-last) and backward (last-to-first), fusing intermediate results at every step for superior temporal coherence. To accommodate bidirectional fusion, the original denoising formula—where each step depends on the previous one—has been redesigned, allowing non-continuous, step-wise integration of forward and backward denoised states.

- **Prompt-Adapted Temporal Attention**: During the reverse pass, temporal self-attention is rotated to align backward generation with the prompt, enabling bidirectionally refined, prompt-consistent video sequences.

With these improvements, we achieve First–Last–Frame-to-Video generation (FLF2V), enabling controllable and consistent video synthesis given the first and last frames as constraints.
## Demo
<div align="center">
  <video src="https://github.com/user-attachments/assets/0e72ac8b-75a2-47fa-9530-c89650621e92" width="100%" poster=""> </video>
</div>

## Run WanFM
### Enviroment Preparation
Please see Wan2.2 (https://github.com/Wan-Video/Wan2.2?tab=readme-ov-file#installation).
### Model Download
| Models              | Download Links                                                                                                                              | Description |
|--------------------|---------------------------------------------------------------------------------------------------------------------------------------------|-------------|
| I2V-A14B    | 🤗 [Huggingface](https://huggingface.co/Wan-AI/Wan2.2-I2V-A14B)    🤖 [ModelScope](https://modelscope.cn/models/Wan-AI/Wan2.2-I2V-A14B)    | Image-to-Video MoE model, supports 480P & 720P |
### Run First-Last-Frame-to-Video Generation
- Single-GPU inference
```sh
python generate.py \
    --task flf2v-A14B \
    --size 832*480 \
    --ckpt_dir ./Wan2.2-I2V-A14B \
    --offload_model False \
    --frame_num 81 \
    --sample_steps 40 \
    --sample_shift 16 \
    --sample_guide_scale 5 \
    --prompt <prompt> \
    --first_frame <first frame path> \
    --last_frame <last frame path> \
    --save_file <output path> \
    --bidirectional_sampling
```
- Multi-GPU inference using FSDP + DeepSpeed Ulysses
```sh
torchrun --nproc_per_node=8 --master_port 39550 generate.py \
    --task flf2v-A14B \
    --size 832*480 \
    --ckpt_dir ./Wan2.2-I2V-A14B \
    --offload_model False \
    --convert_model_dtype \
    --frame_num 81 \
    --sample_steps 40 \
    --sample_shift 16 \
    --sample_guide_scale 5 \
    --dit_fsdp \
    --t5_fsdp \
    --ulysses_size 2 \
    --prompt <prompt> \
    --first_frame <first frame path> \
    --last_frame <last frame path> \
    --save_file <output path> \
    --bidirectional_sampling
```
If you encounter OOM (Out-of-Memory) issues, you can use the `--offload_model True`, `--convert_model_dtype` and `--t5_cpu` options to reduce GPU memory usage.
## More Examples

WanFM

WanFM Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

WanFM

WanFM Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README