ComfyUI-DiffSynth-Studio - GitLite

KO | EN

gitlite — search

#hacktoberfest #react #golang #deep-learning #linux #docker #awesome #kubernetes #awesome-list #security #pytorch #ios #mcp #openai

ComfyUI-DiffSynth-Studio

(★ 101)

make DiffSynth-Studio avialbe in ComfyUI

File Explorer

Download Latest Version (.zip)

- - __init__.py
  - controlnet_unit.py
  - processors.py
- - __init__.py
  - video.py
- - __init__.py
  - attention.py
  - downloader.py
  - hunyuan_dit.py
  - hunyuan_dit_text_encoder.py
  - sd_controlnet.py
  - sd_ipadapter.py
  - sd_lora.py
  - sd_motion.py
  - sd_text_encoder.py
  - sd_unet.py
  - sd_vae_decoder.py
  - sd_vae_encoder.py
  - sdxl_ipadapter.py
  - sdxl_motion.py
  - sdxl_text_encoder.py
  - sdxl_unet.py
  - sdxl_vae_decoder.py
  - sdxl_vae_encoder.py
  - svd_image_encoder.py
  - svd_unet.py
  - svd_vae_decoder.py
  - svd_vae_encoder.py
  - tiler.py
- - __init__.py
  - dancer.py
  - hunyuan_dit.py
  - stable_diffusion.py
  - stable_diffusion_video.py
  - stable_diffusion_xl.py
  - stable_diffusion_xl_video.py
  - stable_video_diffusion.py
- - __init__.py
  - base.py
  - FastBlend.py
  - PILEditor.py
  - RIFE.py
  - sequencial_processor.py
- - __init__.py
  - hunyuan_dit_prompter.py
  - sd_prompter.py
  - sdxl_prompter.py
  - utils.py
- - __init__.py
  - continuous_ode.py
  - ddim.py
- __init__.py
- put diffsynth studio models here
- - previewVideo.js
  - uploadVideo.js
- diffutoon_workflow.json
- exvideo_workflow.json
.gitignore
__init__.py
LICENSE
README.md
requirements.txt
studio_nodes.py
util_nodes.py
web.png

# Installation Guide

1. Get the code

git clone https://github.com/AIFSH/ComfyUI-DiffSynth-Studio

Downloads the entire project code from GitHub to your computer.

cd ComfyUI-DiffSynth-Studio

Moves into the project folder you just downloaded.

2. Official Install Script

Easy Recommended

Prerequisites

APT (Debian/Ubuntu 계열) Built into Debian/Ubuntu-based Linux distributions.

apt install ffmpeg

Installs directly from the APT package repository (Debian/Ubuntu-based).

✅ After installing, open a new terminal and run the program's version command (e.g. --version) to confirm it worked.

Pulled directly from this repo's README.

3. Python

Easy

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

pip install -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

✅ If it runs without errors and prints output in the terminal, it worked.

Pulled directly from this repo's README.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ComfyUI-DiffSynth-Studio
make [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio) avialbe in ComfyUI
<div>
  <figure>
  <img alt='webpage' src="web.png?raw=true" width="600px"/>
  <figure>
</div>

## how to use
test on py3.10，2080ti 11gb，torch==2.3.0+cu121

make sure `ffmpeg` is worked in your commandline
for Linux
```
apt update
apt install ffmpeg
```
for Windows,you can install `ffmpeg` by [WingetUI](https://github.com/marticliment/WingetUI) automatically

then!
```
# in ComfyUI/custom_nodes
git clone https://github.com/AIFSH/ComfyUI-DiffSynth-Studio.git
cd ComfyUI-DiffSynth-Studio
pip install -r requirements.txt
```
weights will be downloaded from huggingface or model scope

## Tutorial
- [Demo for Diffutoon](https://b23.tv/z7hEXlX)[DiffSynth-Studio!ComfyUI插件之Diffutoon节点-哔哩哔哩](https://b23.tv/z7hEXlX)
- [Run on 4090](https://www.xiangongyun.com/image/detail/13706bf7-f3e6-4e29-bb97-c79405f5def4)
- wechat：aifsh_98

## Nodes Detail and Workflow
### ExVideo Node
[ExVideo workflow](./workfolws/exvideo_workflow.json)
```
"image":("IMAGE",),
"svd_base_model":("SD_MODEL_PATH",),
"exvideo_model":("SD_MODEL_PATH",),
"num_frames":("INT",{ 
    "default": 128
}),
"fps":("INT",{
    "default": 30
}),
"num_inference_steps":("INT",{
    "default": 50
}),
"if_upscale":("BOOLEAN",{
    "default": True,
}),
"seed": ("INT",{
    "default": 1
})
```
### image synthesis
comming soon
### Diffutoon Node
[Diffutoon workflow](./workfolws/diffutoon_workflow.json)
```
"required":{
    "source_video_path": ("VIDEO",),
    "sd_model_path":("SD_MODEL_PATH",),
    "postive_prompt":("TEXT",),
    "negative_prompt":("TEXT",),
    "start":("INT",{
        "default": 0 ## from which second of your video to be shaded
    }),
    "length":("INT",{
        "default": -1 ## how long you want to shade, -1 name the whole video frames
    }),
    "seed":("INT",{
        "default": 42 
    }),
    "cfg_scale":("INT",{
        "default": 3
    }),
    "num_inference_steps":("INT",{
        "default": 10
    }),
    "animatediff_batch_size":("INT",{
        "default": 32 ## lower it till you can run
    }),
    "animatediff_stride":("INT",{
        "default": 16 ## lower it till you can run
    }),
    "vram_limit_level":("INT",{
        "default": 0 ## meet killed? try to 1
    }),
},
"optional":{
    "controlnet1":("ControlNetConfigUnit",),
    "controlnet2":("ControlNetConfigUnit",),
    "controlnet3":("ControlNetConfigUnit",),
}
```

### Video Stylization

### Chinese Models

## ask for answer as soon as you want
wechat: aifsh_98
need donate if you mand it,
but please feel free to new issue for answering

Windows环境配置太难？可以添加微信：aifsh_98，赞赏获取Windows一键包，当然你也可以提issue等待大佬为你答疑解惑。