VMix

KO | EN

gitlite — search

#javascript #hacktoberfest #ai #typescript #machine-learning #deep-learning #linux #awesome #cli #awesome-list #security #pytorch #ios #openai

(★ 191)

Official code for VMix: Improving Text-to-Image Diffusion Model with Cross-Attention Mixing Control

파일 탐색기

최종 버전 다운로드 (.zip)

- dimension_control.png
- example1.png
- example2.png
- slider.gif
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center">VMix: Improving Text-to-Image Diffusion Model with Cross-Attention Mixing Control</h1>

> Shaojin Wu,1</a> Fei Ding,1,*</a> Mengqi Huang,1,2</a> Wei Liu,1 </a> Qian He1</a> 
>1 ByteDance Inc. &nbsp;&nbsp;2 University of Science and Technology of China

### 📖 Introduction
We propose VMix, a plug-and-play aesthetics adapter, to upgrade the quality of generated images while maintaining generality across visual concepts by (1) disentangling the input text prompt into the content description and aesthetic description by the initialization of aesthetic embedding, and (2) integrating aesthetic conditions into the denoising process through value-mixed cross-attention, with the network connected by zero-initialized linear layers. VMix outperforms other state-of-the-art methods and is flexible enough to be applied to community modules (e.g., LoRA, ControlNet, and IPAdapter) for better visual performance without retraining.

### 🎨 Examples
Qualitative comparison between results with VMix(on the right) and without VMix(on the left)

<img src="./asset/example1.png" width=100% height=100% 
class="center">


<img src="./asset/example2.png" width=100% height=100% 
class="center">

Aesthetic Fine-grained Control
<div align = "center" style="margin-right: 15px; margin-left: 15px;"> 
<img src="./asset/slider.gif" width="35%" />
<img src="./asset/dimension_control.png" width="62%" />
</div>
For more visual results, go checkout our <a href="https://vmix-diffusion.github.io/VMix/" target="_blank">Project Page</a>

### 🔥Updates
We will open source this project as soon as possible. Thank you for your patience and support! 🌟
- [x] Release arXiv paper. Check the details [here](https://arxiv.org/pdf/2412.20800).
- [ ] Release inference code(Coming soon).
- [ ] Release model checkpoints.
- [ ] Release ComfyUI node.

##  Citation
If VMix is helpful, please help to ⭐ the repo.

If you find this project useful for your research, please consider citing our paper:
```bibtex
@misc{wu2024vmix,
    title={VMix: Improving Text-to-Image Diffusion Model with Cross-Attention Mixing Control},
    author={Shaojin Wu and Fei Ding and Mengqi Huang and Wei Liu and Qian He},
    year={2024},
    eprint={2412.20800},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}
```

VMix

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

VMix

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README