naifu

KO | EN

gitlite — search

#hacktoberfest #ai #typescript #go #android #docker #macos #java #kubernetes #awesome-list #vue #security #mcp #ios

(★ 339)

Train generative models with pytorch lightning

File Explorer

Download Latest Version (.zip)

- __init__.py
- deepspeed.py
- fairscale.py
- fsdp.py
- logging.py
- smddp.py
- trainer.py
- utils.py
- train_cascade_stage_c.yaml
- train_diffusers.yaml
- train_dpo.yaml
- train_dpo_diffusers.yaml
- train_dpo_sd15_diffusers.yaml
- train_fairscale.yaml
- train_general_llm.yaml
- train_gpt2.yaml
- train_lcm.yaml
- train_llava.yaml
- train_llava_1.6.yaml
- train_lycoris.yaml
- train_lycoris_scaled.yaml
- train_mistral_lora.yaml
- train_phi2.yaml
- train_pixart.yaml
- train_pixart_sigma.yaml
- train_refiner.yaml
- train_sd15.yaml
- train_sdxl.yaml
- train_sdxl_adaptive.yaml
- train_sdxl_deepspeed.yaml
- train_sdxl_original.yaml
- train_sdxl_v.yaml
- __init__.py
- bucket.py
- conversation.py
- image_storage.py
- llava_dataset.py
- paired_wds.py
- processors.py
- prompt_style.py
- text_dataset.py
- - common.py
  - effnet.py
  - previewer.py
  - quantize.py
  - stage_a.py
  - stage_b.py
  - stage_c.py
- - __init__.py
  - loss_weights.py
  - noise_conditions.py
  - readme.md
  - samplers.py
  - scalers.py
  - schedulers.py
  - targets.py
- - axial_rope.py
  - dctorch.py
  - flags.py
  - flops.py
  - layers.py
  - network.py
  - utils.py
- - constants.py
  - llava_llama.py
  - mm_utils.py
- - configuration_phi.py
  - modeling_opt.py
  - modeling_phi.py
- - __init__.py
  - alpha.py
  - diffusion.py
  - sigma.py
- - __init__.py
  - attention.py
  - autoencoder.py
  - denoiser.py
  - denoiser_scaling.py
  - denoiser_weighting.py
  - discretizer.py
  - encoder_util.py
  - encoders.py
  - model.py
  - model_util.py
  - sigma_sampling.py
  - vae_model.py
- __init__.py
- __init__.py
- cascade_model.py
- config_sdxl_base.py
- config_sdxl_refiner.py
- scheduler_utils.py
- sd_dpo_diffusers.py
- sd_model_diffusers.py
- sdxl_dpo.py
- sdxl_dpo_diffusers.py
- sdxl_model.py
- sdxl_model_diffusers.py
- sdxl_utils.py
- train_cascade_stage_c.py
- train_diffusers.py
- train_general_llm.py
- train_gpt2.py
- train_lcm.py
- train_llava.py
- train_lycoris.py
- train_phi.py
- train_pixart.py
- train_pixart_sigma.py
- train_refiner.py
- train_sd_diffusers.py
- train_sdxl.py
- train_sdxl_edm.py
- train_sdxl_flow.py
- train_sdxl_original.py
- deepdanbooru.py
- encode_latents_xl.py
- gpt2_inference.py
- run_llava.py
- vae_cache_inspector.ipynb
- vae_playground.ipynb
- wd14_tagger.py
.gitignore
LICENSE
README.md
requirements.txt
trainer.py

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Naifu

naifu (or naifu-diffusion) is designed for training generative models with various configurations and features. The code in the main branch of this repository is under development and subject to change as new features are added.

## Installation

To get started with Naifu, follow these steps to install the necessary dependencies:

```bash
# Clone the Naifu repository:
git clone --depth 1 https://github.com/mikubill/naifu

# Install the required Python packages:
cd naifu && pip install -r requirements.txt
```

Make sure you have a compatible version of Python installed (Python 3.9 or above).

## Usage

Naifu provides a flexible and intuitive way to train models using various configurations. To train a model, use the trainer.py script and provide the desired configuration file as an argument.

```bash
python trainer.py --config config/<config_file>

# or (same as --config)
python trainer.py config/<config_file>
```

Replace `<config_file>` with one of the available configuration files listed below.

## Configurations

Choose the appropriate configuration file based on training objectives and environment.

Train SDXL (Stable Diffusion XL) model
```bash
# prepare image data (to latents)
python scripts/encode_latents_xl.py -i <input_path> -o <encoded_path>

# sd_xl_base_1.0_0.9vae.safetensors
python trainer.py config/train_sdxl.yaml

# For huggingface model support
# stabilityai/stable-diffusion-xl-base-1.0
python trainer.py config/train_diffusers.yaml

# use original sgm loss module
python trainer.py config/train_sdxl_original.yaml
```

Train SDXL refiner (Stable Diffusion XL refiner) model
```bash
# stabilityai/stable-diffusion-xl-refiner-1.0
python trainer.py config/train_refiner.yaml
```

Train original Stable Diffusion 1.4 or 1.5 model
```bash
# runwayml/stable-diffusion-v1-5
# Note: will save in diffusers format
python trainer.py config/train_sd15.yaml
```

Train SDXL model with LyCORIS.
```bash
# Based on the work available at KohakuBlueleaf/LyCORIS
pip install lycoris_lora toml
python trainer.py config/train_lycoris.yaml
```

Use fairscale strategy for distributed data parallel sharded training
```bash
pip install fairscale
python trainer.py config/train_fairscale.yaml
```

Train SDXL model with Diffusion DPO  
Paper: Diffusion Model Alignment Using Direct Preference Optimization ([arxiv:2311.12908](https://arxiv.org/abs/2311.12908))
```bash
# dataset: yuvalkirstain/pickapic_v2
# Be careful tuning the resolution and dpo_betas!
# will save in diffusers format
python trainer.py config/train_dpo_diffusers.yaml # diffusers backend
python trainer.py config/train_dpo.yaml # sgm backend
```

Train Pixart-Alpha model  
Paper: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis ([arxiv:2310.00426](https://arxiv.org/abs/2310.00426))
```bash
# PixArt-alpha/PixArt-XL-2-1024-MS
python trainer.py config/train_pixart.yaml
```

Train SDXL-LCM model  
Paper: Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference ([arxiv:2310.04378](https://arxiv.org/abs/2310.04378))
```bash
python trainer.py config/train_lcm.yaml
```

Train StableCascade model ([Sai](https://github.com/Stability-AI/StableCascade/))
```bash
# currently only stage_c (w/ or w/o text encoder)
python trainer.py config/train_cascade_stage_c.yaml
```

Train GPT2 model
```bash
# currently only stage_c (w/ or w/o text encoder)
python trainer.py config/train_gpt2.yaml
```

Train with [Phi-1.5/2](https://huggingface.co/microsoft) model
```bash
python trainer.py config/train_phi2.yaml
```

Train language models ([LLaMA](https://github.com/facebookresearch/llama), [Qwen](https://huggingface.co/Qwen), [Gemma](https://huggingface.co/google) etc)
```bash
# Note that prepare data in sharegpt/chatml format, or define your own dataset in data/text_dataset.py
# See example dataset for reference: function-calling-sharegpt
python trainer.py config/train_general_llm.yaml
```

Train language models with lora or qlora (For example, [Mistral](https://huggingface.co/mistralai))
```bash
python trainer.py config/train_mistral_lora.yaml
```

## Other branches

* sgm - Uses the [sgm](https://github.com/Stability-AI/generative-models) to train SDXL models.
* sd3 - Trainer for SD3 models - use with caution: may produce undesired result
* hydit - Trainer for hunyuan dit models (v1.1 and v1.2)
* main-archived - Contains the original naifu-diffusion code for training Stable Diffusion 1.x models.

For branches without documentation, please follow the installation instructions provided above.

naifu

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

naifu

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README