LlamaFactory
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
File Explorer
Download Latest Version (.zip)- CLAUDE.md
- SKILL.md
- 1-bug-report.yml
- 2-feature-request.yml
- config.yml
- docker.yml
- docker_npu.yml
- docs.yml
- label_issue.yml
- publish.yml
- tests.yml
- tests_cuda.yml
- tests_npu.yml
- CODE_OF_CONDUCT.md
- CONTRIBUTING.md
- copilot-instructions.md
- instructions-v0.md
- instructions-v1.md
- PULL_REQUEST_TEMPLATE.md
- SECURITY.md
- serpapi.svg
- warp.jpg
- colab.svg
- discord.svg
- dsw.svg
- lab4ai.svg
- online.svg
- logo.png
- 1.jpg
- 1.mp3
- 1.mp4
- 2.avi
- 2.jpg
- 2.wav
- 3.flac
- 3.jpg
- 3.mp4
- 4.mp3
- 4.mp4
- alpaca_en_demo.json
- alpaca_zh_demo.json
- c4_demo.jsonl
- dataset_info.json
- dpo_en_demo.json
- dpo_zh_demo.json
- glaive_toolcall_en_demo.json
- glaive_toolcall_zh_demo.json
- identity.json
- kto_en_demo.json
- mllm_audio_demo.json
- mllm_demo.json
- mllm_video_audio_demo.json
- mllm_video_demo.json
- README.md
- README_zh.md
- reason_tool_use_demo_50.jsonl
- v1_dpo_demo.jsonl
- v1_dpo_demo.yaml
- v1_multimodal_demo.jsonl
- v1_multimodal_demo.yaml
- v1_sft_demo.jsonl
- v1_sft_demo.yaml
- wiki_demo.txt
- docker-compose.yml
- Dockerfile
- Dockerfile.base
- Dockerfile.mbridge
- Dockerfile.megatron
- README.md
- docker-compose.yml
- Dockerfile
- OVERVIEW.md
- OVERVIEW.zh.md
- docker-compose.yml
- Dockerfile
- lang-switcher.css
- switcher.js
- custom-kernels.md
- fused-operators.md
- triton.md
- deepspeed.md
- fsdp.md
- fsdpturbo-ep-efsdp.md
- parallel-dp-tp-ep-sp-cp.md
- lora.md
- quantization.md
- ktransformers.md
- data-processing.md
- data-engine.md
- model-engine.md
- trainer.md
- initialization.md
- kernels.md
- rendering.md
- data-plugins.md
- data-argument.md
- model-argument.md
- sample-argument.md
- training-argument.md
- deploy.md
- dpo.md
- sft.md
- conf.py
- getting-started.md
- index.rst
- installation.md
- llamaboard-web-ui.md
- custom-kernels.md
- fused-operators.md
- triton.md
- deepspeed.md
- fsdp.md
- fsdpturbo-ep-efsdp.md
- parallel-dp-tp-ep-sp-cp.md
- lora.md
- quantization.md
- ktransformers.md
- data-processing.md
- data-engine.md
- model-engine.md
- trainer.md
- initialization.md
- kernels.md
- rendering.md
- data-plugins.md
- data-argument.md
- model-argument.md
- sample-argument.md
- training-argument.md
- deploy.md
- dpo.md
- sft.md
- conf.py
- getting-started.md
- index.rst
- installation.md
- llamaboard-web-ui.md
- conf.py
- make.bat
- Makefile
- requirements.txt
- fsdp2_config.yaml
- fsdp2_config_qwen35.yaml
- fsdp2_config_qwen35_moe.yaml
- fsdp_config.yaml
- fsdp_config_multiple_nodes.yaml
- fsdp_config_offload.yaml
- qwen3_5_full_sft_fsdp2.yaml
- qwen3_5moe_lora_sft_fsdp2.yaml
- qwen3_full_sft_fsdp2.yaml
- qwen3moe_full_sft_fsdp.yaml
- qwen3vlmoe_full_sft_fsdp2.yaml
- qwen3vlmoe_lora_sft_fsdp.yaml
- ds_z0_config.json
- ds_z2_autotp_config.json
- ds_z2_config.json
- ds_z2_offload_config.json
- ds_z3_config.json
- ds_z3_fp8_config.json
- ds_z3_offload_config.json
- qwen2_full_sft.yaml
- llama3_full_sft.yaml
- llama2_full_asft.yaml
- qwen2_full_asft.yaml
- llama3_full_sft.yaml
- qwen2_full_sft.yaml
- qwen25_05b_eaft_full.yaml
- llama3_fp8_deepspeed_sft.yaml
- llama3_fp8_fsdp_sft.yaml
- llama3_lora_sft.yaml
- train.sh
- llama3_full_sft.yaml
- expand.sh
- llama3_freeze_sft.yaml
- llama3_lora_sft.yaml
- llama3_full_sft.yaml
- tokens_cfg.yaml
- qwen2_full_sft.yaml
- llama3_lora_predict.yaml
- llama3_oft_sft.yaml
- qwen2_5vl_oft_sft.yaml
- init.sh
- llama3_lora_sft.yaml
- llama3_oft_sft_awq.yaml
- llama3_oft_sft_bnb_npu.yaml
- llama3_oft_sft_gptq.yaml
- mossvl.yaml
- qwen3.yaml
- qwen3_full_sft.yaml
- qwen3_lora_sft.yaml
- qwen3vl.yaml
- fsdp2_kt_bf16.yaml
- fsdp2_kt_int4.yaml
- fsdp2_kt_int8.yaml
- fsdp2_kt_int8_1gpu.yaml
- fsdp2_kt_int8_8gpu.yaml
- deepseek_v2_lora_sft_kt.yaml
- deepseek_v3_int8_lora_sft_kt.yaml
- deepseek_v3_lora_sft_kt.yaml
- qwen3_5moe_lora_sft_kt.yaml
- qwen3moe_lora_sft_kt.yaml
- qwen3vlmoe_lora_sft_kt.yaml
- qwen2_vl_full.yaml
- qwen3_moe_full.yaml
- llama3_sft.yaml
- mossvl_lora_sft.yaml
- qwen3_full_sft.yaml
- qwen3_gptq.yaml
- qwen3_lora_sft.yaml
- qwen3vl_lora_sft.yaml
- mossvl_full_sft.yaml
- qwen3_full_sft.yaml
- qwen3vl_full_sft.yaml
- mossvl_lora_sft.yaml
- qwen3_lora_dpo.yaml
- qwen3_lora_kto.yaml
- qwen3_lora_pretrain.yaml
- qwen3_lora_reward.yaml
- qwen3_lora_sft.sh
- qwen3_lora_sft.yaml
- qwen3_lora_sft_ds3.yaml
- qwen3_lora_sft_ray.yaml
- qwen3_preprocess.yaml
- qwen3vl_lora_dpo.yaml
- qwen3vl_lora_sft.yaml
- llama3_lora_sft_aqlm.yaml
- llama3_lora_sft_awq.yaml
- llama3_lora_sft_gptq.yaml
- qwen3_lora_sft_bnb_npu.yaml
- qwen3_lora_sft_otfq.yaml
- train_full_fsdp2_batching_normal.yaml
- train_full_fsdp2_dynamic_batching.yaml
- train_full_fsdp2_dynamic_padding_free.yaml
- train_full_fsdp2_padding_free.yaml
- train_freeze_sft.yaml
- train_full_deepspeed.yaml
- train_full_fsdp2.yaml
- train_full_liger_kernel.yaml
- train_full_muon.yaml
- train_full_qwen3_moe_fsdpturbo_ep_fsdp.yaml
- train_full_ulysses_cp.yaml
- train_multimodal.yaml
- export_lora.yaml
- train_lora_dpo.yaml
- train_lora_sft.yaml
- train_lora_sft_rank0.yaml
- quantization.yaml
- README.md
- README_zh.md
- adam-mini.txt
- apollo.txt
- aqlm.txt
- badam.txt
- bitsandbytes.txt
- deepspeed.txt
- dev.txt
- eetq.txt
- fp8-te.txt
- fp8.txt
- fsdpturbo.txt
- galore.txt
- gptq.txt
- hqq.txt
- ktransformers.txt
- liger-kernel.txt
- metrics.txt
- minicpm-v.txt
- moss-vl.txt
- npu.txt
- openmind.txt
- sglang.txt
- swanlab.txt
- triton_ascend.txt
- vllm.txt
- test_image.py
- test_toolcall.py
- llamafy_baichuan2.py
- llamafy_qwen.py
- tiny_llama4.py
- tiny_qwen3.py
- cal_flops.py
- cal_lr.py
- cal_mfu.py
- cal_ppl.py
- length_cdf.py
- bench_qwen.py
- dcp2hf.py
- eval_bleu_rouge.py
- hf2dcp.py
- llama_pro.py
- loftq_init.py
- megatron_merge.py
- pissa_init.py
- qwen_omni_merge.py
- vllm_infer.py
- __init__.py
- app.py
- chat.py
- common.py
- protocol.py
- __init__.py
- base_engine.py
- chat_model.py
- hf_engine.py
- sglang_engine.py
- vllm_engine.py
- __init__.py
- feedback.py
- pairwise.py
- pretrain.py
- processor_utils.py
- supervised.py
- unsupervised.py
- __init__.py
- collator.py
- converter.py
- data_utils.py
- formatter.py
- loader.py
- mm_plugin.py
- parser.py
- template.py
- tool_utils.py
- __init__.py
- evaluator.py
- template.py
- __init__.py
- constants.py
- env.py
- logging.py
- misc.py
- packages.py
- ploting.py
- __init__.py
- data_args.py
- evaluation_args.py
- finetuning_args.py
- generating_args.py
- megatron_bridge_args.py
- model_args.py
- parser.py
- training_args.py
- __init__.py
- attention.py
- checkpointing.py
- embedding.py
- kv_cache.py
- liger_kernel.py
- longlora.py
- misc.py
- mod.py
- moe.py
- packing.py
- quantization.py
- rope.py
- unsloth.py
- valuehead.py
- visual.py
- __init__.py
- adapter.py
- loader.py
- patcher.py
- __init__.py
- muon.py
- chunk_delta_h.py
- chunk_gated_delta_rule.py
- chunk_o.py
- chunk_scaled_dot_kkt.py
- cumsum.py
- solve_tril.py
- utils.py
- wy_fast.py
- __init__.py
- __init__.py
- trainer.py
- workflow.py
- __init__.py
- trainer.py
- workflow.py
- __init__.py
- trainer.py
- workflow.py
- __init__.py
- trainer.py
- workflow.py
- __init__.py
- config_builder.py
- dataset_export.py
- workflow.py
- __init__.py
- ppo_utils.py
- trainer.py
- workflow.py
- __init__.py
- trainer.py
- workflow.py
- __init__.py
- metric.py
- trainer.py
- workflow.py
- __init__.py
- metric.py
- trainer.py
- workflow.py
- __init__.py
- callbacks.py
- fp8_utils.py
- test_utils.py
- trainer_utils.py
- tuner.py
- __init__.py
- helper.py
- interface.py
- profiler.py
- __init__.py
- arg_parser.py
- arg_utils.py
- data_args.py
- model_args.py
- sample_args.py
- training_args.py
- __init__.py
- escape.py
- format.py
- rendering.py
- __init__.py
- batching.py
- callback.py
- checkpoint.py
- collation.py
- inference_engine.py
- __init__.py
- base_sampler.py
- base_trainer.py
- data_engine.py
- model_engine.py
- __init__.py
- converter.py
- loader.py
- fla.py
- __init__.py
- cuda_fused_moe.py
- npu_fused_moe.py
- npu_swiglu.py
- triton_grouped_gemm.py
- __init__.py
- npu_rms_norm.py
- __init__.py
- npu_rope.py
- __init__.py
- __init__.py
- base.py
- interface.py
- liger_kernel_ops.py
- gdn_attention.py
- seq_comm.py
- sequence_parallel.py
- ulysses.py
- __init__.py
- add_token.py
- deepspeed_utils.py
- initialization.py
- peft.py
- quantization.py
- __init__.py
- vllm.py
- __init__.py
- base.py
- deepspeed.py
- fsdp2.py
- fsdpturbo.py
- interface.py
- __init__.py
- muon_optimizer.py
- optimizer.py
- __init__.py
- batching.py
- lr_scheduler.py
- __init__.py
- cli_sampler.py
- __init__.py
- dpo_trainer.py
- rm_trainer.py
- sft_trainer.py
- __init__.py
- logging_callback.py
- trainer_callback.py
- __init__.py
- constants.py
- dtype.py
- env.py
- helper.py
- logging.py
- objects.py
- packages.py
- plugin.py
- pytest.py
- types.py
- __init__.py
- launcher.py
- __init__.py
- chatbot.py
- data.py
- eval.py
- export.py
- footer.py
- infer.py
- top.py
- train.py
- __init__.py
- chatter.py
- common.py
- control.py
- css.py
- engine.py
- interface.py
- locales.py
- manager.py
- runner.py
- __init__.py
- cli.py
- launcher.py
- api.py
- train.py
- webui.py
- test_feedback.py
- test_pairwise.py
- test_processor_utils.py
- test_supervised.py
- test_unsupervised.py
- test_collator.py
- test_converter.py
- test_formatter.py
- test_loader.py
- test_mm_plugin.py
- test_moss_vl_plugin.py
- test_moss_vl_training_configs.py
- test_template.py
- test_chat.py
- test_sglang.py
- test_train.py
- test_eval_template.py
- test_add_tokens.py
- test_attention.py
- test_checkpointing.py
- test_embedding.py
- test_misc.py
- test_packing.py
- test_visual.py
- test_base.py
- test_freeze.py
- test_full.py
- test_lora.py
- test_pissa.py
- conftest.py
- test_config_builder.py
- test_dataset_export.py
- test_megatron_bridge_gpu.py
- test_model_support.py
- test_sft_trainer.py
- check_license.py
- conftest.py
- version.txt
- test_interface.py
- test_args_parser.py
- test_rendering.py
- test_batching.py
- test_data_engine.py
- test_model_loader.py
- test_converter.py
- test_init_plugin.py
- test_kernel_plugin.py
- test_peft.py
- test_quantization_plugin.py
- test_ulysses_cp.py
- test_fsdp2.py
- test_fsdp2_weight_convert.py
- test_fsdpturbo_ep.py
- test_cli_sampler.py
- test_dpo_loss_precision.py
- test_fsdp2_dpo_trainer.py
- test_fsdp2_sft_trainer.py
- conftest.py
- .dockerignore
- .env.local
- .gitattributes
- .gitignore
- .pre-commit-config.yaml
- CITATION.cff
- CLAUDE.md
- LICENSE
- Makefile
- MANIFEST.in
- pyproject.toml
- README.md
- README_zh.md
# Installation Guide
1. Get the code
git clone https://github.com/hiyouga/LlamaFactory
Downloads the entire project code from GitHub to your computer.
cd LlamaFactory
Moves into the project folder you just downloaded.
2. Official Install Script
Easy RecommendedPrerequisites
- Python 3 Python is required to use pip.
pip install "huggingface_hub<1.0.0"
Installs the package published on PyPI directly β no need to clone the source.
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
Installs the package published on PyPI directly β no need to clone the source.
pip install bitsandbytes
Installs the package published on PyPI directly β no need to clone the source.
pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.2.post2-py3-none-win_amd64.whl
Installs the package published on PyPI directly β no need to clone the source.
After installing, open a new terminal and run the program's version command (e.g. --version) to confirm it worked.
Pulled directly from this repo's README.
3. Docker
EasyPrerequisites
- Git Needed to download the project code from GitHub.
- Docker Desktop Needed to build and run containers. Install it and keep it running in the background.
docker run -it --rm --gpus=all --ipc=host hiyouga/llamafactory:latest
Runs the built image as an actual container.
docker pull hiyouga/llamafactory:latest-910b-ubuntu
Type this command into your terminal and run it.
docker pull hiyouga/llamafactory:latest-a3-ubuntu
Type this command into your terminal and run it.
docker pull hiyouga/llamafactory:latest-910b-openeuler
Type this command into your terminal and run it.
docker pull hiyouga/llamafactory:latest-a3-openeuler
Type this command into your terminal and run it.
Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.
Pulled directly from this repo's README.
4. Python
EasyPrerequisites
pip install "huggingface_hub<1.0.0"
Installs the package published on PyPI directly β no need to clone the source.
pip install -e .
Installs the Python libraries listed in requirements.txt (or similar).
pip install -r requirements/metrics.txt
Installs the Python libraries listed in requirements.txt (or similar).
pip uninstall torch torchvision torchaudio
Type this command into your terminal and run it.
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
Installs the package published on PyPI directly β no need to clone the source.
If it runs without errors and prints output in the terminal, it worked.
Pulled directly from this repo's README.
5. Make
MediumPrerequisites
- Git Needed to download the project code from GitHub.
- Make Usually pre-installed on Linux/macOS. On Windows, install separately (e.g. via MSYS2 or WSL).
make
Compiles the code based on the generated build configuration to produce an executable.
If it finishes without errors, it worked. Try running the generated executable directly.
Pulled directly from this repo's README.
// repository documentation
Was this content helpful?
(0 ratings)