Finetune-Qwen2.5-VL - GitLite

KO | EN

gitlite — search

#javascript #python #react #golang #go #linux #macos #java #windows #awesome #cli #vue #pytorch #openai

Finetune-Qwen2.5-VL

(★ 170)

Fine-tuning Qwen2.5-VL for vision-language tasks | Optimized for Vision understanding | LoRA & PEFT support.

파일 탐색기

최종 버전 다운로드 (.zip)

- accelerate.yaml
- vlm_config.yaml
- vlm_inference_config.yaml
- vlm_merge_adapter_config.yaml
- dataset_info.json
- evaluation.py
- - __init__.py
- __init__.py
- inference.py
- merge_model.py
- train.py
- version.py
- __init__.py
- test_data_handler.py
- __init__.py
- constants.py
- logger.py
- misc.py
- packages.py
- ploting.py
.dockerignore
.gitignore
.pre-commit-config.yaml
__init__.py
Dockerfile
LICENSE
main.py
README.md
requirements.txt
setup.py

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/sandy1990418/Finetune-Qwen2.5-VL

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd Finetune-Qwen2.5-VL

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Docker

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Docker Desktop 컨테이너를 빌드하고 실행하려면 필요합니다. 설치 후 실행해서 백그라운드에 켜두세요.

docker build --no-cache -t vlm_finetune:latest .

Dockerfile을 기반으로 실행 가능한 이미지를 빌드합니다.

docker run -it --name CONATINER_NAME -v LOCAL_PATH:/VLM vlm_finetune:latest

빌드된 이미지를 실제 컨테이너로 실행합니다.

✅ 터미널에 docker compose ps 를 입력해 컨테이너들이 Up 상태인지 확인하세요. README에 포트 번호가 적혀있다면 브라우저에서 http://localhost:포트번호 로 접속해보세요.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

3. Python

쉬움

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install -r requirements.txt

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Finetune-Qwen2.5-VL

A comprehensive toolkit for finetuning the Qwen2.5-VL (Visual Language) model using LoRA. This project provides easy-to-use scripts for Supervised Fine-Tuning (SFT), LoRA merging, and inference.

## Requirements

```bash
# Core dependencies
torch>=2.0.0
transformers>=4.37.0
peft>=0.7.0
accelerate>=0.21.0
```

## Installation

```bash
git clone https://github.com/sandy1990418/Finetune-Qwen2.5-VL.git
cd Finetune-Qwen2.5-VL
pip install -r requirements.txt
```

## Usage

### 1.1 Supervised Fine-Tuning (SFT) - Single GPU

Run the following command to start the fine-tuning process:

```bash
python src/train.py config/vlm_config.yaml

or

python main.py config/vlm_config.yaml
```

The `vlm_config.yaml` should contain your training configurations such as:
- Model parameters
- Training hyperparameters
- Dataset configurations
- LoRA settings

Check the configurations `distributed_type: "NO"` in `accelerate.yaml`.

### 1.2 Supervised Fine-Tuning (SFT) - Multiple GPU

Run the following command to start the fine-tuning process:

```bash
python main.py config/vlm_config.yaml config/accelerate.yaml
```

The `vlm_config.yaml` should contain your training configurations and  `accelerate.yaml` should contain `accelerate` configurations. If you want to use multiple GPUs, set `use_accelerate: true` in `vlm_config.yaml` and `distributed_type: "MULTI_GPU", gpu_ids: all` in `accelerate.yaml`

A better approach would be to use Ray, but I am currently facing some issues that have yet to be resolved. I plan to work on this further in the future.

### 2. Merge LoRA Weights

After training, merge the LoRA weights with the base model:

```bash
python src/merge_model.py config/vlm_merge_adapter_config.yaml
```

### 3. Inference

Run inference with your fine-tuned model:

```bash
python src/inference.py config/vlm_inference_config.yaml
```

### 4. Evaluation
```bash
python evaluation/evaluation.py config/vlm_inference_config.yaml
```

### 5. Dockerfile
```bash
docker build --no-cache -t vlm_finetune:latest .
docker run -it --name CONATINER_NAME -v LOCAL_PATH:/VLM vlm_finetune:latest
```

## TODO
- [ ] Resolve issues with Ray for multi-GPU training
- [ ] Implement evaluation pipeline for fine-tuned models  
- [ ] Add test cases for training, merging, and inference  
- [ ] Load Data may be more flexible
- [ ] Load Data support Image_url in Trainig stage
- [ ] Finetune JSON Dataset

## Contributing

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

## Acknowledge
This project is based on [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory/tree/main). Special thanks to the original authors and contributors!