Fine-Tune-VLMs

KO | EN

gitlite — search

#python #hacktoberfest #react #ai #typescript #llm #linux #docker #nodejs #windows #kubernetes #vue #pytorch #open-source

(★ 26)

No description available.

Fine-Tune-VLMs Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Instruct-Tune TinyLlama

<div align="center">
    <a href="">
        <img alt="open-source-image"
		src="https://img.shields.io/badge/Open%20Source%20❤%EF%B8%8F-%2325A162.svg?style=flat"
        style="height: 30px"/>
    </a>
    <a href="https://youtu.be/6XeTk8cZUsM">
        <img alt="youtube-tutorial"
        src="https://img.shields.io/badge/YouTube Tutorial-%23FF0000.svg?logo=youtube&logoColor=white&style=flat"
        style="height: 30px"/>
    </a>
</div>
<div align="center">
    <a href="https://ko-fi.com/uygarkurt" target="_blank">
        <img src="https://storage.ko-fi.com/cdn/brandasset/v2/support_me_on_kofi_blue.png?_gl=1*gypxqi*_gcl_aw*R0NMLjE3MzU5Mzk3MjAuQ2owS0NRaUFzdDY3QmhDRUFSSXNBS0tkV09raHpzemtTUGxadnNOamVpN0FBRDhOdVloTVhuRHJrampCRGhac3FyT2pCY29iRWV0TmNKY2FBbWxiRUFMd193Y0I.*_gcl_au*MjUwODAwNDk1LjE3MzU5NDExMjA.*_ga*MTM3MTUxODgyMS4xNzM1OTQwODYy*_ga_M13FZ7VQ2C*MTczNTk0NTM3MC4yLjEuMTczNTk0NTk4OS40MS4wLjA." alt="Buy Me A Coffee" 
        style="height: 38px"/>
    </a>
</div>
<br/>
<div align="center">
    <p>Liked our work? give us a ⭐!</p>
</div>
<p align="center">
  <img src="./assets/sample.png" height="70%" width="70%"/>
</p>

This repository contains an easy-to-use and understand code to fine-tune VLMs (Visual Language Models).

With Vision-Language Models (VLMs), you can ask questions about an image and receive answers. We'll do it on images with visual data representations. Such as graphs and charts. We'll use [HuggingFaceM4/ChartQA](https://huggingface.co/datasets/HuggingFaceM4/ChartQA) as the dataset.

In this case, we'll be fine-tuning [Qwen/Qwen2-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct). We will be using LoRA adapter and 4-bit quantization. Refer to the `fine-tune-vlms-qwen.ipynb` file.

## YouTube Tutorial
<div align="center">
    <a href="https://www.youtube.com/watch?v=3ypHZayanBI&feature=youtu.be">Fine-Tune Visual Language Models (VLMs) - HuggingFace, PyTorch, LoRA, Quantization, TRL</a>
    <br>
    <br>
    <a href="https://www.youtube.com/watch?v=3ypHZayanBI&feature=youtu.be">
        <img src="./assets/TN-GIT.png" height="85%" width="85%%"/>
    </a>
</div>

### References
- https://huggingface.co/learn/cookbook/en/fine_tuning_vlm_trl
- https://huggingface.co/docs/trl/en/sft_trainer
- https://huggingface.co/docs/transformers/main/en/tasks/visual_question_answering
- https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct

Fine-Tune-VLMs

Fine-Tune-VLMs Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Fine-Tune-VLMs

Fine-Tune-VLMs Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README