Llama3.2_python_dataset

KO | EN

gitlite — search

#javascript #go #android #machine-learning #rust #nodejs #awesome #cli #kubernetes #awesome-list #pytorch #security #ios #open-source

Llama3.2_python_dataset

(★ 12)

Finetune Llama 3.2 1B model on the python code dataset

Llama3.2_python_dataset Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## Fine-tuning LLAMA 3.2-1B on Python Dataset

This repository demonstrates the process of fine-tuning **LLAMA 3.2 1B** on a Python instruction dataset from hugging face https://huggingface.co/datasets/iamtarun/python_code_instructions_18k_alpaca. The goal is to enhance the model's capability in generating and understanding Python code.

### Training Details
- Training Framework: The training uses the SFTTrainer from the trl (Transformer Reinforcement Learning) library.
- Parameter Optimization: QLoRA (Low-Rank Adaptation) is applied to reduce the number of parameters and improve efficiency during the fine-tuning process.

### Evaluation
Run eval_ollama_8B.ipynb to score the model's performance.

### Interactive API with chainlit
Interact with the fine-tuned model through a web API by running the command **chainlit run app.py**. This will launch an interactive interface for the model.

### Example
Below is an example of the model solving a leetcode question

![Alt text](https://raw.githubusercontent.com/dtdo90/Llama3.2_python_dataset/main/experiments/leetcode.png)

Llama3.2_python_dataset

Llama3.2_python_dataset Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Llama3.2_python_dataset

Llama3.2_python_dataset Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README