mentor-kd

KO | EN

gitlite — search

#python #hacktoberfest #ai #typescript #deep-learning #linux #windows #cli #kubernetes #vue #awesome-list #pytorch #mcp #openai

(★ 10)

Official repository for "Mentor-KD: Making Small Language Models Better Multi-step Reasoners", EMNLP 2024

mentor-kd Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Mentor-KD

### Mentor-KD: Making Small Language Models Better Multi-step Reasoners
Hojae Lee*, Junho Kim*, SangKeun Lee (*: equal contribution)

This repository contains the code and datasets used for our study ["Mentor-KD: Making Small Language Models Better Multi-step Reasoners"](https://aclanthology.org/2024.emnlp-main.977/) accepted to EMNLP 2024 (main). In this paper, we improve reasoning distillation by leveraging task-specific intermediate-sized mentor models which complements insufficient distillation signals from the teacher model.

## Overview
![overview](https://raw.githubusercontent.com/2hojae/mentor-kd/master/assets/main_figure.png)

## Requirements
### Required Tools
- Python >= 3.10
- torch >= 2.0.0
- transformers >= 4.41.2
- peft >= 0.4.0

### Data
All data used in our study can be found in the `data/` directory. As mentioned in Appendix A, we use CoT annotations as well as train-test dataset splits previously collected by MCC-KD[^mcckd], specifically for CommonsenseQA, GSM8K, ASDiv, SVAMP. For other datasets, we newly prompt `gpt-3.5-turbo-instruct` and collect CoT annotations, while following the train-test dataset splits from Fine-tune-CoT[^ftcot].

Please refer to each studies' GitHub repository for the original data: [[MCC-KD]](https://github.com/homzer/MCC-KD) [[Fine-tune-CoT]](https://github.com/itsnamgyu/reasoning-teacher)

## Reproduction
We provide shell scripts to reproduce the results reported in our paper as below.

A task-specific intermediate-sized mentor model must be trained before conducting reasoning distillation to the student. We provide shell scripts for training the mentor model, using it for means of rationale augmentation, and finally for training the student models via our Mentor-KD framework.

### Training the Mentor Model
```
bash shells/fine_tune_cot/run_fine_tune_cot.sh
```

### Rationale Augmentation via the Mentor Model
```
bash shells/data/augment_train_data.sh
```

### Reasoning Distillation (Training the Student Model)
```
bash shells/kd/run_mentor_kd.sh
```

## Citation
```
@inproceedings{lee-etal-2024-mentor,
    title = "Mentor-{KD}: Making Small Language Models Better Multi-step Reasoners",
    author = "Lee, Hojae  and
      Kim, Junho  and
      Lee, SangKeun",
    editor = "Al-Onaizan, Yaser  and
      Bansal, Mohit  and
      Chen, Yun-Nung",
    booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.emnlp-main.977",
    pages = "17643--17658",
    abstract = "Large Language Models (LLMs) have displayed remarkable performances across various complex tasks by leveraging Chain-of-Thought (CoT) prompting. Recently, studies have proposed a Knowledge Distillation (KD) approach, reasoning distillation, which transfers such reasoning ability of LLMs through fine-tuning language models of multi-step rationales generated by LLM teachers. However, they have inadequately considered two challenges regarding insufficient distillation sets from the LLM teacher model, in terms of 1) data quality and 2) soft label provision. In this paper, we propose Mentor-KD, which effectively distills the multi-step reasoning capability of LLMs to smaller LMs while addressing the aforementioned challenges. Specifically, we exploit a mentor, intermediate-sized task-specific fine-tuned model, to augment additional CoT annotations and provide soft labels for the student model during reasoning distillation. We conduct extensive experiments and confirm Mentor-KD{'}s effectiveness across various models and complex reasoning tasks.",
}
```

## Acknowledgements
We thank the authors of MCC-KD[^mcckd] and Fine-tune-CoT[^ftcot] for open-sourcing the data used for their respective works.

## Point of Contact
For personal communication related to Mentor-KD, please contact Hojae Lee via <22leehojae@korea.ac.kr>.

[^mcckd]: Chen et al., "MCC-KD: Multi-CoT Consistent Knowledge Distillation", Findings of EMNLP 2023.
[^ftcot]: Ho et al., "Large Language Models are Reasoning Teachers", ACL 2023.

mentor-kd

mentor-kd Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

mentor-kd

mentor-kd Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README