SWE-Fixer

KO | EN

gitlite — search

#python #hacktoberfest #llm #golang #android #machine-learning #linux #nodejs #awesome #cli #kubernetes #vue #pytorch #ios

(★ 139)

No description available.

파일 탐색기

최종 버전 다운로드 (.zip)

파일 수가 많아 일부만 표시됩니다. 전체 파일은 위 다운로드 버튼으로 확인해 주세요.

- - teaser.jpg
- bm25_eval.py
- code_edit.py
- pipeline.py
- prompt.py
- - report.json
- lite_no_p2p_result.jsonl
- lite_with_p2p_filtering_result.jsonl
.gitignore
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center"> SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution </h1>

<a href="https://yitianlian.github.io/" style="text-decoration: none;">Chengxing Xie*,1,2</a>, 
 <a href="https://scholar.google.com/citations?user=RLWXNf8AAAAJ&hl=en" style="text-decoration: none;">Bowen Li*,1</a>,
 <a href="https://gao-xiao-bai.github.io/" style="text-decoration: none;">Chang Gao*,1,3</a>,
 <a href="https://kinza99.github.io/" style="text-decoration: none;">He Du1</a>,
 <a href="https://www.se.cuhk.edu.hk/people/academic-staff/prof-lam-wai/" style="text-decoration: none;">Wai Lam3</a>,
 <a href="https://difanzou.github.io/" style="text-decoration: none;">Difan Zou4</a>,
 <a href="https://chenkai.site/" style="text-decoration: none;">Kai Chen^,1</a>

1Shanghai AI Laboratory, 2Xidian University, 3The Chinese University of Hong Kong, 4The University of Hong Kong 
 *Equal contribution, ^Corresponding author

<a href="https://arxiv.org/abs/2501.05040" style="text-decoration: none;">
 <span style="background-color: #333; color: white; padding: 10px 20px; border-radius: 20px; margin: 8px; font-size: 14px; font-weight: bold; display: inline-block; box-shadow: 2px 2px 6px rgba(0, 0, 0, 0.2);">
 📄 Paper
 
 </a>

SWE-Fixer is a simple yet effective solution for addressing real-world GitHub issues by training open-source LLMs. It features a streamlined retrieve-then-edit pipeline with two core components:  
🔍 **A Code File Retriever** and ✏️ **A Code Editor**.

For implementation, we finetune **Qwen2.5-7b** and **Qwen2.5-72b** for the retriever and the editor respectively, leveraging a curated dataset of **110K instances**. SWE-Fixer achieves **state-of-the-art performance** among open-source solutions with open-source models.

| **Method** | **Model** | **Type** | **Verified** | **Lite** | **#Model Calls** |
|------------|------------|------------|------------|------------|------------|
| AutoCodeRover [[Liu et al., 2024]](https://arxiv.org/pdf/2408.03910) | Qwen2-72B-Instruct | Agent | - | 9.3 | 4† |
| SWE-Gym (Best@1) [[Pan et al., 2024]](https://arxiv.org/abs/2412.21139) | SWE-Gym-32B | Agent | 20.6 | 15.3 | 29.3‡ |
| SWE-SynInfer [[Ma et al., 2024]](https://arxiv.org/pdf/2411.00622) | Lingma-SWE-GPT-72B | Agent | 30.2 | 22.0 | 6† |
| SWE-Search [[Antoniades et al., 2024]](https://arxiv.org/pdf/2410.20285) | Qwen2.5-72b-Instruct | Agent | - | 24.7 | 200* |
| SWE-Gym (Best@8 w/ Verifier) [[Pan et al., 2024]](https://arxiv.org/abs/2412.21139) | SWE-Gym-32B | Agent | 29.8 | **26.0** | 29.3 |
| **SWE-Fixer** | SWE-Fixer-72B | Pipeline | 30.2 | 22.0 | 2† |
| **SWE-Fixer + P2P Filtering** | SWE-Fixer-72B | Pipeline | **32.8** | 24.7 | 2† |

†: The minimum number of model calls needed. 
‡: The average number of model calls needed. 
*: See Appendix in our paper for detailed calculation.

# Models and Datasets

Models:

- **[🤗 SWE-Fixer-Retriever-7B](https://huggingface.co/internlm/SWE-Fixer-Retriever-7B)**  
  🔍 Finetuned for the **code file retrieval task**, this model takes issue descriptions and BM25-retrieved results as input and identifies the defective files related to the issue.

- **[🤗 SWE-Fixer-Editor-72B](https://huggingface.co/internlm/SWE-Fixer-Editor-72B)**  
  ✏️ Designed for the **code editing task**, this model processes issue descriptions and corresponding file content to generate modification patches for resolving the issue.

Datasets:
- **[🤗 SWE-Fixer-Train-110K](https://huggingface.co/datasets/internlm/SWE-Fixer-Train-110K)**  
  📂 This dataset contains nearly **110K detailed instances** collected from real-world GitHub repositories, forming the training data of our model training pipeline.

- **[🤗 SWE-Fixer-Eval](https://huggingface.co/datasets/internlm/SWE-Fixer-Eval)**  
  📊 This evaluation dataset includes **SWE-Bench Lite and Verified instance**, BM25-retrieval results for SWE-Bench Lite and Verified, and code structure for each instance, enabling convenient evaluation.

# Run the Pipeline

## Environment Setup

Download and install our inference environment package [SWE_Fixer.tar.gz](https://drive.google.com/file/d/1slyfDUEA5lrfr4IO79xezjyZqLIKPIvh/view?usp=sharing). Use the following commands:
```bash
mkdir {your_conda_environment_dir/SWE_Fixer}
tar -xzvf SWE_Fixer.tar.gz -C {your_conda_environment_dir/SWE_Fixer}
```
Activate the environment:
```bash
conda activate SWE_Fixer
```

## Prepare Models and Evaluation Datasets

Download the models and datasets and save them to the default locations:
```bash
mkdir model
huggingface-cli login
huggingface-cli download --resume-download internlm/SWE-Fixer-Retriever-7B --local-dir ./model/retrieval_model
huggingface-cli download --resume-download internlm/SWE-Fixer-Editor-72B --local-dir ./model/editing_model
huggingface-cli download internlm/SWE-Fixer-Eval --repo-type dataset --local-dir ./eval_data
```
Alternatively, specify paths in the scripts by modifying `MODEL_DIR` in `scripts/run_evaluation.sh` and `EVAL_DATA_DIR` in `scripts/run_evaluation.sh`.

## Run the Retrieval Model

Run the retrieval pipeline (default to the `lite` dataset):
```bash
scripts/run_evaluation.sh --mode retrieval
```
To use the `verified` dataset, execute:
```bash
scripts/run_evaluation.sh --mode retrieval --dataset verified
```
Retrieval results will be saved in the `result` directory.

## Run the Editing Model

After completing the retrieval step, run the editing pipeline based on the retrieval results:
```bash
scripts/run_evaluation.sh --mode editing
```
To use the `verified` dataset, execute:
```bash
scripts/run_evaluation.sh --mode editing --dataset verified
```
Editing results will also be saved in the `result` directory.

## Evaluate the Results

We evaluate the pipeline results using the **all-hands evaluation approach**. Refer to the evaluation guide here: [Evaluation Guide Link](https://github.com/All-Hands-AI/OpenHands/tree/main/evaluation/benchmarks/swe_bench#swe-bench-evaluation-with-openhands-swe-bench-docker-image).

## Notes

- Ensure all scripts are executable. Use `chmod +x` if necessary.
- The exact paths for scripts and datasets must be updated to match your local setup.
- If you encounter issues during deployment or execution, refer to the respective repositories and documentation.
- The inference results may vary depending on your device or settings.

# Citation
```
@article{xie2025swefixer,
  title={SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution}, 
  author={Xie, Chengxing and Li, Bowen and Gao, Chang and Du, He and Lam, Wai and Zou, Difan and Chen, Kai},
  journal={arXiv preprint arXiv:2501.05040},
  year={2025}
}
```

# Acknowledgements
- [Agentless](https://github.com/OpenAutoCoder/Agentless)
- [OpenHands](https://github.com/All-Hands-AI/OpenHands)
- [SWE-Bench](https://www.swebench.com/)

SWE-Fixer

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

SWE-Fixer

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README