KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #typescript #dsh #dsh-plugin #ai-agent #codex #ai-agents #deepseek-harness #claude-code #cli #react #mcp

daVinci-Dev

★ 73 Open GitHub ↗

[ICML 2026 Oral] Agent-native Mid-training for Software Engineering

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

[![ICML 2026](https://img.shields.io/badge/ICML%202026-Oral-8A2BE2.svg)](https://icml.cc/virtual/2026/oral/71032)
[![arXiv](https://img.shields.io/badge/arXiv-2601.18418-b31b1b.svg)](https://arxiv.org/pdf/2601.18418)
[![GitHub](https://img.shields.io/badge/GitHub-Repository-green)](https://github.com/GAIR-NLP/daVinci-Dev)
[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Dataset-blue)](https://huggingface.co/datasets/GAIR/daVinci-Dev)
[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue)](https://huggingface.co/GAIR/daVinci-Dev-72B)

</div>

<h1 align="center">daVinci-Dev: Agent-native Mid-training for Software Engineering</h1>

## Table of Contents

- [News](#news)
- [Overview](#overview)
- [Model Zoo](#model-zoo)
- [Datasets](#datasets)
- [Pipeline](#pipeline)
- [License](#license)
- [Citation](#citation)

## News

- **2026-05:** 🎉 daVinci-Dev was accepted as an **oral presentation** at ICML 2026.
- **2026-01:** daVinci-Dev paper, code and dataset were released!

## Overview

`daVinci-Dev` is a family of large language models trained for **agentic software engineering**.

This repo provides:

- The paper PDF: [`daVinci-Dev.pdf`](./daVinci-Dev.pdf)
- A high-performance data processing pipeline under [`pipeline/`](pipeline:1) that calls the GitHub API to construct **contextually-native PR trajectories** $\mathcal{D}^{\text{ctx}}_{\text{py}}$

## Model Zoo

We will open-source model checkpoints on Hugging Face:

| Model | Description | Link |
|------|-------------|------|
| `daVinci-Dev-72B` | Final model (agent-native mid-training + env native SFT) | https://huggingface.co/GAIR/daVinci-Dev-72B |
| `daVinci-Dev-32B` | Final model (agent-native mid-training + env native SFT) | https://huggingface.co/GAIR/daVinci-Dev-32B |
| `daVinci-Dev-72B-MT` | **MT checkpoint** (after agent-native mid-training, **before SFT**) | https://huggingface.co/GAIR/daVinci-Dev-72B-MT |
| `daVinci-Dev-32B-MT` | **MT checkpoint** (after agent-native mid-training, **before SFT**) | https://huggingface.co/GAIR/daVinci-Dev-32B-MT |

## Datasets

Datasets are released through Hugging Face:

| Dataset | Description | Link |
|---------|-------------|------|
| `daVinci-Dev` | Agent-native trajectories used in our training recipe (as permitted) | https://huggingface.co/datasets/GAIR/daVinci-Dev |

High-level composition (see the paper for details):

- **Contextually-native trajectories** $\mathcal{D}^{\text{ctx}}_{\text{py}}$ (PR-derived, Python variant)
- **Environmentally-native trajectories** $\mathcal{D}^{\text{env}}_{\text{pass}}$ (executable rollouts, test-passing subset)

## Pipeline

The directory [`pipeline/`](pipeline:1) contains a high-performance pipeline that calls the GitHub API and constructs the structured PR representation used to build $\mathcal{D}^{\text{ctx}}_{\text{py}}$.

| Pipeline | Description | Link |
|----------|---------|-------------|
| daVinci-Dev Pipeline | a high-performance pipeline used to build $\mathcal{D}^{\text{ctx}}_{\text{py}}$ | [`pipeline/`](pipeline:1) |

## Utilities for Environmentally-native Trajectories

The directory [`env_traj_utils/`](env_traj_utils/) provides utilities for converting environmentally-native trajectories ($\mathcal{D}^{\text{env}}$) to LLM-trainable formats:

| Script | Description |
|--------|-------------|
| [`convert_trajectories.py`](env_traj_utils/convert_trajectories.py) | Convert SWE-agent trajectories to XML function calling format |
| [`tokenize_trajectories.py`](env_traj_utils/tokenize_trajectories.py) | Tokenize trajectories and filter by length |

See the [env_traj_utils README](env_traj_utils/README.md) for quickstart instructions on converting [`env-native.jsonl`](https://huggingface.co/datasets/GAIR/daVinci-Dev/blob/main/env-native.jsonl) to formats compatible with training frameworks like [SLIME](https://github.com/THUDM/slime).

## License

This project is a **mixed** release:

- **PR-derived subset:** only includes PRs from repositories detected as having a **permissive license** in the open-source release.
- **Executable rollout subset:** derived from [SWE-rebench](https://huggingface.co/datasets/nebius/SWE-rebench), licensed under **CC-BY-4.0**.
- **daVinci-Dev models:** released under [Qwen](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE) license. Users should verify the licensing status of any generated code before using it in production.
- **daVinci-Dev pipeline:** released under the [Apache-2.0](https://github.com/GAIR-NLP/daVinci-Dev/blob/main/LICENSE) license.

Downstream users are responsible for ensuring their usage complies with the licenses of the underlying sources.

## Citation

If you use this work, please cite the daVinci-Dev paper.

```
@misc{zeng2026davincidevagentnativemidtrainingsoftware,
      title={daVinci-Dev: Agent-native Mid-training for Software Engineering},
      author={Ji Zeng and Dayuan Fu and Tiantian Mi and Yumin Zhuang and Yaxing Huang and Xuefeng Li and Lyumanshan Ye and Muhang Xie and Qishuo Hua and Zhen Huang and Mohan Jiang and Hanning Wang and Jifan Lin and Yang Xiao and Jie Sun and Yunze Wu and Pengfei Liu},
      year={2026},
      eprint={2601.18418},
      archivePrefix={arXiv},
      primaryClass={cs.SE},
      url={https://arxiv.org/abs/2601.18418},
}
```

daVinci-Dev

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README