RankSurprisalRatio

KO | EN

gitlite — search

#hacktoberfest #react #ai #typescript #go #golang #android #machine-learning #docker #windows #cli #kubernetes #awesome-list #openai

RankSurprisalRatio

(★ 16)

[ACL 2026 Main] Official Repo for Paper "Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment“

RankSurprisalRatio 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h2 align="center"> Rank-Surprisal Ratio (RSR) </h2>
 From the ACL 2026 paper "Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment"

<img src="/assets/rsr_def.png" alt="RSR definition." width="350"/>
 
 
 RSR measures the suitability of a teacher trajectory (training sample) for a given student model. Rank values are clipped; see Eq. 7 in the paper for the formal definition.

## 📋 Overview

In this work, we investigate data–student suitability in reasoning distillation and introduce **Rank-Surprisal Ratio** (RSR), a simple yet effective metric for **identifying suitable reasoning trajectories for a given student**.

RSR is defined as the ratio of a trajectory’s average token-wise rank to its average negative log-likelihood (surprisal), and **is straightforward to compute and interpret**. Although our work focuses solely on reasoning tasks, RSR is not specifically designed for reasoning trajectories (CoTs) and can be **applied to general text data**.

- 📖 **Paper**: [Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment](https://arxiv.org/abs/2601.14249)
  - Accepted to **ACL 2026** (Main Conference). 🎉
- 🛠️ **Code**: Available in this repository, providing **a clean and efficient implementation for computing the RSR metric**.
  - Update (2026-03-13): Added support for multi-round chat messages, agent data, and multimodal inputs!
- 📀 **Data**: Available on [Hugging Face](https://huggingface.co/datasets/Umean/RSR_data). We release **33 math reasoning trajectory datasets** used in our experiments, generated by 11 different teacher models. We also release RSR-selected datasets for 5 student models. These datasets can be selectively used for reasoning distillation.

## 💡 Why Use RSR?

<img src="/assets/rsr_informal_example.png" alt="A hypothetical example to help illustrate the basic idea of RSR." width="550"/>
 
 
 Figure 1: A hypothetical example showing the motivation behind RSR. Both overly informative and overly familiar trajectories offer limited learning value.

RSR is motivated by the observation that effective reasoning distillation and SFT require trajectories that strike a **balance between informativeness and familiarity**, as illustrated in Figure 1. Existing assessment methods, such as probability-based metrics, cannot adequately capture this fine-grained balance.

By jointly modeling informativeness and behavioral alignment, RSR favors trajectories with low absolute probabilities but relatively high-ranked tokens (Figure 2), enabling the selection of training data that is both informative and learnable for the student.

<img src="/assets/rsr.png" alt="llustration of the intuition behind RSR." width="450"/>
 
 
 Figure 2: Illustration of the intuition underlying RSR's metric design. Suitable reasoning trajectories should balance informativeness and alignment, having low absolute probability while their tokens remain relatively high-ranked under the student model.

Across five student models and reasoning trajectories from 11 diverse teachers, RSR strongly correlates with post-training reasoning performance (**average Spearman 0.86**), consistently outperforming existing metrics. We further demonstrate its practical utility in both **trajectory selection** and **teacher selection**.

For more information and a detailed introduction to RSR, please refer to our paper.

## 🚀 Quick Start
Our codebase supports the computation of our lightweight suitability metric, the **Rank-Surprisal Ratio** (RSR), given teacher trajectories and student models. The code has been cleaned up from our original implementation and streamlined to focus on RSR computation.

- `rsr_launch.py` serves as the **entry-point** script and batch runner, calling `rsr_cal.py` with your customized configuration. See inline comments for details. By default, the script processes data in the standard Hugging Face chat message format (now support for multimodal messages). You may refer to our provided datasets on Hugging Face for examples of the expected data format.
- `rsr_cal.py` implements the core computation logic. The code runs a forward pass with the provided model and data, then computes the RSR metric. It includes placeholders and explanatory comments to facilitate easy customization.
- `rsr_utils.py` provides utility functions.

To start the RSR computation, simply configure the global variables in `rsr_launch.py` according to your experimental setup and then execute the script:
```
python rsr_launch.py
```
In our experiments, computing RSR over the 5,000-trajectory dataset with a context length of 32,768 using a 7B model typically takes under one hour on a single H200 GPU with FlashAttention enabled. For further computational details, see Appendix C.1 of our paper.
 
The computation results, including dataset-level and sample-level metrics for the given data, will be saved under the specified output root directory. Sample-level results are stored in the `_sample_metrics` folder.

## ⚙️ Environment Setup

The current implementation has minimal dependencies, relying primarily on `torch` and `transformers`. While not strictly required, we recommend using one of the following Python environment configurations:

* torch==2.7.0 (2.6.0)
* transformers==4.57.3 (4.53.3)
* flash-attn==2.8.3 (2.7.4, optional but significantly boosts speed)
* Python 3.12.12 (3.10.12)
* CUDA 12.8 (12.4)

See `requirements.txt` for the complete list of Python package dependencies. You can install them using `pip install -r requirements.txt`. We recommend installing `flash-attn` via pre-built wheels corresponding to your specific Python, CUDA, and PyTorch environment.

## 📝 Citation

If you find our work helpful, please consider citing our paper:

```bibtex
@inproceedings{yang-etal-2026-reasoning,
    title = "Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment",
    author = "Yang, Yuming  and
      Lai, Mingyoung  and
      Zhao, Wanxu  and
      Fan, Xiaoran  and
      Xi, Zhiheng  and
      Wu, Mingqi  and
      Huang, Chiyue  and
      Zhao, Jun  and
      Lv, Haijun  and
      Tong, Jian  and
      Zhou, Yunhua  and
      Zou, Yicheng  and
      Guo, Qipeng  and
      Gui, Tao  and
      Zhang, Qi  and
      Huang, Xuanjing",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.acl-long.1950/",
    pages = "42123--42150",
    ISBN = "979-8-89176-390-6"
}
```

RankSurprisalRatio

RankSurprisalRatio 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

RankSurprisalRatio

RankSurprisalRatio 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README