LP_Bench

KO | EN

gitlite — search

#javascript #python #ai #typescript #llm #rust #deep-learning #linux #docker #macos #windows #cli #vue #openai

(★ 18)

LP_Bench

LP_Bench Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Predicting LLM Output Length via Entropy-Guided Representations

This repository contains the code for the paper **"Predicting LLM Output Length via Entropy-Guided Representations"** (ICLR 2026).

> The long-tailed distribution of sequence lengths in LLM serving and reinforcement learning (RL) sampling causes significant computational waste due to excessive padding in batched inference. We introduce a lightweight framework that reuses the main model's internal hidden states for efficient length prediction, featuring **Entropy-Guided Token Pooling (EGTP)** for static prediction and **Progressive Length Prediction (PLP)** for dynamic estimation. We also release **ForeLen**, the first comprehensive benchmark for length prediction.

## Installation

- **Python >=** 3.9
- **PyTorch >=** 2.4 
- **Dependencies:**

```bash
pip install -r requirements.txt
```

## Usage

```bash
bash EGTP/scripts/run_egtp.sh 
```

## Citation

```bibtex
@inproceedings{xie2026predicting,
  title={Predicting LLM Output Length via Entropy-Guided Representations},
  author={Xie, Huanyi and Chen, Yubin and Wang, Liangyu and Hu, Lijie and Wang, Di},
  booktitle={International Conference on Learning Representations},
  year={2026}
}
```

LP_Bench

LP_Bench Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

LP_Bench

LP_Bench Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README