KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #developer-tools #ai-agent #claude-code #codex #typescript #ai-agents #deepseek-harness #mcp #cli #rust

EditReward

★ 158 Open GitHub ↗

EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing [ICLR 2026]

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ✨[ICLR 2026] EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing
 
[![Project Website](https://img.shields.io/badge/🌐-Project%20Website-deepgray)](https://tiger-ai-lab.github.io/EditReward/)
[![arXiv](https://img.shields.io/badge/arXiv-2509.26346-b31b1b.svg)](https://arxiv.org/abs/2509.26346)
[![Model](https://img.shields.io/badge/🤗-Model-yellow)](https://huggingface.co/collections/TIGER-Lab/editreward-68ddf026ef9eb1510458abc6)
[![Dataset](https://img.shields.io/badge/🤗-Dataset-green)](https://huggingface.co/datasets/TIGER-Lab/EditReward-Data)
[![Benchmark](https://img.shields.io/badge/📊-Benchmark-yello)](https://huggingface.co/datasets/TIGER-Lab/EditReward-Bench)
</div>

We acknowledge the data contribution and support from 
 <a href="https://www.abaka.ai/" target="_blank">
 <img src="./assets/logo_abaka.png"
 alt="Abaka AI"
 style="height:2.1em; vertical-align:text-bottom; position:relative; top:7px;"
 </a>

## 📖 Introduction

This is the official implementation for the paper: [EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing](https://arxiv.org/abs/2509.26346).
In this paper, we introduce **EditReward**, a human-aligned reward model powered by a high-quality dataset for instruction-guided image editing. We first construct **EditReward-Data**, a large-scale, high-fidelity preference dataset for instruction-guided image editing. It comprises over 200K manually annotated preference pairs, covering a diverse range of edits produced by seven state-of-the-art models across twelve distinct sources. Every preference annotation in **EditReward-Data** was curated by trained annotators following a rigorous and standardized protocol, ensuring high alignment with considered human judgment and minimizing label noise. Using this dataset, we train the reward model **EditReward** to score instruction-guided image edits. To rigorously assess **EditReward** and future models, we also introduce **EditReward-Bench** a new benchmark built upon our high-quality annotations, which includes more difficult multi-way preference prediction.

## 📰 News
- **[2026-02-06]** 🔥 We started maintaining a list of <a href="#-awesome-works-using-editreward">Awesome Works</a> using EditReward!
- **[2026-01-27]** 🔥 Add training & inference support for **Qwen3-VL Series**!
- **[2026-01-26]** 🔥 Our paper has been accepted by **ICLR 2026**!
- **[2025-10-29]** 🔥 Release the training guideline of EditReward, see [Training Insctruction](EditReward/TRAIN_README.md)!
- **[2025-10-14]** 🔥 Release the evaluation code and guideline of EditReward-Bench, see [Evaluate Insctruction](EditReward/evaluate/README.md)!
- **[2025-10-10]** 🔥 Release our evaluation benchmark EditReward-Bench, Welcome to use!
- **[2025-10-08]** 🔥 Release our training dataset EditReward-Data, Welcome to use!
- **[2025-10-03]** 🔥 Release inference code and pretrained model.
- **[2025-10-01]** 🎉 We initialize the official repo of EditReward.

## 🚧 TODO List
- [x] Release inference code and pretrained model
- [x] Release evaluation benchmark
- [x] Release training code
- [x] Release training dataset

## 📄 Table of Contents
- [🛠️ Installation](#-installation)
- [👨‍🏫 Get Started](#-get-started)
- [🏋️ Training](#-training)
- [📊 Benchmark](#-benchmark)
- [🖊️ Citation](#-citation)
- [🤝 Acknowledgement](#-acknowledgement)
- [✨ Awesome Works using EditReward](#-awesome-works-using-editreward)
- [🎫 License](#-license)
---

## 🚀 Quick Start

EditReward is a VLM-based reward model trained on EditReward-Data that demonstrates superior alignment with human preferences.

### 💻 Installation

```bash
# Install locally for development or training.
git clone https://github.com/TIGER-AI-Lab/EditReward.git
cd EditReward

conda create -n edit_reward python=3.10 -y
conda activate edit_reward

# Install PyTorch first (CUDA 12.4 example).
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124

# Install the remaining runtime dependencies.
pip install -U datasets pillow openai megfile sentencepiece deepspeed fire omegaconf \
  matplotlib peft trl==0.8.6 tensorboard scipy transformers==4.57.0 accelerate \
  requests packaging pandas opencv-python einops timm huggingface_hub qwen-vl-utils \
  av regex safetensors tqdm

# Optional but recommended.
pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.2.post1/flash_attn-2.7.2.post1+cu12torch2.5cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

# Install EditReward as an editable package.
pip install -e .
```

### 🚀 Usage

#### Basic Command
```python
import torch
from EditReward import EditRewardInferencer
from EditReward.inference_vl_edit import EditRewardVLInferencer

# ------------------------------------------------------------------------------
# Example script for evaluating edited images with EditReward
# ------------------------------------------------------------------------------

# Package-relative config paths work after `pip install -e .`.
# You can also pass an absolute path to a local YAML config.
CHECKPOINT_PATH = "your/local/path/to/checkpoint"
CONFIG_PATH = "config/EditReward-MiMo-VL-7B-SFT-2508.yaml"

# Initialize reward model
inferencer = EditRewardInferencer(
    config_path=CONFIG_PATH,
    checkpoint_path=CHECKPOINT_PATH,
    device="cuda",        # or "cpu"
    reward_dim="overall_detail",    # choose reward dimension if applicable
    rm_head_type="ranknet_multi_head"
)

# (Optional) Unified inferencer for Qwen2.5-VL / Qwen3-VL:
# Just switch CONFIG_PATH to either:
# - "config/EditReward-Qwen2.5-7B-VL.yaml"
# - "config/EditReward-Qwen3-VL.yaml"
# inferencer = EditRewardVLInferencer(
#     config_path=CONFIG_PATH,
#     checkpoint_path=CHECKPOINT_PATH,
#     device="cuda",
#     reward_dim="overall_detail",
#     rm_head_type="ranknet_multi_head",
# )

# Example input data -----------------------------------------------------------
# image_src = [
#     "../assets/examples/source_img_1.png",
#     "../assets/examples/source_img_1.png",
# ]

# image_paths = [
#     "../assets/examples/target_img_1.png",
#     "../assets/examples/target_img_2.png",
# ]
image_src = [
    "your/local/path/to/source_image_1.jpg",
    "your/local/path/to/source_image_2.jpg",
]

image_paths = [
    "your/local/path/to/edited_image_1.jpg",
    "your/local/path/to/edited_image_2.jpg",
]

# example instruction: "Add a green bowl on the branch"
# prompts = [
#     "Add a green bowl on the branch",
#     "Add a green bowl on the branch"
# ]
prompts = [
    "your first editing instruction",
    "your second editing instruction"
]

# ------------------------------------------------------------------------------
# Main evaluation modes
# ------------------------------------------------------------------------------
if __name__ == "__main__":
    mode = "pairwise_inference"  # or "single_inference"

if mode == "pairwise_inference":
        # ----------------------------------------------------------
        # Pairwise comparison: compares two edited images side-by-side
        # ----------------------------------------------------------
        with torch.no_grad():
          rewards = inferencer.reward(
              prompts=prompts,
              image_src=image_src,
              image_paths=image_paths
          )
        scores = [reward[0].item() for reward in rewards]
        print(f"[Pairwise Inference] Image scores: {scores}")

elif mode == "single_inference":
        # ----------------------------------------------------------
        # Single image scoring: evaluates one edited image at a time
        # ----------------------------------------------------------
        with torch.no_grad():
          rewards = inferencer.reward(
              prompts=[prompts[0]],
              image_src=[image_src[0]],
              image_paths=[image_paths[0]]
          )
        print(f"[Single Inference] Image 1 score: {[reward[0].item() for reward in rewards]}")
        
        with torch.no_grad():
          rewards = inferencer.reward(
              prompts=[prompts[0]],
              image_src=[image_src[0]],
              image_paths=[image_paths[1]]
          )
        print(f"[Single Inference] Image 2 score: {[reward[0].item() for reward in rewards]}")
```

---

## 📁 Dataset

### EditReward-Data

 <img src="assets/dataset_stat.png" alt="dataset" width="900"/>

### Download EditReward

```bash
huggingface-cli download --repo-type dataset TIGER-Lab/EditReward-Data --local-dir /your-local-dataset-path
```

## 🏋️ Training

### 🤖 Model Support

- [x] **Qwen2.5-VL Series** 
- [x] **MiMo-VL Series**
- [x] **Qwen3-VL Series**

### 🚀 Training Command

To train **EditReward** model, follow the detail instruction in [Training Insctruction](EditReward/TRAIN_README.md)

#### Unified training entry (Qwen2.5-VL / Qwen3-VL)

We provide a unified training entry that automatically selects the correct model/collator based on `model_name_or_path`:

```bash
# Qwen2.5-VL
python EditReward/train_qwen_vl_edit.py --config EditReward/config/EditReward-Qwen2.5-7B-VL.yaml

# Qwen3-VL
python EditReward/train_qwen_vl_edit.py --config EditReward/config/EditReward-Qwen3-VL.yaml
```

---

## 📊 Benchmark
To evaluate **EditReward preference accuracy**, follow the detail instruction in [Evaluate Insctruction](EditReward/evaluate/README.md)

<summary> Experimental Results: Alignment with Humans </summary>

| Method | GenAI-Bench | AURORA-Bench | ImagenHub | EditReward-Bench (Overall) |
| :--- | :--- | :--- | :--- | :--- |
| Random | 25.90 | 33.43 | -- | 13.84 |
| Human-to-Human | -- | -- | 41.84 | -- |
| ***Proprietary Models*** | | | | |
| GPT-4o | 53.54 | 50.81 | 38.21 | 28.31 |
| GPT-5 | 59.61 | 47.27 | 40.85 | 37.81 |
| Gemini-2.0-Flash | 53.32 | 44.31 | 23.69 | 33.47 |
| Gemini-2.5-Flash | 57.01 | 47.63 | **41.62** | 38.02 |
| ***Open-Source VLMs*** | | | | |
| Qwen2.5-VL-3B-Inst | 42.76 | 30.69 | -2.54 | 26.86 |
| Qwen2.5-VL-7B-Inst | 40.48 | 38.62 | 18.59 | 29.75 |
| Qwen2.5-VL-32B-Inst | 39.28 | 37.06 | 26.87 | 28.72 |
| MiMo-VL-7B-SFT-2508 | 57.89 | 30.43 | 22.14 | 31.19 |
| ADIEE | 59.96 | 55.56 | 34.50 | -- |
| ***Reward Models (Ours)*** | | | | |
| EditReward (on Qwen2.5-VL-7B) | 63.97 | 59.50 | 36.18 | 36.78 |
| EditReward (on MiMo-VL-7B) | **65.72** | **63.62** | 35.20 | **38.42** |
</details>

---

<summary> EditReward-Bench Results </summary>

| Method | EditReward-Bench (K=2) | EditReward-Bench (K=3) | EditReward-Bench (K=4) | EditReward-Bench (Overall) |
| :--- | :--- | :--- | :--- | :--- |
| Random | 25.81 | 11.33 | 1.35 | 13.84 |
| Human-to-Human | -- | -- | -- | -- |
| ***Proprietary Models*** | | | | |
| GPT-4o | 45.69 | 27.33 | 7.31 | 28.31 |
| GPT-5 | 57.53 | 38.51 | 12.84 | 37.81 |
| Gemini-2.0-Flash | 52.43 | 33.33 | **13.51** | 33.47 |
| Gemini-2.5-Flash | **58.61** | 39.86 | 12.16 | 38.02 |
| ***Open-Source VLMs*** | | | | |
| Qwen2.5-VL-3B-Inst | 51.07 | 20.27 | 2.71 | 26.86 |
| Qwen2.5-VL-7B-Inst | 52.69 | 24.67 | 3.38 | 29.75 |
| Qwen2.5-VL-32B-Inst | 50.54 | 25.27 | 4.05 | 28.72 |
| MiMo-VL-7B-SFT-2508 | 49.46 | 30.41 | 9.46 | 31.19 |
| ADIEE | -- | -- | -- | -- |
| ***Reward Models (Ours)*** | | | | |
| EditReward (on Qwen2.5-VL-7B) | 56.99 | 36.00 | 10.81 | 36.78 |
| EditReward (on MiMo-VL-7B) | 56.45 | **42.67** | 11.49 | **38.42** |
</details>

---

## 📚 Citation

Please kindly cite our paper if you use our code, data, models or results:

```bibtex
@article{wu2025editreward,
  title={EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing},
  author={Wu, Keming and Jiang, Sicong and Ku, Max and Nie, Ping and Liu, Minghao and Chen, Wenhu},
  journal={arXiv preprint arXiv:2509.26346},
  year={2025}
}
```

---

## 🙏 Acknowledgements

We would like to thank the [HPSv3](https://github.com/MizzenAI/HPSv3), [VideoAlign](https://github.com/KwaiVGI/VideoAlign) and [GenAI-Bench](https://github.com/TIGER-AI-Lab/GenAI-Bench) codebase for providing valuable references.

---

## <a id="-awesome-works-using-editreward"></a>✨ Awesome Works using EditReward
😊 TIGER-AI-Lab, [RewardHarness: Self-Evolving Agentic Post-Training](https://arxiv.org/abs/2605.08703).

😊 Reve, CUHK, [PromptRL: Prompt Matters in RL for Flow-Based Image Generation](https://arxiv.org/abs/2602.01382).

😊 Adobe, HKU, [Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editing](https://arxiv.org/abs/2512.17909).

😊 Meta, [Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image](https://arxiv.org/pdf/2512.16899).

😊 Google DeepMind, CUHK, [Image Diffusion Preview with Consistency Solver](https://arxiv.org/abs/2512.13592).

---
## ⭐ Star History [🔝](#-table-of-contents)

[![Star History Chart](https://api.star-history.com/svg?repos=TIGER-AI-Lab/EditReward&type=Date)](https://star-history.com/#TIGER-AI-Lab/EditReward&Date)
## 💬 Support

For questions and support:
- **Issues**: [GitHub Issues](https://github.com/TIGER-AI-Lab/EditReward/issues)
- **Email**: wukeming0608@gmail.com & wenhuchen@uwaterloo.ca

EditReward

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README