DINO-R1

KO | EN

gitlite — search

#python #android #docker #machine-learning #javascript #react #python3 #cpp #typescript #classification #kotlin #github

★ 52 Open GitHub ↗

No description available.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1><img src="static/images/dino-r1-icon.jpg" width="20" style="vertical-align:middle;"/> DINO-R1: Incentivizing Reasoning Capability in Vision Foundation Models</h1>

[Chenbin Pan](https://scholar.google.com/citations?user=Ln6sN1IAAAAJ&hl=en)1,2, [Wenbin He](https://hewenbin.github.io/)1,2, [Zhengzhong Tu](https://vztu.github.io/)3, [Liu Ren](https://www.liu-ren.com/)1,2

> 1Bosch Research North America
> 2Bosch Center for Artificial Intelligence (BCAI)
> 3Texas A&M University

[![arXiv](https://img.shields.io/badge/arXiv-DINOR1-critical)](https://arxiv.org/pdf/2505.24025)
[![Project Page](https://img.shields.io/badge/Project_Page-DINOR1-brightgreen)](https://Christinepan881.github.io/DINO-R1)

## 📄 Abstract
> The recent explosive interest in the reasoning capabilities of large language models, such as DeepSeek-R1, has demonstrated remarkable success through reinforcement learning-based fine-tuning frameworks, exemplified by methods like Group Relative Policy Optimization (GRPO). However, such reasoning abilities remain underexplored and notably absent in vision foundation models, including representation models like the DINO series. In this work, we propose DINO-R1, the first such attempt to incentivize visual in-context reasoning capabilities of vision foundation models using reinforcement learning. Specifically, DINO-R1 introduces Group Relative Query Optimization (GRQO), a novel reinforcement-style training strategy explicitly designed for query-based representation models, which computes query-level rewards based on group-normalized alignment quality. We also apply KL-regularization to stabilize the objectness distribution to reduce the training instability. This joint optimization enables dense and expressive supervision across queries while mitigating overfitting and distributional drift. Building upon Grounding-DINO, we train a series of DINO-R1 family models that integrate a visual prompt encoder and a visual-guided query selection mechanism. Extensive experiments on COCO, LVIS, and ODinW demonstrate that DINO-R1 significantly outperforms supervised fine-tuning baselines, achieving strong generalization in both open-vocabulary and closed-set visual prompting scenarios.

![GRQO](https://raw.githubusercontent.com/Christinepan881/DINO-R1/main/static/images/grqo-overview.jpg)

## 📌 Citation
```bibtex
@article{pan2025dino,
  title={DINO-R1: Incentivizing Reasoning Capability in Vision Foundation Models},
  author={Pan, Chenbin and He, Wenbin and Tu, Zhengzhong and Ren, Liu},
  journal={arXiv preprint arXiv:2505.24025},
  year={2025}
}

DINO-R1

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README