MV-RoboBench

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

★ 23 Open GitHub ↗

MV-RoboBench

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes

🔥🔥🔥 ICLR 2026 Accepted 🔥🔥🔥

[ZhiYuan Feng](https://aaronfengzy.github.io/)¹*, Zhaolu Kang²*, Qijie Wang¹*, Zhiying Du³*, Jiongrui Yan⁴, Shi Shubin⁴, Chengbo Yuan¹, Huizhi Liang¹, Yu Deng⁵, Qixiu Li¹, Rushuai Yang⁶, Ruichuan An², Leqi Zheng¹, Weijie Wang⁷, Shawn Chen⁷, Sicheng Xu⁵, Yaobo Liang⁵, Jiaolong Yang⁵†, Baining Guo⁵

*¹Tsinghua University, ²Peking University, ³Fudan University, ⁴Jilin University, ⁵Microsoft Research Asia, ⁶Hong Kong University of Science and Technology, ⁷Zhejiang University*

*(\*Equal Contribution, †Corresponding Author)*

-----

 <a href='https://AaronFengZY.github.io/MV-RoboBench-Webpage/'><img src='https://img.shields.io/badge/Project-Page-Green'></a> <a href='https://arxiv.org/abs/2510.19400'><img src='https://img.shields.io/badge/Arxiv-Page-purple'></a> <a href="https://huggingface.co/datasets/AaronFengZY24/MV_Robobench"><img src='https://img.shields.io/badge/Data-HuggingFace-red'></a> <a href='https://AaronFengZY.github.io/MV-RoboBench-Webpage/'><img src='https://img.shields.io/badge/%F0%9F%94%A5-Hot-orange'></a> <a href='https://AaronFengZY.github.io/MV-RoboBench-Webpage/'><img src='https://img.shields.io/badge/%F0%9F%9A%80-Trending-ff69b4'></a>

## 🎉 News
- [x] [2025.10] 📢📢 Paper and initial project release.
- [x] [2026.01] 📦 Benchmark dataset released on [Hugging Face](https://huggingface.co/datasets/AaronFengZY24/MV_Robobench).
- [x] [2026.01] 🎉🎉 Paper accepted to **ICLR 2026**.
- [x] [2026.03] 🛠️ Evaluation code released (see the [`evaluation`](../../tree/evaluation) branch).

## 📝 To-Do List
- [x] Release Evaluation Code (see the [`evaluation`](../../tree/evaluation) branch)
- [x] Release the benchmark dataset on [Hugging Face](https://huggingface.co/datasets/AaronFengZY24/MV_Robobench)

## MV-RoboBench

<a target="_blank"><img src="./figures/data_pipeline.jpg" alt="Data Pipeline" style="width: 90%; min-width: 200px; display: block; margin: auto;"></a>

**Benchmark Overview:** We introduce *MV-RoboBench*, a benchmark designed to evaluate the multi-view spatial reasoning capabilities of VLMs in robotic scenes. It contains **[Number]** question-answer pairs across **[Number]** diverse robotic scenes. The benchmark comprises **[Number]** challenging tasks, such as **[Task 1 Name]**, **[Task 2 Name]**, and **[Task 3 Name]**. These tasks are designed to probe various aspects of 3D scene understanding, from establishing object correspondences to understanding relative spatial poses.

📌 **A Benchmark for Robotic Scenes:** We introduce *MV-RoboBench*, a comprehensive benchmark designed to evaluate the spatial reasoning of Vision-Language Models in robotic scenes.

📊 **Comprehensive Evaluation:** We evaluate [Number] state-of-the-art VLMs, including models like GPT-4o and Claude 3, revealing a significant performance gap compared to human-level reasoning.

🔍 **Revealing Core Challenges:** Our analysis pinpoints key failure modes for current models in robotic scene understanding, particularly in cross-view correspondence, relative pose estimation, and action planning.

## Contact

For any questions or suggestions, please feel free to contact [Zhiyuan Feng](mailto:t-zhifeng@microsoft.com) or another author.

MV-RoboBench

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README