topviewrs

KO | EN

gitlite — search

#hacktoberfest #react #ai #typescript #golang #machine-learning #java #awesome #cli #awesome-list #pytorch #ios #openai #open-source

(★ 15)

TopViewRS: Vision-Language Models as Top-View Spatial Reasoners (EMNLP 2024 Oral)

topviewrs 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<br />
<p align="center">
  <h1 align="center">👀 TopViewRS: Vision-Language Models as Top-View Spatial Reasoners</h1>
  <h3 align="center">A novel evaluation benchmark for spatial reasoning of vision-language models.</h3>
  
  <p align="center">  
    <a href="https://arxiv.org/abs/2406.02537">📄 [Arxiv]</a>
    ·
    <a href="https://topviewrs.github.io/">🕸️ [Project Page]</a>
    ·
    <a href="https://huggingface.co/datasets/chengzu/topviewrs">🤗 [Data]</a>
    
  </p>
</p>

## Key takeaways

> * Define top-view spatial reasoning task for VLMs via 4 carefully designed tasks of increasing complexity, also encompassing 9 distinct fine-grained sub-tasks with a structured design of the questions focusing on different model abilities.
> * Collect **TopViewRS Dataset** (**Top-View** **R**easoning in **S**pace), comprising 11,384 multiple-choice questions with either _photo-realistic_ or _semantic_ top-view maps of real-world scenarios
> * Investigate 10 VLMs from different model families and sizes, highlighting the _performance gap_ compared to human annotators.

![sicl](https://raw.githubusercontent.com/cambridgeltl/topviewrs/main/figs/main_fig.png)

## Dataset
All benchmark is now available on Huggingface: [https://huggingface.co/datasets/chengzu/topviewrs](https://huggingface.co/datasets/chengzu/topviewrs).

## Citation
If you find TopViewRS useful:
```bibtex
@misc{li2024topviewrs,
      title={TopViewRS: Vision-Language Models as Top-View Spatial Reasoners}, 
      author={Chengzu Li and Caiqi Zhang and Han Zhou and Nigel Collier and Anna Korhonen and Ivan Vulić},
      year={2024},
      eprint={2406.02537},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

topviewrs

topviewrs 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

topviewrs

topviewrs 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README