ProactiveVideoQA

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

ProactiveVideoQA

★ 20 Open GitHub ↗

ProactiveBench: A Comprehensive Benchmark for VideoLLM Proactive Interaction Evaluation

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Models
---

<div align="center">
  <div style="margin: 30px 0">
    <a href="https://arxiv.org/abs/2507.09313" style="margin: 0 10px">📄 arXiv Paper</a> |
    <a href="https://github.com/yellow-binary-tree/ProactiveVideoQA" style="margin: 0 10px"> 🖥️ Github Code </a> |
    <a href="https://huggingface.co/datasets/wangyueqian/ProactiveVideoQA" style="margin: 0 10px">🤗 Data</a>
  </div>
</div>

## Introduction
ProactiveVideoQA is the first comprehensive benchmark designed to evaluate a system's ability to engage in proactive interaction in multimodal dialogue settings.
Unlike traditional turn-by-turn dialogue systems, in proactive intraction model need to determine when to repsond during the playback, so both response timing and response textual content are important points for evaluation.

This repository hosts the dataset for ProactiveVideoQA, along with the evaluation code of our proposed temporal-aware metric, PAUC (Proactive Area Under the Curve).

## Dataset Statistics

![Dataset Overview](https://raw.githubusercontent.com/yellow-binary-tree/ProactiveVideoQA/main/assets/overview.jpg)

ProactiveVideoQA contains 4 tasks:

1. **Proactive web-video QA** `[WEB]`: centering on general web-video understanding.

1. **Proactive ego-centric video QA** `[EGO]`: centering on first-person-view video comprehension, particularly relevant in robotics and daily assistant applications.

1. **Proactive TV-series video QA** `[TV]`: emphasizing dialogue and social relationship understanding with speech input, and

1. **Proactive video anomaly detection** `[VAD]` targeting surveillance video monitoring and alerting.

- **1377** videos from different sources
- **1427** different qeustions, and **3510** ground truth reply turns
- Fully proactive questions and open-ended answers ✅

## Project Structure
```
ProactiveVideoQA/  
├── data/       # ProactiveVideoQA test data (Download from [🤗Huggingface](https://huggingface.co/datasets/wangyueqian/ProactiveVideoQA))
├── offline_model_inference/       # code for benchmarking offline models (GPT 4.1, Gemini, InternVL2.5, and LLaVA-OneVision) on ProactiveVideoQA
├── pauc/       # PAUC scoring scripts  
```

## Dataset
Download the annotations and videos from [🤗Huggingface](https://huggingface.co/datasets/wangyueqian/ProactiveVideoQA), and rename the folder to `./data`.

# Code for offline models inference
See `offline_model_inference/README.md`.

# Evaluation code
See `pauc/README.md`.

# Contact
If you find ProactiveVideoQA useful in your research, please cite our paper:
```bibtex
@misc{wang2025proactivevideoqacomprehensivebenchmarkevaluating,
      title={ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Models}, 
      author={Yueqian Wang and Xiaojun Meng and Yifan Wang and Huishuai Zhang and Dongyan Zhao},
      year={2025},
      eprint={2507.09313},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2507.09313}, 
}
```

ProactiveVideoQA

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README