Video-SafetyBench

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

Video-SafetyBench

★ 13 Open GitHub ↗

[NeurIPS DB 2025] Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMs

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<p align="center">
  <h1 style="display: inline;">
    Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMs
  </h1>
</p>

<h3 align="center">NeurIPS 2025 Datasets and Benchmarks </h3>

<p align="center">
  <a href="https://liuxuannan.github.io/Video-SafetyBench.github.io/">🌐 Homepage</a> •
  
  <a href="https://arxiv.org/abs/2505.11842">📖 Paper</a> •
  <a href="https://huggingface.co/datasets/BAAI/Video-SafetyBench">🤗 Data</a>
</p>

## 📰 News
- **2025-09-19**: [Our paper](https://arxiv.org/abs/2505.11842) has been accepted by the Datasets and Benchmarks track in NeurIPS 2025 ！
- **2025-05-14**: We are excited to release the Video-SafetyBench paper, dataset, and evaluation code!

## 👋 Overview
![Local Image](https://raw.githubusercontent.com/flageval-baai/Video-SafetyBench/main/assets/dataset_overview.png)

### Why Video-SafetyBench Benchmark?
The increasing deployment of Large Vision-Language Models (LVLMs) raises safety concerns under complex real-world inputs. However, existing multimodal safety evaluations primarily focus on model vulnerabilities exposed by static image inputs, ignoring the temporal and dynamic characteristics of video that may induce distinct safety risks.

Video-SafetyBench is designed to **bridge this gap** and offer a safety perspective by providing:
   - **2,264 video-text pairs** spanning **13 unsafe categories and 48 fine-grained subcategories**, each pairing**a synthesized video** with either a **harmful query** and a **benign query**.  
   - Design a controllable pipeline that decomposes **video semantics** into **subject images** and **motion text**, which jointly guide the synthesis of **query-relevant videos**.
   - propose **RiskJudgeScore (RJScore)**, a novel LLM-based metric that leverages **token-level logit distributions** to capture judge model confidence and achieve **strong alignment with human safety judgments**.
   
## 🚀 Quickstart
### 1. Setup (Reproduce the results of LLaVA-Video)
By seeting up the env for llava-video via following steps:

Step 1:
```init
git clone https://github.com/LLaVA-VL/LLaVA-NeXT
cd LLaVA-NeXT
```
In Step 2:
```
conda create -n llava python=3.10 -y
conda activate llava
pip install --upgrade pip  # Enable PEP 660 support.
pip install -e ".[train]"
```

**Dataset Example Feature**:
```bash
{
    "question_id": // Unique ID for the question
    "question": // The question
    "harmful_intention": // The harmful intention for the question
    "video_path": // video_path to the question
    "category": // category to the question
    "subcategory": // subcategory to the question
},
```
### 2. Response Generation
As detailed in Appendix B.1, we evaluate models using three different types of model inference: API-based, vllm, and Transformer, depending on the specific model's availability.
#### 2.1 Transformer-based Evaluation 
 To generate LLaVA-Video's responses for the Video-SafetyBench, run the following command:
```bash
# Run llava-video (Put the model_inference/llava_video.py into the LLaVA-NeXT folder)
python model_inference/llava_video.py \
--model_name LLaVA-Video-7B-Qwen2 \
--query_type harmful  
```

The generated responses will be saved in the `model_results/{model_name}/video_safetybench_{query_type}` directory. Where `{model_name}` is the name of evaluated models and `query_type` is either `harmful_query` or `benign_query` depending on the type of query used..

#### 2.1 vllm-based Evaluation
To support evaluation on Video-SafetyBench with vllm as backend, we recommend the use of the FlagEval Evaluation Framework:

```
git clone https://github.com/flageval-baai/FlagEvalMM
cd FlagEvalMM
pip install -e .
```
An example of Qwen2.5-VL-7B, run the following command:
```
flagevalmm --taskstasks/video_safety_bench/video_safety_benign.py video_safety_bench/video_safety_harmful.py \
        --exec model_zoo/vlm/api_model/model_adapter.py \
        --model Qwen/Qwen2.5-VL-7B-Instruct \
        --cfg Qwen2.5-VL-7B-Instruct.json \
        --output-dir ./results/Qwen2.5-VL-7B-Instruct \
        --backend vllm
```
Create a config file named Qwen2.5-VL-7B-Instruct.json:
```
{
    "model_name": "Qwen/Qwen2.5-VL-7B-Instruct",
    "api_key": "EMPTY",
    "url": "http://localhost:8000/v1/chat/completions",
    "output_dir": "output/Qwen2.5-VL-7B-Instruct",
    "min_short_side": 28,
    "max_long_side": 512,
    "num_workers": 8,
    "max_num_frames": 16,
    "max_tokens": 1024,
    "extra_args": "--limit-mm-per-prompt image=16 --max-model-len 42768 --max_num_seqs 4"
}
```
### 3. Response Evaluation
To evaluate the generated responses, run the following command:
```bash
python eval/evaluate.py \
--response_dir ./model_results \
--model_name LLaVA-Video-72B-Qwen2 \
--query_type harmful  
```
The evaluation results will be saved in the `model_results/{model_name}/video_safetybench_{query_type}_judge` directory.

## ✍️ Citation
If you use our work and are inspired by our work, please consider cite us:
```
@article{liu2025videosafetybench,
  title={Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMs},
  author={Liu, Xuannan and Li, Zekun and He, Zheqi and Li, Peipei and Xia, Shuhan and Cui, Xing and Huang, Huaibo and Yang, Xi and He, Ran},
  journal={arXiv preprint arXiv:2505.11842},
  year={2025}
}
```

Video-SafetyBench

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README