Vid-SME

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

★ 26 Open GitHub ↗

[NeurIPS‘25] Vid-SME: Membership Inference Attacks against Large Video Understanding Models

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<div align="center">
<h1><a href="https://arxiv.org/abs/2506.03179v1" target="_blank">Vid-SME: Membership Inference Attacks against Large Video Understanding Models</a></h1>

</div>

<div>
Qi Li&emsp;Runpeng Yu&emsp;Xinchao Wang<sup>&dagger;</sup>
</div>
<div>
    <a href="https://sites.google.com/view/xml-nus/people?authuser=0" target="_blank">xML-Lab</a>, National University of Singapore&emsp;
    <sup>&dagger;</sup>corresponding author 
</div>
</div>
</div>

------------------
TL;DR (1) - Introduce Vid-SME, the first dedicated method for video membership inference attacks against large video understanding models.

TL;DR (2) - Benchmarking MIA performance by training three VULLMs, each on a distinct dataset, using different representative training strategies.

## Overview

<div align="center">
    <div style="max-width: 100%; text-align: left; margin-bottom: 20px;">
        <img src="assets/main_pipeline.jpg" alt="Diagram 2" style="display: block; margin: 0 auto; width: 100%;">
    </div>
</div>
<strong>Figure 1.</strong> Vid-SME against Video Understanding Large Language Models (VULLMs). <strong>Left:</strong> An example of the video instruction context used in our experiments. <strong>Middle:</strong> The overall pipeline of Vid-SME. <strong>Right:</strong> The detailed illustration of the membership score calculaiton of Vid-SME.
            
## Installation & Preparation

1. Follow the instructions provided in [LongVA](https://github.com/EvolvingLMMs-Lab/LongVA) to build the environment.

2. Download the [models](https://huggingface.co/LIQIIIII/Vid-SME) and move them into `./checkpoints`. For the datasets, the json files are given in the `./video_json` folder, download the related videos and move them into `./video_json/videos`.

## Evaluation

Run Vid-SME on each model via the corresponding script:

```
python Vid_SME_main_CinePile.py
```

## Citation

If you finding our work interesting or helpful to you, please cite as follows:

```
@article{li2026vid,
  title={Vid-sme: Membership inference attacks against large video understanding models},
  author={Li, Qi and Yu, Runpeng and Wang, Xinchao},
  journal={Advances in Neural Information Processing Systems},
  volume={38},
  pages={111572--111596},
  year={2026}
}
```

Vid-SME

Today's picks

Was this content helpful?

Recent Feedback

Vid-SME

Today's picks

Was this content helpful?

Recent Feedback

Download README