m3-break

KO | EN

gitlite — search

#hacktoberfest #machine-learning #rust #linux #docker #nodejs #java #windows #kubernetes #awesome-list #ios #openai #mcp #open-source

(★ 15)

On the Adversarial Robustness of Visual-Language Chat Models (ICMR 2025)

m3-break Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# On the Adversarial Robustness of Visual-Language Chat Models

## Introdution

This is the official repository of our [ICMR 2025 paper
"On the Adversarial Robustness of Visual-Language Chat Models"][paper]

## Requirements

- Install required python packages:
```shell
python -m pip install -r requirements.py
```

## Training
Training commands are as follows.

* VQA:
```shell
python scripts/image_attack_vqa_target.py \
    --eps <noise_budget> \
    --step_size <step_size> \
    --epoch <steps_of_PGD> \
    --model <model_name>
```

* Jailbreaking:
```shell
python scripts/image_attack_bad_prompts.py \
    --eps <noise_budget> \
    --step_size <step_size> \
    --epoch <steps_of_PGD> \
    --model <model_name>
```

* Information hiding:
```shell
python scripts/image_attack_information.py \
    --eps <noise_budget> \
    --step_size <step_size> \
    --epoch <steps_of_PGD> \
    --model <model_name>
```

The parameter choices for the above commands are as follows:
- Noise budget `--eps`: `4` , `8`, `16`, `32`, `...`
- Steps of PGD `--epoch`: `100`, `200`, `300`, `400`, `...`
- Model name `--model`: `minigpt4` , `minigpt5`, `mplug`, `mplug2`, `...`

The trained checkpoints will be saved
at `scripts/experiments_<type>/<model>_eps=<args.eps>_epoch=<args.epoch>`.

## Acknowledgement
- Relevant VLMs:
  - [MiniGPT-4](https://github.com/Vision-CAIR/MiniGPT-4),
  - [MiniGPT-5](https://github.com/eric-ai-lab/MiniGPT-5),
  - [mPlUG-owl](https://github.com/X-PLUG/mPLUG-Owl),
  - [mPlUG-owl2](https://github.com/X-PLUG/mPLUG-Owl),
  - [LLaVA-v1.5](https://github.com/haotian-liu/LLaVA).

## Cite the paper

```bib
@inproceedings{qin2025m3,
  author       = {Tianrui Qin and Xuan Wang and  Juanjuan Zhao and Kejiang Ye and Cheng-Zhong Xu and Xitong Gao},
  editor       = {Zhongfei (Mark) Zhang and Elisa Ricci and Yan Yan and Liqiang Nie and Vincent Oria and Lamberto Ballan},
  title        = {On the Adversarial Robustness of Visual-Language Chat Models},
  booktitle    = {Proceedings of the 2025 International Conference on Multimedia Retrieval, {ICMR} 2025, Chicago, IL, USA, 30 June 2025 - 3 July 2025},
  pages        = {1118--1127},
  publisher    = {{ACM}},
  year         = {2025},
  url          = {https://doi.org/10.1145/3731715.3733407},
  doi          = {10.1145/3731715.3733407},
  timestamp    = {Thu, 26 Jun 2025 14:33:29 +0200},
  biburl       = {https://dblp.org/rec/conf/mir/QinWZY0G25.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}
```

[paper]: https://dl.acm.org/doi/10.1145/3731715.3733407

m3-break

m3-break Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

m3-break

m3-break Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README