Hi-Guard

KO | EN

gitlite — search

#javascript #python #typescript #llm #android #machine-learning #deep-learning #linux #docker #macos #java #windows #awesome #vue

(★ 24)

[KDD 2026 ADS Track] Pytorch implementation of the paper "Hi-Guard: Towards Trustworthy Multimodal Moderation via Policy-Aligned Reasoning and Hierarchical Labeling".

Hi-Guard Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center">Hi-Guard: Towards Trustworthy Multimodal Moderation via Policy-Aligned Reasoning and Hierarchical Labeling (KDD 2026 ADS Track)</h1>

Anqi Li1, Wenwei Jin2, Jintao Tong3, Pengda Qin2, Jiawei Li2, Guo Lu1
 
 1Shanghai Jiao Tong University &nbsp;&nbsp;
 2Xiaohongshu Inc. &nbsp;&nbsp;
 3Huazhong University of Science and Technology

This repository contains the official implementation of **Hi-Guard**, a cascaded multimodal large language model (MLLM) framework designed to improve classification accuracy, interpretability, and policy alignment in real-world content moderation.

**📄 Paper**: [https://arxiv.org/pdf/2508.03296](https://arxiv.org/pdf/2508.03296)

---
## 🧱 Framework Overview

The Hi-Guard framework consists of a cascaded moderation pipeline with two stages:

1. **Binary Guard**: Filters out safe content using lightweight classification.
2. **Hierarchical Guard**: Performs policy-aligned risk path generation on potentially risky content.

## 🔍 Highlights

- **Hierarchical Pipeline**: A two-stage moderation design that first filters safe content, then applies fine-grained risk classification to potentially risky cases.
- **Policy-Aligned Reasoning**: Models are prompted with real rule definitions and produce interpretable outputs with structured reasoning.
- **Soft-Margin Reward Optimization**: Fine-grained classification is optimized using a reward function that penalizes sibling-class confusion with increasing severity by depth.
- **Real-World Efficiency**: Reduces human review by 56.38% in deployment.

---

## 🏗️ Repository Structure

```
Hi-Guard/
├── assets/
├── src/
│   ├── rlvr/                # Stage2: RLVR + GRPO logic
│   ├── sft/                 # Stage1: Supervised fine-tuning 
├── setup1.sh                # Environment setup for stage1
├── setup2.sh                # Environment setup for stage2
└── README.md
```
---

<!-- ## ⚙️ Setup

Install dependencies via:

```bash
sh setup.sh
```

--- -->

## 🚀 Quick Start

Stage 1: Binary Classification (Safe vs. Risky)

```bash
sh setup1.sh
sh src/sft/train/2B_Binary-Guard.sh
```

Stage 2: Hierarchical Risk Path Prediction

```bash
sh setup2.sh
sh src/rlvr/train/7B_Hi-Guard.sh
```

---

<!-- ## 🧠 Method Overview

Hi-Guard addresses key challenges in moderation:
1. Misalignment between model decisions and platform policies
2. Opaque predictions lacking explanation
3. Confusion between fine-grained sibling categories

It resolves these via:
- **Policy-injected prompting** for rule-aware classification
- **Structured output** with `<think>` and `<answer>` format
- **Hierarchical taxonomy** enabling coarse-to-fine prediction
- **Soft-margin reward** to shape learning over sibling categories

All components are optimized using Group Relative Policy Optimization (GRPO).

--- -->

## 📊 Dataset

Hi-Guard uses a two-stage dataset from real-world moderation traffic:

| Stage | Task                  | #Samples | Risky:Safe Ratio |
|-------|-----------------------|----------|------------------|
| 1     | Binary Classification | 38k      | ~1:4             |
| 2     | Hierarchical Classification | 41k      | ~1:1             |

Only Stage 1 risky samples proceed to Stage 2 classification.

---

<!-- ## 🧪 Evaluation

Hi-Guard is evaluated on:
- **In-domain accuracy** (base set)
- **Generalization to unseen categories**
- **Reasoning preference by human moderators**

Hi-Guard outperforms SFT and RLVR baselines in accuracy, recall, and reasoning quality, while reducing GPU time by 22.7%.

--- -->

## 📦 Model Outputs

Structured predictions are formatted as:

```json
{
  "label_path": ["Minor", "Inappropriate Behavior", "Delinquent Social Atmosphere", "Underage Drinking"],
  "reasoning": "The image shows minors in a bar setting, holding drinks, which aligns with platform policy definition of underage drinking..."
}
```

---

## 📌 Citation

```bibtex
@article{li2025towards,
  title={Towards Trustworthy Multimodal Moderation via Policy-Aligned Reasoning and Hierarchical Labeling},
  author={Li, Anqi and Jin, Wenwei and Tong, Jintao and Qin, Pengda and Li, Weijia and Lu, Guo},
  journal={arXiv preprint arXiv:2508.03296},
  year={2025}
}
```

## 📝 License

Apache-2.0 license.

Hi-Guard

Hi-Guard Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Hi-Guard

Hi-Guard Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README