KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #typescript #developer-tools #claude-code #ai-agent #codex #deepseek-harness #ai-agents #cli #react #mcp

reverse-SynthID-text

★ 95 Open GitHub ↗

reverse engineering SynthID for text

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# SynthID Watermark Reverse Engineering

## Overview

This directory contains tools for analyzing and removing SynthID watermarks from AI-generated text. The watermark was developed by Google DeepMind and published in Nature (2024).

## How SynthID Works

### Watermarking Process

1. **N-gram Context**: For each token position, SynthID considers the previous `ngram_len - 1` tokens (default: 4 tokens) as context.

2. **Hash Computation**: It computes a hash of:
   - The context tokens
   - The candidate next token
   - A set of secret watermarking keys (30 by default)

3. **G-value Assignment**: The hash is used to assign a binary g-value (0 or 1) to each possible next token for each key layer.

4. **Probability Modification**: The token probabilities are modified to favor tokens with g-value = 1:
   ```
   new_prob[i] = prob[i] * (1 + g[i] - mean(g * prob))
   ```

5. **Sampling**: The model samples from the modified distribution.

### Detection Process

1. Tokenize the text
2. For each n-gram, compute g-values using the same keys
3. If the mean g-value is significantly > 0.5, the text is watermarked

## Vulnerabilities

### 1. Paraphrasing Attack (Most Effective)
**Effectiveness: 90-100%**

The watermark is embedded in the specific token sequence, not the meaning. Paraphrasing with a non-watermarked model completely regenerates the token sequence.

```python
from reverse_synthid import ParaphrasingAttack
attack = ParaphrasingAttack(model_name="gpt2")
clean_text = attack.paraphrase(watermarked_text)
```

### 2. Token Substitution Attack
**Effectiveness: 50-70%**

Replacing tokens with synonyms breaks the n-gram patterns:

```python
from reverse_synthid import TokenPerturbationAttack
attack = TokenPerturbationAttack()
clean_text = attack.substitute_synonyms(text, rate=0.3)
```

### 3. Homoglyph Attack
**Effectiveness: 95-100%**

Replacing characters with visually identical Unicode characters breaks the hash:

```python
from reverse_synthid import HomoglyphAttack
attack = HomoglyphAttack()
clean_text = attack.apply_homoglyphs(text, rate=0.1)
```

### 4. N-gram Boundary Shift
**Effectiveness: 30-50%**

Inserting or removing words shifts all subsequent n-gram boundaries:

```python
from reverse_synthid import TokenPerturbationAttack
attack = TokenPerturbationAttack()
clean_text = attack.insert_fillers(text, rate=0.1)
```

## Files

- `reverse_synthid.py` - Main attack toolkit with multiple methods
- `analyze_watermark.py` - Watermark detection and analysis tool
- `test_removal.py` - Test script demonstrating watermark removal

## Usage

### Analyze Watermark
```bash
python analyze_watermark.py --input text.txt --verbose
```

### Remove Watermark
```bash
# Using paraphrasing (most effective, requires model)
python analysis/reverse_synthid.py --input text.txt --output clean.txt --method paraphrase

# Using perturbation (no model needed)
python analysis/reverse_synthid.py --input text.txt --output clean.txt --method perturb

# Using combined attack
python reverse_synthid.py --input text.txt --output clean.txt --method combined
```

### Run Test
```bash
python analysis/test_removal.py
```

## Comparing SynthID Watermarked vs De-Watermarked Text
<p align="center">
  <img src="visualizations/synthid_comparison.png" alt="Comparison" width="100%">
</p>

## Technical Details

### Key Insight: The Watermark is Fragile

The watermark relies on:
1. **Exact token sequences** - Any change to tokens affects n-grams
2. **Context windows** - Inserting tokens shifts all contexts
3. **Hash function** - Different bytes produce different hashes

### Why Paraphrasing Works

When you paraphrase text:
1. The semantic meaning is preserved
2. But the exact tokens are completely different
3. The n-gram hashes are completely different
4. The g-value pattern is destroyed

### Theoretical Analysis

For a text of length N with watermark strength W:
- **Substitution rate r** reduces signal by approximately `1 - (1-r)^(ngram_len)`
- **Insertion rate r** shifts `N*r` n-gram boundaries
- **Paraphrasing** essentially sets W → 0 (complete regeneration)

## Limitations

1. Paraphrasing may change nuances in meaning
2. Simple attacks may be detectable
3. Future watermarks may be more robust

## Disclaimer

This code is for educational and research purposes only. It demonstrates vulnerabilities in watermarking schemes to help develop more robust methods.

## References

1. Dathathri et al. "Scalable watermarking for identifying large language model outputs." Nature 634, 818-823 (2024).
2. https://github.com/google-deepmind/synthid-text

reverse-SynthID-text

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README