KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #developer-tools #ai-agent #codex #typescript #cli #deepseek-harness #claude-code #ai-agents #mcp #rust

AIWatermarkDetector

★ 2 Open GitHub ↗

Detect AI Watermarks in text or code

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# AIWatermarkDetector

A self-hostable web app that scans plain text and source code for detectable watermarking, steganography artifacts, Unicode security issues, encoded payloads, URL tracking tokens, and conservative style hints.

The tool is deliberately false-positive-aware. It can prove the presence of specific artifacts, but it cannot prove that a text was generated by AI unless you also have a known watermarking scheme, provenance signal, or external evidence.

**Current structure:** the frontend is no longer a single-file `index.html`. The UI is loaded from `index.html`, while the scanner logic lives in `assets/app.js`. Optional PHP endpoints add server-side checks.

---

## What this tool can show

### Verifiable evidence

The scanner can report objective artifacts that you can independently validate:

- Unicode invisibles and control characters, for example ZWSP, ZWJ, BOM, BiDi controls
- Trojan Source style BiDi controls and pairing issues
- Unicode normalization drift via NFC / NFKC comparison
- Mixed-script or confusable identifiers in code-like input
- Whitespace-channel signals such as systematic trailing spaces or mixed indentation
- URL tracking parameters such as `utm_*`, `gclid`, `fbclid`, `msclkid`, `srsltid`
- Encoded payload candidates such as Base64, Base64URL, hex blobs, escape sequences and percent-encoding runs

### Heuristic hints

The scanner also includes weak triage signals. These are always reported as `HINT`:

- Repetition and token trigram uniformity
- High-entropy runs
- Acrostic samples
- **German AI-style heuristics via `de_ai_style`**

These hints can be useful when comparing texts or prioritizing review work. They are not proof of AI authorship.

---

## Evidence levels

The UI groups findings into four levels:

| Level | Meaning |
| --- | --- |
| `PROOF` | Directly verifiable artifact, for example a specific Unicode code point or tracking parameter. |
| `STRONG` | Validated clear signal, for example a Base64 payload that decodes to printable UTF-8. |
| `MEDIUM` | Conservative indicator that can still occur naturally in some contexts. |
| `HINT` | Weak heuristic for triage only. Never use alone as proof. |

Recommended report wording:

> The text contains the verifiable artifact U+202E RIGHT-TO-LEFT OVERRIDE at index ...

Avoid wording such as:

> The text was written by AI.

---

## Project layout

```txt
AIWatermarkDetector/
├── index.html
├── assets/
│   └── app.js
├── api/
│   ├── analyze.php
│   └── de_ai_style_analyze.php
├── data/
│   └── de_ai_text_indicators.json
└── docs/
    └── de-ai-style-heuristics.md
```

### Main files

- `index.html` – UI shell and CSS
- `assets/app.js` – client-side scanner logic and report rendering
- `api/analyze.php` – optional PHP API for Unicode, ICU Spoofchecker and strict Base64 checks
- `api/de_ai_style_analyze.php` – optional PHP API for German style heuristics
- `data/de_ai_text_indicators.json` – machine-readable German indicator catalog
- `docs/de-ai-style-heuristics.md` – explanation of German style heuristics and limitations

---

## Installation

### Static frontend only

Host these files on any static web server:

```txt
index.html
assets/app.js
```

The client-side checks work without PHP.

### With optional PHP APIs

Host the complete project on a PHP-capable web server:

```txt
index.html
assets/app.js
api/analyze.php
api/de_ai_style_analyze.php
data/de_ai_text_indicators.json
docs/de-ai-style-heuristics.md
```

Recommended PHP setup:

- PHP 8+
- `ext-intl` for better Unicode / ICU checks in `api/analyze.php`
- JSON responses only; do not print PHP warnings into API output
- reasonable request size limits and rate limiting if exposed publicly

Opening `index.html` directly as a local `file://` document is not recommended when using PHP APIs. Use a local web server instead.

---

## Usage

1. Paste text or code into the input field.
2. Choose the input mode: `Auto`, `Text`, or `Code`.
3. Select checks.
4. Click `Run analysis`.
5. Export or copy the JSON report if you need a reproducible record.

Recommended defaults:

- `Mask URLs for payload scans`: on
- `Strict Base64 mode`: on
- `Analyze URL params for tracking/payload`: on
- `Use PHP API`: optional, useful for ICU-based Unicode checks

---

## German AI-style heuristics

**New check:** `de_ai_style` adds German-language style heuristics to the UI.

When selected, the frontend calls:

```txt
./api/de_ai_style_analyze.php
```

This happens independently from the `Use PHP API` checkbox. The checkbox controls the older main API endpoint `api/analyze.php`; `de_ai_style` uses its own endpoint because its output and evidence level are different.

### What it measures

The German style endpoint returns metrics such as:

- word count and sentence count
- average sentence length
- sentence-length standard deviation and coefficient of variation
- type-token ratio
- Yule's K
- hapax ratio
- repeated 3-grams and 4-grams
- density of formulaic transition phrases
- assistant-like caveat and disclaimer phrases
- parallel structures and triplets
- informal German markers
- dash and punctuation patterns

### How it is displayed

The UI merges German style findings into the normal report as:

```json
{
  "method": "de_ai_style",
  "level": "HINT",
  "source": "server"
}
```

Detailed metrics are stored in:

```json
{
  "stats": {
    "de_ai_style": {}
  }
}
```

### Interpretation

German style heuristics are genre-sensitive. They can trigger on completely legitimate text types, including:

- governmental writing
- legal notices
- press releases
- SEO text
- help-center articles
- academic or educational prose
- heavily edited corporate texts

Use them only as triage data. Do not use them as authorship proof.

---

## API endpoints

### `POST /api/analyze.php`

Optional main analysis endpoint for stronger server-side Unicode and payload checks.

Example request:

```json
{
  "text": "...",
  "selected": ["unicode_homoglyph", "unicode_bidi", "payload_base64"],
  "settings": {
    "maskUrls": true,
    "strictBase64": true
  }
}
```

### `POST /api/de_ai_style_analyze.php`

German style heuristic endpoint.

Example request:

```json
{
  "text": "Deutscher Fließtext...",
  "settings": {
    "include_catalog": true
  }
}
```

Example response shape:

```json
{
  "ok": true,
  "meta": {
    "language_focus": "de",
    "evidence_policy": "HINT_ONLY_NOT_AUTHORSHIP_PROOF"
  },
  "style": {
    "metrics": {},
    "findings": [],
    "matches": {}
  },
  "indicator_catalog": []
}
```

---

## Detection methods

### Unicode invisibles and controls

Detects zero-width characters, word joiners, BOM, selected control characters and unusual spaces. Findings include exact code points and positions.

### BiDi / Trojan Source controls

Detects known BiDi control characters and pairing issues that can visually reorder code.

### Unicode normalization drift

Compares text against NFC and NFKC forms to identify compatibility or canonical drift.

### Confusables / mixed-script identifiers

Client-side checks are conservative and code-focused. The optional PHP API can use ICU Spoofchecker for higher fidelity. Normal German characters such as `ä`, `ö`, `ü` and `ß` should not be treated as suspicious in normal prose.

### Whitespace channel signals

Detects systematic trailing whitespace, trailing tabs and mixed indentation patterns.

### Encoded payloads

Detects Base64, Base64URL, hex blobs, escape sequences and percent-encoding runs. Strict Base64 mode requires successful decoding and plausible decoded output.

### URL tracking tokens

Flags known tracking keys only. A normal query string is not treated as tracking by itself.

### Repetition, entropy and acrostic hints

These checks are weak triage signals only and stay in `HINT`.

---

## Why scanning documentation can show findings

If your README or documentation includes example strings containing BiDi controls, ZWSP, `utm_*`, `gclid`, Base64 strings or other artifacts, the scanner should report them. That is correct behavior because the scanner analyzes the exact characters present in the input.

For documentation examples, consider using escaped representations such as `U+200B` instead of embedding the real invisible character.

---

## Security notes

If you expose the PHP APIs publicly:

- enforce a maximum request size
- add basic rate limiting
- keep PHP warnings and notices out of JSON responses
- consider restricting CORS origins
- do not log sensitive pasted text unless that is explicitly intended

The frontend does not execute pasted code.

---

## Limitations

- Generic LLM token-level watermark detection is scheme-dependent and usually requires knowledge of the watermark family, parameters or key.
- Style alone cannot prove AI authorship.
- Watermark robustness varies under paraphrasing, mixing and editing.
- Code watermarking can require model-side cooperation or specialized detectors.
- This project focuses on observable artifacts and conservative indicators.

---

## Method references

### LLM watermarking background

- Kirchenbauer et al., “A Watermark for Large Language Models”
- Kirchenbauer et al., “On the Reliability of Watermarks for LLMs”
- “A Survey of Text Watermarking in the Era of Large Language Models”
- “Watermark under Fire / WaterPark” robustness evaluation
- “A Robustness Evaluation of LLM Watermarking”

### Text steganography and Unicode security

- Trojan Source
- Innamark whitespace replacement
- Unicode whitespace replacement watermarking methods
- Unicode Security Mechanisms, UTS #39
- Unicode Normalization Forms, UAX #15

### Encoding standards and tooling

- RFC 4648 for Base64
- WHATWG URL Standard
- PHP `Spoofchecker`
- PHP `IntlChar`
- PHP `Normalizer`

### Code watermarking background

- “Who Wrote this Code? Watermarking for Code Generation”
- “Practical and Effective Code Watermarking for Large Language Models”

AIWatermarkDetector

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README