bgp-sentinel

KO | EN

gitlite — search

#python #hacktoberfest #react #typescript #golang #go #rust #deep-learning #macos #awesome #awesome-list #security #ios #open-source

(★ 9)

eBPF-powered BGP anomaly detector that catches mass prefix withdrawals in real-time

bgp-sentinel Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# bgp-sentinel

eBPF-powered BGP anomaly detector that catches mass prefix withdrawals in real-time.

~100 lines of BPF C sit inside the kernel, parsing BGP UPDATE messages on port 179 and counting withdrawn prefixes before anything reaches userspace. Zero overhead for non-BGP traffic.

Built after reading the [Cloudflare February 20 postmortem](#the-cloudflare-february-20-incident) - where reviewed, tested, CI-green automation withdrew 1,100 prefixes over 50 minutes and nobody noticed until users did.

![Demo](https://raw.githubusercontent.com/pizzabits/bgp-sentinel/main/demo/demo.gif)

## How it works

```mermaid
graph LR
 K["Linux Kernel tcp_sendmsg() port 179"]
 A["bgp-sentinel sliding window alerter"]
 Y["You GET :9100 /api/v1/withdrawals"]
 K -->|"eBPF perf buffer"| A
 A -->|"HTTP"| Y
 style K fill:#161b22,stroke:#3fb950,color:#c9d1d9
 style A fill:#161b22,stroke:#79c0ff,color:#c9d1d9
 style Y fill:#161b22,stroke:#e3b341,color:#c9d1d9
 linkStyle 0 stroke:#3fb950
 linkStyle 1 stroke:#e3b341
```

An eBPF kprobe on `tcp_sendmsg` filters for BGP traffic (port 179), parses UPDATE headers, and counts prefix withdrawals - all inside the kernel. Zero overhead for non-BGP traffic. No BGP daemon modification needed.

```mermaid
graph TD
 subgraph "Kernel space"
 S["tcp_sendmsg()"] --> F{"dest port == 179?"}
 F -->|no| D["drop"]
 F -->|yes| P["parse BGP UPDATE extract withdrawn routes"]
 P --> B["perf_submit()"]
 end
 subgraph "User space"
 B --> A["alerter 60s sliding window"]
 A --> T{"threshold exceeded?"}
 T -->|no| W["log + expose via API"]
 T -->|yes| AL["ALERT + webhook"]
 end
 style S fill:#161b22,stroke:#3fb950,color:#c9d1d9
 style F fill:#161b22,stroke:#e3b341,color:#c9d1d9
 style D fill:#161b22,stroke:#8b949e,color:#8b949e
 style P fill:#161b22,stroke:#3fb950,color:#c9d1d9
 style B fill:#161b22,stroke:#79c0ff,color:#c9d1d9
 style A fill:#161b22,stroke:#79c0ff,color:#c9d1d9
 style T fill:#161b22,stroke:#e3b341,color:#c9d1d9
 style W fill:#161b22,stroke:#8b949e,color:#c9d1d9
 style AL fill:#161b22,stroke:#f85149,color:#f85149
```

<details>
<summary>BGP UPDATE wire format (RFC 4271) - what the probe parses</summary>

The BPF C code reads 21 bytes from the send buffer - the BGP header plus Withdrawn Routes Length. Then copies up to 256 bytes of raw withdrawn route data into the perf event.

```mermaid
block-beta
    columns 4

block:marker:4
        m["Marker (16 bytes, all 0xFF)"]
    end

block:hdr:4
        len["Length (2B)"]
        type["Type = 2 (UPDATE)"]
    end

block:wrl:4
        wr["Withdrawn Routes Length (2B)"]
    end

block:routes:4
        r["Withdrawn Routes (variable)\nprefix_len (1B) + prefix (ceil(len/8) bytes)\nprefix_len (1B) + prefix (ceil(len/8) bytes)\n..."]
    end

block:tpa:4
        tp["Total Path Attr Length (2B)"]
    end

block:rest:4
        pa["Path Attributes + NLRI (variable)"]
    end

style marker fill:#161b22,stroke:#8b949e,color:#8b949e
    style m fill:#161b22,stroke:#8b949e,color:#8b949e
    style hdr fill:#161b22,stroke:#79c0ff,color:#c9d1d9
    style len fill:#161b22,stroke:#79c0ff,color:#c9d1d9
    style type fill:#161b22,stroke:#79c0ff,color:#c9d1d9
    style wrl fill:#161b22,stroke:#3fb950,color:#3fb950
    style wr fill:#161b22,stroke:#3fb950,color:#3fb950
    style routes fill:#161b22,stroke:#f85149,color:#f85149
    style r fill:#161b22,stroke:#f85149,color:#f85149
    style tpa fill:#161b22,stroke:#8b949e,color:#8b949e
    style tp fill:#161b22,stroke:#8b949e,color:#8b949e
    style rest fill:#161b22,stroke:#8b949e,color:#8b949e
    style pa fill:#161b22,stroke:#8b949e,color:#8b949e
```

> The probe reads bytes 0-20 (marker + length + type + withdrawn length), highlighted in **blue** and **green**. The withdrawn routes themselves (in **red**) are copied into the perf event for userspace parsing.

**Example:** `162.159.128.0/24` on the wire:

| Byte | Value | Meaning |
|------|-------|---------|
| 0 | `24` | prefix length in bits |
| 1 | `162` | first octet |
| 2 | `159` | second octet |
| 3 | `128` | third octet |

For a /16, only 2 prefix bytes are sent. For a /8, just 1. The RFC wastes nothing.

**Why `tcp_sendmsg`:** We hook at the socket layer, not the network layer. The BGP message arrives before fragmentation, checksumming, or encryption. One `sendmsg` call = one BGP message. No reassembly needed.

</details>

## Quick start

### Kubernetes

No Docker build required. The probe runs as a privileged pod with BCC installed at runtime.

```bash
kubectl apply -f k8s/rbac.yaml
kubectl apply -f k8s/pod-inline.yaml
kubectl -n bgp-sentinel wait --for=condition=Ready pod/bgp-sentinel --timeout=120s
kubectl -n bgp-sentinel exec bgp-sentinel -- python3 /code/live.py
```

### Bare metal

```bash
# Install BCC (Ubuntu/Debian)
sudo apt-get install -y python3 python3-bpfcc bpfcc-tools
pip3 install flask requests

# Run (requires root for eBPF)
sudo python3 -m agent.probe --threshold 10 --window 60

# With webhook alerting
sudo python3 -m agent.probe --threshold 10 --webhook https://hooks.slack.com/...

# Custom API port (default: 9100)
sudo python3 -m agent.probe --port 8080
```

## What it catches

```
[2026-02-20 17:56:03] PID=4521 BGP WITHDRAW: 5 prefixes (peer 127.0.0.1:179) 162.159.128.0/24 162.159.129.0/24 162.159.130.0/24 172.64.0.0/24 172.64.1.0/24
[2026-02-20 17:56:03] PID=4521 BGP WITHDRAW: 5 prefixes (peer 127.0.0.1:179) 172.64.2.0/24 104.16.0.0/24 104.16.1.0/24 104.16.2.0/24 104.16.3.0/24
ALERT: 10 withdrawals in 60s (threshold: 10) - possible mass withdrawal event
```

- **Mass prefix withdrawals** - the Cloudflare scenario
- **Withdrawal cascades** - BGP route leaks causing downstream peers to withdraw
- **Automation bugs** - CI/CD or cleanup scripts touching BGP without guards
- **Any anomalous withdrawal rate** - configurable threshold above your baseline

## What it doesn't do

- **Not a blocker.** Detects, doesn't prevent. You still need circuit breakers.
- **Not a BGP daemon.** Doesn't modify, inject, or interfere with BGP messages.
- **Not a config validator.** Watches the wire, not your router configs.
- **Linux only.** eBPF is a Linux kernel feature.

## The Cloudflare February 20 incident

A single missing `=true` in a Go API call, inside a system designed to *prevent* outages.

Cloudflare's "Code Orange: Fail Small" initiative aimed to replace risky manual prefix operations with safe, automated workflows. A cleanup sub-task queried the Addressing API for BYOIP prefixes pending deletion, then withdrew them via BGP.

The sub-task was supposed to query:

```
GET /v1/prefixes?pending_delete=true
```

Instead, it queried:

```
GET /v1/prefixes?pending_delete
```

In Go's `net/url` package, `URL.Query().Get("pending_delete")` returns `""` for *both* URLs:

```go
if v := req.URL.Query().Get("pending_delete"); v != "" {
    // v == "" for ?pending_delete (no value)
    // this branch is SKIPPED - returns ALL prefixes
    prefixes, err := c.RO().IPPrefixes().FetchPrefixesPendingDeletion(ctx)
}
```

The API returned all 4,306 prefixes. The system withdrew every one from BGP. Code reviewed. Tests passed. CI green. 15 days in production before it triggered. A single `=true` was the difference between a routine cleanup and a 6-hour outage.

Source: [Cloudflare blog - Post-mortem on Cloudflare control plane and analytics outage](https://blog.cloudflare.com/cloudflare-outage-february-20-2026/)

bgp-sentinel doesn't care about your code quality. It watches the wire. 847 withdrawals/min would have triggered an alert in the first 60 seconds.

## API

### `GET /api/v1/withdrawals`

```json
{
  "timestamp": "2026-02-20T17:56:03+00:00",
  "window_seconds": 60,
  "threshold": 10,
  "total_withdrawals": 54,
  "event_count": 3,
  "alert_count": 1,
  "alerting": true,
  "events": [
    {
      "time": "2026-02-20T17:56:03+00:00",
      "pid": 4521,
      "peer": "127.0.0.1:179",
      "withdrawn_count": 5,
      "process_name": "addressing-api",
      "prefixes": ["162.159.128.0/24", "162.159.129.0/24", "..."]
    }
  ]
}
```

### `GET /healthz`

```json
{"status": "ok", "ebpf_attached": true}
```

## Architecture

```mermaid
graph TD
 subgraph "k8s/pod-inline.yaml"
 CM["ConfigMap probe.py - BCC + inline C alerter.py - sliding window api.py - Flask HTTP live.py - terminal display"]
 POD["Pod (privileged, hostPID) ubuntu:24.04 + BCC mounts: /proc, debugfs, /lib/modules, /usr/src"]
 SVC["Service ClusterIP :9100"]
 end
 CM --> POD
 POD --> SVC
 style CM fill:#161b22,stroke:#3fb950,color:#c9d1d9
 style POD fill:#161b22,stroke:#79c0ff,color:#c9d1d9
 style SVC fill:#161b22,stroke:#e3b341,color:#c9d1d9
```

Everything ships as a single YAML file. No Docker build. Python code in a ConfigMap, BCC installed at startup, BPF compiled on the node using its own kernel headers.

## Compatibility

> [!NOTE]
> Requires kernel 6.x+, kernel headers, and BCC. The probe uses hardcoded `msghdr` struct offsets from the kernel 6.x embedded `__ubuf_iovec` layout. Requires privileged containers (`CAP_BPF` + `CAP_SYS_ADMIN` + `hostPID`).

| Platform | Status |
|----------|--------|
| AKS | Works |
| EKS | Works |
| GKE | Ubuntu nodes only |
| K3s | Works |
| Bare metal | Works |
| kind / minikube | Works |

## Telemetry

bgp-sentinel sends a single anonymous ping when the probe starts (at most once per 24 hours). No IP addresses, hostnames, prefix data, or cluster information is collected.

To opt out: `BGP_SENTINEL_NO_TELEMETRY=1` or `DO_NOT_TRACK=1`

This is a solo open source project. Telemetry is the only way to know if anyone is actually using it, what kernels to support, and whether to keep investing time. Without it, the project is built blind.

| Field | Example | Why |
|-------|---------|-----|
| tool | bgp-sentinel | Which tool sent the ping |
| version | 0.1.0 | Know which versions are in the wild |
| kernel | 6.17.0-1008-gcp | Know which kernel offsets to support |
| arch | x86_64 | Know if ARM support matters |
| python | 3.12.3 | Know minimum Python version to target |
| deployment_type | pod / daemonset / standalone | Know how people deploy |
| uptime_hours | 48 | Distinguish "tried once" from "running in prod" |
| distinct_id | a1b2c3... (SHA-256 hash) | Count unique installs without identifying anyone |

### How the install ID works

bgp-sentinel reads your cluster's `kube-system` namespace UID and hashes it with SHA-256. This is why `rbac.yaml` includes a ClusterRole with read access to the `kube-system` namespace - it's only used to generate the telemetry hash. The raw UID never leaves your cluster. The hash cannot be reversed. On standalone installs, `/etc/machine-id` is used instead.

</details>

## Limitations

- No persistence - data is lost on pod restart
- No authentication on the HTTP API
- BCC requires kernel headers on every node
- IPv4 only (no IPv6 BGP sessions)
- Single-node - no cross-node aggregation
- Prefix extraction capped at 256 bytes per UPDATE (~64 /24 prefixes)

## License

bgp-sentinel

bgp-sentinel Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

bgp-sentinel

bgp-sentinel Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README