shudhi

KO | EN

gitlite — search

#javascript #react #ai #llm #golang #deep-learning #nodejs #macos #java #windows #vue #awesome-list #security #open-source

(★ 103)

A language agnostic tool for in-memory data management and debugging at runtime.

shudhi Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Shudhi

See, inspect, and clear in-memory caches across all your services and pods — from one dashboard.

Every microservice caches things in memory (local maps, Guava, Caffeine, etc). Once it's in memory, it's invisible — you can't see what's cached, you can't inspect values, and you can't clear a stale entry without restarting the pod. Multiply that by 10 services and 50 pods, and you're flying blind.

Shudhi fixes this. Deploy it as a sidecar, implement 3 simple endpoints in your app, and you get:

- **Visibility** — see every cached key across every pod, from one place
- **Inspection** — read the actual cached value from any specific pod
- **Invalidation** — clear a key (or all keys) across all pods of a service, instantly

```
    Your services with in-memory caches
    ┌─────────┐  ┌─────────┐  ┌─────────┐
    │ App A   │  │ App B   │  │ App C   │
    │ (cache) │  │ (cache) │  │ (cache) │
    └────┬────┘  └────┬────┘  └────┬────┘
         │            │            │
    ┌────┴────┐  ┌────┴────┐  ┌────┴────┐
    │Sidecar A│  │Sidecar B│  │Sidecar C│    ← Shudhi sidecars
    └────┬────┘  └────┬────┘  └────┬────┘
         │            │            │
         └────────────┼────────────┘
                      │
              Redis (coordination)
```

Redis is used internally for sidecar coordination (discovery, message passing). Your app never talks to Redis — it only talks to its local sidecar over HTTP.

## Quick Start

```bash
APP_URL=http://localhost:8080 REDIS_URL=localhost:6379 go run .
```

| Env Var | Default | Description |
|---------|---------|-------------|
| `APP_URL` | `http://localhost:8080` | Your app's base URL |
| `SIDECAR_PORT` | `8900` | Sidecar listen port |
| `REDIS_URL` | `localhost:6379` | Redis address (for sidecar coordination) |
| `POD_IP` | `127.0.0.1` | Pod IP (k8s downward API) |
| `INMEM_TOKEN` | _(empty)_ | Shared auth token between sidecar and app |

## Integrating Your App

Your app keeps its own in-memory cache as-is. No changes to your caching logic. You just expose 3 endpoints so the sidecar can talk to your cache, and optionally make 1 call to register keys for dashboard visibility.

### 1. Tell the sidecar who you are (required)

```
GET /internal/inMem/serverInfo
→ { "serviceName": "rider-app", "podName": "rider-app-7b4f8d6c9-x2k4m" }
```

Called once at startup. The sidecar uses this to identify itself in the network.

### 2. Return a cached value on demand (required)

```
POST /internal/inMem/get
← { "key": "RouteByRouteId:config-id:route-123" }
→ { "found": true, "value": { "vehicleType": "SUV" } }
```

Look up the key in your in-memory cache and return the value.

### 3. Clear cache entries on demand (required)

```
POST /internal/inMem/refresh
← { "keyInfix": "RouteByRouteId" }   // or null to clear all
→ 200 OK
```

Delete any in-memory entries whose key contains the given `keyInfix`. If `null`, clear your entire cache.

### 4. Register cached keys (optional, enables dashboard visibility)

Whenever your app puts something in its in-memory cache, tell the sidecar:

```
POST http://localhost:8900/api/registerKey
{
  "keyName": "RouteByRouteId:config-id:route-123",
  "keySchema": null,
  "ttlInSeconds": 3600
}
```

This makes the key show up in the dashboard so you can browse and inspect it. Without this, get/refresh still work — you just won't see the key listed.

> If `INMEM_TOKEN` is set, the sidecar sends it as `x-inmem-token` header on all calls to your app. Use it to verify requests come from a trusted sidecar. Your app must send the same header back on `/api/registerKey` — see Authentication below.

## Sidecar API

All endpoints work from any sidecar — the dashboard only needs to reach one to interact with any service's in-memory cache.

| Method | Endpoint | What it does | Token |
|--------|----------|-------------|-------|
| `GET` | `/api/services` | List all services with registered caches | required |
| `GET` | `/api/pods?service=X` | List live pods for a service | required |
| `GET` | `/api/keys?service=X&pod=Y` | List cached keys (pod optional) | required |
| `POST` | `/api/pod/get` | Read a cached value from a specific pod's memory | required |
| `POST` | `/api/refresh` | Clear cache entries across all pods of a service | required |
| `POST` | `/api/registerKey` | Register a cache key (called by your app) | required |
| `GET` | `/api/health` | Health check | open |

## Authentication

`INMEM_TOKEN` is a single shared secret used in both directions:

- **Outbound** — the sidecar sends `x-inmem-token` on every call it makes to your app and to peer sidecars.
- **Inbound** — every `/api/*` endpoint except `/api/health` requires the same header. A missing or wrong token gets `401`. Comparison is constant-time.

`/api/health` stays open so k8s liveness/readiness probes work without credentials.

**If `INMEM_TOKEN` is unset, inbound validation is skipped entirely** and every endpoint is open — including reading cached values out of any pod and clearing caches fleet-wide. The sidecar logs a warning at startup when this is the case. Set a token in any shared environment.

> **Upgrading:** if you already run with `INMEM_TOKEN` set, your app must now send `x-inmem-token` on its `/api/registerKey` calls — previously the token was outbound-only and registration was unauthenticated. Without it, key registration gets `401` and the dashboard stops listing that pod's keys (get/refresh keep working).

The dashboard's nginx injects the token server-side via its own `INMEM_TOKEN` env var, so the browser never sees it. Note that this leaves the dashboard itself unauthenticated — anyone who can reach it can act through it. Put auth in front of the dashboard if it's exposed beyond your cluster.

## How It Works

**Your app's in-memory cache is the source of truth.** The sidecar never caches data itself — it's a thin coordination layer that lets you reach into any pod's memory from anywhere.

**Startup**: The sidecar calls your app's `/serverInfo` to learn its identity, then announces itself to other sidecars via Redis. A heartbeat keeps the registration alive (60s TTL, refreshed every 30s). On shutdown, it deregisters immediately.

**Inspecting a value**: When you click "Get" on a key in the dashboard, the sidecar routes the request to the correct pod — either via direct HTTP or via a pub/sub relay if the pod isn't directly reachable. The target pod's sidecar calls the app's `/get` endpoint, reads the value from the app's in-memory cache, and returns it.

**Clearing cache**: When you clear a key, the sidecar broadcasts to all pods of that service. Each pod's sidecar receives the message and calls its local app's `/refresh` endpoint, which removes the entry from the app's in-memory cache. The stale data is gone from every pod within seconds.

**Key registration**: When your app registers a key, the sidecar stores the metadata (key name, schema, TTL) so the dashboard can list it. This is just a registry for browsing — the actual cached data always lives in your app's memory.

**Clears are delivered through a durable log, not pub/sub.** Redis pub/sub is fire-and-forget — a pod that's disconnected when a clear is published never receives it, and reconnecting doesn't replay it. So a refresh is appended to `inmem:refreshlog:<service>` (a Redis Stream capped at 1000 entries, 24h TTL), and every pod of that service reads from it with a blocking `XREAD`.

Each pod keeps its own cursor at `inmem:cursor:<service>:<pod>`, **committed only after its app has actually cleared**. That gives at-least-once delivery: a sidecar that drops its connection — or restarts outright — resumes exactly where it left off. If the app rejects a clear, the cursor stays put and the clear is retried until it lands, so a temporarily unhealthy app doesn't lose the invalidation. Retries are bounded (10 rounds), after which the sidecar logs an error and moves on, so one clear the app permanently rejects can't wedge every clear behind it.

This is deliberately **not** a consumer group. A group distributes each entry to exactly one member, so a single shared group would have one pod clearing while every other pod kept serving stale cache. Per-pod cursors give fan-out with no group metadata to garbage-collect: a dead pod's cursor simply expires.

A freshly started pod begins at the tail rather than replaying history its cache never held. Pub/sub is still used for request/response — reading a value from a specific pod, and collecting per-pod refresh confirmations — where a timeout is the wanted behavior.

> **Rolling upgrade:** refreshes are still published to the old pub/sub channel so sidecars predating the log keep clearing during a rollout. Deploy the version that writes the log before the version that reads it. The publish can be dropped once every sidecar is on a log-reading build.

**Registry cleanup**: Registry entries are removed *before* the app is asked to clear, and put back (with their remaining TTL) if the clear doesn't land — on a transport error, a non-2xx from the app, or all pub/sub retries failing. A stale listing is better than a missing one: the key is still cached, so it should still be visible. After a pod's cache is cleared, that pod's sidecar removes the matching entries from its own registry hash, so the dashboard stops listing keys that are no longer cached. A `keyInfix` clears matching entries (case-insensitive substring, same matching as the `/api/keys` filter); no infix means the whole cache was cleared, so the pod's entire registry hash is dropped. Each sidecar only ever touches its own hash, and pod registration is untouched — the pod still appears in the dashboard, just with no keys until your app registers them again. **Keys come back only when your app calls `/api/registerKey` again**, so if you register keys once at startup, cleared keys won't be listed again until the next restart.

**Redis is only for coordination** — sidecar discovery, message routing, and key metadata. The persistent keys are pod liveness (60s TTL), key metadata (3-day TTL), the refresh log (24h TTL, capped at 1000 entries), and each pod's refresh cursor (24h TTL), all self-cleaning. Nothing piles up.

## Use Cases

**"A config changed in the DB, clear it from all pods"**
You updated a pricing config, but 12 pods still serve the old value from their in-memory cache. Hit Clear in the dashboard and every pod drops the stale entry instantly. No restarts needed.

**"Why is this user seeing stale data?"**
Open the dashboard, find the service, pick the pod, click Get on the cache key. You're looking at the exact value sitting in that pod's memory right now. No port-forwarding, no kubectl exec.

**"We need cache visibility across 5 microservices"**
Each team has their own service with their own in-memory caches. Deploy Shudhi as a sidecar to each, and the dashboard shows all services, all pods, all cached keys in one place.

**"Rolling out a feature flag change"**
Feature flags are cached in memory. Instead of waiting for TTL expiry, clear the flag's cache key and every pod picks up the new value within seconds.

**"Debugging cache inconsistency between pods"**
Pod A returns one value, Pod B returns another. Use the dashboard to Get the value from each pod and see exactly what diverged — without reproducing the issue or adding temporary logging.

shudhi

shudhi Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

shudhi

shudhi Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README