KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#typescript #ai-agents #developer-tools #cli #codex #claude-code #react #mcp #open-source #deepseek-harness #dsh-plugin #ai

kid-mode-jarvis

★ 38 Open GitHub ↗

A JARVIS-style Gemini Live voice console for kids - server-authoritative and safety-walled. The AI can never award a real-world achievement; only a parent can.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Kid Mode — a JARVIS for your kid

A real-time **voice + screen** AI console for a young child, built on
[`google-gemini/live-api-web-console`](https://github.com/google-gemini/live-api-web-console).
It boots a tablet straight into an Iron-Man-style **JARVIS** who teaches, runs an earned
**rank ladder**, controls the house music, and can place a **real call to Dad** — while a
Node server, not the language model, holds authority over everything that has to be *true*.

> **The one idea that makes it safe:** **Gemini performs the character. The server controls reality.**
> The voice model can improvise a mission, tell a story, or turn a lesson into an experiment.
> It **cannot** decide whether the child answered correctly, award a rank, mark a physical
> project complete, claim a song started, or invent a real-world fact. *Rope on actions, rails on truth.*

This is the surface a child actually uses: a locked-down tablet with no browser, no app drawer,
no YouTube — just the console.

---

## What it does

- **Native Gemini Live conversation** with a British-RP, film-JARVIS voice; barge-in + deliberate mute.
- **Grounded answers only.** Any real-world factual question is routed to a grounding LLM (see below)
  or gets an in-character "that's a question for your dad" — the voice model is never a source of truth.
- **Seven-rank service ladder** (Cadet → Chief Engineer) graded **server-side** from an authored
  curriculum. Ranks unlock real capabilities (house music, calling Dad). The top rank is gated on a
  **physical build**, not trivia — a probabilistic transcript can't declare a real object exists.
- **Learn / Do / Build:** exploration bays, hands-on experiments and field missions, and sparring rounds.
- **Dad Link:** a real two-way **WebRTC voice call** to a parent, wrapped in a "satellite search" theater.
- **Dad-authored dossier:** true, rank-gated files about the child's father, revealed as a reward.
- **Approval-gated memory:** the model can *queue* a literal quote/moment; a parent reviews before
  anything is kept. A separate **Dad brief** collects milestones/worries/funny moments.
- **Whole-house music** via Home Assistant (optional), with deterministic start/stop the model can't fake.
- **Child-safety + adult-privacy walls:** a categorical safety system-prompt plus server-side denylists.

## Architecture — one engine, two halves

```
  Tablet browser (HTTPS, secure context)              Cloud / your host
  ┌──────────────────────────────┐                    ┌────────────────────────┐
  │  React app (this repo)        │   WebSocket        │  Google Gemini Live    │
  │   • Gemini Live voice         │◄──────────────────►│  (the PERFORMER)       │
  │   • JARVIS persona + tools    │                    └────────────────────────┘
  │   • on-screen cards           │
  └──────────────┬───────────────┘                    ┌────────────────────────┐
                 │ same-origin /api/*      Bearer key  │  Grounding LLM gateway │
                 ▼                        ┌────────────►│  (OpenAI-compatible    │
  ┌──────────────────────────────┐       │             │   /v1/chat/completions)│
  │  server/index.js (Node)       │──────┘             │  = the source of TRUTH │
  │   • serves the app over HTTPS │                    └────────────────────────┘
  │   • holds ALL secrets/keys    │
  │   • grades ranks, persists    │                    ┌────────────────────────┐
  │     state, enforces lockouts  │──── optional ─────►│  Home Assistant (music)│
  │   • proxies grounded answers  │                    └────────────────────────┘
  └──────────────────────────────┘
```

**Why the server exists:** the browser must not hold API keys, `getUserMedia` needs a secure
(HTTPS) context, and every truth-bearing decision must live somewhere the model can't reach. The
Node server does all four — key isolation, HTTPS serving, durable server-graded state, and a proxy
to the grounding LLM.

## ⚠ The mic gotcha — read this first

Browser voice input (`getUserMedia`) works **only in a secure context — HTTPS or `localhost`.**
Serving the app over a plain LAN IP silently blocks the mic: the app loads, the voice just never
hears anything. You **must** serve it over HTTPS. The easy path is a [Tailscale](https://tailscale.com)
cert for your host (`tailscale cert <host>.<tailnet>.ts.net`); any real HTTPS cert works too. This
bites every browser-voice app, not just this one.

## Requirements

- **A tablet — and the model matters more than you'd think.** Don't buy on price alone; buy on
  these four capabilities, because a tablet that misses any of them fails *silently* (the console
  loads, the voice just never works):
  - **A current Chromium/Android WebView** — Gemini Live is a modern-web workload.
  - **`getUserMedia` in a secure context** — see the mic gotcha above. This is the usual killer.
  - **WebRTC**, if you want Dad Link's voice call.
  - **A powered mount** you intend to leave plugged in — Android power-saving throttles a
    discharging tablet until the whole system *looks* broken (slow, glitchy, drops Wi-Fi,
    vanishes from the network). It has cost a full debugging session. It was the charger.

**Tested:** Samsung Galaxy Tab A9+ (`SM-X210`, ~$250). **Recommended:** its successor, the
  **Galaxy Tab A11+**. **Evaluated and rejected:** cheap Fire HD tablets — the ~$60 wall-tablet
  plan is tempting and it's where the mic dies. Any tablet meeting the four points above should
  work; the two Samsungs are simply the ones with real hours on them.
- **[Fully Kiosk Browser](https://www.fully-kiosk.com/)** for lockdown — the **PLUS** license
  (~one-time $8) enables mic access. (Day-one fallback: plain Chrome over HTTPS gets you the mic
  without kiosk lockdown.)
- **Tailscale** (or any way to serve trusted HTTPS to the tablet).
- **A Gemini API key** (Google AI Studio) for the Live voice.
- **A grounding LLM endpoint** that speaks OpenAI-style `POST /v1/chat/completions` — this can be
  OpenAI, a local runner (Ollama/LM Studio with an OpenAI shim), or a self-hosted agent. It's what
  answers the child's real questions with authority. *(Grounding is optional — without it, factual
  questions simply defer to "ask your dad.")*
- Node 22+.

## Quick start

```bash
# 1. Front-end deps
npm install

# 2. Back-end deps + config. Your Gemini key goes HERE, server-side — the server
#    mints a short-lived ephemeral token per connect, so the key never reaches
#    the browser. Do NOT put it in a root .env; see the note below.
cd server
npm install
cp .env.example .env          # GEMINI_API_KEY, CONSOLE_TOKEN, grounding endpoint, SSL cert paths
cp kid-facts.example.yaml kid-facts.yaml                 # your family facts (gitignored)
cp skills/dad_dossier.example.json skills/dad_dossier.json   # optional dad dossier (gitignored)

# 3. Build the front-end, then run the server (serves build/ over HTTPS)
cd ..
npm run build
cd server && npm start
```

> **Don't bake your key into the bundle.** A root `.env` with
> `REACT_APP_GEMINI_API_KEY` still works and is kept only as a **dev fallback** —
> it compiles your key into `build/`, which is public JavaScript that anyone who
> loads the page can read. The server-side path above (`GEMINI_API_KEY` in
> `server/.env` → `/api/gemini_token`) is the one to use. See **Security model §3**.

Then open `https://<your-host>:8443` on the tablet, arm kid mode from the console (see **SETUP.md**),
tap the glyph, and say hello.

**➡ Before you run it for real, do [SETUP.md](./SETUP.md)** — it walks the ~11 things you personalize
(your child's name, your town map for Dad Link, your family facts, your Home Assistant entity, your
safety denylist, etc.) and the tablet/kiosk lockdown steps.

**➡ Then do [AUTHORING.md](./AUTHORING.md) — this is the actual project.** SETUP gets it running in
forty minutes and leaves you with a very expensive Iron Man soundboard. AUTHORING is the content you
write: the persona, the dad dossier the ladder pays out in, the vocabulary cards, the field missions
and experiments, the trivia, and your family's inside jokes. Everything shipped here is a
`[BRACKETED_CAPS]` placeholder or a starter set — none of it is anyone's real child.

## When something feels wrong: read the transcripts

The voice model talks **browser-direct to Google** — your server is not in that conversation and
cannot see it. What it keeps is a *tee* of both sides at `server/transcripts/YYYY-MM-DD.jsonl`, and
that file is the only real record of what happened. Render it:

```bash
python3 scripts/parse_transcript.py server/transcripts/$(date +%F).jsonl --tz -5
```

Each line carries `t0` (when the utterance started) and `at` (when the server got it). The gap
between them is **latency, not silence** — which is why "it ignored him" is usually a slow tool
call, and why you'd never find it by reading code. Full method, and the mundane causes to check
after (stale bundle, the charger, the mic's HTTPS context), in **[SETUP.md §8](./SETUP.md)**.

## Security model — keeping YOUR child out of YOUR data

Every wall here is built for one job: a child holds this tablet all day, so **the child must never
be able to reach anything sensitive — yours or anyone's.** When you stand it up, these protections
point at *your* family's data. Configure them (see SETUP.md); don't weaken them.

1. **The parent is the only operator (fail-closed).** Arming/disarming, switching mode, approving a
   memory, overriding a lockout, and reviewing generated trivia all require your **`CONSOLE_TOKEN`**.
   The child's same-origin browser has no token, so it cannot do any of these — every gated route
   returns 401 without it. Keep the token secret; it is the master key.
2. **The grounding model must not know your life. ⚠ Most important.** A child's live questions go to
   `KID_ASK_MODEL`. Point it at a **memory-less / isolated** model — NEVER at a personal assistant
   that has your notes, email, or history, or a child can fish your private life out of it. The
   output passes a **family-privacy denylist** (`KID_ANSWER_DENYLIST` — your private names/topics)
   and a **child-safety denylist**, and anything crossing a line becomes an in-character deferral.
   Those are backstops; the memory-less model is the real wall.
3. **The API key never reaches the browser.** The server mints a fresh, short-lived **ephemeral
   token** per connect from a `GEMINI_API_KEY` it holds; the real key isn't in the public client
   bundle. (A static `REACT_APP_GEMINI_API_KEY` remains only as a dev fallback.)
4. **Child memory is isolated and parent-reviewed.** Remembered moments go to a dedicated `child`
   bank — never a shared/operator bank — and only *queue*; nothing is kept until you approve it.
5. **Nothing family-specific is in this repo.** Family facts (`server/kid-facts.yaml`), the dad
   dossier, runtime state, transcripts, and all secrets are **gitignored** — you supply them locally
   from the `.example` templates. Never commit the real files.

## Credits & license

Built on `google-gemini/live-api-web-console` (Apache-2.0, © Google LLC). This project retains that
license — see [LICENSE](./LICENSE). The Kid Mode layer (persona, rank ladder, server authority,
Dad Link, safety walls) is the added work described above.

kid-mode-jarvis

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README