cnos

KO | EN

gitlite — search

#javascript #python #react #machine-learning #rust #linux #macos #java #windows #cli #kubernetes #awesome-list #vue #open-source

(★ 11)

Voice-orchestrated fleet of CLI agents

cnos 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# cnos

Command a fleet of terminals — blank shells and AI coding agents — with your voice.

cnos launches a grid of real terminals — a **blank shell** by default, or a coding
agent (**claude, codex, hermes, opencode**) — each with its own call sign (`jack`,
`zulu`, `echo`, …), driven by voice or text. Agents can be preloaded with a specialist
**role** prompt and start in **auto mode** with **xhigh effort**:

```
claude --permission-mode auto --effort xhigh
```

Voice runs on **Hugging Face's open speech-to-speech stack** — the same pipeline as
[smolagents/hf-realtime-voice](https://huggingface.co/spaces/smolagents/hf-realtime-voice):
**silero-vad** turn detection → **nvidia/parakeet-tdt-1.1b** transcription →
**google/gemma-4-31B-it** (via Cerebras) as the voice brain → **Qwen3-TTS** spoken
replies. The brain gets tools over your fleet, so you can *command* it ("jack, run
the tests") **and converse with it** ("what is jack doing?" — it reads the terminal
and answers out loud). No keys needed; falls back automatically to **local Whisper**
(whisper.cpp, offline, no cloud) whenever the backend is unreachable.

It also runs **right in your terminal** — `cnos` gives you the same live grid + voice
in any shell (see [Terminal CLI](#terminal-cli-cnos)) — and there's a native
**iOS/iPadOS client** (SwiftUI + SwiftTerm) in [`ios/`](ios/README.md). All three
clients share **one live fleet**: spawn in one, see it in the others.

## Quick start

```bash
npm install      # builds node-pty (needs Xcode CLT — already present on most Macs)
npm start

# optional: the offline voice fallback (used when the realtime backend is unreachable,
# and by the terminal CLI's voice)
brew install whisper-cpp ffmpeg
mkdir -p models
curl -L -o models/ggml-base.en.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.en.bin
```

Then open **http://localhost:4173 in Chrome**.

1. Click **+ Add** for a **blank shell** (the default), or pick an agent type first —
   or say *“new terminal”* (shell), *“new claude terminal”*, *“new codex terminal”*,
   *“new opencode terminal”*… Each terminal gets a call sign.
2. Voice **auto-starts** — allow the mic once. Pick your input from the **🎤 selector**
   in the top bar (toggle the **🔎** panel for a live level meter + log). Then talk:
   - *“jack, build a todo app in react”* → typed into **jack**
   - *“everyone, commit your work”* → broadcast to **all** agents
   - *“new codex terminal”* / *“spawn a claude as programmer”* → new agents
   - *“jack, stop”* → interrupts jack
   - *“what is jack doing?”* → the voice brain reads jack's terminal and **answers out loud**
   - you can interrupt it mid-sentence — it stops talking and listens (barge-in)
3. Or use the command bar at the top, or just click a terminal and type.

The UI is **mobile-responsive** — on a phone the top bar stacks, the usage meter
wraps, and agents render one per row. To open it from your phone, browse to
**`http://<your-mac-LAN-IP>:<PORT>`** on the same Wi-Fi (e.g.
`http://192.168.1.20:4173`). Voice needs a secure context (HTTPS or `localhost`),
so over plain LAN http you drive agents by **typing** in the command bar — voice
still works when you open cnos on the host machine itself.

## Agent types

| Type     | Launches                              | Auto mode                |
| -------- | ------------------------------------- | ------------------------ |
| `shell` *(default)* | your `$SHELL` (e.g. `/bin/zsh`) — a plain blank terminal, no agent | — |
| `claude` | `claude --permission-mode auto --effort xhigh` | auto-accept edits |
| `codex`  | `codex --sandbox workspace-write --ask-for-approval on-request` | auto (workspace-write) |
| `hermes` | `hermes`                              | —                        |
| `opencode` | `opencode` (TUI)                    | follows your `opencode.json` (its default is permissive; add `--auto` via `CNOS_OPENCODE_ARGS` for unattended runs) |

**`shell` is the default** — *“new terminal”* / **+ Add** opens a blank shell; name a
type (*“new claude terminal”*, or the top-bar selector) to launch a coding agent. Role
prompts apply to agents only, not blank shells. Only the CLIs you have installed will
launch; others report a clear "not installed" message. Override any type with
`CNOS_<TYPE>_BIN` and `CNOS_<TYPE>_ARGS` (`CNOS_SHELL_BIN` for the shell).

## Working directory

The **📁 folder button** in the top bar picks the directory new agents spawn into —
click it to browse (or paste a path / `~/dev/app`, then **Use this folder**). It
applies to every new agent (manual **+ Add** and voice *"new terminal"*). The
choice is remembered across reloads; the default is `CNOS_WORKDIR` (your home dir).
Each agent card shows the directory it's running in.

## Themes & display

Click **🎨** in the top bar to restyle everything — applied instantly and remembered
across reloads:

- **16 color schemes** — cnos Dark/Light, Solarized (dark & light), Dracula, Nord,
  Tokyo Night, Catppuccin Mocha, Gruvbox, One Dark, Monokai Pro, GitHub Dark,
  **Ubuntu**, High Contrast, **Matrix**, and **Synthwave '84**. Each recolors the
  whole UI *and* the terminal palette (all 16 ANSI colors).
- **Font** — choose a monospace family (JetBrains Mono, Fira Code, IBM Plex Mono,
  Ubuntu Mono, Hack, Cascadia Code, Source Code Pro, Geist Mono, …) or leave it on
  the theme's recommended font. These fonts are **bundled** (`public/fonts/` +
  `public/fonts.css`), so they work offline regardless of what's installed.
- **Text size** — scale the terminal text from Extra Small to Extra Large.

Click **⛶** (or press F11) to toggle **fullscreen**. Themes are data in
`public/themes.js` (`CNOS_THEMES` — add your own there); the engine lives in
`public/app.js`. Terminal title bars are compact and the UI uses restrained
font-weights for a clean, low-chrome look.

## Voice engine — the HF speech-to-speech stack

The browser streams your mic (16 kHz PCM over a WebSocket) to an open
[speech-to-speech](https://github.com/huggingface/speech-to-speech) backend that
chains four open models — every stage swappable or self-hostable:

```
 you speak
   ├─ VAD   silero-vad                      detects when you start/stop talking
   ├─ STT   nvidia/parakeet-tdt-1.1b        transcribes the turn
   ├─ LLM   google/gemma-4-31B-it @ Cerebras  the voice brain — replies + calls fleet tools
   └─ TTS   Qwen/Qwen3-TTS-12Hz-1.7B        speaks the reply back (barge-in supported)
```

The brain sees your live fleet roster and drives it through **tool calls** executed
in the browser: `send_command`, `spawn_terminal`, `control_terminal`,
`read_terminal`, `get_fleet`. Dictated commands are passed to terminals **verbatim**
(it's instructed not to rephrase), and confirmations are terse. Because it can read
terminals, questions like *“is zulu done?”* get real spoken answers.

By default cnos uses the public demo backend documented in the space's README (no
account needed; it may queue at capacity). Self-host the backend and point
`CNOS_S2S_URL` at it to control every stage, pick the reply voice with
`CNOS_S2S_VOICE` (Aiden, Ryan, Dylan, Eric, Ono_Anna, Serena, Sohee, Uncle_Fu,
Vivian), or force the fully-local path with `CNOS_VOICE_ENGINE=classic`.

## Voice grammar (classic fallback + CLI)

When the realtime backend is off or unreachable, the browser falls back to the
classic engine — energy VAD → MediaRecorder → **local whisper.cpp** → a small
deterministic grammar (no LLM, fully offline). The `cnos` terminal CLI always
uses this path today:

The first word is the target. `everyone`, `all`, `team`, `fleet` broadcast.
Common **mishearings** of the agent names are tolerated — *cloud*/*clawed* → claude,
*codec*/*code&nbsp;x* → codex, *hermies* → hermes, *open&nbsp;code*/*opencold* → opencode.

## Loop prompts

Launch any agent **preloaded with a prompt** so it boots straight into a mode or
role. Pick one in the top-bar **Loop Prompt** selector before **+ Add**, or say it —
e.g. *"new terminal, loop"* or *"new codex terminal, architect"*. The prompt is
passed as the agent's first/original prompt, and the agent card shows a badge. The
generic **loop** prompt is ours; the rest are the poc-engineering roles from
[mitsuhiko/agent-prompts](https://github.com/mitsuhiko/agent-prompts/tree/main/poc-engineering).

Bundled prompts (files live in `prompts/`):

| Say | Prompt | Prompt file |
| --- | ------ | ----------- |
| `loop` / `iterate` / `auto` | Loop | `loop_agent` — generic non-stop work loop |
| `orchestrator` / `manager` | Orchestrator | `orchestrator_agent` — delegates to managers → subagents, re-evaluates, repeats |
| `programmer` | Programmer | `implementation_agent` |
| `architect` | Architect | `software_architect_agent` |
| `designer` / `architecture` | Architecture | `architecture_design_agent` |
| `analyst` | Analyst | `problem_analysis_agent` |
| `planner` / `plan` | Planner | `detailed_planning_agent` |
| `breakdown` / `tasks` | Task breakdown | `task_breakdown_agent` |
| `lead` / `research` | Research lead | `programming_lead_agent` |

Add your own: drop a `.md` into `prompts/` and register it in `PROMPT_SPECS` (`server.js`).

For multi-agent work, launch agents with the **Orchestrator** loop prompt (it
delegates to its own subagents) — or just spawn several agents and direct them by
voice/text.

## Terminal CLI (`cnos`)

Prefer to stay in the terminal? cnos ships a CLI that drives the **same fleet** over
the same WebSocket — spawn in the terminal and it shows up in the browser too.

```bash
npm link        # once, to put `cnos` on your PATH (or: npm install -g .)
cnos            # opens the live grid — already listening for your voice
```

`cnos` with no args opens the **web app, in your terminal**: a **live grid** of agent
panes (all tiled and visible at once, each a real terminal so claude's own TUI renders
inside it) plus a **command bar** at the bottom. Each pane's title bar shows its call
sign **in bold, uppercase** so you can tell terminals apart at a glance (with a `▸` on
the focused one); the grid re-tiles as you resize the window.

**Voice is hands-free and on from launch** — the grid opens *listening*, so you can drive
the whole fleet without touching the keyboard. The bottom bar shows a live **mic-level
meter** and a **listening / hearing you / transcribing** indicator, plus a flash of what
it heard and did, so you can see it working. Just talk:

- *“new claude terminal”* · *“new terminal”* — spawn an agent (or a blank shell)
- *“jack, build a login page”* — route to jack · *“everyone, stop”* — broadcast
- *“mute”* / *“listen”* — pause or resume voice · *“zoom”* / *“zoom out”* · *“detach”*

You can also **type** the same commands — routed through the same grammar as the web
(so *"new claude terminal"*, *"jack build a login page"*, *"everyone stop"*, *"kill
zulu"* all work):

| Key | Action |
| --- | ------ |
| *type* + Enter | run the command (routes by name, or to the current target) |
| **Ctrl-V** | mute / unmute **hands-free voice** (mic → local Whisper → routed) |
| **Tab** | cycle the command target (all / each agent) |
| **← / →** | focus a pane (also sets the target) |
| **Ctrl-Z** | zoom into the focused pane for hands-on typing (Esc to return) |
| **Ctrl-K** / **Ctrl-D** | kill the focused pane · detach (fleet keeps running) |

Voice uses `ffmpeg` to capture your mic and the server's `/transcribe` (local Whisper)
— same as the browser. It auto-picks a real input device (skipping virtual ones like
BlackHole); list them with `cnos mics` and override with `--mic <index>`. Start without
voice with `cnos --no-voice` (or `CNOS_NO_VOICE=1`).

One-shot subcommands (scriptable):

```bash
cnos new claude --prompt loop --cwd ~/dev/app   # spawn (prints the call sign)
cnos ls                                          # list terminals
cnos send jack "build a login page"              # type a command into jack ("all" broadcasts)
cnos stop jack       # interrupt  ·  cnos kill jack  ·  cnos usage  ·  cnos mics
cnos serve           # run the server in the foreground
```

It auto-starts the server if one isn't running (or target a remote one with
`--server ws://host:port` / `--port N`). The theme picker and fullscreen stay
browser-only; in the terminal you get your terminal's own font and colors.

## Configuration

| Env var               | Default                                   | Meaning                              |
| --------------------- | ----------------------------------------- | ------------------------------------ |
| `PORT`                | `4173`                                    | HTTP/WebSocket port                  |
| `CNOS_WORKDIR`        | your home dir                             | default working dir for agents       |
| `CNOS_<TYPE>_BIN`     | the CLI name (`claude`, `codex`, …)       | path to that agent's CLI             |
| `CNOS_<TYPE>_ARGS`    | per-type flags (see table; `shell` = none)| flags that terminal type launches with |
| `CNOS_SHELL_BIN`      | `$SHELL` (e.g. `/bin/zsh`)                | shell used for blank terminals       |
| `CNOS_AIUSAGE_BIN`    | `~/.local/bin/ai-usage`                   | path to the `ai-usage` CLI (usage meter) |
| `CNOS_S2S_URL`        | HF's public speech-to-speech demo backend | realtime voice backend (self-host [speech-to-speech](https://github.com/huggingface/speech-to-speech) and point here) |
| `CNOS_S2S_VOICE`      | `Aiden`                                   | Qwen3-TTS reply voice                |
| `CNOS_VOICE_ENGINE`   | `realtime`                                | `classic` forces the local-whisper path |
| `CNOS_S2S_QUEUE_WAIT_MS` | `10000`                                | how long to wait for a voice slot when the backend is at capacity |
| `CNOS_WHISPER_BIN`    | auto (`whisper-cli`)                      | path to the whisper.cpp binary (fallback engine) |
| `CNOS_WHISPER_MODEL`  | `models/ggml-base.en.bin`                 | Whisper model file (fallback engine) |

Examples: `CNOS_WORKDIR=~/dev/myrepo npm start` ·
`CNOS_CLAUDE_ARGS="--permission-mode acceptEdits --effort high" npm start`

## Usage meter

A strip under the top bar shows **Claude + Codex** plan usage, powered by the local
[`ai-usage`](https://). The **basics** sit in the strip — each provider's rate-limit
windows (5h / 7d) with a bar. Click the **▸** arrow to expand the **full detail**:
context-window use, token throughput (avg/peak per minute, cache-hit %), the active
session, spend, and burn-rate trends with sparklines. Click the strip to refresh now;
it also polls every 60s.

`GET /api/usage` runs `ai-usage --once --json` on the server (cached ~25s, serves the
last good snapshot on error). Everything is **read-only** — `ai-usage` reads your own
caches/credentials and never modifies them. It must be installed on the machine running
the server (it lives at `~/.local/bin/ai-usage`; point elsewhere with `CNOS_AIUSAGE_BIN`).
If it isn't present, the strip simply stays empty.

## How it works

```
  Browser   ─┐
  cnos CLI  ─┼─ WebSocket (spawn · input · command · control) ──┐
  iOS app   ─┘   + POST /transcribe  (fallback: local Whisper)   │
      │                                                          ▼
      │ 16 kHz PCM over WS                  server.js (Node + Express)
      ▼                                       │                     │
  HF speech-to-speech backend        ffmpeg + whisper.cpp        node-pty
  silero-vad → parakeet STT            (fallback engine)            │
  → gemma-4 @ Cerebras (tools) ──────── tool calls come back ───────▼
  → Qwen3-TTS reply audio     shell / claude / codex / hermes   (× N — one shared fleet)
```

**Three clients — the browser, the `cnos` terminal CLI, and the iOS app — all speak
the same WebSocket protocol to one server, so they share a single live fleet:** spawn
in one and it shows up in the others; output is broadcast and scrollback is replayed
on connect. The Node server owns the PTYs; `POST /api/voice/session` brokers realtime
voice sessions with the speech-to-speech backend, and the browser streams audio to it
directly. The voice brain acts on the fleet only through tool calls, which the browser
executes as ordinary fleet messages (`command`, `spawn`, `control`) — so voice, typed,
CLI, and iOS actions all take the same path. In fallback mode, spoken commands are
transcribed by local Whisper and parsed by the classic grammar instead. Claude's
first-run "trust this folder" prompt is auto-accepted. Front-end assets in `public/`
(incl. `themes.js`, `fonts.css`, and the bundled fonts) are served with
`Cache-Control: no-cache`, so edits reach the browser on a normal reload.

## ⚠️ Security

Agents run in **auto mode** (auto-accept edits), not full bypass — but they can
still make changes without prompting. Run cnos on a workdir you trust, ideally a
sandbox. **The server listens on all network interfaces** — so other devices on
your LAN (like your phone) can reach it; only run it on a network you trust.

**Voice privacy:** with the realtime engine, mic audio (and terminal excerpts the
brain reads via `read_terminal`) stream to the configured speech-to-speech backend —
HF's public demo endpoint unless you self-host it (`CNOS_S2S_URL`). For fully local,
offline voice set `CNOS_VOICE_ENGINE=classic`: whisper.cpp transcribes on-device and
audio never leaves your machine.

cnos

cnos 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

cnos

cnos 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README