candela

KO | EN

gitlite — search

#python #android #rust #deep-learning #linux #docker #macos #nodejs #java #vue #awesome-list #pytorch #security #open-source

(★ 9)

🕯️ OTel-native LLM observability platform. Trace, cost, and evaluate your LLM calls.

candela Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<div align="center">
  <img src="assets/logo.png" alt="Candela Logo" width="200" />
  <h1>Candela</h1>
</div>
**Open-source, OTel-native LLM Observability & Engineering Platform.**

Candela is a production-grade observatory for your LLM applications. It captures every trace, calculates every cent, and evaluates every output with deep integration into **OpenTelemetry**, **Google Cloud (Vertex AI)**, and the wider GenAI ecosystem.

[![CI](https://github.com/candelahq/candela/actions/workflows/ci.yml/badge.svg)](https://github.com/candelahq/candela/actions/workflows/ci.yml)
[![Go Reference](https://pkg.go.dev/badge/github.com/candelahq/candela.svg)](https://pkg.go.dev/github.com/candelahq/candela)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)

---

## 🚀 Two Ways to Get Observability

Candela offers a dual-mode ingestion strategy to fit any stage of your project:

### 1. Zero-Code Proxy Mode (Quick Start)
Drop Candela into your existing app by just changing your `base_url`. No instrumentation needed.
- **OpenAI**: `http://localhost:8181/proxy/openai/v1`
- **Google Gemini**: `http://localhost:8181/proxy/google/`
- **Gemini (OpenAI-compat)**: `http://localhost:8181/proxy/gemini-oai/v1`
- **Anthropic (via Vertex AI)**: `http://localhost:8181/proxy/anthropic/`
- **Anthropic Vertex (native)**: `http://localhost:8181/proxy/anthropic-vertex` _(Claude Code via Vertex AI)_
- **Anthropic Direct**: `http://localhost:8181/proxy/anthropic-direct` _(client provides API key)_
- **Anthropic Bedrock**: `http://localhost:8181/proxy/anthropic-bedrock` _(Claude via AWS Bedrock + SigV4)_

> **📍 Port Configuration**: Candela uses port `8181` by default. See [Port Configuration](#-port-configuration) for details.

### 2. OTel-Native Agent Mode (Production)
For deep observability into agent frameworks (**ADK**, **LangChain**, **CrewAI**), Candela ingests standard OTLP spans through a custom-built **OTel Collector distro**.

---

## ✨ Key Features

- **🕯️ OTel-Native**: OTLP is our native language. No proprietary SDKs.
- **💰 Real-time Cost Tracking**: Automatic token extraction and USD calculation for OpenAI, Google, and Anthropic — including Anthropic prompt caching (1.25× write at 5m TTL, 2× write at 1h TTL, 0.1× read). Pricing is config-driven via [`pricing.yaml`](pkg/costcalc/pricing.yaml) with `//go:embed`.
- **🚫 Model Allowlist/Blocklist**: `policy.allowed_models` restricts which models users can access, with glob pattern support and audit logging for blocked requests.
- **🏷️ Tag-Based Model Access Control**: Per-user `access_tags`, per-model `required_access`, and tenant isolation via `allowed_tenants` — fail-closed on errors, SAs treated as individuals.
- **🔐 Role-Based Access Control**: Admin vs Developer roles with budget enforcement and grant-based spending with reset countdowns.
- **🔑 Multi-Cloud Auth**: `candela auth login --provider gcp|aws` — native OAuth2 for GCP, SSO/access keys for AWS. No `gcloud` or `aws` CLI dependency.
- **🗄️ Pluggable Storage**: **DuckDB** for high-performance local/edge; **BigQuery** for serverless production scale (with `GROUPING SETS` combined queries); **SQLite** for lightweight dev.
- **📡 SSE Streaming Support**: Captures full streaming responses without interfering with user latency.
- **📦 Single-Binary Edge-Ready**: In-process queuing and processing for low-overhead deployments.
- **🔀 Fan-out Architecture**: CQRS-based design allows writing to multiple sinks simultaneously (e.g., DuckDB + Pub/Sub + OTLP export to Datadog/Tempo/Jaeger).
- **🐳 Production Sidecar**: Minimal `candela-sidecar` binary (<5MB) for container environments — Pub/Sub + OTLP span export with ADC auth.
- **🔗 W3C Trace Context**: Full `Traceparent` / `Tracestate` propagation for unified trace trees across ADK agents and LLM calls.
- **📦 Enrichment SDKs**: Lightweight wrappers for **Python**, **TypeScript**, **Go**, **Kotlin**, and **Rust** — propagate tenant and job metadata via W3C Baggage for multitenant cost attribution.
- **🛡️ eBPF Enforcement**: Kernel-level security ensures all LLM API calls are captured by the proxy — supports transparent iptables redirection with SNI-based routing or Tetragon-based process enforcement for unauthorized egress. [Docs →](https://candelahq.com/governance/ebpf-enforcement/)

---

## 📦 Enrichment SDKs

Candela provides zero-dependency SDKs for 5 languages to propagate tenant and job metadata through your LLM requests. Every SDK injects `X-Candela-*` headers and W3C Baggage — giving you per-tenant, per-job cost attribution automatically.

| Language | Package | Install |
|----------|---------|--------|
| Python | `candela-sdk` | `pip install candela-sdk` |
| TypeScript | `@candelahq/sdk` | `npm install @candelahq/sdk` |
| Go | `github.com/candelahq/candela/sdks/go` | `go get` |
| Kotlin | `com.candelahq:candela-sdk` | Gradle/Maven |
| Rust | `candela-sdk` | `cargo add` |

```python
# Python — works with OpenAI, httpx, requests, or any HTTP client
from candela import CandelaSession

session = CandelaSession(tenant_id="acme-corp", job_id="eval-42")
client = OpenAI(
    base_url="http://localhost:1234/v1",
    http_client=session.httpx_client(),
)
```

```go
// Go — wraps any http.RoundTripper
client := &http.Client{
    Transport: candela.NewTransport(nil,
        candela.WithTenantID("acme-corp"),
        candela.WithJobID("eval-42"),
    ),
}
```

```typescript
// TypeScript — works with fetch, OpenAI SDK, or any HTTP client
const session = new CandelaSession({ tenantId: "acme-corp", jobId: "eval-42" });
const client = new OpenAI({ defaultHeaders: session.headers() });
```

The proxy strips all `X-Candela-*` headers before forwarding upstream — your metadata never leaks to LLM providers.

> 📖 Full SDK docs: [candelahq.com/guides/enrichment-sdks](https://candelahq.com/guides/enrichment-sdks/)

---

## 🐳 Sidecar: Production Container Deployment

For **GKE, Cloud Run, or any container environment**, use the sidecar binary — a stripped-down proxy with zero local dependencies:

```bash
docker pull ghcr.io/candelahq/candela-sidecar:latest

docker run -p 8080:8080 \
  -e GCP_PROJECT=my-project \
  -e VERTEX_REGION=us-central1 \
  -e PUBSUB_TOPIC=candela-spans \
  -e SPAN_FORMAT=proto \
  -e PROVIDERS=google,anthropic \
  ghcr.io/candelahq/candela-sidecar:latest
```

| Env Var | Default | Description |
|---------|---------|-------------|
| `PORT` | `8080` | HTTP listen port |
| `GCP_PROJECT` | — | GCP project for Vertex AI + Pub/Sub |
| `VERTEX_REGION` | `us-central1` | Vertex AI region |
| `PROVIDERS` | all | Comma-separated: `openai,google,anthropic,gemini-oai` |
| `PUBSUB_TOPIC` | — | Pub/Sub topic for span export |
| `SPAN_FORMAT` | `proto` | `proto` (default, BQ-native) or `json` |
| `OTLP_ENDPOINT` | — | OTLP/HTTP endpoint for universal export |
| `OTLP_HEADERS` | — | Comma-separated `key=value` auth headers |
| `CORS_ORIGINS` | `*` | Allowed CORS origins |

**What's excluded** (keeps binary small): Firebase, SQLite, DuckDB, Firestore, ConnectRPC services, runtime management, Next.js UI.

To publish a new release, tag with `sidecar-v*`:
```bash
git tag sidecar-v0.1.0
git push --tags
# → CI builds multi-arch image → ghcr.io/candelahq/candela-sidecar:v0.1.0
```

---

## 🚀 Quick Start

You can get Candela running in less than 60 seconds.

### Option A: Homebrew (Fastest)

Ideal for quick local setup. Uses **SQLite** by default.

```bash
# Install the CLI proxy (macOS / Linux)
brew install candelahq/tap/candela

# Or install the Desktop app (macOS)
brew install --cask candelahq/tap/candela-desktop
```

> **💡 Tip**: `candela start` launches the proxy on `:8181` (proxy + UI at `/_local/`) + `:1234` (LM-compatible endpoint). Point your IDE or app at it immediately.

### Option B: Windows ZIP

Download the ZIP archive that matches your Windows CPU architecture from the
GitHub Release and extract it:

- `candela_<version>_Windows.zip` for Intel/AMD x64 PCs.
- `candela_<version>_Windows_arm64.zip` for Windows on Arm PCs.

Add the extracted folder to `PATH` if you want `candela` available from any
terminal, or run `candela.exe` from the extracted directory:

```powershell
Expand-Archive .\candela_0.1.0_Windows.zip -DestinationPath .\candela
cd .\candela
.\candela.exe start
Start-Process "http://127.0.0.1:8181/_local/"
```

The Windows desktop experience uses the same embedded browser UI as macOS and
Linux. This branch does not install a desktop shell, installer, or launcher
integration; that work is deferred to a desktop-specific branch.

### Option C: From Source

Ideal for contributors or custom builds. Uses **DuckDB** by default.

```bash
# Clone and enter the nix shell (or ensure Go 1.26 is installed)
nix develop

# Start the Candela server (defaults to DuckDB + Port 8181)
nix develop -c go run ./cmd/candela-server

# Start the UI (separate terminal)
cd ui && pnpm install && pnpm run dev
```

> **💡 Quick Start Tip**: Copy `config.example.yaml` to `config.yaml` to use port 8181 and enable all features.

---

## 🛠️ Route an LLM Call

Once Candela is running, point your favorite LLM client at the Candela proxy (Port 8181) to start capturing observability data instantly.

> **🔧 Setup Required**: For Anthropic/Claude via Vertex AI, see [GCP Setup Guide](#-gcp-setup-for-anthropic) below.

### OpenAI Example
```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8181/proxy/openai/v1",
    api_key="sk-..."
)

# Call as usual — Candela handles the rest
response = client.chat.completions.create(...)
```

### Anthropic (via Vertex AI) Example
```python
from anthropic import Anthropic

client = Anthropic(
    base_url="http://localhost:8181/proxy/anthropic",
    api_key="YOUR_GCP_TOKEN" # Uses ADC for GCP authentication
)

response = client.messages.create(...)
```

---

## 🏗️ Architecture

```mermaid
graph TD
    subgraph "Developer Tools"
        JB[JetBrains / Cline]
        GemCLI[Gemini CLI]
        App[App / Agent]
        SDK[OTel SDK]
    end

subgraph "candela"
        LM["LM Compat Listener (:1234)<br/>Unified /v1/models"]
        SC[Span Capture]
        LP[Local Proxy]
        RP[Remote Proxy + Auth]
        SP["SpanProcessor<br/>(Solo Mode)"]
        SQLite[(traces.db)]
    end

subgraph "Local Runtimes"
        Ollama["Ollama (:11434)"]
        VLLM["vLLM (:8000)"]
        LMS["LM Studio (:1234)"]
    end

subgraph "Candela Cloud (Team Mode)"
        Server[Go Backend Server]
        Proxy[LLM API Proxy]
        Processor[Span Processor<br/>Fan-out to Writers]
        DuckDB[(DuckDB)]
        BQ[(BigQuery)]
    end

subgraph "Upstream LLMs"
        VAI[Vertex AI / Google]
        ANT[Anthropic]
        OAI[OpenAI]
        BDK[AWS Bedrock]
    end

### W3C Trace Context Integration

Candela implements W3C Trace Context (via `Traceparent` / `Tracestate` headers) for end-to-end distributed tracing across agent frameworks:

| Header | Direction | Description |
|--------|-----------|-------------|
| `Traceparent` | Client → Proxy → Upstream | Carries trace ID + parent span ID |
| `Tracestate` | Client → Proxy → Upstream | Vendor-specific trace context |
| `X-Request-ID` | Client ↔ Proxy | Request correlation (auto-generated if absent) |
| `X-Session-Id` | Client → Proxy | Session grouping for conversations |

When a caller (e.g., ADK agent) sends a `Traceparent` header, the proxy:
1. Parses the incoming trace ID and parent span ID
2. Creates a child span under that trace
3. Forwards a new `Traceparent` to the upstream LLM with the proxy's span ID

This produces a unified trace tree: `Agent Span → Proxy Span → LLM Call`.

### Storage Architecture (CQRS)

Candela uses a **Command Query Responsibility Segregation** pattern:

| Interface | Purpose | Implementations |
|-----------|---------|-----------------|
| `SpanWriter` | Write-only ingestion | DuckDB, SQLite, BigQuery, Pub/Sub |
| `SpanReader` | Read-only queries | DuckDB, SQLite, BigQuery |
| `TraceStore` | Convenience (both) | DuckDB, SQLite, BigQuery |

The processor fans out writes to **all configured writers** concurrently. Only one reader is active (the primary backend).

---

## 🔧 Port Configuration

Candela uses port `8181` by default.

| Setup | Port | When Used |
|-------|------|-----------|
| **Default** | `8181` | Running `go run ./cmd/candela-server` |

### Quick Setup
```bash
# Copy example config to enable all features on port 8181
cp config.example.yaml config.yaml

# Or create minimal config
cat > config.yaml << EOF
server:
  port: 8181
storage:
  backend: "duckdb"
EOF
```

---

## ⚙️ Configuration

Candela is configured via `config.yaml` (or `$CANDELA_CONFIG`):

```yaml
server:
  host: "0.0.0.0"
  port: 8181

storage:
  backend: "duckdb"  # duckdb | sqlite | bigquery
  duckdb:
    path: "candela.duckdb"
  sqlite:
    path: "candela.db"
  bigquery:
    project_id: "my-gcp-project"
    dataset: "candela"
    table: "spans"         # default: "spans"
    location: "US"         # default: "US"

cors:
  allowed_origins:
    - "http://localhost:3000"
    - "http://localhost:8181"

sinks:
  pubsub:
    enabled: false
    project_id: "my-gcp-project"
    topic: "candela-spans"

proxy:
  enabled: true
  project_id: "default"
  providers:
    - openai
    - google
    - anthropic
    - anthropic-vertex
    - anthropic-direct
    - gemini-oai
    # - anthropic-bedrock
  vertex_ai:
    project_id: "my-gcp-project"
    region: "us-east5"
    anthropic:
      caching_mode: "auto"    # off | auto | system-only
      cache_ttl: 5m           # 5m (default) or 1h
  # policy:
  #   allowed_models:
  #     - provider: openai
  #       models: ["gpt-4o", "gpt-4o-mini"]
  #     - provider: anthropic
  #       models: ["claude-sonnet-4-*"]

catalog:
  backend: "config"           # "config" (default) or "firestore"

pricing:
  discount_percent: 0.0       # global discount (0.15 = 15% off)
  # models:
  #   - provider: openai
  #     model: gpt-4o
  #     input_per_million: 2.00
  #     output_per_million: 8.00

auth:
  dev_mode: false
  # allowed_service_accounts:
  #   - "candela-ci@my-project.iam.gserviceaccount.com"

budget:
  timezone: "America/New_York"  # IANA timezone for period resets (default: UTC)

worker:
  batch_size: 100
  flush_interval: "2s"
```

---

## 🖥️ UI Development

The web interface is a Next.js 16 app in `ui/` with a dark-themed dashboard.

```bash
cd ui
pnpm install          # install deps (included in nix shell)
pnpm run dev           # start dev server → http://localhost:3000
pnpm run build         # production build (includes TypeScript type-check)
pnpm run test:e2e      # run Playwright E2E tests (27+ tests)
pnpm run test:e2e:ui   # Playwright interactive UI mode
```

For **language-agnostic HTTP tests** (works against Go or Rust binary):

```bash
# Start the mock upstream, then:
./test/functional/run.sh --go   # run against Go binary
./test/functional/run.sh --rust # run against Rust binary
```

See [`test/functional/README.md`](test/functional/README.md) for full setup.

The UI communicates with the backend via **ConnectRPC v2** on port `8181`. Pages gracefully handle offline backend state.

> [!TIP]
> **Proto Generation**: Proto definitions live in [`candelahq/candela-protos`](https://github.com/candelahq/candela-protos). Run `buf generate` from the repo root to regenerate stubs from BSR.

---

## 🕹️ candela — Local Development Proxy

`candela` is an auth-injecting proxy + runtime manager for developer machines.
It operates in three modes, with the following subcommands:

```
candela start          Start proxy in background
candela stop           Stop the background proxy
candela status         Show proxy status
candela run [flags]    Run in foreground
candela auth login     Login via browser
candela auth status    Show credential status
candela auth token     Print a fresh access token
candela version        Print version
```

### 🔑 Authentication (Multi-Cloud)

Candela supports multiple cloud providers. Use `--provider` to specify which:

```bash
# GCP — opens browser for Google OAuth
candela auth login --provider gcp

# AWS — SSO login or validates access keys
candela auth login --provider aws

# Show status for all configured providers
candela auth status
# ● gcp: ✅ user@example.com (expires in 58 min)
# ● aws: ✅ arn:aws:iam::123456:user/dev

# Get a GCP token for scripts
candela auth token --provider gcp
```

> **💡 Tip**: If `--provider` is omitted, `candela auth login` infers the provider from your config. If multiple cloud providers are configured, it prompts you interactively. After login, the provider is auto-saved to your config file, and if the proxy is running, you'll be prompted to restart it.

GCP credentials are stored in ADC-compatible format at `~/.config/gcloud/application_default_credentials.json`.
AWS credentials use the standard AWS credential chain (`~/.aws/credentials`, env vars, SSO, or instance roles).

### 🏠 Solo Mode (Zero-Config)

Run local models with full observability — **no cloud account needed**.

```yaml
# ~/.config/candela/config.yaml
runtime_backend: ollama
```

```bash
candela start   # background daemon on :8181 (proxy) + :1234 (LM compat)
```

- Local models via Ollama/vLLM on `:1234`
- Every call traced to `~/.candela/traces.db` (SQLite)
- Management UI at `http://localhost:8181/_local/`

### ☁️ Solo + Cloud Mode (Direct Cloud)

Call Gemini and Claude directly using your **Google identity** — no server deployment needed.

```yaml
# ~/.config/candela/config.yaml
runtime_backend: ollama

providers:
  - name: google
    models: [gemini-2.5-pro, gemini-2.0-flash]
  - name: anthropic
    models: [claude-sonnet-4-20250514]

vertex_ai:
  project: my-gcp-project
  anthropic:
    caching_mode: auto    # off | auto | system-only — injects cache_control (~10x cost reduction)
    cache_ttl: 5m         # 5m (default, 1.25x write cost) or 1h (2x write, ideal for long coding sessions)
```

- Local + cloud models merged into `/v1/models`
- Smart routing: local stays local, cloud goes direct to Vertex AI via ADC
- All calls (local + cloud) traced to SQLite — full observability
- Prerequisites: `candela auth login --provider gcp`

#### With AWS Bedrock

```yaml
# ~/.config/candela/config.yaml
runtime_backend: ollama

providers:
  - name: anthropic-bedrock
    models: [anthropic.claude-3-5-sonnet-20241022-v2:0]

aws:
  region: us-east-1
  # profile: dev-sso  # Optional: AWS CLI profile
```

- Claude via AWS Bedrock with SigV4 request signing
- Prerequisites: `candela auth login --provider aws`
- **Note**: Use exact Bedrock Model IDs (e.g., `anthropic.claude-3-5-sonnet-20241022-v2:0`)

### 🌐 Team Mode (Governance + Budgets)

Connect to a shared Candela server for **budget enforcement, RBAC, and team-wide cost tracking**.

```yaml
# ~/.config/candela/config.yaml
runtime_backend: ollama
remote: https://candela.example.com
audience: "12345678.apps.googleusercontent.com"        # IAP OAuth Client ID
iap_service_account: candela-server@project.iam.gserviceaccount.com  # SA to impersonate for IAP
```

- Local + cloud models merged into `/v1/models`
- Smart routing: local models stay local, cloud models route through Candela server
- **Dual-token IAP auth** when `iap_service_account` is configured (see below)
- Team-wide cost tracking, budget enforcement, and centralized governance
- Prerequisites: `candela auth login --provider gcp`

#### Dual-Token Authentication (IAP)

When `iap_service_account` is configured, the CLI uses a **dual-token strategy** to authenticate through Identity-Aware Proxy:

| Header | Token | Consumed By |
|--------|-------|-------------|
| `Authorization` / `Proxy-Authorization` | IAP ID token (OIDC token from impersonated SA) | IAP at the load balancer |
| `X-Candela-Auth` | User's ADC OAuth2 access token | `candela-server` (identifies the actual user) |

This is necessary because **IAP replaces the `Authorization` header** with its own JWT before forwarding requests to the backend — the original user identity would be lost without the second token.

**Server-side resolution order:**
1. `X-Candela-Auth` header → CLI traffic behind IAP (the typical team mode flow)
2. `Authorization` header → browser or direct traffic (no IAP in the path)

> [!TIP]
> See [docs/candela.md](docs/candela.md) for the full setup guide.

### Unified Model Discovery

The LM-compatible listener on `:1234` merges **local**, **cloud**, and **remote** models into a single `/v1/models` response:

```bash
# JetBrains, Cline, or any OpenAI-compatible client sees everything:
curl http://localhost:1234/v1/models
# → local: llama3.2:3b, mistral:7b (from Ollama)
# → cloud: gemini-2.5-pro, claude-sonnet-4 (direct via ADC)  [Solo + Cloud]
# → remote: gpt-4o (from Cloud Run)  [Team Mode]
```

`/v1/chat/completions` automatically routes to the right backend:
- **Local model** → Ollama/vLLM/LM Studio (no round-trip)
- **Cloud model** → Vertex AI directly (via Google ADC)
- **Remote model** → Cloud Run proxy (with OIDC auth injection)

### Runtime Management UI

Embedded management UI at `/_local/` for controlling local LLM runtimes:

- **Runtime control** — Start/stop Ollama, vLLM, or LM Studio; health monitoring with auto-polling
- **Model management** — List, load, unload, and **delete** models from disk
- **Pull models** — Download models with real-time progress tracking and **cancel** support
- **Traces** — Recent LLM calls with tokens, cost, duration (Solo Mode)
- **Backend discovery** — Auto-detect installed runtimes and show install hints
- **State persistence** — Settings, pull history, and runtime state persisted to `~/.candela/state.db`

All features are accessible via **ConnectRPC** (`RuntimeService`) and the embedded vanilla JS dashboard.

---

## 🗺️ Roadmap

- **Phase 1: Foundation** ✅ (Ingestion, Proxy, Cost Calc, Docs)
- **Phase 2: Storage & Architecture** ✅ (DuckDB, CQRS, BigQuery, Pub/Sub, CORS)
- **Phase 3: Visual Explorer** ✅ (Next.js UI, Dashboard, Traces, Costs, Settings, E2E Tests)
- **Phase 4: Multi-User Platform** ✅ (IAP Auth, Token Budgets, Admin Panel, RBAC)
  - Terraform infrastructure (Cloud Run, BigQuery, Firestore, IAP)
  - Proto-first user/budget/grant schemas with `protovalidate`
  - Role-based access control (admin guard interceptor)
  - Admin UI: user management, budget explainer, audit logs
  - Client-side form validation (`@bufbuild/protovalidate`)
  - Budget enforcement with grant-first waterfall
  - `candela` auth-injecting proxy for developer machines
  - `candela` embedded runtime management UI (`/_local/`)
- **Phase 5: Hardening** ✅ (Security audit, input validation, CORS hardening, race condition fixes)
  - 16 critical/high issues remediated across Go and Rust (XSS, DoS, billing bypass, info leak)
  - 47 new tests (25 unit + 9 integration + 13 Rust) with race detector
  - W3C Trace Context for unified ADK ↔ sidecar trace trees
  - Fan-out sink isolation with per-sink timeouts
  - Rust sidecar CORS enforcement from env config
  - eBPF enforcement: Tetragon + Cilium + iptables transparent proxy (Phases 0–4 shipped)
  - Multi-cloud auth: native `candela auth login` for GCP OAuth2 and AWS SSO
  - Phase A observability hardening: circuit breaker, fuzz tests, runtime coverage
  - Tetragon gRPC audit pipeline with `MultiSink`, graceful shutdown, and hardened error handling
  - Expanded model catalog: Gemini 3.5 Flash, Opus 4.7, Sonnet 4.6, Haiku 4.5
- **Phase 6: Ecosystem & Polish** 📋 (Agent DAGs, Rust SNI listener, Multi-region, Alerting)

---

## 📂 Project Structure

```
candela/
├── buf.gen.yaml                 # Buf code gen config (pulls from BSR)
├── gen/                         # Generated code (Go, TypeScript, BQ schema)
├── cmd/candela-server/          # Server entry point
├── cmd/candela/                 # Local dev proxy + runtime manager
│   ├── lm_handler.go            # Smart model routing (local ↔ cloud)
│   ├── span_capture.go          # Request/response capture middleware
│   ├── traces_handler.go        # /_local/api/traces REST endpoint
│   └── ui/                      # Embedded management UI (vanilla JS)
├── pkg/
│   ├── processor/               # Shared span processor (Solo + Server)
│   ├── storage/                 # Storage interfaces (SpanWriter, SpanReader)
│   │   ├── duckdb/              # DuckDB driver (default, OLAP-optimized)
│   │   ├── sqlite/              # SQLite driver (lightweight)
│   │   ├── bigquery/            # BigQuery driver (production scale)
│   │   └── pubsub/              # Pub/Sub sink (write-only fan-out)
│   ├── proxy/                   # LLM API reverse proxy
│   │   └── ids.go               # W3C Trace Context propagation
│   ├── costcalc/                # Token cost calculation engine
│   ├── connecthandlers/         # ConnectRPC service handlers
│   ├── runtime/                 # Local LLM runtime abstraction (Ollama, vLLM, LM Studio)
│   ├── session/                 # Session resolution (header + fingerprint)
│   ├── notify/                  # Budget threshold notifications
│   ├── billing/                 # Storage-agnostic billing types and Service interface
│   └── ingestion/               # OTel span ingestion
├── cmd/candela-sidecar/         # Production sidecar binary
├── terraform/                   # GCP infrastructure (OpenTofu/Terraform)
│   ├── cloud_run.tf             # Cloud Run + IAP
│   ├── bigquery.tf              # Spans storage
│   ├── firestore.tf             # Users, budgets, grants
│   └── iam.tf                   # Service accounts + roles
├── collector/                   # Custom OTel Collector distro
├── docs/                        # Deep-dive documentation
├── ui/                          # Next.js 16 web interface
│   ├── src/app/                 # App Router pages (dashboard, traces, admin, etc.)
│   ├── src/gen/                 # Generated TS proto stubs
│   ├── src/hooks/               # React hooks (useCurrentUser, useProtoValidation)
│   ├── src/lib/                 # ConnectRPC transport config
│   ├── e2e/                     # Playwright E2E tests (app + admin)
│   └── playwright.config.ts     # Playwright config
├── .github/workflows/ci.yml    # CI pipeline (Go + UI + Playwright)
├── rust/                          # Rust workspace (incremental rewrite)
│   ├── crates/candela-core/       # Domain types, SpanWriter trait
│   ├── crates/candela-proxy/      # LLM reverse proxy engine
│   ├── crates/candela-processor/  # Batched span processor + cost calc
│   ├── crates/candela-storage/    # Pub/Sub + OTLP sinks (WIP)
│   └── bins/candela-sidecar/      # Production sidecar binary
├── sdks/                            # Language-specific enrichment SDKs
│   ├── python/                      # candela-sdk (PyPI)
│   ├── typescript/                  # @candelahq/sdk (npm)
│   ├── go/                          # Go module
│   ├── kotlin/                      # Kotlin/JVM
│   └── rust/                        # Rust crate
├── test/functional/               # Hurl HTTP test suite (Go/Rust parity)
│   ├── mock/upstream.go           # Mock LLM server (OpenAI/Anthropic/Vertex AI)
│   ├── proxy/                     # Proxy round-trip + error tests
│   ├── billing/                   # Budget gate + pricing gate tests
│   ├── compat/                    # /v1/models + compat routing tests
│   ├── security/                  # Header injection + auth bypass tests
│   └── run.sh                     # Runner (--go / --rust flags)
└── config.yaml                  # Server configuration
```

---

## 🌩️ GCP Setup for Anthropic

To use Claude models via Vertex AI, follow these steps:

### 1. GCP Project Setup
```bash
# gcloud CLI is optional — only needed for project creation/API enablement.
# Authentication uses `candela auth login` (no gcloud required).
# To install gcloud (if not already installed):
curl https://sdk.cloud.google.com | bash

# Create a new project (or use existing)
gcloud projects create YOUR-PROJECT-ID

# Set as default project
gcloud config set project YOUR-PROJECT-ID

# Enable required APIs
gcloud services enable \
  aiplatform.googleapis.com \
  compute.googleapis.com
```

### 2. Authentication
```bash
# Native OAuth2 (recommended — no gcloud CLI needed)
candela auth login

# Verify credentials
candela auth status

# Or using gcloud (also works)
gcloud auth application-default login
```

### 3. Enable Claude Models
1. Go to [Vertex AI Model Garden](https://console.cloud.google.com/vertex-ai/model-garden)
2. Search for "Claude"
3. Enable Claude Sonnet 4 and Claude Opus 4
4. Accept Google's terms for Anthropic models

### 4. Update Config
```yaml
# In your config.yaml
proxy:
  enabled: true
  vertex_ai:
    project_id: "YOUR-PROJECT-ID"  # Replace with your actual project
    region: "us-east5"             # or us-central1, us-west1
```

### 5. Test Connection
```bash
# Start Candela
go run ./cmd/candela-server

# Test in another terminal
curl -X POST http://localhost:8181/proxy/anthropic/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer test" \
  -d '{
    "model": "claude-3-51022",
    "messages": [{"role": "user", "content": "Hello!"}],
    "max_tokens": 100
  }'
```

> **💡 Project ID Tips**:
> - Use `gcloud config get project` to see your current project
> - Project IDs must be globally unique (numbers/letters/hyphens only)
> - Billing must be enabled for Vertex AI usage

---

## ☁️ AWS Setup for Bedrock

To use Claude models via AWS Bedrock:

### 1. AWS Account Setup
```bash
# Configure AWS credentials (pick one):
aws configure                     # Access keys
aws configure sso                 # SSO profile
export AWS_ACCESS_KEY_ID=...      # Environment variables
```

### 2. Enable Bedrock Models
1. Go to [Amazon Bedrock Console](https://console.aws.amazon.com/bedrock)
2. Navigate to **Model access** → **Manage model access**
3. Request access to Anthropic Claude models
4. Wait for access to be granted (usually immediate)

### 3. Authenticate
```bash
candela auth login --provider aws

# Verify
candela auth status
```

### 4. Update Config
```yaml
# ~/.config/candela/config.yaml
providers:
  - name: anthropic-bedrock
    models:
      - anthropic.claude-3-5-sonnet-20241022-v2:0
      - anthropic.claude-3-haiku-20240307-v1:0

aws:
  region: us-east-1      # Region with Bedrock access
  # profile: my-profile  # Optional AWS CLI profile
```

### 5. Test Connection
```bash
candela start

# Claude Code with Bedrock
export ANTHROPIC_BASE_URL=http://localhost:8181/proxy/anthropic-bedrock
```

> **⚠️ Model IDs**: Use exact Bedrock Model IDs (e.g., `anthropic.claude-3-5-sonnet-20241022-v2:0`), not friendly names like `claude-3-5-sonnet`.

---

## 🐛 Troubleshooting

### Common Issues

#### Port Already in Use
```bash
# Check what's using the port
lsof -i :8181

# Kill the process or use a different port
kill -9 <PID>
```

#### Backend Not Starting
```bash
# Check if config file is valid
go run ./cmd/candela-server --help

# Verify with minimal config
echo "server:\n  port: 8181" > test-config.yaml
CANDELA_CONFIG=test-config.yaml go run ./cmd/candela-server
```

#### UI Can't Connect to Backend
1. **Check backend is running**: Visit `http://localhost:8181/healthz`
2. **Verify port configuration**: Backend and UI must use same port
3. **Check CORS settings**: Add your UI URL to `cors.allowed_origins`

#### Anthropic/Vertex AI Errors
```bash
# Verify ADC is working
candela auth status
candela auth token

# Check project/region settings (requires gcloud)
gcloud config list

# Test Vertex AI access (requires gcloud)
gcloud ai models list --region=us-east5
```

#### Permission Denied (GCP)
```bash
# Check IAM roles
gcloud projects get-iam-policy YOUR-PROJECT-ID

# Add required role if missing
gcloud projects add-iam-policy-binding YOUR-PROJECT-ID \
  --member="user:YOUR-EMAIL" \
  --role="roles/aiplatform.user"
```

#### OpenCode

If OpenCode hangs, shows stale models, or behaves unexpectedly after config changes, delete the OpenCode database:

```bash
rm -rf ~/.local/share/opencode/opencode.db*
```

Then restart OpenCode. This clears cached state and forces a fresh sync.

> For more OpenCode troubleshooting tips (model not found errors, pricing issues, connection refused), see [docs/proxy.md](docs/proxy.md#-opencode-integration).

### Getting Help
- **GitHub Issues**: [Report bugs](https://github.com/candelahq/candela/issues)
- **Documentation**: Check `docs/` directory for detailed guides
- **Logs**: Run with `CANDELA_LOG_LEVEL=debug` for verbose output

---
## 🤝 Contributing

We are in early development! See [CONTRIBUTING.md](./CONTRIBUTING.md) for local setup instructions and architectural deep dives.

## 📄 License

Apache License 2.0. See [LICENSE](./LICENSE) for details.

candela

candela Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

candela

candela Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README