KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #typescript #ai-agent #developer-tools #claude-code #codex #deepseek-harness #ai-agents #react #mcp #rust

dbxlite

★ 82 Open GitHub ↗

Open-source SQL workbench powered by DuckDB. Query local files, BigQuery, and Snowflake right in your browser. Drop-in UI for duckdb -ui. AI SQL assistant included.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# dbxlite

[![CI](https://github.com/hfmsio/dbxlite/workflows/CI/badge.svg)](https://github.com/hfmsio/dbxlite/actions)
[![npm](https://img.shields.io/npm/v/dbxlite-ui.svg)](https://www.npmjs.com/package/dbxlite-ui)
[![npm downloads](https://img.shields.io/npm/dm/dbxlite-ui.svg)](https://www.npmjs.com/package/dbxlite-ui)
[![Docker Hub](https://img.shields.io/docker/v/hfmsio/dbxlite?sort=semver&logo=docker&label=docker%20hub&color=2496ED)](https://hub.docker.com/r/hfmsio/dbxlite)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://raw.githubusercontent.com/hfmsio/dbxlite/main/LICENSE)
[![Node](https://img.shields.io/badge/node-%3E%3D18-green.svg)](https://raw.githubusercontent.com/hfmsio/dbxlite/main/package.json)

A SQL workbench that handles real data, in your browser. WASM mode reads local files through File System Access handles on demand instead of loading them into RAM, so you can query files larger than browser memory. The result grid streams arbitrarily large sets via virtual scrolling. One UI for DuckDB, BigQuery, and Snowflake. Server mode is a drop-in replacement for `duckdb -ui`.

**Try it now (no install):** [sql.dbxlite.com](https://sql.dbxlite.com)

## Highlights

- **Files larger than RAM in WASM mode.** DuckDB reads local files through File System Access handles on demand (Parquet by range reads) instead of loading them into memory, so multi-gigabyte CSV / Parquet / JSON files open without hitting the 2-4 GB browser RAM ceiling. The database itself is in-memory, so `CREATE TABLE` results are session-only.
- **Virtual-scrolled result grid.** Streams arbitrarily large result sets without freezing the renderer.
- **One UI for three engines.** DuckDB, BigQuery, and Snowflake share the same explorer, query editor, grid, and export flow. The Snowflake explorer mirrors Snowsight (databases, schemas, tables, column preview, compute status, query history).
- **Server mode is a drop-in for `duckdb -ui`.** Run dbxlite as the UI for your local DuckDB CLI. Full extensions, unlimited memory, direct filesystem access.
- **Cross-engine Parquet export.** Pull data out of BigQuery or Snowflake into a portable Parquet file. Same export path for every engine.
- **Excel sheets as first-class.** Multi-sheet `.xlsx` files appear in the explorer with every sheet's columns and types surfaced as proper tables.
- **Cell viewer for large values.** Press `Enter` on any cell with long text or structured data to open it in an inline viewer.
- **Polished by default.** Ten color themes, light/dark modes, multi-level explorer, configurable layout, fast keyboard navigation.

## Privacy & license

- **Local files stay local.** The File System Access API gives DuckDB a handle, not an upload. The browser never copies your file off your machine.
- **Credentials encrypted at rest** in IndexedDB (AES-GCM). Threat model: protects against casual localStorage reads, not XSS or filesystem access on your machine.
- **No account, no signup, no telemetry.**
- **Open source, MIT.**

## Quick Start

### WASM Mode (zero install)

The hosted version at **[sql.dbxlite.com](https://sql.dbxlite.com)** is the same app served as static assets, with DuckDB compiled to WebAssembly. The URL serves UI only; queries run locally on your machine. Or run a copy yourself: `pnpm dev`.

### Server Mode (full DuckDB power)

Use dbxlite as a drop-in replacement for `duckdb -ui`:

```bash
# Start the local asset server (downloads from npm on first run)
npx dbxlite-ui                              # Serves UI on port 8080

# In another terminal, launch DuckDB with the local UI
export ui_remote_url="http://localhost:8080"
duckdb -unsigned -ui
```

Prefer a persistent install? Run `npm install -g dbxlite-ui`, then `dbxlite-ui`.

> **Troubleshooting:** if `npx dbxlite-ui` reports `command not found` or seems to run a stale build (common with conda's bundled Node or a cached npx entry), clear the cache and pin the latest version:
> ```bash
> npx clear-npx-cache && npx --yes dbxlite-ui@latest
> ```

Open http://localhost:4213 in your browser. (DuckDB's `ui` extension serves its UI on port 4213 by default; `ui_remote_url` points it at dbxlite's assets on 8080.) You get full native DuckDB with all extensions, unlimited memory, and direct filesystem access.

**With an existing database:**
```bash
duckdb mydata.duckdb -unsigned -ui
```

> The `-unsigned` flag is required for custom UI URLs (DuckDB security measure).

### Mode Comparison

| | Server Mode | WASM Mode |
|---|-------------|-----------|
| **Memory** | Unlimited | ~2-4GB working set; larger files read on demand via file handles |
| **Extensions** | All (httpfs, spatial, iceberg, etc.) | Limited subset |
| **Filesystem** | Direct access | File handles only |
| **BigQuery** | Via DuckDB extension | Browser OAuth connector |
| **Snowflake** | Via DuckDB extension | Browser OAuth connector (SQL REST API v2) |
| **Install** | DuckDB CLI required | Zero install |
| **Offline** | Requires CLI | Works after first load |

---

## How to use it

### Local files (the core)

Query CSV, Parquet, Excel, JSON, and JSONL. Local files via the File System Access API stay on disk; remote URLs are read via DuckDB's `httpfs` extension. The editor is Monaco with SQL autocomplete; results stream via Arrow IPC.

Server mode gives you full native DuckDB with every extension. WASM mode runs the same engine in a Web Worker; because DuckDB reads files through their File System Access handle on demand, it can query files much larger than browser RAM.

### Warehouses (optional)

BigQuery and Snowflake connect directly from the browser. BigQuery talks to GCP because Google APIs serve CORS. Snowflake doesn't, so it routes through a thin proxy that strips response headers Snowflake adds (CSRF cookies, content-encoding) and rewrites the URL. Both OAuth 2.0 PKCE and Programmatic Access Tokens are supported. PATs are the lower-friction path for users without ACCOUNTADMIN.

### AI assistant (optional, opt-in)

Bring your own key for OpenAI, Anthropic, Gemini, or Groq, or use Snowflake Cortex if you're already on Snowflake. Keys are encrypted with AES-GCM, and a one-time consent dialog gates the first send to each external provider. Toggle with `Cmd/Ctrl+Shift+A`. **AI is off until you enable it.**

### Sharing

Queries are shareable as URLs that run on click: `?example=`, `?sql=`, `?share=gist:...`, plus `&theme=`, `&layout=`, and `&run=true`.

---

## Development

```bash
git clone https://github.com/hfmsio/dbxlite.git
cd dbxlite
pnpm install
pnpm build

# Run with local DuckDB (Server mode)
cd apps/cli && node scripts/build.js && node dist/cli.js
# Then: export ui_remote_url="http://127.0.0.1:8080" && duckdb -unsigned -ui

# Run standalone (WASM mode)
pnpm dev  # Opens http://localhost:5173
```

Requirements: Node.js 18+, pnpm 10+ (the `packageManager` field pins the exact version; run `corepack enable` to use it automatically). On pnpm 11+, dependency build scripts (`esbuild`, `@swc/core`) are pre-approved via `allowBuilds` in `pnpm-workspace.yaml` — no manual `pnpm approve-builds` step needed.

## Docker (self-hosted)

dbxlite is entirely client-side (DuckDB runs as WASM in the browser), so a deployment is a single static-web container with no database or backend to wire up. Ideal for internal use such as SQL training: point everyone at one URL and they get a full SQL workbench, sandboxed in their own browser tab.

```bash
# Prebuilt multi-arch image (no build needed), then open http://localhost:8080
docker run -p 8080:80 ghcr.io/hfmsio/dbxlite      # or: hfmsio/dbxlite (Docker Hub)

# Or build it yourself from source
docker compose up -d                               # build + run on :8080
docker build -t dbxlite . && docker run -p 8080:80 dbxlite
```

The image is a small multi-stage build (Node compiles the assets, nginx serves them; unused DuckDB wasm bundles are dropped). The nginx config sets the cross-origin isolation headers DuckDB needs for OPFS persistence and file export:

- `Cross-Origin-Opener-Policy: same-origin`
- `Cross-Origin-Embedder-Policy: credentialless` (keeps the context isolated while still letting `httpfs` fetch remote files/CSVs)
- `Cross-Origin-Resource-Policy: cross-origin`

Core querying (local files, remote URLs, all the built-in examples) works fully offline of any cloud. BigQuery/Snowflake OAuth needs redirect URIs registered for your own origin and is not required for the DuckDB workflow.

### Server mode against the container

The container serves the same UI assets as `dbxlite-ui`, so a power user can drive it with a **native** DuckDB engine running on their own machine (full extensions, unlimited memory, direct filesystem access) while the container just serves the UI:

```bash
export ui_remote_url="http://localhost:8080"   # the container
duckdb -unsigned -ui                            # native engine + dbxlite UI on :4213
```

The engine runs locally next to the user's files; the container only ships the interface. (Do not run a shared DuckDB engine inside the container: arbitrary SQL means filesystem and extension access, and one engine shared across users has no isolation. Keep the engine on each user's machine.)

## Screenshots

**Main Interface**
![Main Interface](https://raw.githubusercontent.com/hfmsio/dbxlite/main/apps/web-client/public/screenshots/main-interface-dark.png)

**Query Remote Files (CSV, Parquet via HTTP)**
![Query Remote Files](https://raw.githubusercontent.com/hfmsio/dbxlite/main/apps/web-client/public/screenshots/query-remote-files.png)

**Schema Explorer with Multi-Theme Support**
![Multi-Theme Explorer](https://raw.githubusercontent.com/hfmsio/dbxlite/main/apps/web-client/public/screenshots/explorer-multi-themes.png)

**URL Sharing**
![Share Links](https://raw.githubusercontent.com/hfmsio/dbxlite/main/apps/web-client/public/screenshots/share-links.png)

**Excel File Support**
![Excel Query](https://raw.githubusercontent.com/hfmsio/dbxlite/main/apps/web-client/public/screenshots/excel-query-any-sheet.png)

**Export to Parquet/CSV/JSON**
![Export Status](https://raw.githubusercontent.com/hfmsio/dbxlite/main/apps/web-client/public/screenshots/export-status.png)

**Server Settings (HTTP Mode)**
![Server Settings](https://raw.githubusercontent.com/hfmsio/dbxlite/main/apps/web-client/public/screenshots/server-settings.png)

## Core Commands
- `pnpm dev`: start the web client locally
- `pnpm build`: build all workspaces
- `pnpm lint` / `pnpm lint:fix`: Biome lint (errors only) and auto-fix
- `pnpm test`: Vitest suite
- `pnpm e2e` / `pnpm e2e:headed` / `pnpm e2e:ui`: Playwright end-to-end runs

## Project Structure
```
dbxlite/
├─ apps/
│  ├─ web-client/           # React/Vite frontend
│  │  ├─ src/
│  │  │  ├─ components/     # UI components (EditorPane, TabBar, Header, etc.)
│  │  │  ├─ containers/     # Composite components (DialogsContainer, MainContent)
│  │  │  ├─ contexts/       # React contexts (TabContext, QueryContext)
│  │  │  ├─ hooks/          # Custom hooks (useQueryExecution, useTabManager, etc.)
│  │  │  ├─ services/       # Data services (data-source-store, settings-store, ai/)
│  │  │  ├─ stores/         # Zustand stores (settingsStore, aiChatStore)
│  │  │  └─ utils/          # Utilities (formatters, dataTypes, logger)
│  │  └─ App.tsx            # Main orchestrator
│  └─ cli/                  # dbxlite-ui npm package (for duckdb -ui integration)
├─ packages/
│  ├─ connectors/           # Data connectors (DuckDB, BigQuery, Snowflake)
│  ├─ duckdb-wasm-adapter/  # Worker/engine bridge
│  ├─ storage/              # Credential and handle storage
│  └─ plugins/              # Extensible plugin surface
├─ docs/                    # Architecture and usage docs
└─ scripts/                 # Tooling (e.g., download DuckDB WASM)
```

### Frontend Architecture
The web-client uses a layered architecture:
- **Providers**: ToastProvider → SettingsProvider → DataSourceProvider → TabProvider → QueryProvider
- **Contexts**: TabContext (tab state + refs), QueryContext (connector state)
- **Hooks**: 15+ custom hooks handling query execution, file operations, auto-save, keyboard shortcuts
- **Containers**: DialogsContainer groups modals; MainContent handles editor/results layout

See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for detailed diagrams.

## Data & Workflow Notes
- Local files use the File System Access API; prefer zero-copy paths for speed. Remote URLs are fetched on demand. DuckDB `.db` files can be attached and reused.
- Query execution runs in a Web Worker; large results stream via Arrow for responsive grids.
- Keyboard shortcuts: `Cmd/Ctrl+Enter` to run, `Cmd/Ctrl+Shift+F` to format, `Cmd/Ctrl+Shift+A` to toggle AI assistant, `Cmd/Ctrl+Home/End` to jump pages in the grid.

## URL Sharing

Share queries via URL parameters or GitHub Gists:

```bash
# Load built-in example
http://localhost:5173/?example=wikipedia&run=true

# Direct SQL (URL-encoded)
http://localhost:5173/?sql=SELECT%20*%20FROM%20range(10)&run=true

# GitHub Gist
http://localhost:5173/?share=gist:abc123&run=true

# With theme
http://localhost:5173/?example=covid&run=true&theme=dracula

# Results beside the editor (layout=right, or the alias horizontal)
http://localhost:5173/?example=covid&run=true&layout=right
```

**Parameters:** `example`, `sql`, `share`, `run`, `tab`, `theme`, `explorer`, `layout`

**Layout:** `bottom` (alias `vertical`), `right` (alias `horizontal`), `hidden`

**Themes:** `vs-dark`, `dracula`, `nord`, `tokyo-night`, `catppuccin`, `vs-light`, `github-light`, `solarized-light`, `ayu-light`, `one-dark`

See [docs/URL-SHARING.md](docs/URL-SHARING.md) for full reference.

## Limitations

Browser apps can only reach databases over HTTP/REST. PostgreSQL, MySQL, and other TCP-only databases aren't supported and won't be without a server-side proxy. See [CONTRIBUTING.md](CONTRIBUTING.md#adding-new-connectors).

Cloud Parquet export (BigQuery / Snowflake) round-trips chunks through JSON into DuckDB's Parquet writer; native `parquetjs` / Arrow is on the list. DuckDB exports use native `COPY`. Most credentials are encrypted at rest with AES-GCM, but a few legacy paths still use plain `localStorage` and are being migrated.

What's next: Supabase (via PostgREST, since that's HTTP-shaped) and native Parquet export. The AI assistant likely gets a Databricks backend once we have a Databricks connector at all.

## Contributing & Community
- See [CONTRIBUTING](CONTRIBUTING.md) for setup, workflow, and testing guidance.
- Please follow the [Code of Conduct](CODE_OF_CONDUCT.md).
- Security issues: report privately via [SECURITY](SECURITY.md).

## License
MIT. See [LICENSE](LICENSE).

dbxlite

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README