MasterSelects

KO | EN

gitlite — search

#php #android #machine-learning #java #mqtt #nodejs #api #python #security #javascript #chrome #dependency-injection

★ 442 Open GitHub ↗

Legacy Community Edition — frozen at v2.4.5-mit-final under the MIT License.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# MasterSelects

<h3>Local-first Media Editor for Video, Audio, Vector Animation & 3D</h3>

<table><tr><td align="center" style="border:none;background:#0d1117;">
<h1>&#9889; ~2 MB compressed</h1>
compressed editor shell
</td></tr></table>

GPU-first media editing with 34 effects, 74 runtime transitions, 37 blend modes, 23 audio FX, stem separation, native WebGPU 3D, and 20 direct runtime dependencies. 
 Built from scratch in 4.1k+ lines of WGSL and 330k+ lines of TypeScript/TSX app code. 
 Import .lottie, .riv, Lottie JSON, OBJ, glTF, GLB, PLY, SPLAT, KSPLAT, SPZ, SOG, LCC assets and play PLY / GLB sequences directly on the timeline.

# you are home
<img width="2025" height="693" alt="Screenshot 2026-06-12 232723" src="https://github.com/user-attachments/assets/31600199-a90e-4a73-b5a0-d998584ae376" />

</div>

---

## Supported Formats

Decoding depends on what the **browser** supports — the container is just the wrapper, the codec inside is what matters.

<table>
<tr><th colspan="2">Import (Decode)</th></tr>
<tr><td>Video files</td><td>MP4, WebM, MOV, AVI, MKV, WMV, M4V, FLV</td></tr>
<tr><td>Video codecs</td><td>H.264 (AVC), H.265 (HEVC)¹, VP8, VP9, AV1</td></tr>
<tr><td>Audio files</td><td>WAV, MP3, OGG, FLAC, AAC, M4A, WMA, AIFF, OPUS</td></tr>
<tr><td>Audio stems</td><td>On-device Demucs/HTDemucs separation creates Vocals, Drums, Bass, and Other stems as project-local WAV media when the project write path is available</td></tr>
<tr><td>Image</td><td>PNG, JPG/JPEG, WebP, GIF, BMP, SVG</td></tr>
<tr><td>Vector animation</td><td><code>.lottie</code> packages, <code>.riv</code> files, and Lottie JSON files (content-sniffed)</td></tr>
<tr><td>3D Models</td><td>OBJ, glTF, GLB - rendered through the native WebGPU shared-scene path</td></tr>
<tr><td>3D sequences</td><td>PLY and GLB frame sequences played as timeline media</td></tr>
<tr><td>Gaussian Splats</td><td>PLY, compressed PLY, SPLAT, KSPLAT, SPZ, SOG, LCC, SOG-style ZIP payloads</td></tr>
<tr><td>Download</td><td>YouTube, TikTok, Instagram, Twitter/X, Vimeo + <a href="https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md">all yt-dlp sites</a> via Native Helper</td></tr>
<tr><th colspan="2">Export (Encode)</th></tr>
<tr><td>Video containers</td><td>MP4, WebM via WebCodecs / HTMLVideo; MOV, MKV, AVI, MXF via FFmpeg WASM</td></tr>
<tr><td>FFmpeg codecs</td><td>ProRes, DNxHR, FFV1, UTVideo, MJPEG</td></tr>
<tr><td>Video codecs</td><td>H.264, H.265¹, VP9, AV1 — GPU-accelerated via WebCodecs</td></tr>
<tr><td>Image export</td><td>PNG, JPG/JPEG, WebP, BMP (current playhead frame)</td></tr>
<tr><td>Audio-only export</td><td>WAV, plus AAC or OGG/Opus depending on browser codec support</td></tr>
<tr><td>Interchange</td><td>FCPXML (Final Cut Pro / DaVinci Resolve)</td></tr>
</table>

¹ H.265 decode/encode depends on OS & hardware — full support on Windows, partial on macOS/Linux.

> **MOV** files work because they share the same ISO BMFF container as MP4 — any MOV with H.264/H.265 inside plays fine. **MKV** works if it contains browser-decodable codecs (H.264, VP9, etc.). Native Helper decode is an opt-in companion path for browser gaps when the helper is connected and the source path is resolvable; unsupported codecs do not automatically switch to it.

---

## What Makes This Different

Most browser-based video editors share a pattern: Canvas 2D compositing, heavyweight dependency trees, and CPU-bound rendering that falls apart at scale. This project takes a fundamentally different approach.

**GPU-first architecture.** Preview, scrubbing, and export all run through the same **WebGPU ping-pong compositor**. Video textures are imported as `texture_external` (**zero-copy**, no CPU roundtrip). **37 blend modes**, 3D rotation, and inline color effects all execute in a **single WGSL composite shader** per layer. The 3D layer system now renders through the native WebGPU shared-scene path for 3D planes, primitive meshes, 3D text, OBJ/glTF/GLB models, camera clips, and gaussian splats - no GSAP, no Canvas 2D fallback in the hot path.

**Current startup footprint.** A fresh production build loads an editor shell of about **2 MB compressed** on the main editor start path. The largest contributors are the native 3D / gaussian-splat render path, browser-side media parsing/runtime code, and AI/media helper modules.

**Zero-copy export pipeline.** Frames are captured as `new VideoFrame(offscreenCanvas)` directly from the GPU canvas. **No `readPixels()`**, no `getImageData()`, no staging buffers in the default path. The GPU renders, **WebCodecs encodes**. That's it.

**3-tier scrubbing cache.** **300 GPU textures in VRAM** for instant scrub (Tier 1), per-video last-frame cache for seek transitions (Tier 2), and a **900-frame RAM Preview** with CPU/GPU promotion (Tier 3). When the cache is warm, **scrubbing doesn't decode at all**.

**Audio workstation in the timeline.** Audio Focus turns the timeline into a detailed waveform and spectrogram editor with non-destructive region edits, gain fades, silence removal, repair operations, spectral selections, image-in-spectrum layers, recording, and bake/unbake. The docked Audio Mixer adds track/master strips, Peak/RMS meters, stereo phase correlation, track sends rendered into the master return mix, FX stacks, limiter controls, and export preflight. Stem separation runs a browser ONNX Demucs/HTDemucs path and publishes Vocals, Drums, Bass, and Other stems as normal WAV media when the project media write succeeds, with artifact-backed fallback state when it cannot.

**20 direct runtime dependencies.** React/React DOM, Zustand, MediaBunny and its MP3 encoder, mp4box, PlayCanvas / splat-transform helpers, dotLottie, Rive WASM, HuggingFace Transformers, ONNX Runtime, SoundTouch, FFmpeg WASM packages, fflate, gifenc, Tabler icons, and WebGPU types. **Everything else is custom-built from scratch**: the WebGPU compositor, all 34 GPU effects, the 23-effect audio registry, the keyframe animation system, the export engine, the audio mixer, the text renderer, the mask engine, the video scope renderers, the dock/panel system, the timeline UI, and the native shared 3D scene path. Zero runtime abstraction layers between your timeline and the GPU.

**Nested composition rendering.** Compositions within compositions, each with their own resolution. The normal 2D/nested compositor path renders them to **pooled GPU textures** with frame-level caching and composites them in the parent's ping-pong pass. Native 3D and gaussian-splat auxiliary paths can submit their own GPU work when those layer types are active.

**On-device AI and analysis.** SAM2 (Segment Anything Model 2) runs entirely in-browser via ONNX Runtime. Click to select objects in the preview, propagate masks across frames. Stem separation writes project-local WAV media, while local transcription, waveform pyramids, spectrogram tiles, loudness envelopes, beat/onset maps, and frequency/phase summaries are generated as project artifacts instead of being baked into timeline clips. Large neural models are loaded on demand.

---

## Why I Built This

No Adobe subscription, no patience for cracks, and no template-first online editor. MasterSelects is fast in the browser, GPU-first, built for real editing, and open enough that AI can steer the timeline instead of just suggesting ideas.

**The vision:** an editor where AI can directly operate the tool. The built-in chat exposes editor operations across timeline, media, editing, playback, stats, and node-workspace actions. Developers can connect a self-hosted external agent through the token-authenticated local MCP/dev bridge. Multi-output routing is built in.

~330k lines of TypeScript/TSX app code, ~4.1k lines of WGSL shader files, and a Rust native helper cover the browser gaps that still matter.

---

## AI Control

MasterSelects centers on the idea that AI should be able to *do the edit*, not just talk about it.

- **Built-in editor chat:** Hosted agent/kernel execution with a fixed application prompt and the reviewed editor-operation surface
- **External agents in development:** Claude Code, Codex, or another self-hosted agent can use `npm run mcp`; the local MCP server owns no model, provider key, or billing relationship
- **AI video, image, and audio generation:** FlashBoard uses hosted generation routes; provider credentials never enter browser settings or project files
- **Multicam and analysis:** Audio-based sync, manual multicam alignment, and transcript-based sync when transcripts are available
- **On-device AI:** SAM2 segmentation in-browser via ONNX Runtime, MatAnyone2 via Native Helper, plus local Whisper transcription via Transformers.js

Run the development-only MCP adapter while the Vite app is open:

```bash
npm run mcp
```

This adapter talks only to the token-authenticated local development bridge. MasterSelects does not ship a production agent harness; users who want another model or agent host it themselves.

---

## What It Does

| Feature | Description |
|---|---|
| [**Multi-track Timeline**](docs/Features/Timeline.md) | Cut, copy, paste, multi-select, JKL shuttle, nested compositions |
| [**34 GPU Effects**](docs/Features/Effects.md) | Color correction, blur, distort, stylize, keying - all real-time |
| [**Video Scopes**](docs/Features/UI-Panels.md#video-scopes-panels) | GPU-accelerated Histogram, Vectorscope, Waveform monitor |
| [**Keyframe Animation**](docs/Features/Keyframes.md) | Bezier curves, copy/paste, tick marks, 5 easing modes |
| [**Vector Masks**](docs/Features/Masks.md) | Pen tool, edge dragging, feathering, multiple masks per clip |
| [**SAM2 Segmentation**](docs/Features/AI-Integration.md) | AI object selection in preview - click to mask, propagate across frames |
| [**Transitions**](docs/Features/UI-Panels.md#transitions-panel) | 74 GPU-rendered transitions across 2D and 3D families |
| [**AI Integration**](docs/Features/AI-Integration.md) | Hosted editor agent plus a development-only MCP adapter for self-hosted external agents |
| [**FlashBoard**](docs/Features/FlashBoard.md) | Media Panel generation tray for text-to-video, image-to-video, images, speech, and music |
| [**Multicam & Analysis**](docs/Features/Audio.md#multicam-and-analysis) | Sync selected clips by audio and align multicam clips manually or from available transcripts |
| [**Export Pipeline**](docs/Features/Export.md) | WebCodecs Fast/Precise, FFmpeg intermediates, image/audio-only export, FCPXML, and project-persistent presets |
| [**Audio Workstation**](docs/Features/Audio.md) | Audio Focus, detailed waveforms and spectrograms, 23 Audio FX, track sends into the master return mix, stem separation, recording, and export preflight |
| [**Download Panel**](docs/Features/Download-Panel.md) | YouTube, TikTok, Instagram, Twitter/X, Vimeo, and other yt-dlp-supported sites via Native Helper |
| [**Vector Animation**](docs/Features/Vector-Animation.md) | `.lottie`, `.riv`, and Lottie JSON clips with bounce playback, render resolution overrides, state-machine keyframes, Rive data binding, and preview/export |
| [**Text & Solids**](docs/Features/Text-Clips.md) | 50 Google Fonts, stroke, shadow, and solid color clips |
| [**Proxy System**](docs/Features/Proxy-System.md) | GPU-accelerated proxies with resume and cache indicator |
| [**Output Manager**](docs/Features/Preview.md) | Multi-window outputs, source routing, corner pin warping, slice masks |
| [**Signal IR**](docs/Features/Signal-IR.md) | Universal import layer for files that become binary, table, model, splat, or text-summary signals |
| [**Slot Grid**](docs/Features/Slot-Grid.md) | Resolume-style 12x4 grid with multi-layer live playback and slot-clip trims |
| [**Preview & Playback**](docs/Features/Preview.md) | RAM Preview, transform handles, multiple render targets |
| [**Project Storage**](docs/Features/Project-Persistence.md) | Local folders, raw media auto-copy, continuous save by default, interval mode, backups |
| [**Interactive Tutorial**](docs/Features/UI-Panels.md) | Guided onboarding with animated Clippy mascot |

<details>
<summary>See Keyframe Editor</summary>
 
<img src="docs/images/screenshot-curves.png" alt="Bezier Curve Editor" width="400">
</details>

---

## Quick Start

```bash
npm install
npm run dev     # http://localhost:5173
```

**Requirements:** Chrome or Edge with WebGPU support. A dedicated GPU is recommended.

> **Firefox:** project storage requires the Native Helper backend because Firefox does not support the File System Access API flow used by Chrome.

> **Linux:** Enable Vulkan for smooth 60fps: `chrome://flags/#enable-vulkan`

---

## Native Helper

Cross-platform Rust companion app for the parts browsers still can't do well. Required for Firefox project storage and for yt-dlp-based downloads.

```bash
cd tools/native-helper
cargo run --release    # WebSocket :9876, HTTP :9877
```

| Capability | Details |
|---|---|
| **Storage** | Native project persistence backend for Firefox |
| **Local AI** | MatAnyone2 video matting and MuScriptor transcription sidecars |
| **Download** | yt-dlp integration for YouTube, TikTok, Instagram, Twitter/X, Vimeo, and other supported sites |

**Platforms:** Windows, Linux, macOS. Building the Native Helper requires Rust. The Windows MSI bundles `yt-dlp.exe`; source builds and non-Windows archive installs use `yt-dlp` from the helper folder or PATH. See [Native Helper docs](tools/native-helper/README.md) for platform-specific setup.

---

## Security

MasterSelects is a **local-first editor**. Editing, rendering, caching, and most analysis stay in the browser unless you explicitly invoke an external provider or the Native Helper.

- **Provider credentials:** hosted AI credentials stay server-side; the optional YouTube integration key is encrypted in browser IndexedDB
- **Native Helper:** binds to `127.0.0.1` only, requires a random startup Bearer token for HTTP and WebSocket
- **Dev bridge:** Vite `/api/ai-tools` and local file routes require a per-session token and reject non-loopback origins
- **Local file access:** restricted to explicit allowed roots (project root, temp, Desktop, Documents, Downloads, Videos)
- **AI tool policy:** external bridge calls run through caller restrictions and approval gates
- **Secret handling:** logs redact common secret/token patterns; AI provider keys are not accepted from browser settings or project files
- **CI checks:** secret scanning, JS and Rust security audits, dedicated tests for bridge auth and file access policy

**Known boundary:** this is not perfect sandboxing. Same-user local processes, malicious browser extensions, and compromised same-origin code can still be dangerous. The goal is **clear, test-covered local trust boundaries**.

See [Security.md](docs/Features/Security.md) for the full trust model and limitations.

---

## Known Issues

This is alpha software. Features get added fast, things break.

- FFmpeg WASM export is blocking and constrained by browser memory
- Stem separation downloads a large ONNX model on first use and depends on WebGPU/WASM runtime performance
- Firefox project storage requires the Native Helper backend
- Video downloads require Native Helper; the Windows MSI bundles yt-dlp, while source/non-Windows installs need yt-dlp beside the helper or on PATH
- Audio waveforms may not display for some video formats
- Very long videos (>2 hours) may cause performance issues

If something breaks, refresh. If it's still broken, [open an issue](https://github.com/Sportinger/MasterSelects/issues).

---

## Tech Stack

- **Frontend:** React 19, TypeScript, Zustand, Vite 7.3
- **Rendering:** WebGPU + 4.1k+ lines of WGSL shader files
- **Video:** WebCodecs, MediaBunny, mp4box, HTMLVideo fallback, and FFmpeg WASM export
- **Audio:** Web Audio API, AudioWorklet recording, 23 registry-backed Audio FX, flexible EQ, track sends rendered into the master mix, stem separation through ONNX Runtime, artifact-backed waveform/spectrogram/loudness/beat/frequency analysis, and export preflight
- **AI:** Hosted agent/kernel editor chat, development-only MCP control for self-hosted external agents, hosted multicam analysis and media generation, SAM2 via ONNX Runtime, MatAnyone2 via Native Helper, and local Whisper via Hugging Face Transformers
- **Native:** Rust helper for Firefox storage backend, local MatAnyone2 and MuScriptor sidecars, and bundled/system yt-dlp downloads
- **Storage:** File System Access API on Chrome, Native Helper backend on Firefox, IndexedDB, local project folders with raw media

---

## Shortcuts

| Key | Action |
|-----|--------|
| `Space` | Play/Pause |
| `J` / `K` / `L` | Reverse / Pause / Forward (shuttle) |
| `C` | Cut at playhead |
| `I` / `O` | Set in/out points |
| `Ctrl+C/V` | Copy/Paste clips or keyframes |
| `Shift+Click` | Multi-select clips |
| `Tab` | Toggle edit mode |
| `Ctrl+Z/Y` | Undo/Redo |
| `Ctrl+S` | Save project |

[All 89 shortcuts](docs/Features/Keyboard-Shortcuts.md)

---

## Documentation

Detailed docs for each feature: **[docs/Features/](docs/Features/README.md)**

---

## Development

```bash
npm run dev              # Dev server with HMR
npm run dev:changelog    # Dev server with changelog dialog
npm run build            # Production build (tsc + vite)
npm run build:deploy     # Production build (vite only, skip tsc)
npm run lint             # ESLint
npm run preview          # Preview production build
npm run test             # Run tests (vitest)
npm run test:security    # Security-focused test suite
npm run test:watch       # Run tests in watch mode
npm run test:ui          # Run tests with UI
npm run test:coverage    # Run tests with coverage
npm run test:unit        # Run unit tests only
```

<details>
<summary>Project Structure</summary>

```
src/
├── components/          # React UI
│   ├── timeline/        # Timeline editor (hooks/, components/, utils/)
│   ├── panels/          # Properties, Media, AI, Download, Export, Scopes, Transitions
│   ├── preview/         # Canvas + overlays + transform handles + SAM2 overlay
│   ├── outputManager/   # Multi-window output with slices
│   ├── export/          # Export dialog, codec selector, FFmpeg section
│   ├── dock/            # Panel/tab system
│   ├── common/          # Dialogs, tutorial, settings, shared components
│   └── mobile/          # Mobile-responsive layout
├── stores/              # Zustand state management
│   ├── timeline/        # Slices: track, clip, keyframe, mask, playback, selection, transitions, ...
│   └── mediaStore/      # Slices: import, folder, proxy, composition, slot, selection
├── engine/              # WebGPU rendering pipeline
│   ├── core/            # WebGPUContext, RenderTargetManager
│   ├── render/          # Compositor, RenderLoop, LayerCollector, NestedCompRenderer
│   ├── export/          # FrameExporter, VideoEncoder, ClipPreparation
│   ├── audio/           # AudioMixer, AudioEncoder, TimeStretch
│   ├── ffmpeg/          # FFmpegBridge, codecs
│   ├── pipeline/        # CompositorPipeline, EffectsPipeline, OutputPipeline, SlicePipeline
│   ├── scene/           # Shared-scene 3D runtime
│   ├── native3d/        # Native WebGPU 3D renderer helpers
│   ├── gaussian/        # Gaussian splat loading/render support
│   ├── texture/         # TextureManager, ScrubbingCache, MaskTextureManager
│   ├── managers/        # CacheManager, ExportCanvasManager, OutputWindowManager
│   ├── analysis/        # Histogram, Vectorscope, Waveform scopes
│   ├── video/           # VideoFrameManager
│   ├── stats/           # PerformanceStats
│   └── structuralSharing/ # SnapshotManager for undo/redo
├── effects/             # 34 GPU effects (color/, blur/, distort/, stylize/, keying/)
├── transitions/         # Transition definitions (crossfade)
├── artifacts/           # Content-addressed artifact storage
├── importers/           # Universal media importers
├── signals/             # Signal IR contracts and adapters
├── runtime/             # Runtime renderer adapters
├── extensions/          # Extension/provider integration points
├── marketing/           # Public/landing surfaces
├── routing/             # Route-level app shells
├── styles/              # Shared styles
├── shims/               # Browser/runtime shims
├── services/            # Audio, AI, Project, NativeHelper, Logger, LayerBuilder, MediaRuntime
│   ├── aiTools/         # 86 exported AI tool definitions + handlers
│   ├── sam2/            # SAM2 model manager + service
│   ├── project/         # Project persistence, save/load
│   ├── nativeHelper/    # Native decoder + WebSocket client
│   ├── layerBuilder/    # Layer building + video sync
│   ├── mediaRuntime/    # Media runtime bindings + playback
│   ├── vectorAnimation/ # Lottie/Rive metadata + runtime canvas playback
│   └── export/          # FCPXML export
├── shaders/             # WGSL (composite, effects, output, optical flow, slice)
├── hooks/               # React hooks (useEngine, useGlobalHistory, useMIDI, useTheme)
├── utils/               # Keyframe interpolation, mask renderer, file loader
├── types/               # TypeScript type definitions
├── workers/             # Transcription worker
└── test/                # In-browser test components
```

```
tools/
├── native-helper/       # Rust binary (FFmpeg + yt-dlp bridge)
│   └── src/             # WebSocket server, decode/encode sessions
├── ffmpeg-build/        # FFmpeg build scripts
├── ffmpeg-wasm-build/   # FFmpeg WASM build configuration
├── qwen3vl-server/      # Qwen3 VL server for scene description
└── visitor-tray/        # Windows tray notifier for hosted visit events
```

</details>

---

<details>
<summary>License Scan (FOSSA)</summary>

[![FOSSA Status](https://app.fossa.com/api/projects/custom%2b61097%2fmasterselects.svg?type=large)](https://app.fossa.com/projects/custom%2b61097%2fmasterselects)

The linked FOSSA report tracks direct and transitive dependencies across npm, Cargo, and pip. The current npm runtime surface in `package.json` has **20 direct runtime dependencies**.

| Category | Count | Status |
|----------|-------|--------|
| License Issues | 35 flagged | All reviewed — no violations |
| Vulnerabilities | 6 | All in dev-dependencies, fixable via `npm audit fix` |
| Outdated Deps | 4 | Non-critical |

**Flagged licenses (all compliant):**

| Package | License | Why it's OK |
|---------|---------|-------------|
| `soundtouch-ts` | LGPL-2.1 | Used as unmodified npm dependency |
| `sharp` / `libvips` (15 platform binaries) | LGPL-3.0 | Used as unmodified prebuilt binary |
| `mediabunny` | MPL-2.0 | Used as unmodified npm dependency |
| `torch`, `pillow` | BSD/PIL | Python tooling only (`tools/qwen3vl-server`), not shipped |
| Cargo crates (`r-efi`, `ring`, `rustix`, `wit-bindgen`, ...) | Apache-2.0 / MIT | Standard Rust ecosystem, no copyleft issues |

No source code of any dependency has been modified. No GPL/AGPL dependencies. All copyleft packages (LGPL, MPL) are used strictly as libraries via their published APIs.

[View full FOSSA report](https://app.fossa.com/projects/custom%2B61097%2Fmasterselects?utm_source=share_link) · [Attribution report (HTML)](docs/FOSSA-Attribution.html)

</details>

---

**MIT License**

</div>

MasterSelects

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README