beachdb

KO | EN

gitlite — search

#react #typescript #llm #golang #rust #linux #docker #cli #kubernetes #vue #awesome-list #pytorch #mcp #open-source

(★ 27)

🏖️ A Raft-distributed NoSQL database and storage engine in pure Go.

beachdb 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

**BeachDB is a toy distributed NoSQL database. Built for learning and education, not production.**

It starts life as a small, inspectable storage engine, then deliberately grows “real-system bones”: a server API, a failure model, and a Raft-replicated core. The point isn’t to win benchmarks — it’s to understand, measure, and explain what’s actually happening.

### Backstory

I’ve been fond of distributed systems and databases for a long time. I wrote my first Hadoop and Apache Spark pipeline back in 2016, then went on to solve hairy stream-processing problems at Shopify, and later worked on Apache HBase at HubSpot where I helped build and operate database infrastructure on top of Kubernetes at massive scale.

BeachDB is my attempt to re-learn the fundamentals by building them from scratch in Go. I’m prioritizing **simplicity, clarity, and understanding** over scalability, speed, and micro-optimizations.

## Architecture

- **LSM storage engine** (WAL → memtable → SSTables → compaction)
- **Single-node API** (server wrapper for Get/Put/Delete/Scan with timeouts + backpressure)
- **Distributed replication with Raft** (single group: leader writes + leader reads; log entry == `WriteBatch`)
- **Inspectability-first** (dump tools + crash tests as part of the architecture)

## Key features (shipped as a checklist)

> This list is ordered to match the build + blog sequence. I’ll tick these off as they land.

### Engine (storage truth)
- [x] **Scope + semantics contract** (snapshots, iterators, durability), see: [intro blog post](https://aalhour.com/posts/building-beachdb/)
- [x] **WAL v1**: checksums + deterministic crash recovery (**fsync per committed batch**), see: [durability blog post](https://aalhour.com/posts/beachdb-wal-v1-milestone/)
- [x] **Memtable v1**: sorted structure + tombstones, see: [memtable blog post](https://aalhour.com/posts/beachdb-memtable-v1/)
- [x] **Reference-model randomized tests** (model vs implementation)
- [x] **SSTables v1**: immutable sorted files + `sst_dump`, see: [sstables blog post](https://aalhour.com/posts/beachdb-sstables-v1/)
- [x] **Crash-loop harness**: kill mid-write, reopen, validate invariants, see: [crash-testing, part 1 blog post](https://aalhour.com/posts/beachdb-crash-testing-part1/)
- [x] **Manifest v1**: durable SSTable catalog (`CURRENT` + `VersionEdit` log) for startup reconstruction + `manifest_dump`, see: [manifest blog post](https://aalhour.com/posts/beachdb-manifest-v1/)
- [ ] **Merge iterators** (memtable + SSTs) + **snapshot reads** (seqno-based)
- [ ] **Read path acceleration**: block index + bloom filters + benchmark evidence
- [ ] **Compaction v1**: one strategy, minimal knobs + amplification measurements
- [ ] **Adversarial testing**: fault injection + fuzzing (WAL/SST decode paths)

### Server (systems truth)
- [ ] **Binary protocol** (framed) + timeouts + backpressure
- [ ] **Load generator** + p50/p99 latency reporting
- [ ] **Metrics/tracing hooks** that make performance explainable

### Replication (distributed truth)
- [ ] **Raft (single group)** where a log entry == serialized `WriteBatch`
- [ ] **Deterministic apply** + restart safety
- [ ] **Snapshotting** for fast catch-up

### Sequel teaser (maybe)
- [ ] **Tables & Regions**: table-ish encoding + scans + key-range routing (minimal, no rabbit holes)

## Non-goals (by design)

To keep BeachDB small and finishable, these are intentionally out of scope for Season 1:

- Production readiness, multi-year maintenance guarantees, or compatibility promises
- Multi-writer concurrency in the engine (single-writer early on)
- Background compaction early on (added only after invariants are rock-solid)
- SQL, query planner, joins, secondary indexes
- Full transactions / serializable isolation
- Auto sharding, region split/merge, rebalancing, quorum reads, gossip/repair

## Philosophy

> Every chapter ends with evidence: a dump tool, a crash test, a benchmark, or a diagram.

See [docs/principles.md](docs/principles.md) to see how I'm keeping this project from turning into a second job :)

## License

Apache 2.0 (see: [LICENSE](LICENSE))

beachdb

beachdb 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

beachdb

beachdb 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README