A log-structured key-value store, built from scratch in Go.
This is a learning project. I'm working through Chapter 3 of Designing Data-Intensive Applications and building each idea as I go, starting from the simplest possible database (append to a file, scan to read) and growing it stage by stage into a small LSM-tree storage engine.
The rule: each stage stays as small as possible, and only grows when a limitation actually bites. The design lessons live in the failure paths, so every stage includes crash experiments, not just happy-path tests.
logkv -dir ./data set <key> <value>
logkv -dir ./data get <key>
logkv -dir ./data delete <key>
- Stage 1 - Append-only log. Sets append to a file, gets scan the whole file and return the last match.
- Stage 2 - Hash index. In-memory map of key to byte offset, rebuilt on startup. Gets become a single seek.
- Stage 3 - Segments and compaction. Roll to new segment files at a size threshold, merge old segments in the background, tombstones for deletes.
- Stage 4 - Memtable, WAL, and SSTables. Sorted in-memory buffer flushed to sorted segments, write-ahead log for crash recovery, sparse index. A baby LSM-tree.
- Stage 5 - Beyond. Bloom filters, a binary record format, and eventually a TCP front with leader-follower replication.
cmd/logkv/ CLI entry point, nothing else
internal/store/ the storage engine
docs/adr/ one decision record per stage
scripts/ crash test scripts (kill -9 and friends)
Each stage gets a short ADR in docs/adr/: what limitation forced the
change, what was chosen, what was rejected. Stages are tagged in git
(stage-1, stage-2, ...) so the progression stays browsable.