How MVCC and Transactions Work in RocksDB
5 hours ago
- RocksDB uses an LSM-tree that creates new versions of keys on every write, providing half of MVCC; the other half requires snapshots and garbage collection aware of in-flight reads.
- Each write assigns a monotonically increasing sequence number; reads capture the published sequence at start to see a consistent snapshot, skipping higher sequence numbers.
- The memtable is a concurrent skip list ordered by user key ascending and sequence number descending; atomic writes use write batches to allocate sequence numbers atomically, ensuring no partial updates are visible.
- Snapshots pin a sequence number and prevent compaction from removing versions still needed; a reference-counted SuperVersion structure coordinates safe cleanup of memtables and SSTs.
- Pessimistic transactions lock keys on write operations for early conflict detection; optimistic transactions defer conflict checking to commit time, using lightweight hashed mutexes for validation.
- Isolation levels range from read committed (no snapshot) to snapshot isolation and serializable (using get_for_update to lock read keys and prevent write skew).
- A balance transfer example demonstrates how transactions with snapshots and get_for_update prevent lost updates and race conditions.
- Performance comparison: pessimistic transactions outperform on contended keys by blocking and avoiding rework; optimistic transactions excel when conflicts are rare and retries are cheap.
- RocksDB-native transactions are used by MyRocks and ArangoDB; distributed databases like TiDB and CockroachDB implement their own transactional layers above the key-value store.
- The core complexity lies in concurrency control, not MVCC itself; sequence numbers are the fundamental primitive for snapshots, atomic batches, and conflict detection.