- The author started with the belief that OLTP databases were solved, but building Databricks revealed they were clunky and fragile, leading to the creation of Lakebase.
- Traditional monolithic databases like Postgres use a write-ahead log (WAL) for fast writes and data files for reads, but this architecture causes issues like data loss, scaling difficulties, and workload interference.
- Lakebase addresses these by externalizing storage: the WAL goes to a distributed SafeKeeper service for durability via Paxos replication, and data files go to a PageServer service that materializes pages into cloud object storage.
- This stateless compute architecture enables unlimited storage, serverless elastic compute, durable writes with zero data loss, simpler high availability, and instant branching/cloning without physical copying.