Disaggregation: A New Architecture for Cloud Databases
2 days ago
- #cloud databases
- #distributed systems
- #disaggregation
- Disaggregation in cloud databases allows for elastic scalability by decoupling compute and storage.
- Key motivations for disaggregation include cost asymmetry between compute and storage, fluctuating compute demand, and easier scaling of stateless compute.
- Early examples like Snowflake and Amazon Aurora separate compute and storage, while modern systems like Socrates further split storage into specialized services.
- Disaggregation introduces performance tradeoffs due to communication overhead, but optimizations can mitigate these.
- New protocols like Cornus 2PC leverage disaggregated storage to solve traditional distributed database problems.
- Pushdown techniques (e.g., PushdownDB, FlexPushdownDB) reduce data movement by executing operators closer to storage, improving performance and cost.
- Disaggregation enables new capabilities like HTAP systems (e.g., Hermes) and easier adoption of specialized hardware (e.g., GPUs, RDMA).
- Research opportunities include transforming monolithic databases into disaggregated systems and rethinking core distributed protocols in a disaggregated context.