S3 Is the Future, S3 Is the Past
2 days ago
- S3's dominance is due to infinite capacity, high durability, low cost, and shared namespace, making it the foundation of modern cloud data systems.
- S3's disk-based hardware imposes limitations: high latency (tens of ms), low per-request bandwidth (<100 MB/s), requiring workarounds like caching, batching, and separate metadata stores.
- SSD prices have dropped to roughly 3× that of disks, enabling much faster performance: microsecond latency, millions of IOPS, small random access, and in-place updates.
- SSD-based disaggregated storage with fast networks could eliminate many S3 workarounds, making caching optional and batching unnecessary.
- Amazon's S3 Express One Zone is a limited SSD offering: single availability zone, high cost, still multi-millisecond latency, not a general-purpose replacement.
- There is no technological barrier to a true SSD-based replacement; the obstacle is cloud provider inertia due to existing revenue from disks.
- The architectural patterns built around S3 are becoming obsolete as hardware constraints fade, suggesting we should build new infrastructure primitives instead of waiting for providers.