a day ago
- PlanetScale automates backups for sharded Postgres and MySQL databases to minimize production impact by spinning up temporary EC2 instances per shard.
- The steady-state backup process involves restoring the previous backup from object storage (e.g., S3), replaying WAL (mostly from S3, with recent segments directly from primary), and saving the consistent, encrypted snapshot.
- Massive parallelism across shards enables high-speed backups (e.g., 32 TB spread over 8 shards backs up in ~2.8 hours; over 32 shards in 42 minutes), achieving rates over 50 GB/s and supporting petabyte-scale databases.
- Initial backups use pg_basebackup to seed nodes and wal-g for durable storage, ensuring consistency with the steady-state format.