Thundering herd problem: Preventing the stampede
a day ago
- #thundering herd
- #caching
- #distributed systems
- The thundering herd problem occurs when multiple concurrent requests for the same record result in cache misses, leading to increased database load.
- A typical solution involves using a cache-aside pattern, but this fails under high concurrency for the same key.
- The problem can be reproduced using a Spring Boot application with Redis caching and Postgres database.
- Two solutions are proposed: distributed locks using Redis and in-process synchronization with CompletableFuture and ConcurrentHashMap.
- Distributed locks ensure only one request accesses the database, while others retry cache lookups.
- In-process synchronization avoids network calls but lacks coordination across multiple nodes.
- Both solutions aim to prevent database overload by ensuring only necessary queries are made.