The power of two random choices
6 days ago
- #distributed-systems
- #performance
- #load-balancing
- Large-scale web services use multiple layers of stateless and stateful services separated by load balancers.
- Load balancing can be achieved via hardware, software, DNS tricks, or client-side mechanisms.
- Centralized load balancers efficiently distribute load but add cost, latency, and potential single points of failure.
- Distributed load balancing with stale data leads to herd behavior, causing servers to oscillate between busy and quiet states.
- Random host selection can degrade performance by unevenly distributing load.
- The 'best of two random choices' method effectively balances load and avoids herd behavior.
- Simulations show 'best of two' outperforms random, best, and 'best of three' approaches, especially with delayed data updates.
- The method combines real load information with resistance to herd behavior, making it highly effective.