- Zalando's engineering team developed an in-process client-side load balancer for their high-throughput Product Read API handling ~1M req/s.
- The client-side balancer routes high fan-out internal traffic, while Skipper remains for edge and single-GET traffic, reducing latency and infrastructure costs.
- The solution uses consistent hashing (xxHash64, 100 virtual nodes per endpoint) to match Skipper's routing, minimizing cache churn during migration.
- Key improvements include watch-based Kubernetes informers, fast reversible deploys, N-ring fade-in for cache warming, and richer observability.
- Skipper fleet shrank from 50+ to 8 pods, deployment daily cost dropped from $450 to $110, and brief node-level freezes are now automatically routed around.
- The team warns against building custom load balancers unless at extreme scale, suggesting use of mature proxies like Skipper or Envoy.
- Hacker News discussion questioned not contributing back to Skipper, as the new client-side balancer remains internal and unpublished.