Automatic K8s pod placement to match external service zones
7 days ago
- #Network Optimization
- #Latency Improvement
- #Kubernetes
- Kubernetes lacks awareness of external network topology, leading to suboptimal Pod scheduling.
- A solution introduces a lightweight lookup service and mutating webhook to optimize Pod placement based on external resource locations.
- The lookup service resolves domain names to IPs and maps them to network zones (e.g., AWS Availability Zones).
- A mutating webhook (e.g., Kyverno) injects node affinity rules into Pods at creation time to ensure they are scheduled in the same zone as their external dependencies.
- This approach can improve performance by 175% to 375% for latency-sensitive workloads.
- The solution is generic and works for any external resource (on-prem or cloud) with a single A-record FQDN.
- Manual node affinity rules are unreliable as external resource locations can change (e.g., RDS maintenance events).
- The solution requires Kyverno/OPA, a workload communicating via FQDN, and predefined subnet zone information.
- AWS subnets are zonal, making IP-to-zone mapping straightforward; GCP/Azure require additional complexity due to regional subnets.
- Alternative solutions (CI/CD modifications, sidecars, custom controllers) were considered but deemed less optimal.