- OpenAI uses WebRTC to serve voice AI to 900 million weekly users, but faces challenges with Kubernetes due to WebRTC's need for stable IPs and ports.
- Their solution splits the architecture into a stateless relay at the geographic edge for packet routing and a stateful transceiver that manages WebRTC protocol state.
- The relay uses the ICE ufrag field from WebRTC setup packets as a routing key to direct the first packet to the correct transceiver, avoiding database lookups.
- Global Relay deploys distributed ingress points to reduce latency by handling media close to users, while signaling is geo-steered via Cloudflare.
- The relay is implemented in Go with optimizations like SO_REUSEPORT, runtime.LockOSThread, and minimal copying to handle traffic efficiently without kernel bypass.
- This design is optimized for 1:1 sessions, with tradeoffs including custom infrastructure and reliance on controlling both ends of signaling for the ufrag trick.