How to escape the Linux networking stack

5 days ago

Copy Link

Cloudflare's soft-unicast method allows sharing IP subnets across data centers, enhancing performance and efficiency.
The Linux networking stack's design limitations led to the creation of SLATFATF ("fish"), a service for managing soft-unicast IP space.
Challenges include managing IP/port combinations, avoiding conntrack interference, and ensuring socket and packet forwarding coexistence.
Solutions explored include Netlink for conntrack manipulation, TCP_REPAIR for socket control, and routing rule adjustments for packet forwarding.
Early demux in Linux's routing decision process was identified as a hurdle, with disabling it showing minimal performance impact.
Despite efforts, Cloudflare opted for TCP connection termination within servers for simplicity and reliability over pure IP forwarding.

Hasty Briefsbeta