How fast are Linux pipes anyway?

10 months ago

Unix pipes in Linux are optimized by iteratively improving a test program's performance, starting from 3.5GiB/s to match a FizzBuzz program's ~35GiB/s.
The optimization process involves profiling with Linux's perf tooling and using vmsplice and splice syscalls to bypass some IO slowness.
Initial attempts with write and read syscalls achieve only 3.7GiB/s, significantly slower than the target, due to copying and allocating pages.
Linux pipes are implemented as ring buffers with references to pages, where head and tail pointers manage write and read operations respectively.
vmsplice and splice syscalls allow moving data between user memory and pipes without copying, improving throughput to 32.8GiB/s.
Further optimizations include using huge pages (2MiB) to reduce page table walking overhead, increasing speed to 51.0GiB/s.
Busy looping with SPLICE_F_NONBLOCK avoids synchronization costs, achieving a final throughput of 62.5GiB/s.
Key concepts covered include zero-copy operations, ring buffers, virtual memory paging, and synchronization overhead.

Hasty Briefsbeta