Accelerating Linux Pipes with Solarflare Onload
11 days ago
- #linux-pipes
- #kernel-bypass
- #performance-optimization
- Kernel bypass techniques like Solarflare Onload reduce latency and improve throughput by shifting work to user space.
- Solarflare Onload is easy to integrate, exposing standard APIs like BSD sockets and glibc wrappers without requiring application rewrites.
- Linux pipes, an IPC mechanism, involve kernel-managed buffers and syscalls, introducing overhead.
- Onload accelerates pipes by using shared memory in user space, intercepting syscalls like pipe(), read(), and write().
- Benchmarking shows a ~5X reduction in average RTT for non-blocking pipes with Onload, confirmed by strace showing fewer syscalls.
- Blocking mode with Onload shows no improvement due to kernel scheduling overhead, but tuning with spin-waiting can reduce latency.
- Onload's spin-before-blocking behavior (EF_PIPE_RECV_SPIN) can achieve near non-blocking performance in blocking mode.
- Onload's benefits extend beyond pipes to networking acceleration, especially with specialized Solarflare NICs, bypassing the kernel entirely.
- For lowest latency IPC, custom lock-free queues may still outperform Onload-accelerated pipes.