The Tokio/Rayon Trap and Why Async/Await Fails Concurrency
4 hours ago
- #concurrency
- #async_await
- #system_design
- Async/await simplifies writing asynchronous code by making it look synchronous, but hides complexity in scheduling and control flow, shifting burden to developers.
- The conflation of asynchrony and concurrency in async/await can cause production issues, such as latency spikes from CPU-bound tasks blocking cooperative runtimes.
- Developers must manually separate I/O and compute tasks into different runtimes like Tokio and Rayon, acting as 'human schedulers' and undermining the abstraction's promise.
- Async/await runtimes often default to unbounded capacity, leading to out-of-memory crashes during traffic spikes due to unchecked task spawning and queuing.
- Work-stealing schedulers aimed at fairness can reduce throughput by destroying CPU cache locality and causing contention, as seen in large-scale systems like Erlang BEAM.
- Project Tina is introduced as an alternative concurrency framework based on state machines, featuring thread-per-core design, strict bounds, and deterministic simulation testing for predictability.