Tokio Gives Progress, Not Ordering: Scheduling 1M Tasks
7 hours ago
- Tokio's multi-threaded scheduler uses local and global queues without guaranteeing tasks are polled in creation order.
- Spawning tasks early does not ensure early polling or completion; tasks from different events compete for worker queues and can get delayed.
- Without bounds on task creation, large numbers of live tasks increase peak memory because earlier tasks and their parents remain alive longer.
- Using a Semaphore to limit concurrent events provides event-level fairness, reducing memory spikes while maintaining throughput.
- Application code must enforce its own fairness boundaries; Tokio treats all tasks equally and does not know about application-level units like events.