How Amazon SQS fair queues work

5 hours ago

Amazon SQS fair queues reduce noisy-neighbor impact in multi-tenant environments.
Tenants are identified by the MessageGroupId property; messages without it are treated as unique tenants.
Noisy neighbors are detected using concurrency share (in-flight messages) and processing time share.
Detection thresholds are approximate (10% share, with min 30 messages for concurrency, or 10% processing time).
When a noisy neighbor is detected, messages from quiet tenants are prioritized for delivery.
Noisy neighbor messages aren't dropped; their dwell time increases until the backlog clears.
Multiple noisy neighbors are balanced by prioritizing the one with fewer in-flight messages.
A tenant is considered quiet again after full backlog consumption or no in-flight messages for 5 minutes.
Effective detection requires adequate consumer capacity and concurrent message processing.
Using Lambda as a consumer involves considering concurrency and batch size for optimal performance.

Hasty Briefsbeta