We used a database as a message queue. Now we use Kafka
13 hours ago
- Used FoundationDB as both primary database and message queue, implementing Apple's QuiCK design.
- Encountered write limits and read load issues when using FoundationDB as a queue, prompting a move of async tasks to Kafka.
- QuiCK uses time-based job identity and leasing to ensure durable, retryable message processing without dual-write problems.
- Custom adaptations included skipping Record Layer, using fixed queues, supporting long-running jobs with checkpoints, and implementing cronjobs via self-rescheduling.
- Challenges with the approach include transaction conflicts, chronically unlucky jobs, write pressure at scale, and clock synchronization requirements.
- Moved asynchronous tasks like garbage collection to Kafka to reduce load on FoundationDB and simplify custom code.