Durable queues, streams, pub/sub, and a cron scheduler – inside your SQLite file
4 hours ago
- #SQLite
- #Pub/Sub
- #Task Queue
- Honker adds Postgres-style NOTIFY/LISTEN semantics to SQLite, enabling durable pub/sub, task queues, and event streams without client polling or a daemon/broker.
- It functions as a SQLite loadable extension, allowing any language that can load it to share the same on-disk queue, streams, and notifications, with bindings for Python, Node, Rust, Go, Ruby, Bun, and Elixir.
- The queue lives in the same SQLite file as the primary datastore, ensuring that inserts into business tables and queue operations commit in the same transaction, eliminating dual-write problems and rollback consistency issues.
- Honker polls SQLite's PRAGMA data_version every millisecond, a low-cost monotonic counter that provides a precise wake signal for subscribers, with one poller thread per database regardless of listener count.
- It avoids the operational overhead of adding external brokers like Redis + Celery, leveraging SQLite's durability for applications already using it as their primary datastore, such as Bluesky's PDS and Fly's LiteFS.