Multigres Supports Listen/Notify Across Pooled Connections
3 hours ago
- #Pub/Sub
- #Connection Pooling
- #Postgres
- Multigres solves the scaling problem of Postgres LISTEN/NOTIFY by maintaining a shared listener connection that handles subscriptions on behalf of all clients, avoiding per-client session dependencies.
- The pooler uses a single, reserved backend connection for listening, with refcounting for channels to issue one LISTEN/UNLISTEN per channel regardless of client count, ensuring efficient resource use.
- Notifications are delivered asynchronously through a two-level fan-out: from the pooler to gateways via gRPC streams, then to client connections with bounded queues to prevent blocking and match Postgres's best-effort delivery.
- Multigres ensures transactional behavior for LISTEN/UNLISTEN by buffering actions and applying net changes only at commit time, and closes race conditions with readiness handshakes to guarantee no lost notifications.
- The system handles edge cases like reconnection by replaying active channels, matches Postgres quirks such as channel name truncation and quoting, and plans future improvements like cross-shard NOTIFY and stream consolidation.