Building reliable (and fast) directory sync
4 hours ago
- Directory sync copies users, groups, and members from an identity provider into an app and keeps that copy updated.
- SCIM standardizes the protocol but not provider-specific behavior, leading to inconsistent implementations across providers.
- Pull-based sync gives the app control by fetching data on a schedule, handling pagination, stable IDs, cycles, and flattening.
- Syncing larger directories requires handling rate limits, bad responses, nested groups, overlapping jobs, and distinguishing temporary from permanent errors.
- Delta syncs were rejected due to issues with transitive groups, missed deletes, expiring tokens, and unrecoverable mistakes.
- Firezone uses a hybrid approach combining provider real-time APIs with periodic full syncs for near-real-time updates and reliability.