Operationalizing Macaroons
a year ago
- #fly.io
- #macaroons
- #security
- Fly.io uses Macaroon tokens for security, which allow users to scope down their own tokens for minimal privilege.
- Macaroons are online-stateful, requiring a database for nonce lookup, leading to the creation of tkdb, a simple system managing SQLite databases with LiteFS and Litestream.
- Token verification involves caching, with a 98% cache ratio, reducing the need for frequent database access.
- Revocation is handled via a blacklist table in tkdb, ensuring tokens stay dead when revoked, with a polling mechanism for cache updates.
- Service tokens are managed to minimize risk, with transformations and attenuations to limit their use to specific environments.
- OpenTelemetry and Honeycomb provide comprehensive telemetry for the token system, ensuring stability and traceability.
- Despite initial skepticism, Macaroons have proven effective internally, with tkdb and Petsem (a Vault replacement) handling security and secrets management efficiently.