When the machine boots but the reply disappears
a day ago
- When a machine boots but the reply disappears, retrying is reasonable but starting a second machine would make recovery more expensive than the failure.
- Each boundary (command execution, network membership, disk capture) needs a decision about what a retry is allowed to do; idempotency keys and receipts help prevent duplicate charges or actions.
- Account coordination uses Cloudflare Durable Objects to serialize admission (reserve slot, monthly start, compute allowance) before booting, allowing parallel provisioning while maintaining exclusive admission.
- A receipt is created atomically with the charged reservation before the machine boots; retries find the receipt and return 'starting' or reconcile with actual state after a 90-second window.
- Reservation numbers and generation identifiers (id, createdAt) protect against delayed messages arriving after cancellation or slot reuse; old dispatches are rejected.
- Compute budgets are reserved at launch and enforced via hard deadlines; unused reservation is released on confirmed stop, but failed stops keep the reservation to prevent double spending.
- Managed executions have their own idempotency keys and sequence numbers; output events allow reconnection to missed data; runtime restarts interrupt unfinished executions and destroy the guest to avoid untracked work.
- Private service HTTP routing uses outbound interception with trusted headers; destination rechecks liveness and membership before delivering response, preventing unauthorized access from stale arrangements.
- File transfers write to a temporary file then rename; they reject symlink targets and have a 1 MiB limit; application must verify artifact correctness beyond file transfer.
- Workspace snapshots cross three owners and cannot be atomic; the runtime saves a receipt before capture, and retries can recover the handle if interrupted, but an ambiguous capture cannot be repeated and leaves the source intact.
- Lifecycle tests deliberately delay dispatch, drop replies, and reuse slots to verify that fences and idempotency work correctly, including in the same millisecond.