Offloading FFmpeg with Cloudflare

6 days ago

The author initially processed podcast episodes directly on the Fly.io server running their website, which worked for 226 episodes until a particularly long episode caused CPU overload.
The issue prompted a move to offload FFmpeg processing to Cloudflare, using queues and containers to isolate the workload from the primary server.
The new architecture involves enqueuing jobs to Cloudflare Queue, processing in Cloudflare Containers, and using callbacks to update the episode status, significantly reducing load on the primary server.
The author reflects on the decision-making process, emphasizing the value of starting simple and iterating when necessary, rather than over-engineering solutions prematurely.
Cost analysis reveals that Cloudflare's pay-as-you-go model is cost-effective for sporadic workloads, though the primary benefit is operational isolation rather than cost savings.
Initial implementation issues included unnecessary fallback paths and suboptimal container lifecycle management, which were later refined for efficiency.
The author concludes that the transition was worthwhile, both for operational improvements and the learning experience with Cloudflare's services.

Hasty Briefsbeta