We Built a Video Rendering Engine by Lying to the Browser About What Time It Is
2 days ago
- #web-development
- #browser
- #video-rendering
- Browsers are real-time systems, making screen recording for videos challenging due to frame timing mismatches.
- Remotion was considered but rejected due to constraints requiring capturing arbitrary web content without framework limitations.
- A JavaScript solution was developed to replace browser time-related APIs with a controlled virtual clock for deterministic frame rendering.
- A warmup loop was implemented to prevent Chrome's compositor from going stale by continuously issuing 'skip frames'.
- A complex workaround for <video> elements involves server-side preprocessing, demuxing, decoding, and rendering to canvas synchronized to the virtual clock.
- Audio capture is achieved by intercepting Web Audio API and HTMLMediaElement calls to track playback intent and reconstruct audio server-side.
- Additional measures ensure determinism, such as disabling OffscreenCanvas and validating subresource requests against SSRF patterns.
- The solution is inspired by WebVideoCreator but adapted for modern Chrome APIs and specific cloud infrastructure needs.
- The implementation is planned to be open-sourced, offering techniques for programmatic video creation from web content.