Processing Piano Tutorial Videos in the Browser
6 days ago
- #computer-vision
- #web-tool
- #piano
- Most piano tutorial videos use falling raindrop visuals with highlighted keys, often created using Synthesia.
- PianoReader is a web tool that converts piano tutorial videos into readable notes and chords directly in the browser.
- The tool uses HTML Canvas for client-side video processing without needing a server.
- Users manually position rectangles over piano keys to help the tool detect key placements.
- The tool samples key regions on each video frame to detect pressed keys using grayscale filtering.
- Detected keys are processed with tonal.js to identify chords.
- Limitations include inability to process cross-origin videos (e.g., YouTube links directly) and slow processing speed due to frame-by-frame analysis.
- Currently, the tool only works with white keys and may require manual tuning for accuracy.
- PianoReader is available at pianoreader.app and is open-sourced on GitHub.