Audio Reactive LED Strips Are Diabolically Hard
a day ago
- #audio visualization
- #human perception
- #LED strips
- Started with audio-reactive LED strips in 2016, expecting a few weeks but turned into a decade-long project.
- Initial method used volume detection on RGB channels, which was easy but boring and lost frequency information.
- Switched to naive FFT mapping frequency bins to LEDs, but energy concentrated in few pixels, leaving most dark.
- Key challenge: 'Pixel Poverty'—LED strips have limited pixels (e.g., 144 per meter), requiring high perceptual relevance for each.
- Breakthrough came with the mel scale, mapping frequencies to perceptual space, making every LED meaningful.
- Applied exponential smoothing and convolutions to reduce flicker and enable spatial smoothing on the strip.
- Needed perceptual models on both sides: mel scale for sound and gamma correction/color theory for light perception.
- Developed three main visualizations: Spectrum, Scroll, and Energy, using real-time processing with latency tradeoffs.
- Project supports Raspberry Pi and ESP8266, with code on GitHub; gained popularity with 2.8k stars and diverse user applications.
- Unsolved problems: making it work well on all music genres and capturing the human response (e.g., foot-tapping).