Magenta RealTime 2: Open and Local Live Music Models
6 hours ago
- #Real-Time Audio
- #Machine Learning
- #AI Music Generation
- MRT2 is an open real-time music model enabling low-latency AI instruments on Apple Silicon MacBooks.
- It supports interactive control via MIDI, audio, and text, with inference performed on-device for immediate response.
- The release includes an open-weights model, C++ inference engine via MLX, and example applications for musicians and developers.
- Key improvements over MRT include reduced frame size (40ms vs. 2s), lower control latency (~200ms), and expanded hardware compatibility.
- Technical enhancements involve frame-level autoregression, sliding window attention, and multi-signal conditioning for precise control.
- Future updates will include finetuning capabilities and more example tools, with ongoing development for live music AI.