- Apple's new SpeechAnalyzer is the most accurate on-device speech engine tested, beating all Whisper models (including Whisper Small) on both clean and noisy LibriSpeech with about 3x faster speed.
- The legacy API SFSpeechRecognizer performed worst on clean speech, with SpeechAnalyzer reducing word error rate by 3.5 to 4 times, making migration highly recommended for accuracy improvements.
- SpeechAnalyzer outperforms Whisper Small on English transcription on Apple hardware, but Whisper retains advantages in multilingual support and cross-platform compatibility.
- All engines ran faster than real-time on an M2 Pro, with SpeechAnalyzer being roughly 3x faster than Whisper Small while maintaining higher accuracy.
- The benchmark methodology ensures reproducibility with public raw transcripts and validation against OpenAI's Whisper numbers, and it helped uncover a bug in Inscribe's implementation.