- Epilude Model 4 is the fourth generation on-device model for Local Mode, offering the largest quality jump since its launch while maintaining the same 1.5 GB download size and speed, with no data leaving the user's Mac.
- Local Mode uses two models: a speech model to convert voice to text and a cleanup model to refine transcripts by removing filler words, fixing punctuation, handling self-corrections, and matching tone; Model 4 is a new fine-tuned cleanup model based on the Qwen family, focusing on dictation cleanup without changing the speech model.
- Evaluation is based on a zero-tolerance approach for critical errors that change meaning or leak instructions, using an internal benchmark of 90 dictation scenarios; Model 4 passes nine more scenarios than Model 3 and reduces critical meaning errors by 56%, excelling in mid-sentence self-corrections, dates, mixed-language speech, and casual tone.
- Key lessons from building Model 4 include that label quality is more important than data volume, as fixing mislabeled training examples improved self-correction, and weight averaging multiple model runs outperformed individual models by eliminating rare failure modes.
- Model 4 has limitations, such as struggling with long, rambling passages and spoken questions, where Cloud mode still leads in overall performance; it requires Apple Silicon and 16 GB of memory for full AI cleanup, but works offline with basic formatting on smaller machines, ensuring privacy by processing all data locally at no extra cost.