Running local LLMs on your Mac: what fits, what's free, and what's overkill
5 hours ago
- Every Apple Silicon Mac can run local LLMs, with models from 1B to 70B parameters depending on RAM.
- Unified memory on Apple Silicon allows models to run that wouldn't fit in PC video cards.
- Model size guide: 1-3B models run anywhere, 7-8B need 8GB+, 14B needs 16GB+, 30B+ needs 32GB+, 70B+ needs 64GB+.
- Free tools: Ollama (command line) and LM Studio (GUI) run the same open models.
- Local models are not as good as frontier models for hard reasoning, but excel in latency, privacy, cost, and reliability.
- The most useful local model is not a chat but embedded in tools, like autocomplete (e.g., TypeTab).
- Practical division: use cloud models for the hard 5% of thinking, local models for the constant 95% of typing.
- FAQ highlights: Macs can run open models like Llama, Mistral, and Gemma; RAM requirements vary; local models beat cloud on privacy and cost for focused tasks.