Bonsai 27B (1-bit LLM): The First 27B-Class Model to Run on a Phone
3 hours ago
- #AI Model Compression
- #On-Device AI
- #Multimodal AI
- Bonsai 27B is a new multimodal model from the Bonsai family, based on Qwen3.6 27B, and the first of its capability class to run on a phone.
- It comes in two variants: Ternary Bonsai 27B (5.9 GB) for quality on laptops and 1-bit Bonsai 27B (3.9 GB) for footprint on phones, both using low-bit weights end-to-end with no high-precision escapes.
- The model retains 95% (Ternary) and 90% (1-bit) of full-precision baseline performance across benchmarks, excelling in math, coding, and tool calling for agentic workloads.
- Bonsai 27B enables local execution for agentic AI, reducing costs and enhancing privacy by keeping data on-device, and supports hybrid deployments with cloud models.
- It achieves high speeds (up to 163 tok/s on RTX 5090) and fits phone memory constraints, with a 262K-token context, multimodal vision, and speculative decoding.
- The release represents a paradigm shift in intelligence density, advancing AI deployment on everyday devices under the Apache 2.0 License, with plans for larger models.