2 days ago
- Neutrino-1 models use a proprietary ternary-family weight format, delivering three models from one engine across datacenter GPUs, Apple Silicon, and desktop CPUs.
- The flagship Neutrino-1 8B has 8.19 billion parameters stored in a single 3.88 GB file, with weights eight times smaller than fp16, fitting on 8 GB GPUs or 16 GB laptops.
- Architecture: dense decoder-only transformer with grouped-query attention, requiring 144 KiB per token of KV cache.
- Weight distribution: 62.63% zeros, with sign-balanced plus and minus splits, and early feed-forward layers having 70-72% zeros.
- Performance: 24.9 tok/s on Apple M5 (CPU), 30.7 tok/s on NVIDIA L4, 33.7 tok/s on base M5 MacBook.
- Speculative decoding with a 0.6B draft model (328 MB) achieves full draft acceptance on counting prompts and 96.5% acceptance on factual prompts, boosting throughput.
- Open weights under Apache License 2.0, derivative of Qwen3-8B, with one-command installation via pip and support for multiple runtimes.