Kolibri Has Landed: A Sovereign Open-Weight Model
6 hours ago
- Aleph Alpha released Kolibri, a sovereign open-weight English-German Mixture-of-Experts model with 78B total and 3B active parameters, on German Reunification Day.
- Kolibri supports context lengths up to 1M tokens and is available under Apache 2.0 with full weights on Hugging Face.
- Built via an automated Model Factory pipeline, Kolibri went from Kolibri Origin (30B, 65k context) to 78B params, 1M context, and 20T training tokens in just three months.
- The model is specialized for German, reasoning, math, agentic behavior, and regulated sectors like public administration, aerospace, and automotive.
- Sovereignty is a core principle: full supply-chain transparency, EU AI Act/GDPR compliance, and no foreign control over data or infrastructure.
- Kolibri is trained to abstain when the answer is not in context using the Merlin-Arthur protocol, cutting hallucinations dramatically (44% abstention vs 15% for Origin).
- With only 3B active parameters, Kolibri matches or beats much larger models (up to 4x its size) on math, coding, grounding, and long-context benchmarks.
- German data strategy used 1.3T curated German tokens from Common Crawl, 1T tokens via LLM rephrasing, and minimal translation, preserving authentic German cultural context.
- Introduced UniBPE, a bilingual English-German tokenizer combining BPE and Unigram, improving German compression and inference efficiency.
- Post-training included 268B SFT tokens and reinforcement learning on 1.2M+ tasks, supporting controllable reasoning effort (none/low/medium/high).
- Kolibri outperforms competing open-weight models on the Honeypot agentic-RAG benchmark and leads in 4 of 5 customer verticals.