German AI consortium releases Soofi S, an open 30B model
7 hours ago
- #German Language Model
- #Open-Source AI
- #Mixture-of-Experts
- A German research consortium released the open-source 30B language model Soofi S, trained entirely on Deutsche Telekom's AI cloud.
- Soofi S uses a resource-efficient Mixture-of-Experts (MoE) hybrid architecture, activating only 3.2B of its 31.6B parameters per token for constant processing speed with long inputs.
- The model outperforms other fully open models like Olmo 3 32B and Apertus 70B in German, English, and programming benchmarks, with a training mix deliberately weighted toward German data.
- Critics argued Soofi S was 'overtrained' by Chinchilla scaling law standards, but the technical leadership countered that those laws don't apply to MoE architectures due to data repetition benefits.
- Soofi S shows strong throughput in long-context generation, maintains near-constant performance from 4,000 to 256,000 tokens, and leads in German-specific knowledge tests while having some weaknesses in math and factual retrieval.
- The model was trained on sovereign infrastructure using renewable energy, with openness in releasing weights, code, and data inventory, though it doesn't fully meet a strict European open-data definition due to a small commercially licensed dataset.