Bilingual

Tencent Hy3
15 days ago
- The text provides instructions on how to use the tencent/Hy3 model with various libraries, inference providers, notebooks, and local apps.
- Detailed examples are given for using Hy3 with Transformers, vLLM, SGLang, and Docker, including code snippets for installation, server setup, and API calls.
- Hy3 is a 295B-parameter Mixture-of-Experts (MoE) model with 21B active parameters, developed by Tencent Hy Team, offering strong capabilities in agent tasks, reasoning, and long-context handling.
- The model shows improvements in stability, knowledge accuracy, and multi-turn intent tracking, with reduced hallucination and error rates based on internal evaluations.
- Deployment recommendations include using vLLM or SGLang on 8 GPUs with specific configurations, and the model is open-sourced under Apache License 2.0, available on multiple platforms.
German AI consortium releases Soofi S, an open 30B model
6 days ago
- A German research consortium released the open-source 30B language model Soofi S, trained entirely on Deutsche Telekom's AI cloud.
- Soofi S uses a resource-efficient Mixture-of-Experts (MoE) hybrid architecture, activating only 3.2B of its 31.6B parameters per token for constant processing speed with long inputs.
- The model outperforms other fully open models like Olmo 3 32B and Apertus 70B in German, English, and programming benchmarks, with a training mix deliberately weighted toward German data.
- Critics argued Soofi S was 'overtrained' by Chinchilla scaling law standards, but the technical leadership countered that those laws don't apply to MoE architectures due to data repetition benefits.
- Soofi S shows strong throughput in long-context generation, maintains near-constant performance from 4,000 to 256,000 tokens, and leads in German-specific knowledge tests while having some weaknesses in math and factual retrieval.
- The model was trained on sovereign infrastructure using renewable energy, with openness in releasing weights, code, and data inventory, though it doesn't fully meet a strict European open-data definition due to a small commercially licensed dataset.
Motif-3-Beta
13 hours ago
- Motif-3-Beta is a preview checkpoint of the upcoming Motif-3 Mixture-of-Experts (MoE) language model, not the final release.
- It has approximately 314 billion total parameters with only about 13 billion active per token, using sparse MoE with 384 experts (8 activated per token plus 1 shared expert).
- Supports a native context length of 256K tokens (262,144) and is built for multilingual, general-purpose tasks.
- The model introduces custom in-house components: Grouped Differential Latent Attention (GDLA), Grouped PolyNorm activation per expert, and a modified mHC architecture.
- Can be used via Hugging Face Transformers pipeline with trust_remote_code=True, vLLM server, SGLang server, or Docker Model Runner.
- Weights are openly available for download without access request; license permits personal, educational, and non-commercial research use only, with commercial use requiring prior permission.
- Scored an AAII benchmark of 44 according to Artificial Analysis.

Hasty Briefsbeta

#mixture-of-experts

Tencent Hy3

German AI consortium releases Soofi S, an open 30B model

Motif-3-Beta