Kimi K3, and what we can still learn from the pelican benchmark

5 hours ago

Moonshot AI released Kimi K3, a 2.8 trillion parameter model described as their most capable to date.
K3 is currently available via website and API, with an open weight release promised by July 27, 2026.
Self-reported benchmarks show K3 outperforms Claude Opus 4.8 max and GPT-5.5 high but loses to Claude Fable 5 and GPT-5.6 Sol.
Artificial Analysis reports K3's Elo is 1547 on long-horizon knowledge work, with cost per task similar to GPT-5.6 Sol.
Pricing is $3/million input tokens and $15/million output tokens, making it the most expensive Chinese AI lab model.
The author uses a 'pelican riding a bicycle' SVG generation test to explore model characteristics.
K3's test revealed high reasoning token usage (13,241 tokens), costing 25 cents, and a potential hidden system prompt.
The pelican benchmark is limited, especially regarding agentic tool calling, and shouldn't be used to compare models.
Despite limitations, the test serves as a forcing function to try models and reveals details like cost and SVG capability.

Hasty Briefsbeta