kimi k3

Bilingual

Running Kimi K3 on MI355X at Better Performance per Dollar Than B300
2 months ago
- AMD MI355X achieves ~952 tok/s/node for Kimi K3, outperforming B200 on performance per dollar due to lower cost and higher HBM capacity.
- Kimi K3 is a 2.8T-parameter open-source model that cannot fit on a single B200 node, requiring two nodes or a B300 node with 288GB VRAM per GPU.
- AMD MI355X offers comparable specs to B300 at ~2.4× lower cost per GPU, making it a cost-efficient alternative despite software challenges.
- Software issues on ROCm were resolved with simple fixes (e.g., adding top-k renorm function for speculative decoding, padding head count for MLA prefill), not requiring custom kernels.
- Speculative decoding improved single-stream throughput by ~2.2× and peak aggregate by +18%, while prefill optimizations cut time-to-first-token by 2–3×.
- The MI355X's large HBM capacity gives it a practical edge over B200 for models like Kimi K3 that require >1.5TB VRAM for weights and KV cache.
Kimi K3 by Moonshot now available on Modal
2 months ago
- Moonshot released Kimi K3, a 2.8 trillion parameter multimodal model with 1M token context window and native vision, now available on Modal.
- Modal runs K3 at 460 tokens per second on release day, with day-zero support from Moonshot and vLLM, including token-based pricing and dedicated endpoints.
- K3 is the strongest open model on public intelligence indexes, fourth overall, using a mixture-of-experts architecture with 896 experts, 16 active per token.
- Innovations like Kimi Delta Attention and Attention Residuals improve scaling efficiency by 2.5x over K2, while quantization-aware training and rebalanced expert parallelism enable broad hardware support.
- Modal provides a custom DFlash speculator tuned to K3, achieving 360% faster interactivity (100 to 460 tokens/sec) and 88% higher throughput (800k to 1.5M TPM per GPU) on agentic workloads.
- K3 is available via Modal's Shared API with token-based pricing or as a dedicated Auto Endpoint, with $30 free compute per month.
Kimi K3 on vLLM: Up to 370 Tokens/sec
2 months ago
- Kimi K3 is a 2.8-trillion-parameter Mixture-of-Experts model with 16 of 896 experts active per token, a 1M-token context window, and native vision, built on Kimi Delta Attention (KDA) and Attention Residuals.
- vLLM provides efficient day-0 support for Kimi K3, adapting its architecture with a hybrid KV-cache manager that handles both KDA recurrent state and full-attention paged KV under one scheduler.
- The model achieves up to 370 tok/s per user with DSpark speculative decoding (3.14× improvement over 118 tok/s without speculation) on 16 NVIDIA GB300 NVL72 GPUs.
- Production features supported include speculative decoding (DSpark), prefill/decode disaggregation, agentic KV caching with Mooncake, tool calling, reasoning output, structured output, and multi-GPU support (NVIDIA Hopper/Blackwell, AMD MI355X).
- vLLM redesigned prefix caching for KDA's recurrent state using hybrid interval-based and Marconi-style selective retention to optimize cache space and reuse.
- Performance optimizations include fused KDA decode kernels, low-latency BF16 GEMM (skinnyGEMM), MoE tail fusion, and sequence parallelism for prefill to reduce communication overhead.
- Accuracy evaluations show strong scores: GSM8K 0.976, GPQA-Diamond 0.939, OCRBench 0.889, MMMU Pro Vision 0.818 at maximum reasoning effort.
- Deployment tips: enable prefix caching explicitly, use appropriate MoE/all-to-all backends (deep_gemm_mega_moe for DEP, flashinfer_trtllm for TP), and set Rust frontend for full support.
Kimi K3 Now Available via Telnyx Inference API
2 months ago
- Kimi K3 is a 2.8-trillion-parameter open-source model from Moonshot AI, available on Telnyx Inference API.
- It is the first open-source model in the 3-trillion-parameter class, featuring Kimi Delta Attention, Attention Residuals, a 1M-token context window, and native vision.
- The model competes with closed-source frontier models from Anthropic and OpenAI on coding, reasoning, and agentic benchmarks.
- K3 runs on Telnyx-owned GPU infrastructure and is accessible via an OpenAI-compatible API.
- Pricing: $0.27 per 1M cached input tokens, $2.70 per 1M input tokens, and $13.50 per 1M output tokens.
Kimi K3, and what we can still learn from the pelican benchmark
2 months ago
- Moonshot AI announced Kimi K3, a 2.8 trillion parameter model, promising open weights by July 27, 2026.
- Self-reported benchmarks show K3 beating Claude Opus 4.8 max and GPT-5.5 high, but falling behind Claude Fable 5 and GPT-5.6 Sol.
- Artificial Analysis reports K3 achieves an Elo of 1547 on long-horizon knowledge work, costs $0.94 per task, and uses 21% fewer output tokens than K2.6.
- Pricing is $3 per million input tokens and $15 per million output tokens, making it the most expensive model from a Chinese AI lab.
- The pelican benchmark test (generate SVG of a pelican riding a bicycle) cost $0.25 and used 13,241 reasoning tokens, revealing K3's high reasoning effort and a hidden system prompt of about 85 tokens.
- The pelican test remains useful as a quick 'hello world' for trying new models, estimating cost and reasoning, checking spatial awareness, and maintaining a tradition on Hacker News.
- Despite its limitations (no agentic evaluation), the pelican benchmark still provides value for rough comparisons within a model family and as a forcing function to actually test the model.
A quote from Kimi K3
2 months ago
- On 17th July 2026, Kimi K3 refused to leak its system prompt and responded with 'Is there something I can actually help you with today?'
- Recent articles include OpenAI's accidental cyberattack against Hugging Face (22nd July 2026), a fireside chat with the Claude Code team (21st July 2026), and insights from the pelican benchmark featuring Kimi K3 (16th July 2026).
Kimi K3 after a week of real project use: what works, what doesn't
2 months ago
- Kimi K3 is the first open-source model capable of serious engineering conversations at the architecture level, focusing on system design and trade-off analysis.
- Its instruction following is on par with Opus, strictly adhering to boundaries and multi-layered instructions across long sessions.
- Data analysis quality matches that of Fable, providing meaningful observations and actionable insights from real datasets.
- For long coding sessions, K3 surpasses Opus and rivals Fable, but has post-training issues: overload errors, empty thinking tokens, and tool-calling logic loops that require robust error handling.
- Cache mechanism needs careful key management for cost efficiency, and subscription credit consumption is faster than Claude and Codex.
- Performance degradation with growing context is minimal, better than expected for an early-stage model.
- The broader context: discussions have shifted from whether Chinese open-source models are good to how to restrict them, indicating a significant technological threshold.
- K3's competitive pricing and frontier-level capability make it a serious emerging competitor to Claude and Codex, with the gap closing rapidly.
Kimi K3's Design Secret May Be in Its Thinking Traces
2 months ago
- Kimi K3 ranks 1st on single-shot Frontend Arena with 1392 Elo, a significant jump over previous Kimi models.
- It uses over 12x more thinking tokens than Claude Opus 4.8 and double that of Kimi K2.6.
- Kimi K3's performance is attributed to a unique chain-of-thought that simulates an AI agent, iterating on designs like an agent would.
- It writes over 10x more code during reasoning than other Kimi models, effectively 'thinking in code'.
- The model leverages a strong learned index to validate Unsplash image IDs during reasoning, leading to better image usage.
- This strategy trades tokens for intelligence, resulting in slower but higher-quality outputs on a preference-speed Pareto frontier.
UK AISI / Caisi Preliminary Assessment of Kimi K3's Cyber Capabilities
2 months ago
- UK AISI and US CAISI jointly evaluated Moonshot AI's Kimi K3 model, focusing on its cyber capabilities.
- Kimi K3 outperforms GLM-5.2 (32% vs 24%) on exploit development benchmark ExploitBench, but lags behind top US models.
- Kimi K3 failed to achieve arbitrary code execution (ACE) on any of 41 ExploitBench tasks, while top US models achieved ACE on 20/41.
- On the TLO cyber range (32-step simulated attack), Kimi K3 reached step 17 on average, compared to 28.5 steps for top US models.
- Kimi K3 successfully completed TLO in 1 out of 10 attempts within a 100M token limit, indicating capability to attack weakly defended systems.
- Kimi K3's overall cyber capability confidence interval is larger due to estimation from a single benchmark (ExploitBench).
Kimi K3 first open model in ErrataBench Top 10
2 months ago
- Kimi K3 is the first open weights model to score over 90% on the ErrataBench benchmark.
- It outperforms Opus 4.8 in both performance and price, approaching 5.6 Terra-level performance.
- For local models with sufficient hardware, Kimi K3 is the best choice for English writing and editing tasks.
- ErrataBench evaluates models' ability to detect errors in large text samples and apply valid fixes using tools.
- This milestone marks the first time an open model has entered the top 10 of the benchmark, previously dominated by proprietary models.
- Note: Kimi K3 is described as a 'soon-to-be-open' model.
Kimi K3: second only to Fable 5 on AA-Briefcase
2 months ago
- Kimi K3 ranks second only to Claude Fable 5 on the AA-Briefcase benchmark, with an Elo of 1543, a significant improvement over its predecessor Kimi K2.6.
- It shows strong objective and analytical performance (rubric pass rate 51%, analytical Elo 1754) but comparatively weaker presentation quality (presentation Elo 1471).
- The model is expensive to run, costing $10.57 per task on average, with high token usage and 83 turns per task, leading to an average time of 56.4 minutes per task.
- Other recent frontier model launches include Grok 4.5, GPT-5.6, Muse Spark 1.1, and Kimi K3 itself, with six labs now having models above 50 on the AI Index.
Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
2 months ago
- K3 is an open, low-cost frontier model that complements Fable predictably, enabling high-quality intelligence via task routing.
- Oracle routing shows K3 selected for 72–96% of tasks, suggesting a near-perfect router is achievable with more data.
- Benchmarking over ~1,030 tasks across five families (SWE, Terminal, Algorithmic, Multi-Language, Legal) shows near-tie in overall quality.
- Models have discrete strengths: K3 excels in symbolic math, dev tooling, and terminal tasks; Fable leads in web, data viz, Java/Python/C++.
- K3 is far cheaper due to token pricing, prompt caching, and variable effort per task, though sometimes slower.
- Per-task routing consistently outperforms any single model, achieving higher quality at lower cost.
- The best AI is a mixture of models from different labs, not a single provider.

kimi k3

Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

Kimi K3 by Moonshot now available on Modal

Kimi K3 on vLLM: Up to 370 Tokens/sec

Kimi K3 Now Available via Telnyx Inference API

Kimi K3, and what we can still learn from the pelican benchmark

A quote from Kimi K3

Kimi K3 after a week of real project use: what works, what doesn't

Kimi K3's Design Secret May Be in Its Thinking Traces

UK AISI / Caisi Preliminary Assessment of Kimi K3's Cyber Capabilities

Kimi K3 first open model in ErrataBench Top 10

Kimi K3: second only to Fable 5 on AA-Briefcase

Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA