Self-hosting Kimi K3: 20% more hardware cost, 20% better task resolution
13 hours ago
- Kimi K3 achieves 86.4% resolution rate but requires 8×B300 nodes and is slower than Claude Code.
- Token costs for AI coding agents are rising sharply, with 90th percentile users spending ~$7,300/year.
- Over 70% of API revenue from major model providers comes from coding use cases.
- Alternatives to frontier APIs include API routers, cheaper model tiers, or self-hosting with GPUs.
- Self-hosting involves paying for infrastructure 24/7, with average GPU utilization of 15-35%.
- Hardware must be sized for peak usage, and utilization is key to cost-effectiveness.
- Different hardware options (DGX Spark, H200, H200 cluster, B200 rack) support varying concurrency and model quality.
- Developer experience degrades at high concurrency; each setup has a comfortable concurrency limit (e.g., GLM-5.2 struggles beyond 8 users).
- Cost analysis shows API can be cheaper than renting for some models (e.g., DeepSeek V4 Flash), while self-hosting can beat API pricing for others (e.g., GLM-5.2 at 15% utilization).
- Open-weight models are closing the quality gap with frontier models but require powerful hardware like B200 racks.