- Achieved a record of 511.6 tokens per second serving Kimi-K2.6, a 1 trillion-parameter model, on four B200 GPUs with lossless speculative decoding.
- The record was set at 505.9 tokens per second and raised via blind re-runs using publicly available components, outperforming other providers on public leaderboards by up to 13.4%.
- Measured the 'verify tax' at 0.397ms per token in trillion-parameter MoE models, showing that verification cost scales with draft length due to expert routing.
- Developed Fovea, a trained drafter head that improves performance by 5.8% over public heads, demonstrating the potential of domain-specific optimization.
- Identified and fixed a training defect where distillation teacher was misaligned by one transformer layer, highlighting the importance of correct hidden-state capture in drafter training.