Everyone is building LLM routers, we deprecated ours
4 hours ago
- The author deprecated their LLM router after four months of mixed results, concluding that sticking to a single battle-tested model is better for most use cases.
- Complexity of a task cannot be deduced from the prompt alone; context emerges later through tool calls, web searches, etc.
- Cache is more effective than routing for cost reduction, as cached reads are 75–90% cheaper and prefix caching benefits system prompts and conversation history.
- LLM routers break behavior consistency; engineers should understand model trade-offs and select tools intentionally, similar to craftsmen.
- Unpredictability from routing adds cost in automated workflows, making evals, system prompts, and observability harder to maintain.
- Overall, the savings from routing are often offset by hidden costs, making it not worthwhile in most observed use cases.