- An AI Routing Policy is a documented team decision that assigns specific tasks to the most cost-effective execution path, whether it's a cheaper AI model, a frontier model, deterministic code, human review, or no automation, ensuring accountability and traceability.
- The policy requires defining sufficiency criteria—clear standards for acceptable output, similar to acceptance criteria—for each task class, along with escalation triggers and a designated owner to maintain and enforce the policy.
- Implementing a minimal routing log alongside the policy creates a record of actual usage, costs, and outcomes, enabling task-class attribution and calculation of Return on Tokens (ROT), which helps justify AI spend to finance teams.
- A key insight is that routing decisions should consider whether tasks need probabilistic AI judgment every time or can be handled by deterministic code created once, moving beyond just choosing among AI models.
- The policy makes trade-offs between cost, quality, risk, throughput, autonomy, and adaptability explicit, transforming AI usage from ad-hoc habits into governed, repeatable processes that survive individual departures.