AI-assisted analytics now 10x cheaper

6 hours ago

Small AI models are now as smart as prior flagship models, enabling new strategies for agentic analytics that prioritize speed and cost over raw intelligence.
OpenAI's GPT 5.6 Luna, with an 80% price cut, achieves 99.8% accuracy on SQL benchmarks at a 5x lower price than Gemini-3-Flash, and even 10x cheaper on semantic modeling benchmarks due to model caching improvements.
Luna occupies a new section of the price/performance curve, offering speed and cost efficiency for tasks that don't require maximum intelligence, such as SQL generation.
Practical steps include switching to Luna on max effort for low-intelligence tasks and asking more questions (Jevons Paradox) to leverage the technology's improved performance.
Larger shifts involve adopting faster analytical databases to avoid latency bottlenecks, building detailed context layers for weaker models, and running frequent evals with natural language questions to catch regressions and adapt to business changes.
Eval workflows should be lab-agnostic to avoid lock-in and allow quick switching to optimal models as prices and performance evolve.
The text introduces Guides, MotherDuck's context layer for AI agents, to convert raw intelligence into accurate answers with lower token spend.

Invest in upfront alignment by making implicit assumptions explicit in the initial prompt to avoid unexpected AI outputs.
Restart with better context rather than steering AI when initial outputs are far off; avoid compounding errors through path dependence.
Equip AI agents with the same tools you use (CLI, MCP, browser control) to handle setup, testing, and instrumentation tasks.

Tokenless reduces inference costs by routing requests to the most suitable model, canceling unnecessary ones to halve costs.
The service maintains output quality comparable to top models (e.g., Opus 4.8) while significantly lowering cost per task.
It offers drop-in compatibility with OpenAI/Anthropic APIs and provides model options like PRO, GPT 5.5, Opus 4.8, and MAX for different quality/savings balances.

AI models have reached an inflection point where their output is worth more than the cost for real development work, leading to increased inference spending.
The author's annual AI inference spend grew from $100 in 2023 to $5000 in 2025, reflecting growing utility.
o1-pro was the first model consistently useful for professional coding, capable of debugging large files and saving hours per week.

The transition from frontier model APIs to building your own open-source LLM inference stack often starts due to financial, legal, or engineering concerns.
The initial decision point is whether to buy from managed inference providers or build with open weights and your own hardware, with this post focusing on the build path.
Starting with a single replica using serving engines like vLLM, SGLang, or TensorRT-LLM typically works well, handling batching and KV cache management effectively.

The author initially used Claude Fable 5 for deep research on tokenomics, but it burned through the token limit in 30 minutes without producing results, prompting a need for optimization.
To reduce costs and improve efficiency, the author leveraged existing subscriptions (Claude, Codex, Antigravity) with shared memory via a custom setup, allowing multiple models to work together without extra payment.
A model-orchestration pattern was implemented, assigning specific roles to cheaper models (e.g., Claude Sonnet 5 for finding, Claude Opus 4.8 for verification) and using Claude Fable 5 only for planning, which extended research time from 30 minutes to several hours.

Invest in upfront alignment by making implicit assumptions explicit in the initial prompt to avoid unexpected AI outputs.
Restart with better context rather than steering AI when initial outputs are far off; avoid compounding errors through path dependence.
Equip AI agents with the same tools you use (CLI, MCP, browser control) to handle setup, testing, and instrumentation tasks.

Tokenless reduces inference costs by routing requests to the most suitable model, canceling unnecessary ones to halve costs.
The service maintains output quality comparable to top models (e.g., Opus 4.8) while significantly lowering cost per task.
It offers drop-in compatibility with OpenAI/Anthropic APIs and provides model options like PRO, GPT 5.5, Opus 4.8, and MAX for different quality/savings balances.

AI models have reached an inflection point where their output is worth more than the cost for real development work, leading to increased inference spending.
The author's annual AI inference spend grew from $100 in 2023 to $5000 in 2025, reflecting growing utility.
o1-pro was the first model consistently useful for professional coding, capable of debugging large files and saving hours per week.

The transition from frontier model APIs to building your own open-source LLM inference stack often starts due to financial, legal, or engineering concerns.
The initial decision point is whether to buy from managed inference providers or build with open weights and your own hardware, with this post focusing on the build path.
Starting with a single replica using serving engines like vLLM, SGLang, or TensorRT-LLM typically works well, handling batching and KV cache management effectively.

The author initially used Claude Fable 5 for deep research on tokenomics, but it burned through the token limit in 30 minutes without producing results, prompting a need for optimization.
To reduce costs and improve efficiency, the author leveraged existing subscriptions (Claude, Codex, Antigravity) with shared memory via a custom setup, allowing multiple models to work together without extra payment.
A model-orchestration pattern was implemented, assigning specific roles to cheaper models (e.g., Claude Sonnet 5 for finding, Claude Opus 4.8 for verification) and using Claude Fable 5 only for planning, which extended research time from 30 minutes to several hours.

Hasty Briefsbeta