Is the "cost of inference" going up or down?
7 days ago
- #AI economics
- #LLM inference
- #cost analysis
- The cost of LLM inference has decreased in terms of price per quality, similar to how TVs have become cheaper for better quality over time.
- Despite lower costs per quality, overall spending on LLMs has increased because users opt for higher-quality models and more extensive usage.
- Ed Zitron argues that the cost of inference has gone up, but this is misleading as it conflates total spending with cost per unit or quality.
- Users have choices: use cheaper older models, pay the same for better results, or spend more for superior performance—many choose the latter two.
- Model providers and applications can sustain profitability by maintaining margins on increased usage, not by reducing costs alone.
- Zitron's analysis often frames any pricing or usage change negatively, suggesting bias rather than objective evaluation of industry trends.
- The debate highlights the importance of distinguishing between cost per unit, cost per quality, and total spending when analyzing industry trends.