Ways to think about token pricing
24 days ago
- Token prices are currently in a state of supply crunch and instability, with all variables in play, leading to an uncertain future equilibrium.
- Supply is increasing due to massive data center and semiconductor investments, improved inference efficiency, and variable token efficiency in new models.
- Demand surge is driven mainly by software development, a relatively small field, leaving future use cases, scale, and token needs unknown.
- Inference currently has high gross margins, but profitability depends on covering training costs and uncertain future demand ROI.
- Bottom-up modeling of token pricing is challenging due to unknown variables like supply, demand, marginal costs, and ROI, similar to forecasting broadband in 1998.
- Top-down analysis considers factors like frontier model adoption, ROI for high-cost models, competition, and value capture by models versus surrounding tooling.
- Key uncertainties include whether frontier models will maintain competitive advantages, see reduced competition, or face commoditization like databases.
- Comparisons with fiber, mobile data, and semiconductors highlight risks of low-margin infrastructure with value captured elsewhere, but analogies lack predictive power.
- Structural uncertainty in AI is heightened by a lack of theoretical understanding of model improvements, making predictions about compute needs and demand volatile.
- For foundation models to avoid commoditization and achieve market dominance, significant changes such as network effects, reduced competition, or regulatory shifts are needed, but current dynamics point toward commodity infrastructure.