The AI Superforecasters Are Here
5 hours ago
- #prediction markets
- #future technology
- #AI forecasting
- AI superforecasters have emerged, with startups like FutureSearch and Preseen claiming significant success in prediction markets and stock market outperformance.
- These AI systems typically use large language models like ChatGPT or Claude, enhanced with 'scaffolds' for improved forecasting through extensive research and subagents.
- Current AI superforecasters are competitive with top human superforecasters, with performance metrics showing near parity in forecasting competitions like the Metaculus Cup.
- AI superforecasters offer advantages over humans: they are faster, more diligent, standardized, and can be accessed easily without the logistical challenges of hiring human experts.
- In finance, AI may have a special advantage, as evidenced by Preseen's bot outperforming humans in finance-focused forecasting competitions.
- The trend suggests AI forecasting capabilities are improving rapidly, with predictions that they will surpass human experts within the next year or two.
- AI superforecasters could transform decision-making in finance, business, government, and personal life by providing accessible, high-quality probabilistic forecasts.
- Prediction markets may remain important for epistemic infrastructure, providing canonical forecasts that aggregate AI opinions and mitigate biases across different AI systems.
- Publicly available AI superforecasters include FutureSearch (open for use) and Preseen (in closed beta), with standard AIs also offering some forecasting capability behind the state of the art.
- A potential future role for AI superforecasters is as an 'opinion layer' for AIs, enabling them to provide forecasts on controversial topics while remaining grounded in empirical benchmarks.