- Time series forecasting is fundamentally harder than classical ML because data comes from a single dependent path (a Data Generating Process) rather than i.i.d. samples from a distribution.
- The three main challenges are low signal-to-noise ratio (feature poor), lack of effective data (auto-correlation shrinks sample size, cycles matter more than timestamps), and severe distribution shift (forecasting is extrapolation, not interpolation).
- Stationarity, ergodicity, and autocorrelation are key properties that determine whether a time series process is learnable, but even then, white noise shows that learnable properties do not guarantee forecastability.
- Different forecasting models (Naive, ARIMA, neural nets, foundation models) essentially differ in which lags they read and what kind of dependence they exploit, with limited success on inherently low-forecastability series.
- The main takeaway is that throwing more complex models or pooling data does not solve fundamental limitations; instead, incorporating external (exogenous) features may be a more promising direction for real-world forecasting.