The Knowable Future
The Knowable Future:
Forecastability and the
Limits of Prediction
The limit on prediction is set by the information in the data, not the sophistication of the model.
A research programme by Dr Peter Catt
Auckland, Aotearoa New Zealand.
​
The core research question: How much of the future is knowable from the information available now?
How much of the future can actually be known? Forecastability answers that question. It measures how much the information available today can tell us about what comes next, and how that changes the further ahead we look.
More precisely, forecastability is the predictive information available about a target at a given forecast horizon, given a declared information set: the data a forecast is allowed to use, stated in advance. That may be the target’s own history alone, or it may add other observed variables and external signals available when the forecast is made. The Knowable Future studies how forecastability varies across horizons, how it can be estimated, and what it implies for forecasting and decisions.
There is a limit to how well anything can be forecast, and it is set by the information available, not by the forecasting model. No method, however sophisticated, can extract more predictive information than its data contains. Measured across forecast horizons, forecastability forms a profile, written F(h). Under logarithmic loss, mutual information provides an exact information-theoretic representation of that profile. In practice the profile is estimated from data, so any measured value carries some uncertainty. The formal statement is on the Definitions page.
Forecastability comes before model selection. It characterises the predictive-information environment a forecasting problem presents: how much there is to learn, and at which horizons. Model evaluation then determines which methods exploit that information effectively.
How Much of the Future Is Knowable from the Past?
Why prediction has limits
In 1814 Pierre-Simon Laplace imagined an intellect that knew the position and momentum of every particle in the universe, a figure now known as Laplace's demon. For such an intellect, he argued, nothing would be uncertain; the future would lie open like the past. The demon did not survive the twentieth century.
The first failure is quantum mechanical. Bell's theorem, confirmed by loophole-free experiments in 2015, rules out any account in which measurement outcomes are locally predetermined. Deterministic interpretations of quantum mechanics survive, but only by abandoning locality, and none restores a world in which the future is simply read off the present.
The second failure is classical. Even a perfectly deterministic system defeats prediction when its dynamics are chaotic. The smallest error in the starting state grows exponentially, at a rate set by the Lyapunov exponent, and no measurement is ever fine enough to pin the state down. That is why atmospheric predictability ends near two weeks: not for want of better models, but because the atmosphere forgets its own past.
Information theory turns that forgetting into a number. The past holds a measurable quantity of information about the future, and the data processing inequality guarantees that no method, and no amount of computation, can extract more. What survives Laplace is not the demon but the question of degree: how much of the future the past still contains. Prediction has a ceiling, and the ceiling belongs to the data. This programme measures it.
​
​