The Knowable Future
The Knowable Future:
Forecastability and the
Limits of Prediction
The limit on prediction is set by the information in the data, not the sophistication of the model.
A research programme by Dr Peter Catt
Auckland, Aotearoa New Zealand.
The core research question: How much of the future is knowable from the past?
Forecastability is the extent to which the past contains exploitable information about the future. It is a property of the series and the horizon, not of any model. The forecastability profile F(h) makes this precise: the predictive information the past carries about the future at each horizon h.
F(h) sets the exploitability ceiling: the limit, at each horizon, on what any forecasting method drawing on the same past can achieve. No method can exceed it; good ones approach it. The Knowable Future estimates F(h) from training data alone, before any model is chosen, and reads forecasting performance against it.
The consequence is forecast triage. Where information is rich, sophistication earns its keep; where it is thin, simple baselines are near-optimal; where it is absent, the real work is decision design. Model choice is the second question; how much of the future is knowable is the first.
How Much of the Future Is Knowable from the Past?
Why prediction has limits
In 1814 Pierre-Simon Laplace imagined an intellect that knew the position and momentum of every particle in the universe, a figure now known as Laplace's demon. For such an intellect, he argued, nothing would be uncertain; the future would lie open like the past. The demon did not survive the twentieth century.
The first failure is quantum mechanical. Bell's theorem, confirmed by loophole-free experiments in 2015, rules out any account in which measurement outcomes are locally predetermined. Deterministic interpretations of quantum mechanics survive, but only by abandoning locality, and none restores a world in which the future is simply read off the present.
The second failure is classical. Even a perfectly deterministic system defeats prediction when its dynamics are chaotic. The smallest error in the starting state grows exponentially, at a rate set by the Lyapunov exponent, and no measurement is ever fine enough to pin the state down. That is why atmospheric predictability ends near two weeks: not for want of better models, but because the atmosphere forgets its own past.
Information theory turns that forgetting into a number. The past holds a measurable quantity of information about the future, and the data processing inequality guarantees that no method, and no amount of computation, can extract more. What survives Laplace is not the demon but the question of degree: how much of the future the past still contains. Prediction has a ceiling, and the ceiling belongs to the data. This programme measures it.