Monday, August 11, 2014

Simon Wren-Lewis — On Macroeconomic Forecasting

Macroeconomic forecasts produced with macroeconomic models tend to be little better than intelligent guesswork. That is not an opinion – it is a fact. It is a fact because for decades many reputable and long standing model based forecasters have looked at their past errors, and that is what they find. It is also a fact because we can use models to generate standard errors for forecasts, as well as the most likely outcome that gets all the attention. Doing so indicates errors of a similar magnitude as those observed from past forecasts. In other words, model based forecasts are predictably bad. 
The sad news is that this situation has not changed since I was involved in forecasting around 30 years ago. During the years before the Great Recession (the Great Moderation) forecasts might have appeared to get better, but that was because most economies became less volatile. As is well known, the Great Recession was completely missed. Forecasting has not improved, because our ability to explain variables like consumption or investment has not improved. 
Does that mean that macroeconomics is not making any progress? I do not want to get sidetracked on this issue, but it could just be that as macroeconomists understand the economy as it was a little better, the nature of the economy also changes because of factors like financial innovation or technical progress. Does this mean macroeconomics is useless? No, in much the same way as medicine cannot predict year by year how your health changes but is quite good at responding to these changes.
Frank admission, but false analogy with medicine. Conventional economic remedies are more like nostrums than medicine — if not snake oil based on ideology rather than science.
The rather boring truth is that it is entirely predictable that forecasters will miss major recessions, just as it is equally predictable that each time this happens we get hundreds of articles written asking what has gone wrong with macro forecasting. The answer is always the same - nothing. Macroeconomic model based forecasts are always bad, but probably no worse than intelligent guesses.


Mainly Macro
On Macroeconomic Forecasting
Simon Wren-Lewis | Professor of Economics, Oxford University

5 comments:

Ryan Harris said...

"medicine cannot predict year by year how your health changes but is quite good at responding to these changes"

And medicine must be proven to work before it can be approved for use. Economic solutions, like lowering interest rates, and QE, and austerity have been proven to NOT work very well.

Brian Romanchuk said...

His arguments are reasonable, but fairly subversive from the point of view of macro theory.

If you cannot come up with good forecasts, how can the optimising representative household?

Monetary policy is viewed as a panacea, but how can it be effective if the central bank cannot forecast the effect of its policies?

Tom Hickey said...

Shows the limits of formalization in economics. Business gets along with out relying heavily on formalized models by anticipation based on experience, and distinguishing between signal and noise, and interpreting signals tentatively, adapting to context.

There are three big reasons that highly formalized econometric models don't work as forecasting tools.

First, they need to be simplified to be mathematically tractable, resulting in unrealistic and even unreasonable assumptions.

Secondly, the data is not real time for the most part and is not that accurate when reported. By the time adjustments are made, time has moved on.

These introduce epistemic uncertainty.

The third factor is the ontological uncertainty inherent in complex adaptive social systems due to reflexivity and emergence, on one hand, and on the other, surprises and shocks.

Having to make important decisions under epistemic and ontological uncertainty is like juggling with knives. Experience, intuition, feeling, and luck are more significant than the information available from formal models and data about the past.

Looking for an econometric model to yield precise forecasts that are borne out is a fool's errand, at least before the roll out of much more highly developed AI, and that is still unlikely to introduce precision, since it is beyond the tolerance of the data given the context.

I would compare this with prosecution of a war. There are no models capable of precise prediction of future events. Moreover, intelligence is never as good as commanders would like it to be. So that have to be adaptive and agile, while preserving resilience.

This doesn't mean that commanders don't rely on strategy and tactics, but those are loose rather than tight. They don't want to find themselves reactive instead of proactive. So they have to a ready to seize opportunity and able to recognize and act on it quickly before the window closes.

Macro is a policy science, just a strategy is a military science. Economic response has to be adaptive, agile and resilient, too. Models might provide some insights and an overall lay of the land, but they are not pilots and even less homing missiles.

Macro economists are like intelligence officers. Their job is to provide those responsible for policy, strategy and tactics, that is the executive and legislative branches, with the best information along with a clear statement of its limitations and deficiencies.

This is not what we hear from the pontificators, who may admit that the precision of a model's output may not be accurate descriptively or predictively and yet go to act as if it were in making policy pronouncements.

Business people have learned this, and so has the military. Judging from Fed guidance and minutes, they, too, take an adaptive approach, even though central banks have the resources to run highly sophisticated models and have fastest access to the best data that governments can collect and process into information.

Most importantly, however, this admission undermines the retort of conventional economists that while heterodox economists may have foreseen the crisis developing, they had no model, with the implication that it was a lucky guess.

But there's a lot of room between precise prediction in terms of a formalized model and just guessing. Steve Keen, Michael Hudson and Wynne Godely didn't pull their their early warnings out of hat. There was a clear rationale, even if not an econometric model acceptable to the mainstream.

NeilW said...

So in summary it is still better to be roughly right than precisely wrong.

:)

Ryan Harris said...
This comment has been removed by the author.