21 Jan Be critical when you encounter predictions (Rolf Dobelli)
‘Regime shift in North Korea in two years.’ ‘Sour grapes for France as Argentinian wines expected to dominate.’ ‘Euro collapse likely.’ ‘Low-cost space flights by 2025.’ ‘No more crude oil in 15 years.’
Every day, experts bombard us with predictions, but how reliable are they? Until a few years ago, no one bothered to check. Then along came Philip Tetlock. Over a period of ten years, he evaluated 28,361 predictions from 284 self-appointed professionals. The result: in terms of accuracy, the experts fared only marginally better than a random forecast generator. Ironically, the media darlings were among the poorest performers; and of those the worst were the prophets of doom and disintegration. Examples of their far-fetched forecasts included the collapse of Canada, Nigeria, China, India, Indonesia, South Africa, Belgium and the E.U. None of these countries has imploded.
‘There are two kinds of forecasters: those who don’t know, and those who don’t know they don’t know,’ wrote Harvard economist John Kenneth Galbraith. With this he made himself a ﬁgure of hatred in his own guild. Fund manager Peter Lynch summed it up even more cuttingly: ‘There are 60,000 economists in the U.S., many of them employed full-time trying to forecast recessions and interest rates, and if they could do it successfully twice in a row, they’d all be millionaires by now [?. . .?] As far as I know, most of them are still gainfully employed, which ought to tell us something.’ That was ten years ago. Today, the U.S. could employ three times as many economists – with little or no effect on the quality of their forecasts.
The problem is that experts enjoy free rein with few negative consequences. If they strike it lucky, they enjoy publicity, consultancy offers and publication deals. If they are completely off the mark, they face no penalties – either in terms of ﬁnancial compensation or in loss of reputation. This win-win scenario virtually incentivises them to churn out as many prophecies as they can muster. Indeed, the more forecasts they generate, the more will be coincidentally correct. Ideally, they should have to pay into some sort of ‘forecast fund’ – say $1,000 per prediction. If the forecast is correct, the expert gets his money back with interest. If he is wrong, the money goes to charity.
So what is predictable and what is not? Some things are fairly simple. For example, I have a rough idea of how many pounds I will weigh in a year’s time. However, the more complex a system, and the longer the time frame, the more blurred the view of the future will be. Global warming, oil prices or exchange rates are almost impossible to foresee. Inventions are not at all predictable because if we knew what technology we would invent in the future we would already have invented it.
So, be critical when you encounter predictions. Whenever I hear one, I make sure to smile, no matter how bleak it is. Then I ask myself two questions. First, what incentive does the expert have? If he is an employee, could he lose his job if he is always wrong? Or is he a self-appointed guru who earns a living through books and lectures? The latter type of forecaster relies on the media’s attention so, predictably, his prophecies tend to be sensational. Second, how good is his success rate? How many predictions has he made over the past ﬁve years? Out of these, how many have been right and how many have not? This information is vital yet often goes unreported. I implore the media: please don’t publish any more forecasts without giving the pundit’s track record.
Finally, since it is so ﬁtting, a quote from former British prime minister Tony Blair: ‘I don’t make predictions. I never have, and I never will.’
The Art of Thinking Clearly