Have you ever made a prediction that turned out to be wrong? What mental errors thwart accurate forecasting?<\/p>\n\n\n\n
In The Signal and the Noise<\/em>, Nate Silver examines mental errors that make your predictions inaccurate. These include making faulty assumptions<\/a>, being overconfident, trusting data and technology too readily, seeing what you want to see, and following the wrong incentives.<\/p>\n\n\n\n Keep reading to learn how these mental errors cause inaccurate predictions.<\/p>\n\n\n\n\n\n\n\n One of the mental errors that influence predictions is our tendency to make faulty assumptions. Our predictions tend to go awry when we don\u2019t have enough information or a clear enough understanding of how to interpret our information. This problem gets even worse, Silver says, when we assume that we know more than we actually do. He argues that we seldom recognize when we\u2019re dealing with the unknown because our brains tend to make inaccurate predictions based on analogies to what we do<\/em> know<\/strong>.\u00a0<\/p>\n\n\n\n (Shortform note: In Thinking, Fast and Slow<\/em><\/a>, Daniel Kahneman explains that the brain uses these analogies to conserve energy by substituting an easier problem in place of a hard one<\/a>. For example, determining whether a stock is a good investment is a complicated question that requires research, analysis, and reflection. Left to its own devices, your brain is likely to skip all that work and instead answer an easier question such as whether you have a good feeling about the company or whether the stock has done well recently. According to Kahneman, you can overcome this quick, associative reasoning by deliberately engaging your brain\u2019s slower, more analytical faculties<\/a>.)<\/p>\n\n\n\n To illustrate the danger of assumptions, Silver says that the 2008 financial crisis resulted in part from two flawed assumptions made by ratings agencies who gave their highest ratings to financial products called CDOs that depended on mortgages not defaulting.<\/p>\n\n\n\n Silver argues that these assumptions exacerbated a bad situation because they created the illusion that these products were safe investments when in fact, no one really understood their actual risk.<\/p>\n\n\n\n (Shortform note: These assumptions also highlight an especially pernicious thinking error called availability bias<\/a><\/em>. In Thinking, Fast and Slow<\/em><\/a>, Kahneman explains that availability bias occurs when we misjudge an event\u2019s likelihood based on how readily we can think of examples of that event<\/a>. This error is, in part, what led to the assumption that CDOs were safe. As Michael Lewis explains in The Big Short<\/em><\/a>, investors and analysts repeatedly reasoned that because mortgages hadn\u2019t tended to default recently, they were unlikely to do so going forward<\/a>\u2014even though the CDO<\/a> bubble was created by an unprecedented laxity in lending practices.)<\/p>\n\n\n\n According to Silver, the same faulty assumptions that make our predictions less accurate also make us too confident in how good these predictions are. One dangerous consequence of this overconfidence is that by overestimating our certainty, we underestimate our risk<\/strong>. Silver argues that we can easily make grievous mistakes when we think<\/em> we know the odds but actually don\u2019t. You probably wouldn\u2019t bet anything on a hand of cards if you didn\u2019t know the rules of the game\u2014you\u2019d realize that your complete lack of certainty would make any bet too risky. But if you misunderstood<\/em> the rules of the game (say you mistakenly believed that three of a kind is the strongest hand in poker), you might make an extremely risky bet while thinking it\u2019s safe.<\/p>\n\n\n\n (Shortform note: One way to avoid this overconfidence trap is to literally think of our decisions as bets. In Thinking in Bets<\/em><\/a>, former professional poker player Annie Duke argues that most of the decisions we make carry some degree of uncertainty and therefore risk\u2014after all, we rarely have complete information about any given situation. She argues that, as a result, making a good decision isn\u2019t about picking the \u201cright\u201d choice or avoiding a \u201cwrong\u201d choice: It\u2019s a matter of determining which option is most likely<\/em> to give you the outcome you want<\/a>. When you think this way, you can act knowing that you\u2019ve given yourself the best chance of success\u2014but that success isn\u2019t a sure thing.)<\/p>\n\n\n\n Silver further argues that it\u2019s easy to become overconfident if you assume a prediction is accurate just because it\u2019s precise<\/strong>. With information more readily available than ever before and with computers to help us run detailed calculations and simulations, we can produce extremely detailed estimates that don\u2019t necessarily bear any relation to reality, but whose numerical sophistication might fool forecasters and their audiences into thinking they\u2019re valid. Silver argues that this happened in the 2008 financial crisis, when financial agencies presented calculations whose multiple decimal places obscured the fundamental unsoundness of their predictive methodologies. <\/p>\n\n\n\n (Shortform note: Whereas Silver argues specifically that precise <\/em>predictions inspire overconfidence, in The Black Swan<\/em><\/a>, Nassim Nicholas Taleb<\/a> argues that numerical projections of any kind\u2014even imprecise ones\u2014tend to lead to (and reflect) overconfidence<\/a> because random events make most predictions fundamentally unreliable given enough time. For example, Taleb cites forecasts from 1970 that predicted that oil prices would stay the same or decline in the next decade. In reality, events like the Yom Kippur War (1973) and the Iranian Revolution (1979) caused unprecedented oil price spikes during that period. Because it\u2019s impossible to predict events like these<\/a>, Taleb suggests that we should have very little confidence in predictions in general.)<\/p>\n\n\n\n As noted earlier, one of the challenges of making good predictions is a lack of information. By extension, the more<\/em> information we have, the better our predictions should be, at least in theory. By that reasoning, today\u2019s technology should be a boon to predictors\u2014we have more data than ever before, and thanks to computers, we can process that data in ways that would have been impractical or impossible in the past. However, Silver argues that data and computers both present their own unique problems that can hinder predictions as much as they help them.<\/p>\n\n\n\n For one thing, Silver says, having more data doesn\u2019t inherently<\/em> improve our predictions. He argues that as the total amount of available data has increased, the amount of signal (the useful data) hasn\u2019t<\/strong>\u2014in other words, the proliferation of data means that there\u2019s more noise (irrelevant or misleading data<\/a>) to wade through to find the signal. At worst, this proportional increase in noise can lead to convincingly precise yet faulty predictions when forecasters inadvertently build theories that fit the noise rather than the signal.<\/p>\n\n\n\n Likewise, Silver warns that computers can lead us to baseless conclusions when we overestimate their capabilities or forget their limitations<\/strong>. Silver gives the example of the 1997 chess series between grandmaster Garry Kasparov and IBM\u2019s supercomputer Deep Blue. Late in one of the matches, Deep Blue made a move that seemed to have little purpose, and afterward, Kasparov became convinced that the computer must have had far more processing and creative power than he\u2019d thought\u2014after all, he assumed, it must have had a good reason for the move. But in fact, Silver says, IBM eventually revealed that the move resulted from a bug: Deep Blue got stuck, so it picked a move at random.<\/p>\n\n\n\n According to Silver, Kasparov\u2019s overestimation of Deep Blue was costly: He lost his composure and, as a result, the series. To avoid making similar errors, Silver says you should always keep in mind the respective strengths and weaknesses of humans and machines<\/a><\/strong>: Computers are good at brute-force calculations, which they perform consistently and accurately, and they excel at solving finite, well-defined, short- to mid-range problems (like a chess game). On the other hand, Silver says, computers aren\u2019t flexible or creative, and they struggle with large, open-ended problems. Conversely, humans can do several things computers can\u2019t: We can think abstractly, form high-level strategies, and see the bigger picture<\/a>.<\/p>\n\n\n\n Another challenge of prediction is that many forecasters have a tendency to see what they want or expect to see in a given set of data\u2014and these types of forecasters are often the most influential ones<\/strong>. Silver draws on research by Phillip E. Tetlock (author of Superforecasting<\/em><\/a>) that identifies two opposing types of thinkers: hedgehogs and foxes<\/a>.<\/p>\n\n\n\n Silver argues that although foxes make more accurate predictions than hedgehogs, hedgehog predictions get more attention because the hedgehog thinking<\/a> style is much more media-friendly<\/strong>: Hedgehogs give good sound bites, they\u2019re sure of themselves (which translates as confidence and charisma), and they draw support from partisan audiences who agree with their worldview.<\/p>\n\n\n\n Furthermore, Silver points to the media\u2019s preference for hedgehog-style predictions as an example of how our predictions can be warped by bad incentives<\/strong>. In this case, for forecasters interested in garnering attention and fame (which means more airtime and more money), bad practices pay off. That\u2019s because all the things that make for good predictions\u2014cautious precision, attentiveness to uncertainty, and a willingness to change your mind\u2014are a lot less compelling on TV than qualities that lead to worse predictions\u2014broad, bold claims, certainty, and stubbornness.<\/p>\n\n\n\n (Shortform note: In The Art of Thinking Clearly<\/em><\/a>, Rolf Dobelli explains why<\/em> we\u2019re drawn to predictors who exude certainty and confidence. For one thing, he says, we dislike ambiguity and uncertainty<\/a>\u2014in fact, that\u2019s why we try to predict the future<\/a> in the first place. Meanwhile, we\u2019re quick to assume that experts know what they\u2019re doing (a phenomenon called \u201cauthority bias\u201d<\/a>) and we struggle to understand the probabilities<\/a> involved in careful forecasts. Taken all together, these mental biases combine to make simplistic, confident predictions appear more convincing than more nuanced (but possibly more accurate) ones.)<\/p>\n\n\n\n Similarly, Silver explains that predictions can be compromised when forecasters are concerned about their reputations<\/strong>. For example, he says, economic forecasters from lesser-known firms are more likely to make bold, contrarian predictions in hopes of impressing others by getting a tough call right. Conversely, forecasters at more esteemed firms are more likely to make conservative predictions that stick closely to the consensus view because they don\u2019t want to be embarrassed by getting a call wrong.<\/p>\n\n\n\n (Shortform note: Another reason forecasters might deliberately make bold predictions is to spur audiences into action. For example, in Apocalypse Never<\/em><\/a>, Michael Shellenberger argues that some climate activists employ this tactic by deliberately making alarmist predictions<\/a> in hopes of prompting policy and behavioral changes. The problem with this approach, Shellenberger says, is that it can backfire<\/a>: In the case of climate change, if you successfully convince people that we\u2019ve damaged the planet beyond repair, they might simply give up hope rather than enacting the change you\u2019d hoped for.)<\/p>\n","protected":false},"excerpt":{"rendered":" Have you ever made a prediction that turned out to be wrong? What mental errors thwart accurate forecasting? In The Signal and the Noise, Nate Silver examines mental errors that make your predictions inaccurate. These include making faulty assumptions, being overconfident, trusting data and technology too readily, seeing what you want to see, and following the wrong incentives. Keep reading to learn how these mental errors cause inaccurate predictions.<\/p>\n","protected":false},"author":14,"featured_media":29492,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[9,160,25],"tags":[1032],"class_list":["post-103067","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-psychology","category-science","category-statistics","tag-the-signal-and-the-noise","","tg-column-two"],"yoast_head":"\n1. We Make Faulty Assumptions<\/strong><\/h2>\n\n\n\n
2. We\u2019re Overconfident<\/strong><\/h2>\n\n\n\n
3. We Trust Too Much in Data and Technology<\/strong><\/h2>\n\n\n\n
4. Our Predictions Reflect Our Biases<\/strong><\/h2>\n\n\n\n
5. Our Incentives Are Wrong<\/strong><\/h2>\n\n\n\n