Read below for a brief overview of The Signal and the Noise<\/em>.<\/p>\n\n\n\n\n\n\n\n The early 21st century has already seen numerous catastrophic failures of prediction: From terrorist attacks to financial crises to natural disasters to political upheaval, we routinely seem unable to predict the events that change the world<\/a>. In The Signal and the Noise<\/em><\/a>, Nate Silver sets out to explain why our predictions typically fail and how we can do better.\u00a0<\/p>\n\n\n\n Silver is the creator of FiveThirtyEight<\/em><\/a>, a political analysis website known for its track record of accurate forecasts of US elections. He\u2019s also worked as a professional poker player and a baseball analyst\u2014during which time he developed a well-respected system (PECOTA<\/a>) for predicting players\u2019 future performances.\u00a0<\/p>\n\n\n\n In The Signal and the Noise<\/em>, Silver explores a wide range of fields that depend on predictions, from politics, poker and baseball to economics, meteorology, and military intelligence. Though the book focuses mostly on these sorts of high-stakes examples, Silver suggests that its lessons are valuable for all readers\u2014after all, decisions as mundane as what to wear on a given day are, in essence, predictions (for example, about the weather). <\/p>\n\n\n\n The book was originally published in 2012, though our guide is based on the updated 2020 version which includes a new preface discussing the 2016 US Presidential election and the early stages of the Covid-19 pandemic.<\/p>\n\n\n\n Before we discuss our prediction mistakes and Silver\u2019s advice for how to avoid them, it\u2019s worth acknowledging that even under the best circumstances, prediction is an inherently challenging endeavor. In this section, we\u2019ll explore Silver\u2019s analysis of how insufficient information, system complexity, and the tendency of small errors to compound all limit the accuracy of our predictions.<\/p>\n\n\n\n To make good predictions, Silver says, you need information about the phenomenon you\u2019re trying to predict as well as a good understanding of how that phenomenon works<\/strong>. For example, today\u2019s meteorologists have abundant information about atmospheric conditions as well as a good understanding of the physical laws by which those conditions develop. Accordingly, Silver says, they can make reasonably accurate predictions about the weather. <\/p>\n\n\n\n The problem is that in many situations, we don\u2019t have much useful data to go on<\/strong>. Silver illustrates this point by comparing meteorology to seismology, explaining that seismologists have no way to predict the timing, location, or strength of specific earthquakes. That\u2019s because earthquakes happen rarely and on a geological timescale, which makes it hard to discern any patterns that might be at play. And whereas meteorologists can directly observe atmospheric conditions, there\u2019s no way to measure factors such as the pressures currently at work on a given fault line nor to foresee future tectonic activity.<\/p>\n\n\n\n Also, Silver argues that even when we have rich data and a good understanding of the underlying principles, many systems still defy accurate prediction because of their inherent complexity. Silver discusses economic forecasts\u2014which he says are notoriously unreliable\u2014to lay out several ways complex systems<\/a> hinder forecasting:<\/p>\n\n\n\n Finally, Silver explains that even when you have good data and you\u2019re not dealing with an inherently unpredictable system like the economy, your predictions are still limited by the fact that even minuscule mistakes in the predictive process compound into large errors over time<\/strong>. Silver argues that this principle of compounding error is at work even in fields with relatively accurate predictions, such as meteorology. Because each day\u2019s weather depends on the previous day\u2019s conditions, even the smallest inaccuracy in the initial data you plug into a simulation will add up to big discrepancies over time. Silver says that\u2019s why weather forecasts are generally accurate a few days out but become increasingly unreliable the further into the future they try to predict.<\/p>\n\n\n\n So far, we\u2019ve discussed challenges that are inherent to prediction\u2014but according to Silver, these challenges don\u2019t tell the whole story. Instead, he argues, we exacerbate these challenges through a series of mental errors<\/a> that make our predictions even less accurate. In this section, we\u2019ll examine these mental errors, which include making faulty assumptions<\/a>, being overconfident, trusting data and technology too readily, seeing what we want to see, and following the wrong incentives.<\/p>\n\n\n\n As we\u2019ve seen, our predictions tend to go awry when we don\u2019t have enough information or a clear enough understanding of how to interpret our information. This problem gets even worse, Silver says, when we assume that we know more than we actually do. He argues that we seldom recognize when we\u2019re dealing with the unknown because our brains tend to make faulty assumptions based on analogies to what we do<\/em> know<\/strong>. <\/p>\n\n\n\n To illustrate the danger of assumptions, Silver says that the 2008 financial crisis resulted in part from two flawed assumptions made by ratings agencies who gave their highest ratings to financial products called CDOs<\/a> that depended on mortgages not defaulting.<\/p>\n\n\n\n Silver argues that these assumptions exacerbated a bad situation because they created the illusion that these products were safe investments when in fact, no one really understood their actual risk.<\/p>\n\n\n\n According to Silver, the same faulty assumptions that make our predictions less accurate also make us too confident in how good these predictions are. One dangerous consequence of this overconfidence is that by overestimating our certainty, we underestimate our risk<\/strong>. Silver argues that we can easily make grievous mistakes when we think<\/em> we know the odds but actually don\u2019t. You probably wouldn\u2019t bet anything on a hand of cards if you didn\u2019t know the rules of the game\u2014you\u2019d realize that your complete lack of certainty would make any bet too risky. But if you misunderstood<\/em> the rules of the game (say you mistakenly believed that three of a kind is the strongest hand in poker), you might make an extremely risky bet while thinking it\u2019s safe.<\/p>\n\n\n\n Silver further argues that it\u2019s easy to become overconfident if you assume a prediction is accurate just because it\u2019s precise<\/strong>. With information more readily available than ever before and with computers to help us run detailed calculations and simulations, we can produce extremely detailed estimates that don\u2019t necessarily bear any relation to reality, but whose numerical sophistication might fool forecasters and their audiences into thinking they\u2019re valid. Silver argues that this happened in the 2008 financial crisis, when financial agencies presented calculations whose multiple decimal places obscured the fundamental unsoundness of their predictive methodologies. <\/p>\n\n\n\n As noted earlier, one of the challenges of making good predictions is a lack of information. By extension, the more<\/em> information we have, the better our predictions should be, at least in theory. By that reasoning, today\u2019s technology should be a boon to predictors\u2014we have more data than ever before, and thanks to computers, we can process that data in ways that would have been impractical or impossible in the past. However, Silver argues that data and computers both present their own unique problems that can hinder predictions as much as they help them.<\/p>\n\n\n\n For one thing, Silver says, having more data doesn\u2019t inherently<\/em> improve our predictions. He argues that as the total amount of available data has increased, the amount of signal (the useful data) hasn\u2019t<\/strong>\u2014in other words, the proliferation of data means that there\u2019s more noise (irrelevant or misleading data<\/a>) to wade through to find the signal. At worst, this proportional increase in noise can lead to convincingly precise yet faulty predictions when forecasters inadvertently build theories that fit the noise rather than the signal.<\/p>\n\n\n\n Likewise, Silver warns that computers can lead us to baseless conclusions when we overestimate their capabilities or forget their limitations<\/strong>. Silver gives the example of the 1997 chess series between grandmaster Garry Kasparov and IBM\u2019s supercomputer Deep Blue. Late in one of the matches, Deep Blue made a move that seemed to have little purpose, and afterward, Kasparov became convinced that the computer must have had far more processing and creative power than he\u2019d thought\u2014after all, he assumed, it must have had a good reason for the move. But in fact, Silver says, IBM eventually revealed that the move resulted from a bug: Deep Blue got stuck, so it picked a move at random.<\/p>\n\n\n\n According to Silver, Kasparov\u2019s overestimation of Deep Blue was costly: He lost his composure and, as a result, the series. To avoid making similar errors, Silver says you should always keep in mind the respective strengths and weaknesses of humans and machines<\/a><\/strong>: Computers are good at brute-force calculations, which they perform consistently and accurately, and they excel at solving finite, well-defined, short- to mid-range problems (like a chess game). On the other hand, Silver says, computers aren\u2019t flexible or creative, and they struggle with large, open-ended problems. Conversely, humans can do several things computers can\u2019t: We can think abstractly, form high-level strategies, and see the bigger picture<\/a>.<\/p>\n\n\n\n Another challenge of prediction is that many forecasters have a tendency to see what they want or expect to see in a given set of data\u2014and these types of forecasters are often the most influential ones<\/strong>. Silver draws on research by Phillip E. Tetlock (author of Superforecasting<\/em><\/a>) that identifies two opposing types of thinkers: hedgehogs and foxes<\/a>.<\/p>\n\n\n\n Silver argues that although foxes make more accurate predictions than hedgehogs, hedgehog predictions get more attention because the hedgehog thinking<\/a> style is much more media-friendly<\/strong>: Hedgehogs give good sound bites, they\u2019re sure of themselves (which translates as confidence and charisma), and they draw support from partisan audiences who agree with their worldview.<\/p>\n\n\n\n Furthermore, Silver points to the media\u2019s preference for hedgehog-style predictions as an example of how our predictions can be warped by bad incentives<\/strong>. In this case, for forecasters interested in garnering attention and fame (which means more airtime and more money), bad practices pay off. That\u2019s because all the things that make for good predictions\u2014cautious precision, attentiveness to uncertainty, and a willingness to change your mind\u2014are a lot less compelling on TV than qualities that lead to worse predictions\u2014broad, bold claims, certainty, and stubbornness.<\/p>\n\n\n\n Similarly, Silver explains that predictions can be compromised when forecasters are concerned about their reputations<\/strong>. For example, he says, economic forecasters from lesser-known firms are more likely to make bold, contrarian predictions in hopes of impressing others by getting a tough call right. Conversely, forecasters at more esteemed firms are more likely to make conservative predictions that stick closely to the consensus view because they don\u2019t want to be embarrassed by getting a call wrong.<\/p>\n\n\n\n Although prediction is inherently difficult\u2014and made more so by the various thinking errors we\u2019ve outlined\u2014Silver argues that it\u2019s possible to make consistently more accurate predictions by following the principles of a statistical formula known as Bayes\u2019 Theorem. Though Silver briefly discusses the mathematics of the formula, he\u2019s most interested in how the theorem encourages us to think<\/em> while making predictions. According to Silver, Bayes\u2019 Theorem suggests that we make better predictions when we consider the prior likelihood of an event and update our predictions in response to the latest evidence<\/strong>.<\/p>\n\n\n\n Bayes\u2019s Theorem\u2014named for Thomas Bayes, the English minister and mathematician who first articulated it\u2014posits that you can calculate the probability of event A with respect to a specific piece of evidence B<\/strong>. To do so, Silver explains, you need to know (or estimate) three things:<\/p>\n\n\n\n Bayes\u2019 Theorem uses these values to calculate the probability of A given B\u2014P(A|B)\u2014as follows:<\/p>\n\n\n\n Now that we\u2019ve explored the mathematics of Bayes\u2019 Theorem, let\u2019s look at the broader implications of its underlying logic. First, building on the principle of considering prior probabilities, Silver argues that it\u2019s important to be open to a wide range of possibilities<\/strong>, especially when you\u2019re dealing with noisy data. Otherwise, you might develop blind spots that hinder your ability to predict accurately.<\/p>\n\n\n\n To illustrate this point, Silver argues that the US military\u2019s failure to predict the Japanese attack on Pearl Harbor in 1941 shows that it\u2019s dangerous to commit too strongly to a specific theory when there\u2019s scant evidence for any<\/em> particular theory<\/strong>. He explains that in the weeks before the attack, the US military noticed a sudden dropoff in intercepted radio traffic from the Japanese fleet. According to Silver, most analysts concluded that the sudden radio silence was because the fleet was out of range of US military installations\u2014they didn\u2019t consider the possibility<\/a> of an impending attack because they believed the main threat to the US Navy was from domestic sabotage by Japanese Americans.<\/p>\n\n\n\n Silver explains that one reason we sometimes fail to see all the options is that it\u2019s common to mistake a lack of precedence for a lack of possibility<\/strong>. In other words, when an event is extremely uncommon or unlikely, we might conclude that it will never<\/em> happen, even when logic and evidence dictate that given enough time, it will. For example, Silver points out that before the attack on Pearl Harbor, the previous foreign attack on US territory came in the early 19th century\u2014a fact that made it easy to forget that such an attack was even possible.<\/p>\n\n\n\n In addition to emphasizing the importance of considering all possibilities, Bayesian logic also highlights the need to stay focused on the evidence rather than getting sidetracked by other factors such as your emotional responses or the trends you observe.<\/p>\n\n\n\n For one thing, Silver argues that when you give in to strong emotions, the quality of your predictions suffers<\/strong>. He gives the example of poker players going on tilt<\/em>\u2014that is, losing their cool due to a run of bad luck or some other stressor (such as fatigue or another player\u2019s behavior). Poker depends on being able to accurately predict what cards your opponent might have\u2014but Silver argues that when players are on tilt, they begin taking ill-considered risks (such as betting big on a weak hand) based more on anger and frustration than on solid predictions. <\/p>\n\n\n\nThe Signal and the Noise<\/em> by Nate Silver<\/strong><\/h2>\n\n\n\n
Part 1: Why Prediction Is Hard<\/strong><\/h3>\n\n\n\n
We Often Lack Sufficient Information<\/strong><\/h4>\n\n\n\n
Many Systems Are Too Complex to Fully Understand<\/strong><\/h4>\n\n\n\n
Small Errors Compound<\/strong><\/h4>\n\n\n\n
Part 2: Why Our Predictions Are Worse Than They Could Be<\/strong><\/h3>\n\n\n\n
We Make Faulty Assumptions<\/strong><\/h4>\n\n\n\n
We\u2019re Overconfident<\/strong><\/h4>\n\n\n\n
We Trust Too Much in Data and Technology<\/strong><\/h4>\n\n\n\n
Our Predictions Reflect Our Biases<\/strong><\/h4>\n\n\n\n
Our Incentives Are Wrong<\/strong><\/h4>\n\n\n\n
Part 3: Better Predictions Through Bayesian Logic<\/strong><\/h3>\n\n\n\n
The Principles of Bayesian Statistics (bayesian principles)<\/strong><\/h4>\n\n\n\n
![\"\"](\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2023\/05\/image-5.png\")
<\/figure>\n\n\n\nBayesian Lesson #1: Consider All Possibilitie<\/strong>s<\/h4>\n\n\n\n
Bayesian Lesson #2: Follow the Evidence, Not Emotions or Trends<\/strong><\/h4>\n\n\n\n