Do you know bullshit when you see it? What do you do about it? Should you call it out?<\/p>\n\n\n\n
With the proliferation of misinformation online, in the news, and even in academia, the modern world is replete with nonsense and flat-out lies. In their book Calling Bullshit<\/em>, Carl T. Bergstrom and Jevin D. West contend that anyone can learn how to detect and refute bullshit in its many forms.<\/p>\n\n\n\n Continue reading for an overview of this book that’s more important than ever.<\/p>\n\n\n\n\n\n\n\n Bergstrom and West define bullshit<\/em> as the use of misleading evidence to persuade an audience. But, they argue in their 2020 book Calling Bullshit<\/em><\/a> that we aren\u2019t defenseless against this tactic.<\/p>\n\n\n\n Bergstrom and West\u2019s complementary backgrounds are evident throughout Calling Bullshit<\/em>. Bergstrom\u2019s expertise in evolutionary biology lends him insight into the ways that bullshit can sneak into formal scientific models. In a similar vein, West\u2019s experience as the co-founder of the University of Washington\u2019s DataLab<\/a>\u2014a hub for data science and analytics research\u2014informs his discussion of how data can be misappropriated to create bullshit. Bergstrom and West have long co-taught a class at the University of Washington called \u201cCalling Bullshit,\u201d and this book codifies the lessons from their class. <\/p>\n\n\n\n We\u2019ll begin with an introduction to bullshit, examining Bergstrom and West\u2019s definition of bullshit<\/a> and explaining why bullshit pervades the news and the internet. Next, we\u2019ll discuss the varieties of bullshit most prevalent in contemporary society\u2014namely, bullshit based on data. We\u2019ll analyze how data can be miscollected and misinterpreted, creating bullshit that infects science and big data. Finally, we\u2019ll examine Bergstrom and West\u2019s concrete strategies for identifying and calling bullshit.<\/p>\n\n\n\n Bergstrom and West note that to recognize specific forms of bullshit, you have to understand what bullshit is more generally. To that end, we\u2019ll begin by discussing their general thesis about the nature of bullshit and outlining why bullshit is so pervasive online and in the news.<\/p>\n\n\n\n Bergstrom and West define bullshit as the use of misleading evidence, without regard for the truth, to sway an audience by confusing or overwhelming them<\/strong>. <\/p>\n\n\n\n This definition, they point out, has two key aspects. First, the bullshitter appeals to evidence that obscures <\/em>the truth, rather than illuminating it. For example, this might involve using bombastic language that makes it difficult for an audience to follow the argument. Alternatively, this could involve using statistics or data taken out of context.<\/p>\n\n\n\n Second, Bergstrom and West say, the bullshitter bears no commitment to the truth. This lack of commitment<\/a> manifests in two ways. Most obviously, it can manifest when someone actively seeks to undermine <\/em>the truth, such as by intentionally making false claims to advance a political agenda. The lack of commitment can also manifest when someone is simply indifferent<\/em> to the truth, such as a comedian who embellishes her stories for the sake of entertaining a crowd. For such entertainers, the truth is simply irrelevant.<\/p>\n\n\n\n Crucially, Bergstrom and West add that black boxes <\/em>can immunize bullshit from criticism. <\/strong>Black boxes, they explain, are highly complex sources of evidence that laypeople are not equipped to evaluate. For instance, a climate change activist might appeal to projections from the Dynamic Integrated Climate Change Model<\/a> to argue that the threat of climate change<\/a> is more serious than most assume. Because this multivariable climate model lies beyond the expertise of most laypeople, it constitutes a black box that is effectively beyond reproach. <\/p>\n\n\n\n Although bullshit can exist in any era, Bergstrom and West contend that two components of contemporary society\u2014the internet and the mainstream media\u2014are particularly potent vehicles for spreading bullshit. They argue that because the internet and the media are incentivized to earn revenue through clicks and views,<\/strong> they often propagate bullshit to meet this end since bullshit often sounds far more interesting than the unalloyed truth.<\/p>\n\n\n\n First, Bergstrom and West point out that the internet created an entirely new marketplace of ideas that inundated us with information. But, unlike previous information sources that earned money via subscriptions\u2014such as paper magazines and newspapers\u2014website owners mostly earn money via advertising revenue. Consequently, online stories seek to generate clicks to earn this revenue. <\/p>\n\n\n\n Further, the authors note that according to one 2017 study, the internet headlines that generate the most clicks are emotive<\/em>, rather than factual. In practice, this means that facts are not enough to compete in the online marketplace because online authors care more about clicks (and therefore money) than truth. For this reason, the internet is a fertile ground for bullshit since bullshit allows you to write emotionally charged headlines with no regard for the truth.<\/p>\n\n\n\n Additionally, Bergstrom and West contend that similar incentives have influenced the media since the 1987 repeal of the Fairness Doctrine. The Fairness Doctrine, they explain, required news outlets to include competing viewpoints when discussing contentious ideas. Bergstrom and West write that, since its repeal, mainstream media sources have become increasingly partisan, promoting bullshit to more easily push a partisan agenda. Moreover, the authors observe that social media algorithms have amplified this partisanship, as they only expose users to sources that reinforce their existing views. Bergstrom and West contend that because these algorithms aim to generate views, they likewise promote bullshit at the expense of truth.<\/p>\n\n\n\n The perverse incentives that infect the internet and the media aren\u2019t the only reason bullshit is so widespread. Bergstrom and West maintain that another reason is codified in Brandolini\u2019s principle<\/em>: It takes exponentially more energy to refute bullshit than to produce it<\/strong>. Thus, bullshit online<\/a> and in the news accumulates faster than we can eradicate it.<\/p>\n\n\n\n In addition to its intuitive plausibility, Bergstrom and West note that Brandolini\u2019s principle is supported by several studies examining the spread of rumors on social media platforms<\/a>. For example, one study found that even after being \u201cfact-checked\u201d by agencies like Snopes, false rumors continued to spread more rapidly on Facebook than true rumors.<\/p>\n\n\n\n Historically, bullshit was often transmitted via rhetorical devices like hyperbole, ad hominem <\/em>attacks, and false analogies. However, Bergstrom and West point out that the proliferation of data-based arguments in contemporary society has given rise to new variants of bullshit that rely heavily on data analytics. We\u2019ll outline three such types of bullshit\u2014bullshit arising from improper data collection<\/em>, bullshit arising from improper data interpretation<\/em>, and bullshit arising in science and big data in particular. <\/p>\n\n\n\n According to Bergstrom and West, bullshit often arises when data-based arguments rely on data that was itself flawed. Specifically, they argue that selection bias<\/a> <\/em><\/strong>can lead to bullshit because it justifies faulty conclusions based on unrepresentative samples<\/strong>. <\/p>\n\n\n\n Selection bias, Bergstrom and West explain, occurs whenever the population sampled for a research study doesn\u2019t represent the broader population that you\u2019re interested in. For example, if you wanted to know how the US population would vote in the 2024 election, you could commit selection bias by only polling (say) senior citizens. And although selection bias takes many forms, Bergstrom and West focus on two in particular: the observation selection<\/a> effect and data censoring.<\/p>\n\n\n\n Bergstrom and West maintain that selection bias can creep into data via the observation selection effect<\/em><\/strong>, which occurs when our data collection process is correlated with the variable that we\u2019re collecting data on. For example, if we could only conduct presidential polls via smartphones, the correlation between smartphone ownership and presidential preference would undermine the integrity of these polls\u2014smartphone owners may be wealthier, on average, and therefore more likely to vote for a candidate who cuts taxes for the rich.<\/p>\n\n\n\n The observation selection effect, Bergstrom and West argue, can yield misleading conclusions that bullshitters can prop up. Returning to the previous example, bullshitters could use poll data conducted exclusively by smartphone to spread a false narrative<\/a> that one candidate was guaranteed <\/em>to win, when in fact their chances might be much lower than the poll suggests.<\/p>\n\n\n\n Bergstrom and West explain that in addition to the observation selection effect, data censoring <\/em>can be another source of selection bias<\/strong>. They relate that data censoring occurs whenever an initially random sample becomes non-random at the completion of a study because a non-random subset of that initial sample was ineligible for inclusion in the study\u2019s results. <\/p>\n\n\n\n For instance, imagine that we conducted a study in 2023 assessing the life expectancy of individuals born in the 1900s versus those born in the 2000s. Because individuals who are still alive <\/em>can\u2019t figure into our results, the 2000s sample would appear to have a drastically lower life expectancy because it could only include those who had died by 2023 (and thus were at most 23 years old). Although data censoring is rare, Bergstrom and West caution that its misleading results can be propagated by bullshitters.<\/p>\n\n\n\n Even when safe from selection bias, data can nonetheless be manipulated to promote bullshit. According to Bergstrom and West, bullshitters often misinterpret data<\/a> to draw unjustified conclusions<\/strong>, either by making invalid inferences from valid data or by visually misrepresenting valid data in graphic form.<\/p>\n\n\n\n Bergstrom and West write that bullshitters employ various strategies to infer unsound conclusions from sound data. We\u2019ll examine three such strategies: inferring causation from correlation, discussing numbers out of context, and presenting misleading percentages.<\/p>\n\n\n\n Humans naturally seek out causal narratives, write Bergstrom and West. For this reason, we often judge that one phenomenon causes <\/em>another<\/strong>, even when our evidence only suggests the two phenomena are correlated<\/em>. For example, it\u2019s well known that increasing ice cream sales are positively correlated with increasing drowning rates<\/a>. However, it\u2019d be foolish to conclude that ice cream is causing <\/em>the uptick in drownings\u2014according to a more likely explanation, ice cream sales increase in the summer, and drownings also increase in the summer because more people swim when the weather is warmer. <\/p>\n\n\n\n Consequently, bullshit often arises when people make causal conclusions based on mere correlation. This bullshit is especially pervasive in mainstream news outlets, with one analysis finding that almost half of all news articles that report on research studies identifying correlations<\/em> between two variables framed these studies as having established a causal link.<\/p>\n\n\n\n In a similar vein, Bergstrom and West contend that taking numbers out of context can lead to bullshit<\/strong>. For example, if a politician proudly announces that his economic plan has increased the average citizen\u2019s income by $10,000, the economic plan could seem beneficial for the average citizen. However, if this plan only increased the income of the wealthiest 1% of citizens, thus inflating the average, then it may be less universally beneficial than it appears. Thus, by presenting the average income increase without<\/em> the context of how this increase was distributed across the population, the politician can promote a false view of his economic plan.<\/p>\n\n\n\n Further, Bergstrom and West explain that using percentages can obscure true values and promote bullshit<\/strong>. For instance, a tobacco company might point out that in 2022, of 330 million Americans, only 0.2% died of cancer<\/a>. While technically correct, this percentage obscures the fact that cancer accounted for almost 20% of deaths among Americans in that same year, as it caused 600,000 deaths out of 3.2 million<\/a>. Percentages can therefore be a powerful vehicle for disguising bullshit.<\/p>\n\n\n\n According to Bergstrom and West, bullshitters can further manipulate valid data by using strategies to create misleading representations<\/strong>. Although Bergstrom and West list an array of strategies for misleading viewers, we\u2019ll focus on three particularly illuminating ones: prioritizing form over function, shoehorning data in suboptimal visualizations<\/a>, and using disproportionate visualizations.<\/p>\n\n\n\n Bergstrom and West argue that prioritizing form over function can create bullshit that obscures the data that a visualization attempts to convey. <\/strong>For instance, imagine that you wanted to represent the proportion of Americans associated with a specific religious affiliation in graphical form. If, instead of representing this information via a simple bar chart, you chose to represent it via a shaded map of the United States, that shaded map would be neglecting function for the sake of form. After all, although the shaded map of the United States might look cooler, it would also be more difficult to interpret.<\/p>\n\n\n\n While an excessive focus on form can be innocuous, Bergstrom and West contend that shoehorning data into inappropriate forms is an unequivocal attempt to create bullshit. <\/strong>The clearest example of this is the proliferation of visual representations that parallel the periodic table; for instance, there are periodic tables of marketing techniques<\/a>, cryptocurrencies, and typefaces. But, while the original periodic table was carefully designed to convey the chemical groupings of different elements, these mock periodic tables are not. Rather, they attempt to convey an appearance of rigor which their underlying data lack.<\/p>\n\n\n\n Finally, the authors argue that certain data visualizations spread bullshit by violating statistician Edward Tufte\u2019s requirement of proportionate visualizations. In his 1983 book, The Visual Display of Quantitative Information<\/em><\/a>, Tufte argues that the size of regions that represent data should be proportionate to the values they represent. <\/strong>For instance, if you\u2019re creating a bar chart that illustrates the number of Nobel Prize winners by country, then the bar representing Germany\u2019s 111 Nobel laureates<\/a> should be just over five times as large as the bar representing Austria\u2019s 22 Nobel laureates. <\/p>\n\n\n\n Bergstrom and West point out that, although the principle of proportionate visualization seems commonsensical, violations of it are common\u2014any time a bar chart\u2019s vertical axis doesn\u2019t start at zero, it will violate this principle. Such charts can convey bullshit because they misrepresent the scale of the difference between two values, providing a ripe ground for false inferences. For example, imagine we created a bar chart measuring Nobel laureates by country that was evenly spaced but started at 20 rather than zero. Then, Germany\u2019s bar would be 91 units tall (111 minus 20) while Austria\u2019s would be 2 units tall (22 minus 2), creating a misleading picture of the difference between the two countries.<\/p>\n\n\n\n While there are various ways to mistakenly collect and interpret data<\/a>, it\u2019s tempting to think that official forms of data analysis are immune from bullshit. But, according to Bergstrom and West, this couldn\u2019t be further from the truth. They argue that bullshit is widespread both in science and in so-called \u201cbig data.\u201d<\/strong><\/p>\n\n\n\n According to Bergstrom and West, even the institution of science isn\u2019t immune from bullshit. On the contrary, they argue that science\u2019s focus on <\/strong>statistical significance<\/em><\/strong> gives rise to bullshit<\/strong> for two reasons: We can easily misinterpret what statistical significance means, and we\u2019re only exposed to statistically significant findings because of publication bias<\/a>.<\/p>\n\n\n\n For context, they explain that a statistically significant finding<\/em> is one with a certain p-value<\/em>\u2014a statistical measure of how likely it is that a study\u2019s result happened by pure chance. <\/em><\/p>\n\n\n\n For example, imagine you wanted to see if there was a relationship between smoking cigarettes daily and getting lung cancer. You could perform a statistical analysis comparing the rates of people with lung cancer who did, and did not, smoke cigarettes daily. If you found a positive correlation between smoking and cancer and the resulting p-value of that correlation was less than 0.05, scientists would normally consider that a statistically significant result<\/em>\u2014one that is unlikely to occur from chance alone. If your analysis yielded a p-value of 0.01, that would mean there would be a 1% chance of it occurring if there weren\u2019t<\/em> a correlation between smoking and cancer.<\/p>\n\n\n\n However, Bergstrom and West point out that many people misinterpret the p-value, taking it to be the likelihood that there\u2019s no correlation between the variables tested, which leads to bullshit when they overstate the results of statistical analyses. For example, just by sheer chance, it\u2019s possible that if we flipped two coins 100 times, they would land on the same side 60 times, yielding a p-value of around 0.03<\/a> (in other words, there\u2019s about a 3% chance of getting this result by pure luck). But we would be mistaken to conclude that the likelihood the two coins are connected is 0.97 because we know that barring any funny business, two simultaneous coin flips are independent events. So, we would instead be justified in concluding that the low p-value was a statistical anomaly.<\/p>\n\n\n\n Further, Bergstrom and West contend that publication bias <\/em>can promote bullshit by creating a distorted view<\/a> of scientific studies. Publication bias refers to scientific journals\u2019s tendency to only publish statistically significant results since such results are considered more interesting than non-significant results. In practice, this means published scientific studies often report statistically significant results even when these results don\u2019t necessarily indicate a meaningful connection. <\/p>\n\n\n\n For example, even though there isn\u2019t a connection between astrological signs and political views, if 100 studies attempted to test this relationship, we should expect about five to have a p-value below 0.05. Because these five studies would likely get published while the other 95 wouldn\u2019t, scientific journals would inadvertently promote the bullshit view that there\u2019s a connection between astrology and politics because of publication bias.<\/p>\n\n\n\n In a similar vein, Bergstrom and West argue that big data<\/em><\/strong>\u2014a technological discipline that deals with exceptionally large and complex data sets<\/a> using advanced analytics\u2014can foster bullshit because it can incorporate poor training data and find illusory connections by chance. <\/strong><\/p>\n\n\n\n For context, Bergstrom and West explain how big data generates computer programs. They relate that researchers input an enormous amount of labeled training data<\/em> into an initial learning algorithm. For instance, if they were using big data to create a program that could accurately guess people\u2019s ages from pictures, they would feed the learning algorithm pictures of people that included their age. Then, by establishing connections between these training data, the learning algorithm generates a new program for predicting people\u2019s ages. If all goes well, this program will be able to correctly assess new test data<\/em>\u2014in this case, unfamiliar pictures of people whose ages it attempts to predict.<\/p>\n\n\n\n However, Bergstrom and West argue that flawed training data can lead to bullshit programs. For example, imagine that we used big data to develop a program that allegedly can predict someone\u2019s socioeconomic status based on their facial structures, using profile pictures from Facebook as our training data. One reason why this training data could be flawed is that people from higher socioeconomic backgrounds typically own better cameras, and thus have higher-resolution profile pictures. Thus, our program might not be directly identifying socioeconomic status, but rather camera resolution. In turn, when exposed to training data not <\/em>sourced from Facebook, the big data program would likely fail to distinguish between socioeconomic status.<\/p>\n\n\n\n In addition, Bergstrom and West point out that, when given enough training data, these big data programs will often find chance connections that don\u2019t apply to test data. For instance, imagine that we created a big data program that aimed to predict the presidential election based on the frequency of certain keywords in Facebook posts. Given enough Facebook posts, chance connections between certain terms may appear to predict election outcomes. For example, it\u2019s possible that posts including \u201cTom Brady\u201d have historically predicted Republican victories, just because the Patriots have happened to win on the verge of Republican presidential elections. <\/p>\n\n\n\n Although the modern world is rife with bullshit, we aren\u2019t defenseless against it. On the contrary, Bergstrom and West argue that several strategies can help us recognize and refute bullshit. We\u2019ll examine these strategies in depth, first focusing on ways to identify bullshit before concluding with ways to call bullshit.<\/p>\n\n\n\n We\u2019ll focus on three of Bergstrom and West\u2019s key strategies for identifying bullshit: Evaluate information<\/a> sources, scrutinize claims that are \u201ctoo good to be true,\u201d and be wary of confirmation bias<\/a>.<\/strong><\/p>\n\n\n\n Bergstrom and West explain that we should assess information sources by asking who <\/em>the information is coming from and what <\/em>their possible motivations are. After all, many information sources have ulterior motives, meaning they\u2019re more likely to use bullshit to support their aims. For example, in light of conclusive evidence linking smoking to lung cancer in the 1950s, the tobacco industry<\/a> conducted a marketing campaign<\/a> that sought to undermine this scientific consensus<\/a> (and thus retain their massive profits). This campaign was rife with bullshit, but by asking what the tobacco industry\u2019s underlying motivations were, you could have easily detected this bullshit.<\/p>\n\n\n\n Next, Bergstrom and West recommend being careful when you encounter claims that seem too good to be true. In other words, if a claim seems absolutely implausible, there is a good chance that it\u2019s bullshit. For instance, in so-called \u201cNigerian prince scams,\u201d scammers email potential targets claiming to be international royalty in need of financial assistance, promising to quickly return your loans with sizable interest. These scams are too good to be true\u2014Nigerian royalty certainly wouldn\u2019t solicit loans via anonymous emails. Thus, by treating such claims with a healthy amount of skepticism, you\u2019ll often be able to detect bullshit.<\/p>\n\n\n\nOverview of Calling Bullshit<\/em><\/h2>\n\n\n\n
An Introduction to Bullshit<\/strong><\/h3>\n\n\n\n
What Is Bullshit?<\/strong><\/h4>\n\n\n\n
Bullshit Online and in the News<\/strong><\/h4>\n\n\n\n
Brandolini\u2019s Principle<\/h5>\n\n\n\n
Varieties of Contemporary Bullshit<\/strong><\/h3>\n\n\n\n
How Improper Data Collection Creates Bullshit<\/strong><\/h4>\n\n\n\n
Selection Bias #1: The Observation Selection Effect<\/h5>\n\n\n\n
Selection Bias #2: Data Censoring<\/h5>\n\n\n\n
How Misinterpreting Data Creates Bullshit<\/strong><\/h4>\n\n\n\n
Shoddy Inferences From Sound Data<\/h5>\n\n\n\n
Strategy #1: Causation From Correlation<\/h6>\n\n\n\n
Strategy #2: Numbers Without Context<\/h6>\n\n\n\n
Strategy #3: Misleading Percentages <\/h6>\n\n\n\n
Misleading Data Visualizations<\/h5>\n\n\n\n
Misleading Strategy #1: Prioritizing Form Over Function<\/h6>\n\n\n\n
Misleading Strategy #2: Shoehorning Data Into Inappropriate Visualizations<\/h6>\n\n\n\n
Misleading Strategy #3: Using Disproportionate Visualizations<\/h6>\n\n\n\n
How Experts Give Rise to Bullshit<\/strong><\/h4>\n\n\n\n
Scientific Bullshit<\/h5>\n\n\n\n
Big Data Bullshit<\/h5>\n\n\n\n
How to Deal With Bullshit<\/strong><\/h3>\n\n\n\n
How to Identify Bullshit<\/strong><\/h4>\n\n\n\n
Strategy #1: Evaluate Information Sources<\/h5>\n\n\n\n
Strategy #2: Scrutinize Implausible Claims<\/h5>\n\n\n\n
Strategy #3: Remain Cognizant of Confirmation Bias<\/h5>\n\n\n\n