How much damage can improper data collection create? How can manipulators leverage it for their own advantage?
In Calling Bullshit, Carl T. Bergstrom and Jevin D. West contend that bullshit often arises when data-based arguments rely on flawed data. They explain how the observation selection effect is an example of this and show how some people take advantage of it to deliberately deceive others.
Continue reading to learn about the observation selection effect and how it can wreak havoc.
The Observation Selection Effect
Bergstrom and West maintain that selection bias can creep into data via the observation selection effect, which occurs when our data collection process is correlated with the variable that we’re collecting data on. For example, if we could conduct presidential polls only via smartphones, the correlation between smartphone ownership and presidential preference would undermine the integrity of these polls—smartphone owners may be wealthier, on average, and therefore more likely to vote for a candidate who cuts taxes for the rich.
(Shortform note: The observation selection effect causes notorious issues in philosophy and science, where the fact that we exist to observe certain phenomena presupposes certain conditions about the universe. For example, some philosophers take the observation that the universe is surprisingly hospitable to sentient life as evidence of a creator, even though this hospitability is necessary for us to observe it in the first place.)
The observation selection effect, Bergstrom and West argue, can yield misleading conclusions that bullshitters can prop up. Returning to the previous example, bullshitters could use poll data conducted exclusively by smartphone to spread a false narrative that one candidate was guaranteed to win, when in fact their chances might be much lower than the poll suggests.
(Shortform note: In the case of polling, one reason bullshitters might want to use biased polls is that such polls can influence elections via the contagion effect—the tendency of voters to rally around candidates who are polling well. For example, if a misleading poll suggested that one candidate was crushing the other candidates in a primary election, that could cause more voters to view that candidate’s victory as inevitable and therefore vote for them.)
