This article is an excerpt from the Shortform book guide to "Naked Statistics" by Charles Wheelan. Shortform has the world's best summaries and analyses of books you should be reading.
Like this article? Sign up for a free trial here .
What are the different types of bias in statistics? What are some ways bias can creep into a research project?
As individuals and as a society, we rely on scientific research to make informed decisions and to understand the world around us. Therefore, researchers have an ethical obligation to identify and address sources of bias in their research. Statistical bias can make its way into a research project anywhere along the way, from the study’s conception to the research question, the data collection, the statistical analysis, the reporting of findings, and the study’s publication.
Keep reading to learn about the most common sources of bias in statistics.
Sources of Bias
Biased data can sabotage otherwise sound research methods and statistical calculations. Sources of bias in data may be glaringly obvious or so subtle as to go unnoticed. If we want our data to be reliable, we should be aware of and take steps to mitigate potential sources of bias.
| Reducing Bias Is a Researcher’s Responsibility As individuals and as a society, we rely on scientific research to make informed decisions and understand the world around us. Therefore, researchers have a practical and ethical obligation to identify and address sources of bias in their research. Bias can make its way into a research project anywhere along the way, from the study’s conception to the framing of the research question, the data collection, the statistical analysis, the reporting of findings, and the study’s publication. Therefore, keeping bias out of research takes effort and attention, beginning with an awareness of the myriad sources of bias, both glaringly obvious and inadvertent. |
Wheelan highlights the following types of bias in statistics:
Selection Bias
Selection bias happens when our sample is not random, and certain subsets of the population are over- or underrepresented. Selection bias can be subtle. If researchers are not cognizant of selection bias when developing data collection methods, the fact that a sample is not truly random might go unnoticed.
For example: Say you wanted to collect data on people’s political leanings before an election, and you decided to collect your data at an art show outside of town. You might think that your sample was random because the art show was a public event, the crowd was a mix of people from different parts of town, and people of all ages were represented. However, it’s likely that your data would be biased towards the opinions of wealthier residents because the people at the art show can afford the cars they used to drive out of town and the art for sale.
Selection bias can also happen when people are able to self-select into (or out of) a study. When we allow the people who feel strongly enough about a study to become the sample, our data is automatically skewed. For example, if you were to stand on the sidewalk with a banner promoting a local dog park and asked “random” people to take your survey, chances are that dog lovers strongly in favor of a dog park would be over-represented in your results, since they would take the time to come over.
Recall Bias
Recall bias happens when we ask people to give us data on the effect of a treatment or event retroactively. The challenge with obtaining reliable data from the past is that memory is not static. Wheelan explains that when we try to recall data from the past, our memory will be influenced by the meaning and emphasis our mind has placed on the event. For example, a person who fails calculus in college might be more likely to report that they “have always hated math,” even if they enjoyed math classes in high school, because their negative experience in college calculus is affecting their memory of prior classes.
———End of Preview———
Like what you just read? Read the rest of the world's best book summary and analysis of Charles Wheelan's "Naked Statistics" at Shortform .
Here's what you'll find in our full Naked Statistics summary :
- An explanation and breakdown of statistics into digestible terms
- How statistics can inform collective decision-making
- Why learning statistics is an exercise in self-empowerment
