PDF Summary:May Contain Lies, by Alex Edmans

In today's era of accessible data and unprecedented connectivity, the proliferation of misinformation has become a global issue with serious implications. In May Contain Lies, Alex Edmans brings to light the widespread dissemination of falsehoods, concocted studies, and deceptive assertions across various sectors such as business, healthcare, and politics.

Edmans emphasizes the necessity of developing critical thinking skills to navigate this "post-truth" landscape. He explores cognitive biases that lead individuals to accept false claims readily, delves into methods for distinguishing assertions from verifiable facts, and offers strategies to evaluate evidence effectively. The book provides a comprehensive guide for individuals, organizations, and society to combat misinformation and cultivate an analytical mindset.

(continued)...

• In certain contexts, exceptions can be more instructive than the norm, as they might highlight emerging trends or shifts in patterns that are not yet widely recognized.

Accumulated facts constitute a compilation.

Edmans emphasizes the significance of recognizing that while anecdotes may offer valuable perspectives, they do not equate to data, and in the same vein, gathering singular data points does not amount to the creation of an exhaustive dataset. Selecting a hundred unbiased data points is still insufficient to establish causality when attempting to elucidate a specific phenomenon.

Investigating a range of methods to ascertain the ways in which manipulation or influence could be applied.

Edmans proposes various methods for altering data. Researchers adjust the parameters and measurements in their investigations to correspond with their preferred outcomes, even when there is no real correlation. Researchers have considerable leeway in classifying their data and choosing a statistical model.

Practical Tips

• Create a personal journal to track your own biases in decision-making. Whenever you make a significant decision, write down the factors you considered and the outcome you desired. After the results are in, reflect on whether you subconsciously altered or emphasized certain information to align with your hoped-for outcome. This self-awareness exercise can help you recognize and adjust for your own cognitive biases.
• Experiment with free online statistical tools to see how different models affect outcomes. Take a simple dataset, like the number of steps you take each day over a month, and apply various statistical models available on these platforms. Notice how the choice of model can change the trends or insights you derive, which will give you a hands-on appreciation for the flexibility researchers have.

Careful examination of the methods used to collect and assess information is crucial.

To confirm the reliability of the findings, Edmans recommends checking if the input and the outcomes are assessed in a way that is as unforced and authentic as possible. Does the research explore alternative metrics to verify its findings' solidity? Determining whether researchers have transformed continuous measurements into a dichotomous categorization is crucial, as this necessitates scrutinizing the consistency of the outcomes when analyzed through regression.

Other Perspectives

• Checking for authenticity does not address potential biases introduced by the researchers' interpretations or the statistical methods used to analyze the data.
• Some research questions are best answered by a specific metric, and the introduction of alternative metrics could be irrelevant or misleading.
• Dichotomous categorization can be useful for policy-making and practical applications where decisions are often binary.
• Regression models can be sensitive to outliers, which can disproportionately affect the assessment of outcome consistency if not properly addressed.

Supporting data that confirms a specific hypothesis or discovery.

To advance from mere data to definitive proof, it is essential to lessen the impact of common factors, as Edmans emphasizes, by employing a method that both supports your hypothesis and rules out alternative causes.

Differentiating causation from mere correlation.

Possessing data does not equate to having evidence, since a correlation between two elements does not automatically imply causation. Research on breastfeeding has suggested a link with increased intelligence in children, but this may simply be due to other characteristics of mothers who opt for breastfeeding, such as providing a supportive home environment. To account for these shared factors, researchers must incorporate them into a unified regression analysis.

Context

• Evidence is more robust when findings can be consistently reproduced in different studies and settings, confirming that results are not due to chance or specific conditions of a single study.
• A statistically significant correlation might not have practical implications. Researchers must consider whether the relationship is meaningful in real-world terms, beyond just statistical measures.
• These are often used in such research to track children over time, providing more comprehensive data on how early-life factors like breastfeeding might impact long-term cognitive development.
• Cultural norms and values that encourage breastfeeding might also emphasize early childhood education and development, contributing to higher intelligence outcomes.
• Access to strong support networks, such as family and community resources, can enhance the ability to provide a supportive home environment.
• In the context of regression analysis, this refers to the likelihood that the relationship observed in the data occurred by chance. Researchers use p-values and confidence intervals to determine statistical significance.

Acknowledging the constraints when utilizing evidence to establish absolute truths.

Edmans emphasizes our inclination to embrace research findings without scrutinizing their broader significance. He demonstrates the success of a specific strategy in industrial environments, where there is frequently a single best way to operate, by discussing the principle known as Scientific Management. This method proved to be counterproductive within the educational sector, where instructional strategies and objectives for student achievement differ.

Practical Tips

• Create a simple flowchart for household chores to streamline the process. Draw out the steps involved in regular household tasks, such as cleaning the kitchen or organizing your workspace. Look for steps that can be eliminated, combined, or simplified. This might lead to discovering that cleaning as you cook prevents a pile-up of dishes, or that decluttering your desk at the end of each day saves time when starting work the next morning.
• Start a conversation with educators in your community to learn about their goals for student achievement. This will give you a clearer picture of the personalized objectives they set for each student, which often extend beyond quantifiable metrics. For instance, a teacher might aim to build a student's confidence in public speaking, a goal that isn't easily measured but is crucial for personal development.

Methods for establishing causality and evaluating evidence.

The meticulous gathering and examination of data typically marks the beginning of a broader investigative procedure and is essential for a thorough assessment of the facts. The aim of these tactics is to eliminate alternative explanations for the observed outcomes and to evaluate the probability that the input is the causal factor.

Controlled and randomized experiments are broadly acknowledged as the most dependable approach for determining causality.

Edmans contends that to ascertain causality with the highest degree of certainty, a method should be employed where participants are randomly assigned to either the treatment cohort or a comparison group. The significance of the placebo is rooted in its ability to separate the true effects of a treatment from those that are psychological.

How RCTs Use Random Assignment to Establish a Causal Link

Randomized allocation plays a pivotal role in establishing cause and effect in clinical studies. The stochastic characteristic of the input ensures that it remains unaffected by any common underlying element. It's not always feasible to carry out randomized control trials. Administering a potentially harmful treatment to a group for control purposes is sometimes considered unethical.

Context

• While randomization is a powerful tool, ethical considerations must be taken into account, especially when withholding potentially beneficial treatments or exposing participants to harm.
• Random assignment is crucial for establishing a causal relationship because it helps isolate the effect of the independent variable (e.g., a treatment) on the dependent variable (e.g., health outcome), minimizing the influence of external factors.
• For rare diseases, finding a sufficiently large sample size to conduct an RCT can be difficult, limiting the feasibility of such studies.
• International guidelines, such as the Declaration of Helsinki, provide a framework for conducting ethical research, emphasizing the need to protect human subjects from harm.

The difficulties in implementing randomized control trials highlight the importance of alternative approaches.

In certain situations, conducting research dependent on intervention is unfeasible. A tool is necessary that naturally separates individuals into different categories for the purpose of experimentation and analysis.

Other Perspectives

• The challenges of implementing RCTs can sometimes be mitigated through careful planning and design, suggesting that the difficulties are not insurmountable.
• Some alternative methods may not be as well-understood or as rigorously validated as RCTs, potentially leading to less confidence in the findings they produce.
• Advances in technology and methodology may reduce the barriers to conducting intervention-dependent research, making it more feasible than previously thought.
• Ethical concerns may arise from categorizing individuals without their consent, even if the categorization is 'natural'.

Alternative Approaches to Determining Causality: Experiments Utilizing Instruments and Nature

The research conducted by Caroline Hoxby is often cited to demonstrate the impact that a variety of educational opportunities have on the scholarly success of students. Hoxby's methodology evaluates the variety of educational choices by assessing the density of rivers within a region, recognizing that regions abundant in rivers often have school districts limited by these natural boundaries, which is believed to create a competitive atmosphere among schools that can improve academic outcomes for students.

Employing tools to pinpoint the direct influence of a variable.

Tools can isolate the influence of a specific factor. A family-run business tends to favor the eldest son of the CEO when choosing a successor from within its own ranks. Handing over the reins of the company to an offspring, commonly the eldest male child, usually results in diminished earnings.

Practical Tips

• Experiment with a single variable in your budget to understand its financial impact. Choose one discretionary spending category, like dining out, and eliminate or significantly reduce it for a specific period. Monitor your savings and observe how this change affects your overall budget. This will help you understand the weight of that expense in your financial life without the complexity of multiple simultaneous changes.
• Create a skills development plan for your potential successors by identifying key areas of the business they need to understand. Instead of focusing on the eldest male child, assess each child's interests and strengths, and tailor internships or mentorship programs within different departments of the company to broaden their experience.

Identifying Valid Natural Experiments and Avoiding "Unnatural" Ones

A different approach to determining causality, which does not involve direct intervention, is to take advantage of naturally occurring events that sort individuals into separate categories. Edmans illustrates his point by discussing a study that observed the effects of New Jersey's minimum wage hike in 1992, in contrast to Pennsylvania, which maintained its wage levels at that time. The research, based on empirical trials, found that, against conventional beliefs, increasing the baseline salary led to an uptick in job availability.

Context

• These can include policy changes, natural disasters, or other events that affect some groups but not others, providing a comparison between affected and unaffected groups.
• Unlike controlled experiments, where researchers assign subjects to treatment or control groups, natural experiments rely on pre-existing conditions that mimic random assignment.
• During the early 1990s, the U.S. economy was recovering from a recession, and debates about the impact of minimum wage laws on employment were particularly relevant as policymakers sought ways to stimulate job growth.
• The study used data from fast-food restaurants, which are often sensitive to wage changes, to assess employment effects.
• Establishing causality is crucial in research to determine whether one factor directly affects another, rather than just being correlated.

Establishing causation beyond mere correlation necessitates accounting for shared factors.

Edmans underscores the necessity of employing regression analysis to adjust for commonly encountered variables. In regression analyses, multiple variables can be included to assess their distinct impacts while controlling for the influence of all remaining variables.

Recognizing the importance of contributions from within.

However, regression analysis is limited by how many inputs you can measure. It is possible to consider a chief executive's age, earnings, and educational background, but it is not practical to account for characteristics like patience. Three essential factors can render a factor intrinsically motivated: it might stem from an individual's deliberate decision, associate with an alternate characteristic, or emerge due to an independent occurrence, such as a city experiencing elevated pollution due to its substantial populace.

Context

• When multiple inputs are highly correlated, it can be challenging to determine the individual effect of each variable, complicating the analysis.
• Evaluating a CEO's impact on a company involves understanding a complex interplay of measurable and immeasurable factors, requiring more than just statistical analysis.
• Unlike static characteristics, patience can change over time due to personal growth or changing circumstances, complicating its inclusion in a model that assumes stable inputs.
• These are traits or attributes that might not be directly measurable but can influence behavior or outcomes. For example, a leader's charisma or resilience might impact their effectiveness, even if these traits are not easily quantified.
• Managing pollution in large cities can be challenging due to the complexity of implementing and enforcing environmental regulations across diverse and densely populated areas.

Analysis is pivotal in assessing elements that are broadly acknowledged to have an impact.

Regression analysis stands out as the optimal approach for compensating for common factors, even though it's not feasible to consider every variable that might distort the results. The investigators would have considered this specific link only if it was the most sensible approach to investigate, provided that the connection made logical sense and was supported by the evidence. An element that seems improbable is likely unrelated to the known determinants of the result.

Other Perspectives

• In time-series data, regression analysis may not account for autocorrelation, where past values influence current values, without modifications or additional techniques being applied.
• Modern machine learning techniques can handle high-dimensional data, which includes many variables, and can uncover complex patterns that traditional regression analysis might miss.
• Investigators might consider a specific link based on preliminary data or hypotheses, even if it's not initially perceived as the most sensible approach.
• Logical sense is subjective and can vary significantly between different schools of thought or disciplines, meaning what is logical in one context may not be considered so in another.
• Historical precedents show that many scientific breakthroughs were initially considered improbable.

Grasping the importance of subtle reasoning while recognizing the limitations associated with evidence.

Even with substantial evidence and a persuasive argument connecting cause and effect, Edmans observes that this does not amount to definitive proof.

The concept of 'external validity' pertains to the applicability of the findings across various situations.

Difficulties emerge when contemplating the possibility that the evidence may not be relevant across various contexts, which could restrict its wider relevance. Edmans highlights the common misunderstanding that tactics which are successful in one area, such as manufacturing, can be equally effective when applied to another sector, like education.

Considering the diverse environments, the range of populations, and the numerous factors involved.

It's essential to assess the varying circumstances when examining evidence: strategies that succeed in large corporations may not be suitable for smaller enterprises; recommendations that assist the majority might not produce identical outcomes for people with distinct characteristics.

Other Perspectives

• Overemphasis on context may inadvertently discount universal principles or best practices that are effective regardless of varying circumstances.
• Smaller enterprises can benefit from the lessons learned by large corporations, avoiding their mistakes and capitalizing on their successful tactics.
• It could be argued that the majority can benefit from certain recommendations because they address common challenges or leverage widely applicable best practices.

The dangers of jumping to overly simplistic conclusions and regarding evidence as beyond dispute.

A specific truth may be applicable to a certain demographic but not extend to others. The book points out the research conducted by Angela Duckworth, which suggests that the deep-rooted persistence, commonly known as grit, is a predictor of one's ability to successfully complete the challenging initial training at the United States Military Academy. For individuals aiming to join the armed forces, their physical fitness may be a more significant predictor of success, considering that every applicant to the Military Academy has demonstrated considerable athletic prowess. The author stresses the importance of considering the context and range when evaluating evidence.

Other Perspectives

• The statement could be seen as implying that demographic factors are the primary determinants of applicability of truths, which may not account for the influence of other factors such as personal choice, chance, or environmental influences.
• The concept of grit itself can be subjective and difficult to measure accurately, which may lead to inconsistencies in how it predicts success.
• Adaptability and the ability to learn and think critically may be just as important as physical fitness, given the rapidly changing nature of modern warfare and military operations.
• There are instances where universal principles or laws are applicable, and the context or range does not significantly alter the interpretation of evidence, such as in certain natural laws or mathematical truths.

The Multidimensional Nature of Decision-Making

Edmans underscores the necessity of acknowledging both the robustness and the limitations of the evidence presented. In our pursuits, we often strive for various objectives, although research usually concentrates solely on a single result. Furthermore, even for a single goal, scientific research gives us an average result, which might not be true for every individual person.

Decisions can be influenced by evidence without being dictated by it.

Decisions can be influenced by evidence without being dictated by it. Leaders might prioritize workforce diversity for ethical reasons rather than focusing exclusively on performance improvement. For infants who have not reached the target weight, the optimal choice remains nursing, even if parents might opt for formula. If solo practice is your passion, pursue it, despite studies indicating that drummers benefit more from group rehearsals.

Context

• In areas like innovation, taking risks might be necessary despite evidence suggesting caution. This can lead to breakthroughs that evidence-based approaches might not achieve.
• Companies that prioritize diversity may improve their reputation and brand image, attracting customers and employees who value inclusivity.
• Breastfeeding is generally more cost-effective than formula feeding and does not require the preparation and sterilization that formula does.
• Solo practice allows for focused repetition and mastery of specific techniques that might not be addressed in a group setting.

It was crucial to steer through a range of goals that could potentially conflict.

Decisions are seldom made with a singular objective in mind; a variety of factors may sway our decision when buying a vehicle, for instance, and individuals who consider themselves to be guided purely by rationality, including those who confidently claim to base their judgments solely on empirical evidence, frequently fall prey to biases they are not aware of. Additionally, the data pertaining to that objective merely illustrates the strategies effective for most individuals. Not everyone is subject to this principle.

Other Perspectives

• The idea that decisions involve multiple conflicting goals may not account for situations where goals are aligned and synergistic, thus not in conflict.
• While it's true that various factors influence decisions, some individuals may prioritize a single factor above all others when making certain decisions, such as cost when buying a vehicle, especially in cases of strict budget constraints.
• The concept of rationality itself can be subjective, and what is considered rational for one person might include considerations that another might label as biases.
• Data on objectives might be skewed by the way it was collected, potentially misrepresenting the strategies that are truly effective.

Strategies for Critical Thinking at the Individual, Organizational, and Societal Levels

Edmans outlines various tactics to enhance and encourage more analytical thought processes. Our approaches vary widely, encompassing small-scale tactics like conducting meetings effectively to large-scale endeavors involving the education of our youth.

Individual Level: Actively Seeking Out Dissenting Viewpoints and Enhancing Numerical Data Interpretation Skills.

Edmans emphasizes the importance of individuals proactively exploring diverse perspectives. What might be viewed as the opposing viewpoint? What evidence would be required to make me reconsider my stance? Moving beyond simply memorizing facts is crucial, as it's important to understand the implications of interpreting numerical data.

Investigating multiple viewpoints and utilizing a range of mental approaches.

The author stresses the need to assess different viewpoints by scrutinizing the strength and stability of the theories we hold. How could I rationally disregard an outcome that challenges my expectations? One must also pay close attention to where any claims stem from. Inquire about the individual presenting this assertion and their qualifications. What would happen to their reputation if it were revealed that their claims were not truthful? Are they intentionally masking subtleties by using extreme language?

Practical Tips

• You can enhance your critical thinking by starting a "Theory Journal" where you document and analyze different theories you come across in daily life. Each time you hear a new theory or a strong opinion, write it down, and then list out the evidence that supports and contradicts it. This practice will help you to habitually assess the strength and stability of theories, much like a scientist evaluating hypotheses.
• Develop a habit of playing the "alternative outcomes game" with friends or family where you discuss various scenarios and imagine different outcomes, regardless of their likelihood. This game can be played casually in conversation and helps to stretch your thinking beyond the expected. For instance, if you're discussing a sports game, talk about unexpected ways the game could end, like a last-minute rule change or an unforeseen play.
• Create a "trust index" for your personal use, scoring sources you frequently encounter on a scale based on their previous reliability, expertise, and evidence provided. Over time, this index will help you quickly assess the credibility of information. For instance, if a blogger consistently provides well-researched health advice, they might score higher on your index than a new, unverified source.
• Engage in conversations with peers where you play the role of a skeptic, politely challenging statements by asking about the qualifications and background of the source. This can be done in study groups, book clubs, or even casual discussions. It encourages a culture of critical inquiry and helps everyone involved to think more deeply about the credibility of the information they share.
• You can create a personal "truth log" to track the accuracy of your statements over time. Start by jotting down claims you make in conversations, emails, or social media. At the end of each week, review your log and verify the truthfulness of each claim. This practice will help you become more aware of the accuracy of your statements and encourage you to fact-check before speaking or posting.
• Develop a habit of asking for clarification when others use extreme language. When you hear someone making a statement that seems overly simplified or extreme, gently ask them to elaborate. For instance, if a friend says, "This is the worst day ever," you could respond with, "What happened that makes you feel this way?" This encourages a more nuanced discussion and helps you understand the subtleties in their experience.

Cultivating an inquisitive mindset and enhancing logical thinking skills.

Prior to disseminating information swiftly, it's crucial to thoroughly evaluate its truthfulness to guarantee comprehension of every involved viewpoint. It's important not to critique an article or individual without fully grasping the argument presented.

Practical Tips

• Set up a "truth delay" protocol for your social media use. Before posting any article, study, or piece of news, wait a predetermined period, like 24 hours, to allow for additional fact-checking and reflection. This pause can prevent the knee-jerk sharing of misinformation and encourage a more thoughtful dissemination of content.
• You can enhance your understanding of different viewpoints by starting a 'perspective journal' where you write down a summary of someone else's opinion after every significant conversation. This practice encourages active listening and reflection on the perspectives you encounter daily. For example, after a debate with a friend about a political issue, jot down their main points and the reasoning behind them, which can help you see the issue from their angle.
• Create a habit of asking at least three clarifying questions during conversations when you're tempted to critique an argument. This practice forces you to engage more deeply with the speaker's perspective and ensures that your critique is based on a complete understanding of their position. For example, if someone presents an opinion on a new city policy, ask them about the policy's intended benefits, its potential drawbacks, and how it compares to alternatives.

Organizational Level: Promoting a culture that values diverse ways of thinking and inclusivity.

Edmans contends that organizations should not only focus on recruiting a varied workforce but also make a concerted effort to foster an environment where each person feels valued and included. This requires recognizing and appreciating diverse perspectives, making certain that they are listened to.

Employing structured techniques like brainstorming and casting anonymous ballots

Edmans recommends specific processes that foster inclusion. At the beginning of the session, participants are provided with initial materials and familiarized with a technique that mandates the substantiation of their predictions, guaranteeing that each participant, even those who are usually reluctant to share their views, has a fair chance to participate.

Practical Tips

• Create a "rotating facilitator" role in your meetings where a different person each time is responsible for guiding discussions and ensuring that everyone has the opportunity to speak. This not only trains individuals in leadership skills but also gives varied perspectives on how to manage inclusion.
• You can enhance your learning experience by creating a personalized starter kit for a new hobby or skill you're interested in. For example, if you're keen on gardening, assemble a basic set of tools like gloves, a trowel, seeds, and a watering can. This approach mirrors the concept of being provided with initial materials, setting you up for a structured and focused beginning.
• Engage with a friend or colleague in a prediction challenge where you both make predictions about a common interest or topic, then meet regularly to discuss the outcomes and the reasoning behind your forecasts. This social interaction not only makes the process more enjoyable but also introduces accountability and the opportunity to learn from each other's thought processes and substantiation techniques.
• Implement a "round-robin" feedback session after collaborative projects. Instead of an open discussion, go around the room and give each person a set amount of time to share their thoughts. This structured approach can help balance participation and give quieter members a defined space to express themselves.

Processes that foster an environment for challenge and dissent are often informal in nature.

Organizations can enhance their shared knowledge by ensuring the removal of predetermined options, flattening hierarchical structures, and appreciating efforts that, although not always successful, yield valuable lessons.

Other Perspectives

• Informal processes may lack the necessary documentation and tracking to ensure that the insights from dissent and challenges are captured and acted upon effectively.
• Predetermined options can sometimes act as a foundation for shared knowledge, providing a starting point from which new ideas can be compared and contrasted.
• In some cases, hierarchy is necessary to ensure accountability and decision-making efficiency.
• There's a risk of creating an environment where failure is too readily accepted, possibly lowering the drive for excellence and the motivation to succeed.

At the societal level, incorporating critical thinking instruction within educational systems and employing fact-checking measures to counteract misinformation.

Education serves as a powerful tool, Edmans asserts, yet the problems stem not from a lack of education but from chasing the wrong educational aims. Education should not only involve absorbing information but also cultivating the ability to think critically and evaluate.

Incorporating skills for analytical reasoning into educational programs.

Edmans suggests that sharpening analytical abilities is crucial and recommends that engaging in EK23 and comparable strategic tasks could be beneficial tools for achieving this goal. Schools ought to not only enhance numerical capabilities but also cultivate the capacity for data comprehension, while prompting children to question the veracity of the information they acquire.

Practical Tips

• Practice interpreting data through gamification by using mobile apps or online platforms that turn data analysis into a game. Seek out games that require you to solve puzzles or make decisions based on numerical information. This not only makes learning fun but also helps you apply data comprehension skills in a low-stakes environment.
• Start a family "Question Jar" where each member can drop in questions about things they've heard or read throughout the week. Dedicate a time during the weekend to pull out questions and collectively research and discuss the answers, fostering a habit of inquiry and critical thinking.

Centralized systems are designed to authenticate information and strategically distribute news among the general populace.

Edmans advocates for the creation of specialized forums to validate the truthfulness of content across different fields, and he proposes that impartial groups or individuals with opposing viewpoints, such as a conservative discussing environmental issues, should deliberate on delicate political subjects.

Practical Tips

• You can verify the authenticity of news by cross-referencing information with multiple sources. When you come across a piece of news, don't take it at face value. Instead, look for the same story on different news platforms, especially those with distinct editorial viewpoints. This helps you to see if the information is consistent across sources, which can be a good indicator of its reliability.
• Engage in 'Virtual Town Halls' by participating in or observing online forums and webinars that aim to bring together individuals with opposing political views. Look for events that encourage civil discourse and are moderated to ensure a balanced exchange of ideas. Take notes on the arguments presented and consider how the different viewpoints expand your understanding of the issue.