PDF Summary:Black Box Thinking, by Matthew Syed

In Black Box Thinking, Matthew Syed argues that learning from failure drives human progress, from innovation in science and business to individual and organizational growth. Yet other areas of society—including the political, judicial, and health care systems—have yet to benefit from this process of rational learning.

As long as these institutions stigmatize failure, they create a culture of misguided exceptionalism that prevents them from improving. And if we neglect to improve our stagnating institutions, they’ll continue to take the lives of innocents—whether it’s death due to preventable medical error or wrongful conviction that leads to life in prison.

Syed explains how an organization’s culture and systems either promote or prevent learning, and he lays out several steps for learning from failure in our complex world. Any organization can change its culture and systems by embracing failure as a powerful source of learning. We’ve rounded out Syed’s ideas with advice from Tribal Leadership and The Fifth Discipline, which discuss the cultures and systems of effective businesses.

(continued)...

On the social level, mistakes damage your reputation. In a culture of expertise, any error calls your competency into question. A large mistake can strip away your credibility overnight— for example, the above doctor might become known for letting that patient die.

(Shortform note: Dave Logan explains in Tribal Leadership that at Stage 3, professionals tend to view colleagues as less competent than them, and they are more interested in controlling the flow of information through one-on-one relationships. Since knowledge is power in a competitive environment, this helps them maintain their image and keep others from knowing too much about them. Unfortunately, in health care, this reputation management can come at the expense of the patient.)

Three Problematic Patterns

Syed explains how these cultures are tough on those who make mistakes, believing that proper punishment enhances performance. But in reality, unjust punishment causes a number of other problems, from scapegoating to cover-ups.

Pattern #1: We scapegoat. When something goes wrong, we instinctively blame someone close to the failure without properly examining the situation. This doesn’t help: Scapegoating oversimplifies the story and exacerbates future problems by creating a climate of fear. Since nobody wants to end up scapegoated, mistakes go unreported, unexamined, and unlearned from.

(Shortform note: Cancel culture, the internet boycotting of individuals deemed problematic by public opinion, shares characteristics with scapegoating. It sometimes removes genuinely harmful individuals from power, but it doesn’t address the systems that put them in power, and the mob mentality can ruin the lives of much less powerful people. To engage in effective discourse, try following Rule Omega: Assume that someone you disagree with has at least a partially valid perspective, and try to empathize and understand before you disagree or fight.)

Pattern #2: We oversimplify the story. Syed explains that when we gloss over the full complexity of mistakes, the “narrative fallacy” is at work. In short, we make sense of complex situations by telling simple, appealing stories. Simple stories help us feel like we understand, but they’re usually wrong. We emphasize the major “story beats,” but we discount the details—thus failing to wrestle with the full complexity of the failure.

(Shortform note: In Antifragile, Nassim Nicholas Taleb explains that thought leaders often engage (knowingly or unknowingly) in “narrative discipline,” or the process of creating stories that sound great and seem to fit the data, but which are false or incomplete. This involves cherry-picking data from statistical studies to fit the narrative, since statistics can seem objective and convincing without contextualization. To avoid this, Taleb suggests running controlled experiments, which can’t be cherry-picked as easily.)

Pattern #3: We commit cognitive distortions. When faced with complex situations, cognitive dissonance—unconscious mental distortion of the facts—often comes into play. As Syed explains, nobody likes to be wrong—it’s a threat to our egos. So to defend our beliefs, our brains often distort information to conform to our beliefs. And the more invested we are, the worse the distortions. We’re all prone to three types of cognitive dissonance:

1. We reframe the evidence. We accept a piece of evidence but tell a story to conform it to our beliefs.
2. We invent justifications. We accept the evidence but find a way to excuse or rationalize it.
3. We ignore the facts outright. Sometimes, cognitive dissonance causes point-blank denial of the facts—especially if they’re too threatening to your sense of self.

(Shortform note: Cognitive dissonance occurs automatically, but we can recognize the following three signs: 1) Feeling discomfort prior to making an important decision may indicate conflicting beliefs, a form of dissonance. 2) Excessive justification or qualification of a decision can indicate that you know it wasn’t quite right. 3) Doing something while hiding it from others, like smoking in a private place, indicates dissonance between your desire to improve and your existing habit.)

Element #2: The Systemic Barrier

Syed explains that unlike learning-oriented organizations, failure-averse organizations don’t gather data, conduct investigations, analyze what went wrong, or implement changes. This poses two problems:

Problem #1: Without feedback, you can’t improve. These organizations lack a feedback loop that enables information to flow from failure to analysis to adjustment to implementation. Since feedback tells you what’s going wrong, an organization can’t improve without it.

Imagine running a marketing campaign but gathering no data about who clicked on your ads. You’d have no clue which demographics they appealed to, and thus no idea of how to better target future campaigns.

(Shortform note: In The Art of Learning, Josh Waitzkin suggests examining the theme of your errors. In short, gather data by noting down the mistakes you repeatedly make. Then, examine your notes and identify any patterns. Look for both the emotional and technical aspects of your error, and work to correct them. For example, a parent who gets impatient with their toddler might work to change their mindset while also studying better childcare techniques. To maintain the feedback loop, continue to note and correct your patterns of error.)

Problem #2: Without feedback, the system stagnates. Syed argues that people can only perform as well as the system enables them to. If an organization’s systems aren’t designed to learn from failures, they won’t get better and will also prevent employees from improving.

For example, if a hospital doesn’t gather information about whether its triage system works, they might be constantly overburdened and have no idea how to improve. Without feedback, the employees are stuck within a limited system.

(Shortform note: In his 3-2-1 newsletter, James Clear explains that we don’t “rise to the level of our goals,” we “fall to the level of our systems.” He defines your personal system as the network of habits you’ve built, and we can extend this to organizations: An organization’s system is the network of habits and habit-enabling tools (like error reporting mechanisms) that all its people engage in. If part of the system is to report errors and openly admit fault, any new employees will default to that behavior, “falling” to the level of the system they’ve stepped into.)

Failure-Averse Institutions Waste Lives and Resources

In the bigger picture, the institutional failure to improve is morally irresponsible, according to Syed. First, a stagnant institution is a missed opportunity to develop better systems that support life and improve our world. Each time a learning opportunity goes unexploited, progress slows.

(Shortform note: Jean-Jacques Rousseau, a 17th-century Enlightenment thinker, argued that if civilization progresses solely through scientific rationality, our morals will degrade. In his Discourse on the Sciences and Arts, Rousseau points to ancient Athens, where the flourishing sciences and arts led to a society focused on leisure, luxury, and decadence. Advanced scientific knowledge, he says, isn’t intrinsically moral, so if we neglect virtue in favor of scientific progress, we’ll end up with a materially advanced but morally bankrupt society.)

Second, mistakes can have life-and-death consequences. For example, consider the health care system: Preventable errors in the US medical system cause hundreds of thousands of deaths annually—making them the US’s third largest cause of death.

(Shortform note: To expand on Syed’s data, The Peterson Center Health System Tracker reveals that, while preventable errors remain problematic, many causes of death have declined: Mortality rates for infant care, respiratory diseases, diseases of the circulatory system, and cancers have all fallen since the 1980s.)

Failing isn’t the problem in and of itself. Rather, the problem is neglecting to learn from those failures. When organizations fail to report and learn from their mistakes, they perpetuate patterns of failure that put lives at risk.

In contrast to courts and hospitals, the airline industry has learned from the hundreds of lives lost in historical crashes. They analyze the failures, extract takeaways, and implement changes. Because of this, flying is now incredibly safe.

(Shortform note: In The Art of Learning, tai chi world champion Josh Waitkzin argues that we improve only as quickly as we learn from our mistakes. He suggests that if you never repeated a mistake—learning from it after one instance—you’d grow incredibly fast. While this is difficult in practice, Waitzkin advises that setting aside your ego can help. If we’re too focused on appearing competent or we’re embarrassed to admit mistakes, we’ll never learn.)

How to Create a Learning-Oriented Organization

Syed refers to psychologist Carol Dweck’s ideas of “growth mindset” versus “fixed mindset,” arguing that how we view failure determines whether we learn from it.

Syed argues that having a fixed mindset—believing that ability is set in stone—correlates with fragile self-esteem. Because failure could reveal your inadequacies, you’ll avoid it for fear of looking foolish or incapable. Regarding failure, people with a fixed mindset are threatened by failure and do everything they can to avoid it.

In contrast, having a growth mindset—believing that ability is malleable—means you understand that errors are intrinsic to learning. So you’ll regard difficulties, like a challenging work project or struggling relationship, as chances to learn. Regarding failure, people with a growth mindset believe failure is to be expected, and it’s a tool.

(Shortform note: In Grit, Angela Duckworth argues that innate talent doesn’t predict success. Instead, “grit”—a combination of resilience and direction—better determines who sticks it out until they reach success, and who drops out early. Like growth mindset, grit depends on the belief that we can learn from our mistakes. The stronger your belief, the grittier you can get.)

Mindset Determines Organizational Culture

As with individuals, an organization can exhibit a growth mindset or a fixed mindset, and that affects its culture. Crucially, an organization’s systems can’t change until its culture does. Since mindset shapes culture, we first need to change our mindsets.

An organization-level fixed mindset creates a culture of fear and blame. Since learning isn’t part of the company’s “DNA,” few mistakes come to light and people generally fear errors.

An organization-level growth mindset creates an open, collaborative culture. Since the founders and employees all understand that mistakes are intrinsic to learning, they’re comfortable erring, sharing those errors, and learning from them.

(Shortform note: In Tribal Leadership, Dave Logan argues that the best way to upgrade an organization is by upgrading its tribes. Each tribe within an organization exhibits a cultural stage, and the lower the stage, the less effective the tribe. To upgrade your organization is to upgrade your tribe’s cultures. At a higher stage, the mindset is, “work is better when we work together,” a teams-focused attitude that complements Syed’s focus on establishing an organizational growth mindset.)

Handle Mistakes Fairly

To create a culture that learns from failures, you need to handle mistakes justly. This creates a culture where employees can trust the higher-ups—the people who decide what happens when mistakes occur—because they’re neither too lax nor too harsh when reacting to mistakes.

Fair leadership prevents an environment of fear by thoroughly investigating any error. When genuine mistakes occur, like forgetting to file a report properly, the person can learn from it. When genuine negligence occurs, like a surgeon coming into work on cocaine, leadership must mete out an appropriate punishment. This helps employees feel comfortable making mistakes, while ensuring that workers remain diligent and honest.

Make Difficult Decisions That Serve the Tribe

In Tribal Leadership, Dave Logan and coauthors suggest that a great leader must understand how to put the tribe first under difficult circumstances. Acting from the tribe’s core values and shared vision, she might need to put big egos in their place, part ways with employees that no longer align with those values, or shut down stagnating projects.

The key is to listen thoroughly to the tribe, much like Syed suggests. By paying attention to the range of perspectives people bring to the table and noticing whether they’re acting in the tribe’s best interests or their own, a leader learns to “sniff out” self-interested individuals who could threaten the tribe’s success. This way, she maintains the tribe’s trust by showing that she has its best interest at heart, even when she needs to make tough choices.

Create a Learning System

After changing an organization’s culture, the final step is to change its systems. Syed offers the following three ideas:

1. Create better feedback mechanisms: Since many organizations lack a built-in error signal, a key step is to create that feedback mechanism. For example, local governments could hold weekly feedback sessions with community members to learn how they’re doing on a week-to-week basis.

(Shortform note: An easy way to gather feedback is to record video of what you’re doing. Josh Waitzkin writes in The Art of Learning that he recorded his tai chi practice sessions to identify mistakes and refine his form. Similarly, imagine recording video of a community meeting, then holding a retrospective with key players to analyze how things went and what you could do better.)

2. Use pilot programs: In politics, Syed suggests running pilot programs—small-scale tests of policy changes or business ideas that allow you to get feedback safely and inexpensively. Set up pilots that mimic real-world conditions, and avoid priming them to succeed. By staffing them with a mix of experienced and inexperienced employees, you’ll get a sense for how an actual team would perform under your program.

(Shortform note: In 2019, the US Congress passed the Foundations for Evidence-Based Policymaking Act, which stipulates that lawmakers and federal agencies must begin developing statistical evidence to support policies. While evidence-based pilots were once thought to be too expensive or time-consuming, the rise of agile methods in software engineering and in the broader business world has shifted that attitude. Now, numerous federal agencies, from the FDA to the Department of Labor, have begun running pilots, much like Syed suggests.)

3. Use “pre-mortems” to strategize: Before launching projects, Syed suggests running a “pre-mortem,” which improves outcomes by helping you discover the flaws in a plan before implementing it. Ask your team to imagine that your plan has failed completely. Then, work backward by asking what went specifically wrong and adjust the plan accordingly.

(Shortform note: In The Obstacle Is The Way, Ryan Holiday suggests that pre-mortems are also a useful tool for personal growth. Hardships, he says, are inevitable. By expecting that bad things will happen sooner or later, we better prepare ourselves for them. And by actively anticipating what could go wrong (such as with your workday, a date, or a difficult conversation) you improve your chance of success. Specifically, he suggests thinking up as many possibilities for failure as you can, and then creating a contingency plan for each.)