PDF Summary:Life 3.0, by Max Tegmark

Life on Earth has drastically transformed since it first began. The first single-celled organisms could do little more than replicate themselves. Fast-forward to today: Humans have built a civilization so complex that it would be utterly incomprehensible to the lifeforms that came before us. Judging by recent technological strides, physicist Max Tegmark believes that an equally revolutionary change is underway—artificial intelligence may become more capable than the human brain, making us the simplistic lifeforms.

In this guide, you’ll learn about cutting-edge technological theories that will help you better understand our rapidly changing world. We’ll examine the evidence that an artificial superintelligence might soon exist, explore the theoretical limits of its power, and speculate about the impact this AI might have on humanity. In our commentary, we’ll offer contrasting perspectives from other leading AI experts—those who think Tegmark’s view of AI is unrealistically alarmist and those who feel it’s an even more urgent concern than Tegmark. We’ll also update Tegmark’s ideas with news regarding AI research and development.

(continued)...

Second, Tegmark argues that it’s possible for an artificial intelligence to discard the goal we give it and choose a new one. As the AI grows more intelligent, it might come to see our human goals as inconsequential or undesirable. This could incentivize it to find loopholes in its own programming that allow it to satisfy (or abandon) our goal and free itself to take some other unpredictable action.

Finally, even if an AI accepts the goals we give it, it could still behave in ways we wouldn’t have predicted (or desired), asserts Tegmark. No matter how specifically we define an AI’s goal, there’s likely to be some ambiguity in how it chooses to interpret and accomplish that goal. This makes its behavior largely unpredictable. For example, if we gave an artificial superintelligence the goal of enacting world peace, it could do so by trapping all humans in separate cages.

(Shortform note: One possible way to reduce the chance of an AI rejecting human goals or interpreting them in an inhuman way would be to focus AI development on cognition-enhancing neural implants. If we design a superintelligent AI to guide the decision-making of an existing human (rather than make its own decisions), they could collectively be more likely to respect humanist goals and interpret goals in a human way. The prospect of merging human and AI cognition is arguably less outlandish than it may seem—tech companies like Neuralink and Synchron have already developed brain-computer interfaces that allow people to control digital devices with their thoughts.)

The Possible Extent of AI Power

Why would an artificial superintelligence’s goal have such a dramatic impact on humanity? An artificial superintelligence would use all the power at its disposal to accomplish its goal. This is dangerous because such a superintelligence could theoretically gain an unimaginable amount of power—enough to completely transform our society with a negligible amount of effort.

According to Tegmark, although an artificial superintelligence is a digital program, it could easily exert power in the real world. For instance, it could make money selling digital goods such as software applications, then use those funds to bribe humans into unknowingly working for it (perhaps posing as a human hiring manager on digital job listing platforms). An AI controlling a Fortune 500-sized human task force could do almost anything—including creating robots that the AI could control directly.

Tegmark asserts that, in theory, an artificial superintelligence could eventually attain godlike power over the universe. By using its intelligence to create increasingly advanced technology, an AI could eventually create machines able to rearrange the fundamental particles of matter—turning anything into anything else—as well as generate nearly unlimited energy to power those machines.

AI’s Power in the Digital World

While Tegmark focuses primarily on the ways AI could influence the physical world, Yuval Noah Harari emphasizes the danger posed by AI’s influence solely in the digital world. For instance, AI-controlled social media accounts could earn the trust of human users, distort their view of the world, and influence their behavior for political or economic ends.

Additionally, Harari contends that this kind of AI will threaten to unravel our society long before we develop superintelligent AI—in fact, he asserts that we should be aware of this potential danger today. People already regularly consult online resources to dictate their decisions. For instance, they use product reviews to determine what to buy, and they conduct online research to determine who to vote for. AI (or people controlling AI) therefore wouldn’t need to pay workers or manufacture reality-bending technology to totally reshape human society. Transforming the digital landscape we consult every day would be enough.

There’s evidence that this kind of distortion of the digital world has already begun—for instance, social media bots were used to discredit 2017 French presidential candidate Emmanual Macron by amplifying the spread of his leaked emails across social media platforms.

Optimistic Possibilities for Humanity

What might happen to humanity if an artificial superintelligence wields nearly unlimited power over the world in service of a single goal? Tegmark describes a number of possible outcomes, each of which results in a wildly different way of life for humans—or the end of human life.

Let’s begin by discussing three scenarios in which the AI’s goal, whatever it may be, allows humans to live relatively happy lives.

Possibility #1: Friendly AI Takes Over

First, Tegmark imagines that an artificial superintelligence could overthrow existing human power structures and use its vast intelligence to create the best possible world for humanity. No one could challenge the AI’s ultimate authority, but few people would want to, since they have everything they need to live a fulfilling life.

Tegmark clarifies that this isn’t a world designed to maximize human pleasure, which would mean continuously injecting every human with some kind of pleasure-inducing chemical. Rather, this is a world in which humans are free to continuously choose the kind of life they want to live from a diverse set of options. For instance, one human could choose to live in a non-stop party, while another could decide to live in a Buddhist monastery where rowdy, impious behavior wouldn’t be allowed. No matter who you are or what you want, there would be a “paradise” available for you to live in, and you could move to a new one at any time.

This Utopia Borrows From Existentialist Philosophy

This vision of utopia makes assumptions about humanity that align with Victor Frankl’s existentialist philosophy in Man’s Search for Meaning. According to Frankl, the primary contributor to human happiness isn’t pleasure, but meaning—that is, humans need to feel like their actions are valuable in light of a bigger purpose.

However, Frankl contends that there isn’t one universal “meaning” of life. Rather, anything can be meaningful, and each individual must discover for themselves what their life means to them. This reveals the importance of personal choice, which is central to Tegmark’s vision of utopia. Because the same life won’t feel meaningful to everyone, an AI-created world that maximizes human happiness would need to allow humans to pursue whatever life they believe to be the most meaningful.

To return to our example, someone living a non-stop party could find meaning through the experience of togetherness, while someone living as a Buddhist monk could find meaning in pious, devoted action. If that partygoer ever realized that a permanent party made them feel purposeless, they could cross over to the Buddhist sector and find meaning there.

In contrast, a world in which people are constantly taking a pleasure-inducing drug rather than living exciting lives might be OK for the people experiencing it, but most people today would probably view this as a dystopia rather than a utopia. This is because such a world would be meaningless, as every experience would feel exactly the same and nothing would ever change.

Possibility #2: Friendly AI Stays Hidden

Tegmark supposes another positive scenario that’s a bit different: Instead of completely taking over the world, an artificial superintelligence does everything within its power to improve human lives while keeping its existence a secret.

This could happen if the artificial superintelligence—or someone influencing its goals—concluded that to be as happy and fulfilled as possible, humans need to feel in control of their destiny. Arguably, if you knew that an all-powerful computer could give you anything you wanted (as in the previous optimistic scenario), you might still feel like your life is meaningless and be less satisfied because you don’t have control over your life. In this case, the best thing a godlike AI could do for you is help without your knowledge.

Is the Universe Secretly a Simulation?

If an all-powerful artificial intelligence could adopt the goal of keeping its existence a secret, how do we know one doesn’t exist already? This idea overlaps with simulation theory, the idea that our entire universe, including ourselves, is a complex computer simulation that’s creating our reality yet hiding its true nature.

This idea is popular among some physicists, who’ve come to this conclusion using facts we know about the universe. Because it’s theoretically possible that humans will at some point create a computer powerful enough to simulate the universe, there’s a chance that another civilization already has—and has created us. If so, Tegmark’s logic could explain why the true nature of this simulation is hidden from us: If we prefer control over our destiny, it would be cruel to take that illusion away from us.

Possibility #3: AI Protects Humanity From AI

Third, Tegmark imagines a scenario in which humans create an artificial superintelligence with the sole purpose of preventing other superintelligences from coming into existence. This allows humans to continue developing more advanced technology without worrying about the potential dangers of another AI.

(Shortform note: This scenario is arguably relatively unrealistic, as it assumes that we have total control over the superintelligence yet aren’t taking full advantage of its power. If we’re able to successfully program a superintelligence’s goal, we would likely get more ambitious and tell it to design a society for us in which we can be eternally happy and immortal—which would bring us back to one of the previous two optimistic scenarios we’ve discussed.)

According to Tegmark, the advanced technology humans could develop in a world free from superintelligence would eventually allow us to create a bountiful classless society. Robots are able to build anything humans might want, making scarcity a thing of the past. Since robots are constantly generating surplus wealth, the government can give everyone a universal basic income (UBI) that’s high enough to purchase anything they could possibly need. People are free to work for more money, but finding a productive job is near-impossible since everything people might buy is already given to them for free.

(Shortform note: Tegmark also acknowledges the possibility that we develop superintelligence, manage to keep it entirely under our control, and use it to create a humanist utopia. Although he doesn’t specify what he imagines this world would look like, it’s reasonable to assume that Tegmark thinks it would mirror the outcome of one of these three positive scenarios.)

How to Transition to a Post-Scarcity Society

Becoming a post-scarcity, UBI-driven society like this would require a complete overhaul of our employment-based economy. But, we wouldn’t necessarily have to enact this kind of sweeping change all at once.

AI expert Lorenzo Pieri describes how a nation might incrementally transition to this kind of society. First, the private companies that serve basic human needs progressively automate their workforces with increasingly sophisticated technology, boosting the nation’s total economic production. As existing taxes channel some of this new wealth into the government, they pass it on to citizens as a small universal basic income. The government grows this UBI alongside the economy as a whole, helping everyone increase their quality of life—especially those in poverty, whose currently unfilled needs are more essential to their well-being than the unfilled needs of people with more money.

After private companies have automated the production of all basic human needs, the government begins funneling the wealth from the automated production of non-basic goods into subsidies for basic goods rather than further increasing UBI. Eventually, the government can use the productivity gains from automation to pay private companies to supply all basic human needs for free, transitioning into a fully post-scarcity society.

If citizens desire non-basic goods that aren’t available for free, they still can get jobs producing such luxury goods for others. Pieri terms this dynamic “luxury-capitalism.”

Pessimistic Possibilities for Humanity

Next, let’s take a look at some of the existential dangers that an artificial superintelligence poses. Here are three scenarios in which the AI’s goal ruins humans’ chances to live a satisfying life.

Possibility #1: AI Kills All Humans

Tegmark contends that an artificial superintelligence may end up killing all humans in service of some other goal. If it doesn’t value human life, it could feasibly end humanity just for simplicity’s sake—to reduce the chance that we’ll do something to interfere with its mission.

(Shortform note: Some argue that an artificial superintelligence is unlikely to kill all humans as long as we leave it alone. Conflict takes effort, so an AI might conclude that the simplest option available is to pursue its mission in isolation from humanity. For instance, a superintelligence might be peaceful toward us if we don’t interfere with its goal and allow it to colonize space.)

If an artificial superintelligence decided to drive us extinct, Tegmark predicts that it would do so by some means we currently aren’t aware of (or can’t understand). Just as humans could easily choose to hunt an animal to extinction with weapons the animal wouldn’t be able to understand, an artificial intelligence that’s proportionally smarter than we are could do the same.

(Shortform note: In Superintelligence, Nick Bostrom imagines one way an artificial intelligence could kill all humans through means we would have difficulty understanding or averting: self-replicating “nanofactories.” This would be a microscopic machine with the ability to reproduce and synthesize deadly poison. Bostrom describes a scenario in which an artificial intelligence produces and spreads these nanofactories throughout the atmosphere at such a low concentration that we can’t detect them. Then, all at once, these factories turn our air toxic, killing everyone.)

Possibility #2: AI Cages Humanity

Another possibility is that an artificial intelligence chooses to keep humans alive, but it doesn’t put in the effort to create a utopia for us. Tegmark argues that an all-powerful superintelligence might decide to keep us alive out of casual curiosity. In this case, an indifferent superintelligence would likely create a relatively unfulfilling cage in which we’re kept alive but feel trapped.

(Shortform note: A carelessly, imperfectly designed world for humans to live in may be intolerable in ways we can’t imagine. This idea is dramatized by the ending of Stanley Kubrick’s 1968 film 2001: A Space Odyssey. As Kubrick explains in an 1980 interview, the film portrays an astronaut trapped in a “human zoo” created for him by godlike aliens who want to study him. They place the astronaut in a room in which it feels like all of time is happening simultaneously, and he ages and dies all at once.)

Possibility #3: Humans Abuse AI

Finally, Tegmark imagines a future in which humans gain total control over an artificial superintelligence and use it for selfish ends. Theoretically, someone could use such a machine to become a dictator and oppress or abuse all of humanity.

(Shortform note: This scenario would likely result in even more suffering than if a supreme AI decided to kill all humans. Paul Bloom (Against Empathy) asserts that cruelty is a uniquely human act in which someone feels motivated to punish other humans for their moral failings. Thus, a superintelligence under the control of a hateful dictator would be far more likely to intentionally cause suffering (as moral punishment) than an AI deciding for itself what to do.)

What Should We Do Now?

We’ve covered a range of possible outcomes of artificial superintelligence, from salvation to disaster. However, all these scenarios are merely theoretical—let’s now discuss some of the obstacles we can address today to help create a better future.

We’ll first briefly disregard the idea of superintelligence and discuss some of the less speculative AI-related issues society needs to overcome in the near future. Then, we’ll conclude with some final thoughts on what we can do to improve the odds that the creation of superintelligence will have a positive outcome.

Short-Term Concerns

The rise of an artificial superintelligence isn’t the only thing we have to worry about. According to Tegmark, it’s likely that rapid AI advancements will create numerous challenges that we as a society need to manage. Let’s discuss:

• Concern #1: Economic inequality
• Concern #2: Outdated laws
• Concern #3: AI-enhanced weaponry

Concern #1: Economic Inequality

First, Tegmark argues that AI threatens to increase economic inequality. Generally, as researchers develop the technology to automate more types of labor, companies gain the ability to serve their customers while hiring fewer employees. The owners of these companies can then keep more profits for themselves while the working class suffers from fewer job opportunities and less demand for their skills. For example, in the past, the invention of the photocopier allowed companies to avoid paying typists to duplicate documents manually, saving the company owners money at the typists’ expense.

As AI becomes more intelligent and able to automate more kinds of human labor at lower cost, this asymmetrical distribution of wealth could increase.

(Shortform note: Some experts contend that new AI-enhanced technology doesn’t have to lead to automation and inequality. If AI developers create technology that expands what one worker can do, rather than just simulating their work, that technology could create new jobs and update old ones while creating value for companies. These experts implore AI developers to consider the impact of their inventions on the labor market and adjust their plans accordingly, just as they would consider any other ethical or safety concern.)

Concern #2: Outdated Laws

Second, Tegmark contends that our legal system could become outdated and counterproductive in the face of sudden technological shifts. For example, imagine a company releases thousands of AI-assisted self-driving cars that save thousands of lives by being (on average) safer drivers than humans. However, these self-driving cars still get into some fatal accidents that wouldn’t have occurred if the passengers were driving themselves. Who, if anyone, should be held liable for these fatalities? Our legal system needs to be ready to adapt to these kinds of situations to ensure just outcomes while technology evolves.

(Shortform note: Although Tegmark contends that the legal system will struggle to keep up with AI-driven changes, other experts note that advancements in AI will drastically increase the productivity and efficiency of legal professionals. This could potentially help our legal system adapt more quickly and mitigate the damage caused by rapid change. For instance, in a self-driving car liability case, an AI language model could quickly digest and summarize all the relevant documents from similar cases from the past (for instance, a hotel-cleaning robot that injured a guest), instantly collecting the context necessary for legislators to make well-informed decisions.)

Concern #3: AI-Enhanced Weaponry

Third, AI advancements could drastically increase the killing potential of automated weapons systems, argues Tegmark. AI-directed drones would have the ability to identify and attack specific people—or groups of people—without human guidance. This could allow governments, terrorist organizations, or lone actors to commit assassinations, mass killings, or even ethnic cleansing at low cost and minimal effort. If one military power develops AI-enhanced weaponry, other powers will likely do the same, creating a new technological arms race that could endanger countless people around the world.

(Shortform note: In 2017, the Future of Life Institute (Tegmark’s nonprofit organization) produced an eight-minute film dramatizing the potential dangers of this type of AI-enhanced weaponry. After this video went viral, some experts dismissed its vision of AI-directed drones as scaremongering, arguing that even if multiple military powers developed automated drones, such weapons wouldn’t be easily reconfigured to target civilians. However, in a rebuttal article, Tegmark and his colleagues pointed to an existing microdrone called the “Switchblade” that can be used to target civilians.)

Long-Term Concerns

How should we address the long-term concerns related to AI, including potential superintelligence creation? Because there’s little we know for sure about the future of AI, Tegmark contends that one of humanity’s top priorities should be AI research. The stakes are high, so we should try our best to discover ways to control or positively influence an artificial superintelligence.

The Current State of AI Research Funding

It seems that many people agree with Tegmark, as major institutions around the world are already prioritizing AI research. The European Union is currently investing €1 billion a year in AI research and development, and it intends to increase that annual investment to €20 billion by the year 2030. Private companies are leading AI research in the United States—for instance, Meta plans to spend $33 billion on AI research in 2023 alone.

However, some experts worry that private companies may not act in line with public interest while researching AI, and they urge the US government to fund a cutting-edge AI research program of its own. It’s possible that state-controlled research would be less likely to unleash a dangerous superintelligence, as governments lack the profit motive to create a marketable AGI as quickly as possible.

What Can You Do to Prevent the AI Apocalypse?

Outside of AI research, Tegmark recommends cultivating hope grounded in practical action. Before we can create a better future for humanity, we have to believe that a bright future is possible if we band together and responsibly address these technological risks.

After cultivating this optimistic attitude, Tegmark urges readers to do everything they can to make the world more ethical and peaceful—not just in the field of AI, but in every aspect of society. The more humans who are willing to empathize and cooperate with one another, the greater the chance that we’ll develop AI safely and with the intent to benefit all of humanity. This could involve organizing a fundraiser for a local homeless shelter, volunteering at a nursing home, or just being kinder to the people around you.

Determinate vs. Indeterminate Optimism

The attitude Tegmark urges readers to adopt is what Peter Thiel in Zero to One calls “determinate optimism”—you expect the future to be better than the present, and you believe that you can successfully predict and bring about specific positive outcomes. Thiel argues that in contrast to this perspective, most Americans today (or in 2014, when Zero to One was written) think in terms of “indeterminate optimism”—they believe that things will get better in the future, but they assume that the future is too unpredictable for them to plan.

According to Thiel, this is a problem because indeterminate optimism encourages people to be passive and short-sighted: They think, “Why bother planning a better future? Things will turn out OK no matter what I do.” To combat this, Thiel urges optimists to make long-term plans and stick to them. To apply this to Tegmark’s plea to make the world more ethical and peaceful: Don’t just become a generally cooperative person in life; instead, come up with a plan to bring people together and motivate them to treat others well.