This article is an excerpt from the Shortform book guide to "Superintelligence" by Nick Bostrom. Shortform has the world's best summaries and analyses of books you should be reading.
Like this article? Sign up for a free trial here.
What’s the negative impact of artificial intelligence on society? How much will a superintelligent AI change the world?
According to Superintelligence by Nick Bostrom, sooner or later, a superintelligent AI will be created. Depending on the superintelligent AI’s behavior, these changes could be very detrimental to humanity.
Let’s look at a more in-depth explanation of how a superintelligent AI will affect society.
The Consequences of Superintelligent AI
If an AI has some measure of general intelligence and the ability to modify its own programming, its intelligence would likely increase at an ever-accelerating rate. This implies that an AI might rise from sub-human to superhuman intelligence very quickly, which is probably the biggest negative impact of artificial intelligence on society.
(Shortform note: Although we’ve not yet witnessed this type of growth in artificial intelligence, there are other applications that demonstrate how self-accelerating growth can cause rapid transformations. One is the “coulombic explosion” reaction between water and alkali metals, where the chemical reaction causes the surface area of the metal to increase and the reaction speed is proportional to the surface area. When this condition occurs, the reaction rate increases so quickly that the metal appears to explode.)
Moreover, as Bostrom points out, superior intelligence is what has allowed humans to dominate the other life forms on planet Earth. Thus, it stands to reason that once a superintelligent AI exists, the fate of humanity will suddenly depend more on what the superintelligent AI does than on what humans do—just as the existence of most animal species depends more on what humans do (either to take care of domestic animals or to preserve or destroy habitat that sustains wild animals) than on what the animals themselves do.
| Superior Intelligence or Superior Communication? There are differences of opinion about exactly what elevated humans above other animals. In Homo Deus, Yuval Noah Harari contends that it wasn’t humans’ greater intelligence, per se, but rather their greater capacity for communication and coordinated work. In fact, he argues that human intelligence really isn’t that different from, or very far above, the intelligence of other animals. But if Harari is correct, this perspective would actually strengthen Bostrom’s conclusions about the rise of AI. This is because computers already have greater communication and coordination capabilities than humans—after all, that’s one of the main things humans use computers for. And if the intelligence gap between humans and animals is small, then an AI with even slightly superhuman general intelligence (and a much greater capacity for communication) might be in a position to bring about sweeping changes, just as humans did for other animals. |
The Abilities of Superintelligent AI
But how would a superintelligent AI actually gain or wield power over the earth if it exists only as a computer program? Bostrom lists some abilities that an AI would have as soon as it became superintelligent.
- It would be capable of strategic thinking. Consequently, it could develop plans to achieve long-term objectives and would take into account any applicable opposition.
- It could manipulate and persuade. It could figure out how to get humans to do what it wanted them to, much like a human might train a dog to play fetch. Humans might not even realize the superintelligent AI was trying to manipulate them.
- It would be a superlative hacker. It could gain access to virtually all networked technology without needing anyone’s permission.
- It would be good at engineering and development. If it needed new technology or other devices that didn’t exist yet in order to achieve its objectives, it could design them.
- It would be capable of business thinking. It could figure out ways to generate income and amass financial resources.
———End of Preview———
Like what you just read? Read the rest of the world's best book summary and analysis of Nick Bostrom's "Superintelligence" at Shortform.
Here's what you'll find in our full Superintelligence summary:
- How an AI superintelligence would make humans the inferior species
- Why AI can't be expected to act responsibly and ethically
- How to make sure a superintelligent AI doesn’t destroy humankind
