This article is an excerpt from the Shortform book guide to "Superintelligence" by Nick Bostrom. Shortform has the world's best summaries and analyses of books you should be reading.
Like this article? Sign up for a free trial here.
How can humans responsibly use AI? What type of motives should humans program in AI?
According to Superintelligence by Nick Bostrom, making sure every superintelligent AI has good ultimate motives may be the most important part of AI development. Ultimately the superintelligent AI’s own motives will be the only thing that constrains its behavior.
Keep reading for a number of approaches for the responsible use of AI.
1. Hard-Coded Commandments
As Bostrom remarks, one approach for the responsible use of AI is to hard-code a set of imperatives that constrain the AI’s behavior. However, he expects that this is not practicable. Human legal codes illustrate the challenges of concretely defining the distinction between acceptable and unacceptable behavior: Even the best legal codes have loopholes, can be misinterpreted or misapplied, and require occasional changes. To write a comprehensive code of conduct for a superintelligent AI that would be universally applicable for all time would be a monumental task, and probably an impossible one.
| Commandments and Free Will The question of free will presents additional complications for this approach. Even if rules and regulations are created to eliminate loopholes, misinterpretations, and so on, they’ll only restrain people if those people choose, using their free will, to obey them. The question is, would AI evolve a free will that would empower it to disobey rules it doesn’t want to follow? Admittedly, there is some debate over whether human free will is real or just an illusion, and more debate about whether it will ever be possible to endow an AI with free will. But some sources assert that free will is an essential component of human cognition, playing a key role in consciousness and higher learning capabilities. If this proves true, then free will might be an essential component of general intelligence, in which case any AI with superhuman general intelligence would have free will. Then the AI could choose to disobey a pre-programmed code of conduct, further complicating the problem of controlling its behavior. This possibility reinforces Bostrom’s assertion that hard-coded commandments are probably not the best approach to giving an AI the right motives. |
2. Existing Motives
Another approach that Bostrom discusses is to create a superintelligent AI by increasing the intelligence of an entity that already has good motives, rather than trying to program them from scratch. This approach might be an option if superintelligent AI is achieved by the method of brain simulation: Choose a person with exemplary character and scan her brain to create the original model, then run the simulation on a supercomputer that allows it to think much faster than a biological brain.
However, Bostrom points out that there is a risk that nuances of character, like a person’s code of ethics, might not be faithfully preserved in the simulation. Furthermore, even a faithful simulation of someone with good moral character might be tempted to abuse the powers of a superintelligent AI.
| Does Power Corrupt? The risk Bostrom identifies that even a person of good character who was given the capabilities of superintelligent AI might abuse those powers calls to mind the old adage that power corrupts people who wield it. A psychological study published the same year as Bostrom’s book found scientific evidence for this. When people were given the choice between options that benefited everyone and options that benefited themselves at others’ expense, initially those with higher levels of integrity tended to choose the options that benefited everyone, while the people with lower levels of integrity chose the opposite. But over time, this difference disappeared, and everyone leaned toward choosing the options that benefited themselves. Thus, the risk of a superintelligent AI based on a simulation of a human brain pursuing its own objectives at other people’s expense appears to be significant, even if the original human was a person of good character. In addition, if the person’s moral code wasn’t completely preserved in the simulation, a risk Bostrom also warns about, the superintelligent AI would probably show selfish tendencies even sooner. |
———End of Preview———
Like what you just read? Read the rest of the world's best book summary and analysis of Nick Bostrom's "Superintelligence" at Shortform.
Here's what you'll find in our full Superintelligence summary:
- How an AI superintelligence would make humans the inferior species
- Why AI can't be expected to act responsibly and ethically
- How to make sure a superintelligent AI doesn’t destroy humankind
