PDF Summary:Rebooting AI, by Gary Marcus and Ernest Davis

Artificial Intelligence could revolutionize every aspect of our lives, but only if we can develop AI that reliably carries out its tasks. While some see AI as a cornucopia that will free the human race from drudgery, others see dangers in ceding so much power to digital, non-human minds. In Rebooting AI, Gary Marcus and Ernest Davis argue that modern AI underdelivers on its promise while AI proponents oversell what it can do. Marcus and Davis are AI advocates who feel that current AI research is heading in the wrong direction, with its narrow focus on “big data” and machine learning.

In this guide, we’ll examine how the public perception of modern AI differs from reality, how AI works, and the many ways in which it fails. We’ll then describe Marcus and Davis’s prescription for producing stronger AI systems to benefit the human race. We’ll also highlight where modern AI trends are leading, other experts’ thoughts on machine cognition, and how AI development is affecting business and society at large.

(continued)...

(Shortform note: In the field of image recognition, especially in terms of fine-tuning AI “vision” so that it can correctly identify objects, the research is ongoing. Since, as Davis and Marcus point out, the process by which neural networks process any given piece of information is opaque, some researchers are working to determine exactly how neural networks interpret visual data and how that process differs from the way humans see. Possible methods to improve computer vision include making AI’s inner workings more transparent, including artificial models with real-world inputs, and developing ways to process images using less data than modern AIs require.)

Hallucinations illustrate a difference between human and machine cognition—we can make decisions based on minimal information, whereas machine learning requires huge datasets to function. Marcus and Davis point out that if AI is to interact with the real world’s infinite variables and possibilities, there isn’t a big enough dataset in existence to train an AI for every situation. Since AIs don’t understand what their data mean, only how those data correlate, AIs will perpetuate and amplify human biases that are buried in their input information. There’s a further danger that AI will magnify its own hallucinations as erroneous computer-generated information becomes part of the global set of data used to train future AI.

Bad Data In, Bad Data Out

The issue that Marcus and Davis bring up about AIs propagating other AIs’ hallucinations is commonly known as model collapse. This danger grows as more AI-generated content enters the world’s marketplace of ideas, and therefore the data pool that other AIs draw from. This danger isn’t theoretical either—media outlets that rely on AI for content have already been caught generating bogus news, which then enters the data content stream feeding other AI content creators.

The problem with relying on purely human content isn’t only that much of it contains human bias, but also that bias is hard to recognize. In Biased, psychologist Jennifer Eberhardt explains that most forms of prejudice are unconscious—if we’re not aware of their existence in ourselves, how can we possibly hope to prevent them from filtering into our digital creations? To make things more difficult, bias is, at its core, a form of classification, which Davis and Marcus freely point out is what narrow AI is getting really good at doing. Parroting and projecting human bias is therefore second nature to data-driven AI.

One solution might be to develop AI that can classify data and draw conclusions with little or no training data examples, cutting out human bias and other AIs’ hallucinations altogether. Known as “less than one” or “zero-shot learning,” techniques are currently being developed to teach AI to categorize images and text that aren’t present anywhere in its training data. As of early 2024, the zero-shot approach is still under development but is showing the potential to eliminate some of the problems inherent in big data AI training techniques.

Problems With Language

A great deal of AI research has focused on systems that can analyze and respond to human language. While the development of language interfaces has been a vast benefit to human society, Davis and Marcus insist that the current machine language learning models leave much to be desired. They highlight how language systems based entirely on statistical correlations can fail at even the simplest of tasks and why the ambiguity of natural speech is an insurmountable barrier for the current AI paradigm.

It’s easy to imagine that when we talk to Siri or Alexa, or phrase a search engine request as an actual question, that the computer understands what we’re asking; but Marcus and Davis remind us that AIs have no idea that words stand for things in the real world. Instead, AIs merely compare what you say to a huge database of preexisting text to determine the most likely response. For simple questions, this tends to work—but if your phrasing of a request doesn’t match the AI’s database, the odds of it not responding correctly increase. For instance, if you ask Google “How long has Elon Musk been alive?” it will tell you the date of his birth but not his age unless that data is spelled out online in something close to the way you asked it.

(Shortform note: In the years since Rebooting AI’s publication, search engine developers have worked to include natural language processing into search algorithms so that search results don’t entirely depend on matching keywords in your queries. Natural language systems are trained to recognize the peculiarities of human speech, including common errors, “filler words,” intent, and context. The business sector is pushing research on stronger natural language systems thanks to the inherent flaws of keyword searching and the friction it causes for customers and stakeholders.)

A Deficiency of Meaning

Davis and Marcus say that though progress has been made in teaching computers to differentiate parts of speech and basic sentence structure, AIs are unable to compute the meaning of a sentence from the meaning of its parts. As an example, ask a search engine to “find the nearest department store that isn’t Macy’s.” What you’re likely to get is a list of all the Macy’s department stores in your area, clearly showing that the search engine doesn’t understand how the word “isn’t” relates to the rest of the sentence.

(Shortform note: Guiding an online search using words like “isn’t” is a form of Boolean logic, a type of algebraic formulation that uses the conjunctions “and,” “or,” and “not” to determine whether a given point of data meets the requested search criteria. Boolean logic is a cornerstone of traditional computer programming which Davis and Marcus say has been discarded in the techniques used to train neural networks. Boolean terms can be used in search engines, but generally not via natural language. In Google, the words “and” and “or” must be typed in ALL CAPS to be used as operators, while the word “not” must be replaced by a minus sign to work, as in “find the nearest department store -Macy’s.”)

An even greater difficulty arises from the inherent ambiguity of natural language. Many words have multiple meanings, and sentences take many grammatical forms. However, Marcus and Davis illustrate that what’s most perplexing to AI are the unspoken assumptions behind every human question or statement. Given their limitations, no modern AI can read between the lines. Every human conversation rests on a shared understanding of the world that both parties take for granted, such as the patterns of everyday life or the basic laws of physics that constrain how we behave. Since AI language models are limited to words alone, they can’t understand the larger reality that words reflect.

(Shortform note: The underlying cause that Marcus and Davis hint at for AI’s lack of real-world understanding is that LLMs have no physical embodiment and interaction from which they can attach meaning and experience to the words and images they process. This isn’t true for other forms of AI, such as those that guide robots’ real-world interactions, but work to merge LLM technology with that of robotics is still in its infancy as of early 2024. If successful, these techniques may prove to be a big step toward teaching robots with AI how to infer meaning rather than relying on specific, spelled-out instructions for every minuscule action they take.)

Problems With Awareness

While language-processing AI ignores the wider world, mapping and modeling objects in space is of primary concern in the world of robotics. Since robots and the AIs that run them function in the physical realm beyond the safety of mere circuit pathways, they must observe and adapt to their physical surroundings. Davis and Marcus describe how both traditional programming and machine learning are inadequate for this task, and they explore why future AIs will need a broader set of capabilities than we currently have in development if they’re to safely navigate the real world.

Robots are hardly anything new—we’ve had robotic machinery in factories for decades, not to mention the robotic space probes we’ve sent to other planets. However, all our robots are limited in function and the range of environments in which they operate. Before neural networks, these machines were either directly controlled or programmed by humans to behave in certain ways under specific conditions to carry out their tasks and avoid a limited number of foreseeable obstacles.

(Shortform note: Despite Davis and Marcus’s statements to the contrary, we’ve developed robots that can overcome obstacles in environments outside of human control—most notably the autonomous rovers sent to explore the surface of Mars. Because of the lightspeed time delay between Earth and Mars, the rovers can’t be driven remotely. Instead, the rovers are empowered to pick objects for study based on their own direct observations, as well as to make their own course corrections to avoid potential hazards and other obstacles. The Perseverance rover autonomously explored nearly 18 kilometers of Mars’s surface in 2021 alone, surviving in a much more open terrain than the ideal laboratory conditions Davis and Marcus write about.)

Real-World Complexity

With machine learning’s growing aptitude for pattern recognition, neural network AIs operating in the real world are getting better at classifying the objects that they see. However, statistics-based machine learning systems still can’t grasp how real-world objects relate to each other, nor can AIs fathom how their environment can change from moment to moment. Researchers have made progress on the latter issue using video game simulations, but Marcus and Davis argue that an AI-driven robot in the real world could never have time to simulate every possible course of action. If someone steps in front of your self-driving car, you don’t want the car to waste too much time simulating the best way to avoid them.

Building Better Models and Simulations

Researchers at MIT have made progress on the object relationship problem by devising a method for AI to interpret individual object relationships—such as a coffee cup sitting next to a keyboard—and combining those individual relationships to build a complete internal picture of a more complicated setting—such as the collection of interrelated objects cluttering your desktop at work. This “one relationship at a time” approach has been more successful than prior attempts to teach AI to process an entire array of object relationships at once, a task that human brains do instinctively.

What the human brain can’t instinctively do is generate a multitude of simulations to explore the outcomes of many variables at once. While, as Marcus and Davis suggest, this may not be a helpful approach to real-time decision-making, generative AI simulations have proven useful in designing solutions to long-term problems in engineering, medicine, and finance. The lessons learned from these simulations are fed back to the AI in the next round of simulations; and while this has led to advancements in fields such as automotive design, it again raises the specter of model collapse if AI simulations reinforce their own mistakes.

Davis and Marcus believe that an AI capable of safely interacting with the physical world must have a human-level understanding of how the world actually works. It must have a solid awareness of its surroundings, the ability to plan a course of action on its own, and the mental flexibility to adjust its behavior on the fly as things change. In short, any AI whose actions and decisions are going to have real-world consequences beyond the limited scope of one task needs to be strong AI, not the narrow AIs we’re developing now.

(Shortform note: Not everyone agrees that more humanlike AIs are even possible or desirable. In Mind Over Machine and What Computers Still Can’t Do, philosopher Hubert Dreyfus argues that too much of human intelligence and reasoning comes from cultural and experiential knowledge, which data-driven computers can’t emulate. This isn’t far from Davis and Marcus’s thesis that machine learning through big data isn’t enough to create strong AI. However, others argue that we don’t need strong AI at all—that narrow AI will soon surpass human abilities in many ways, and the belief that machines must think like us is a limited, human-centric attitude.)

The Road to Strong AI

To be clear, Marcus and Davis aren’t against AI—they simply believe that what the world needs is more research on strong AI development. The path to achieving strong AI systems that can genuinely understand and synthesize information requires drawing on more than big data and current machine learning techniques. The authors advocate for AI developers to make use of current research in neuroscience and psychology to build systems capable of human-level cognition, ones that learn like the human brain does instead of merely correlating data. The authors add that these systems should be developed with more rigorous engineering standards than have been employed in the industry so far.

Davis and Marcus don’t deny that modern AI development has produced amazing advances in computing, but they state that we’re still falling short of AI’s true potential. An AI with the ability to understand data would be able to read all the research in a field—a task no human expert can do—while synthesizing that information to solve problems in medicine, economics, and the environment that stump even the brightest human minds. The advent of strong AI will be transformative for the whole human race, but Marcus and Davis insist that we won’t get there by feeding data to narrow AI systems. The AIs of the future will have to think and learn more like humans do, while being held to higher performance standards than modern AIs can achieve.

(Shortform note: Davis and Marcus’s assertion that strong AI should be modeled on the human brain goes back decades. In The Singularity Is Near, published in 2005, futurist Ray Kurzweil argues that creating a digital simulation of the brain is a necessary step in AI development. Kurzweil observes that the human brain’s major advantage over digital computers is that it’s massively parallel—it uses countless neural pathways operating in tandem, as opposed to traditional computing’s more linear approach. Mapping the brain’s multitude of parallel systems and simulating them in a digital environment may go a long way to addressing the issues that Marcus and Davis have with narrow AI.)

Human-Level Cognition

Narrow AIs are computational black boxes where information goes in, passes through a single (if convoluted) algorithm, then comes out reprocessed as a new result. That’s not how the human brain works, and Marcus and Davis argue that strong AI shouldn’t work like that either. Instead, AI research should draw on the efficiencies of the human brain, such as how it uses different processes for different types of information, how it creates abstract models of the world with which it interacts, and how it goes beyond correlating data to think in terms of causality—using mental models to understand how the external world changes over time.

Unlike narrow, single-system AI, the brain is a collection of systems that specialize in different types of information—sight, sound, and touch, for example—while regulating different forms of output, such as conscious and unconscious bodily functions. Likewise, Marcus and Davis write that strong AI should incorporate multiple processing systems and algorithms to handle the panoply of inputs and problems it will encounter in the real world. Also like the human brain, strong AI must be flexible enough to combine its different systems in whatever arrangement is needed at the moment, as humans do when we associate a memory with a smell, or when an artist engages both her visual and manual skills to produce a painting or a sculpture.

(Shortform note: The idea of modeling machine intelligence on the brain can be traced back to computer scientist Norbert Wiener’s book Cybernetics, published in 1948. However, most brain-based AI research has focused on simulating the behavior of neurons with little attention given to replicating the brain’s larger structures. That trend may be changing, with the development of a new type of neural network called a transformer. It mimics the brain’s hippocampus, which is closely tied to learning and memory. Transformers have proven useful in solving translation issues that plagued older AI frameworks, hinting that Marcus and Davis are correct that closer brain emulation is key to building more powerful AIs in the future.)

Mental Models and Causality

A more challenging but necessary step to strong AI will be designing knowledge frameworks that will let computers understand the relationships among entities, objects, and abstractions. These relations form the bedrock of human thought in the form of the mental models we use to interpret the world around us. Davis and Marcus state that modern AI developers largely disregard knowledge frameworks as an AI component. But without knowing how information interrelates, AI is unable to synthesize data from different sources and fields of knowledge. This ability is as crucial to advancing scientific progress as it is to driving a car in the snow while adjusting your schedule because of school closings—all of which AI could help with.

(Shortform note: Before AI can understand relationships between the real world and abstract concepts, it will first have to be able to perform abstract reasoning—solving problems conceptually through logic and imagination without complete data or exact prior experience. Swiss researchers have made progress in this field using a combination of machine learning and a “vector symbolic” architecture that’s conceptually similar to the knowledge frameworks Davis and Marcus recommend. Meanwhile, software engineer François Chollet has created the Abstraction and Reasoning Corpus, a tool for measuring how effectively AI systems perform basic abstract reasoning tasks as compared to humans presented with the same challenge.)

In addition to having a knowledge framework, strong AI must understand causality—how and why objects, people, and concepts change over time. Marcus and Davis say this will be hard to achieve because causality leads to correlation, but not all correlations are causal. For example, though most children like cookies (a correlation), enjoying cookies doesn’t cause childhood. Furthermore, AI will have to juggle causality in multiple fields of knowledge at once. For instance, an AI working on a legal case will have to understand how human motivations interact with physical evidence. At present, the only way for a computer to model causality is to run multiple simulations, but that’s far less efficient than how the human brain works. Therefore, we must design our strong AIs to learn these concepts the way that humans learn.

(Shortform note: It’s not accurate to say that our human understanding of causality isn’t based on simulations—we just call them stories instead. In Wired for Story, Lisa Cron explains that the human brain uses story structure to store information and make causal predictions, such as what will happen if you poke a sleeping bear. The difference between stories and computer simulations is that stories are largely symbolic, revolving around a few key elements rather than a plethora of details, and are therefore more versatile and efficient than full simulations. AI developers are now starting to use narrative structures to organize AI-generated content and to shape the data used for AI training in much the same way that the human brain parses data.)

Brain-Like Learning

When it comes to teaching computers how to think, we can draw from millions of years of human evolution and incorporate facets of the human brain’s tried-and-true approach to learning into machine cognition. While current machine learning operates from a starting point of pure data, Davis and Marcus argue that preprogrammed knowledge about the world similar to what humans are born with can facilitate stronger AI development. The authors describe how the brain learns by combining first-hand experience with preexisting knowledge, how AIs could use this tactic to construct their own knowledge frameworks, and why a hybrid approach to machine learning would be more powerful than the current data-only techniques.

When learning, people draw from two sources—high-level conceptualizations that are either instinctive or taught to us by others, and low-level details that we absorb through day-to-day experiences. Our preexisting, high-level knowledge provides a vital framework through which we interpret whatever we discover on our own. For example, we’re born knowing that food goes into our mouths and tastes good—as children, we then use this framework to determine what does and doesn’t qualify as food. However, Marcus and Davis report that AI developers shun the idea of preprogramming knowledge into neural networks, preferring their systems to learn from data alone, free of any context that would help it make sense.

(Shortform note: Psychologists classify humans’ instinctive knowledge into two broad categories—individual instincts, such as eating and self-protection, and social instincts like reproduction and play. Instincts are different from autonomic functions such as digestion and blood circulation in that they trigger complex behaviors and are tied to emotional responses. In Instinctive Computing, computer science professor Yang Cai discusses how and what types of instincts should be programmed into AI to bring it closer in line with biological intelligence while laying the groundwork for AI self-awareness, a topic that Marcus and Davis barely broach.)

Preprogrammed knowledge frameworks—like a set of “instincts” an AI would be born with— could greatly advance AI language comprehension. When humans read or listen to language, we construct a mental model of what’s being described based on our prior understanding of the world. Davis and Marcus argue that giving an AI a preprogrammed basis for connecting language to meaning would let it construct its own knowledge frameworks, just as humans learn over time. By insisting that AIs learn from raw data alone, developers tie their hands behind their backs and create an impossible task for AI, like giving a calculator the complete works of Shakespeare and expecting it to deduce the English language from scratch.

The Basis for Language in the Brain

Some experts argue that a preprogrammed basis for comprehending language already exists in the natural world. In The Language Instinct, Steven Pinker explains that our human propensity for language is an evolutionary trait that’s hardwired into our brains. Even though no one’s born speaking English, Arabic, Latin, or Hindi—the specifics must be learned from outside data—we’re born with the ability to recognize patterns, apply meanings, and combine them in infinite variations: skills that form the building blocks of all languages.

Our wiring for language is so strong that children who grow up without a native language instinctively create one of their own. As the history of technology shows, anything in nature can be artificially replicated, such as the brain-like innate language framework that Davis and Marcus recommend for AI.

Marcus and Davis conclude that neither preprogrammed knowledge nor enormous data dumps are sufficient in themselves to teach computers how to think. The road to strong AI will require a combination of engineered cognitive models and large amounts of input data so that artificial intelligence can have a fighting chance to train itself. Knowledge frameworks can give AI the capability for logic and reason beyond their current parlor tricks of generating output from statistical correlations. Meanwhile, absorbing information from big data can give AI the experience to build its own cognitive models, growing beyond its programmers' designs.

(Shortform note: Aligned with Marcus and Davis’s thoughts about modeling AI functions on the regions of the brain, software engineer Jarosław Wasowski writes that LLM cognitive models for processing data should be built on our understanding of how the human brain processes memory. These would include separate modules for encoding sensory and short-term memory, forgetting what’s irrelevant, and indexing what’s important into long-term storage. Once there, the AI could synthesize what it knows with a “knowledge module” comprising organizational tools, procedures, relationship structures, and past experience. In essence, Wasowski is researching exactly the combination of approaches Marcus and Davis recommend.)

Higher Engineering Standards

As AI permeates our lives more and more, the degree to which it functions well or poorly will have more of an impact on the world. However, because many AI applications have been in fields like advertising and entertainment where the human consequences of error are slight, AI developers have grown lackadaisical about performance standards. Davis and Marcus discuss their AI safety concerns, the difficulty of measuring an AI’s performance, and the minimum expectations we should have regarding AI’s reliability before we hand over the reins of power.

In most industries, engineers design systems to withstand higher stressors than they’re likely to encounter in everyday use, with backup systems put in place should anything vital to health and safety fail. Marcus and Davis say that compared to other industries, software development has a much lower bar for what counts as good performance. This already manifests as vulnerabilities in our global information infrastructure. Once we start to put other vital systems in the hands of unreliable narrow AI, a slipshod approach to safety and performance could very well have disastrous consequences, much more so than chatbot hallucinations.

(Shortform note: Some of the problems Marcus and Davis point out regarding the poor state of software engineering standards may be due to a mismatch between how corporate managers and computer programmers understand the process of software development. In The Phoenix Project, Gene Kim, Kevin Behr, and George Spafford demonstrate how efficient and productive software development gets hamstrung by a combination of unrealistic management expectations and engineers who lose sight of the business goals their work supports. Kim, Behr, and Spafford promote a production line model of IT work known as DevOps to bring the standards and practices of manufacturing into the world of software development.)

Exacerbating AI’s issues with performance, when AIs go wrong, they’re very hard to debug precisely because of how neural networks work. For this reason, Davis and Marcus are engaged in research on ways to measure AI’s progress and performance. One method they hope to adapt for AI is “program verification”—an approach that’s been used in classical software to confirm that a program’s outputs match expectations. They also recommend that other AI designers explore a similar approach to improving performance, perhaps by using comparable AI systems to monitor each other’s functionality.

(Shortform note: Quantifying the performance of AI may actually turn out to be the easy part. We now have a variety of metrics to measure AI performance, accuracy, and precision, but determining whether AI is used ethically and safely is becoming a more prominent concern in the field. In 2023, US President Joe Biden signed an executive order directing AI developers and government agencies to share security data, establish safety standards, and protect the interests of consumers and workers through the economic changes brought about by AI.)

It would be unrealistic to expect any computer system to be perfect. However, the weakness of narrow AI is that without human-level comprehension, it’s prone to unpredictable, nonsensical errors outside the bounds of merely human mistakes. Marcus and Davis insist that until we develop stronger AI systems, people should be careful not to project human values and understanding on these purely automated systems. Most of all, if we’re to grant AI increasing levels of control, we should demand that AI have the same shared understanding of the world that we’d expect from our fellow human beings.

(Shortform note: The discussion of how to give AI human values began in science fiction long before it became a practical necessity. Beyond the often-portrayed possibility of strong AI taking over the world, or at least displacing humans from whole sectors of employment, there are other ethical considerations, such as whose human values AI should be aligned with and whether self-aware AI should have legal rights. In The Singularity Is Near, Ray Kurzweil offers the view that if strong AI is modeled on our human understanding of the world, it will include human values as part of its program, just as children learn values from their parents. It follows, then, that as we build strong AI, we must set the best example for it that we can.)