PDF Summary:The Case Against Reality, by Donald D. Hoffman

What if everything you perceive—the objects around you, space, time itself—isn't reality at all, but merely an interface designed by evolution? In The Case Against Reality, cognitive scientist Donald D. Hoffman argues that our perceptions don't reveal objective reality. Instead, they function like icons on a computer desktop, hiding the true nature of reality while guiding us toward survival and reproduction.

Hoffman presents his Interface Theory of Perception, supported by evolutionary game theory and mathematical proofs, to show that natural selection favors perceptions that enhance fitness over those that reveal truth. He explores how our senses compress fitness information into manageable forms, why symmetry in perception doesn't reflect reality's structure, and what this means for our understanding of consciousness, causation, and the fundamental nature of existence.

(continued)...

The crucial thing is that our senses deliver information about fitness payoffs, and it's essential for survival to receive the correct information. Make a mistake with the digits concerning fitness. and it could be a matter of life and death. We can assume evolution integrated redundancy into how we perceive, crafting our spacetime desktop and icons of physical objects as redundant codes for fitness payoffs, allowing for error detection and correction.

(Shortform note: Hoffman’s claim that evolution built redundancy into our perceptual systems is supported by research on population coding in sensory neuroscience. Population coding refers to the way groups of neurons collectively encode information about sensory stimuli. In their book Spikes: Exploring the Neural Code, Fred Rieke and colleagues describe experiments showing that many neurons in the visual and auditory systems respond to the same features of a stimulus, such as the orientation of a line or the frequency of a sound. This redundancy makes our perceptual estimates more robust to noise and loss.)

Bekenstein and Hawking found the same thing regarding spacetime. It repeats itself. The symmetry of how we perceive objects reflects the symmetry of objective reality. The vision scientist Zygmunt Pizlo spells this out. "Consider the shapes of animal bodies. The majority of them, if not every one, have mirror symmetry. What indicates they're symmetrically mirrored? Since we perceive them that way. It's impossible to see an object as symmetrical unless its two halves appear identical in shape. Consider how extraordinary this is: (1) We can only observe the visible fronts of each of the halves, and (2) we see these two halves from opposing directions. Unless shape constancy is a real phenomenon and unless it is close to perfect, we would not even know that symmetrical shapes actually exist. " Whenever we perceive something as symmetrical, this means objective reality must be symmetrical as well.

(Shortform note: Bekenstein and Hawking didn’t discover that spacetime repeats itself. They discovered that information is preserved in black holes. This is a very different thing. The information in a black hole is not spacetime itself. It’s information about the matter and energy that fell into the black hole. The information is stored on the event horizon of the black hole. This is a two-dimensional surface that surrounds the black hole. The information is stored in a way that is consistent with the laws of quantum mechanics. This means that the information is not lost, even though the matter and energy that fell into the black hole are no longer accessible.)

The "Invention of Symmetry Theorem," a conjecture of mine that Chetan Prakash proved, shows the claim to be false. The theorem claims that perceived symmetries have no bearing on the structure of true reality. The demonstration builds on a specific example. It demonstrates exactly how both activities and perceptions can exhibit symmetry—like moving, rotating, reflecting, and Lorentz transformations—even when symmetry is absent. We observe numerous symmetrical objects. Why do we see symmetry if perceptual symmetries don't show reality's symmetries? The explanation, as before, is that algorithms and structures used for reducing data size and fixing mistakes often rely on symmetries. Excess fitness data can be reduced to a manageable amount with symmetries. We utilize symmetries as straightforward algorithms for data compression and error correction.

(Shortform note: The connection between symmetry, data compression, and error correction has been explored in information theory for decades. For example, the classic textbook Elements of Information Theory by Cover and Thomas (1991) discusses how symmetrical code structures can achieve efficient compression and robust error correction. The book shows how carefully designed symmetrical code structures can achieve efficient data compression and robust error correction. For instance, the authors explain how symmetrical codebooks can be used to create efficient source codes that approach the theoretical limits of data compression. They also show how symmetrical parity-check matrices can be used to construct error-correcting codes that can detect and correct errors in transmitted data.)

The symmetrical patterns we perceive demonstrate the way we encode and condense information, though they don't reflect objective reality. We can develop machine vision systems capable of driving vehicles and recognizing the same shapes and symmetries as we do. The Invention of Symmetry Theorem relates to every perceptual system, whether biological or mechanical. The symmetries a computer sees entail nothing about the structure of objective reality. We can create a robot that detects the same symmetries we do. However, this provides us with no understanding of the world's structure.

(Shortform note: Particle physicists like Leon Lederman argue that the symmetries we observe in nature reveal the structure of the universe. In Symmetry and the Beautiful Universe, Lederman explains how symmetry principles have guided the discovery of new particles and forces. For example, the discovery of the Higgs boson was based on predictions from symmetry considerations in the Standard Model of particle physics. Lederman argues that these symmetries aren't just mathematical conveniences but reflect deep truths about the universe's structure. He suggests that by studying symmetries, we can uncover the fundamental laws governing reality, much like how the periodic table's symmetry revealed the structure of atoms before they could be directly observed.)

Pizlo provides an evolutionary argument for accurate spatial perception. "It is inconceivable that animals evolved successfully or were naturally selected effectively without enabling planning and purposive behavior." He contends that to be successful in tasks like farming and gathering, we need to plan and coordinate, which requires perceiving objective reality accurately. Our accomplishments depend on planning and coordination. Yet is an accurate portrayal of true reality necessary?

(Shortform note: Pizlo’s argument that accurate spatial perception is necessary for planning and coordination is part of a long tradition in vision science and hippocampal research. In their 1978 book The Hippocampus as a Cognitive Map, John O’Keefe and Lynn Nadel argued that the hippocampus creates a “cognitive map” of the environment, allowing animals to select distant goals and routes between them. This theory assumes that perception provides a metrically accurate internal representation of space, supporting flexible navigation and planning.)

Now, we will define interface elements and explain how they're constructed by evolution.

Defining Interface Elements

Hoffman explains that interface elements are constructed data frameworks we create to track fitness payoffs. Objects and spacetime serve as a symbolic framework for fitness information. The form of an object conveys potential fitness benefits and indicates ways to obtain them, while its distance represents the energy required to get there. Hues and surface details also encode vital fitness information.

(Shortform note: If you think of objects as interface elements, you can use this to your advantage by arranging your environment so that the objects that represent the highest payoffs are the most salient and easiest to access. In The Design of Everyday Things, Don Norman explains that good design makes the most important actions the easiest to discover and execute, while making less important or dangerous actions less prominent or harder to initiate.)

Interface Mechanisms and Construction

Our perception serves as a medium shaped by evolution to hide reality and guide behavior. According to the perception interface theory, our perceptions are tailored to our species, helping us raise children. Spacetime serves as the interface's desktop, with material things acting as its icons. The FBT Theorem states that the likelihood of our perceptions being accurate—that is, preserving some aspect of objective reality—is smaller than the likelihood of winning the lottery. This likelihood approaches zero as our perceptions and the world increase in complexity. The FBT Theorem explains that successful genes aren't designed to see reality accurately.

(Shortform note: In Behind the Mirror, Nobel Prize-winning ethologist Konrad Lorenz argued that our inborn forms of perception and thought are “phylogenetically a priori” hypotheses about the structure of the external world, shaped and corrected by natural selection. Lorenz’s evolutionary epistemology anticipated Hoffman’s claim that perception is an evolved medium that hides reality. He explains that our perceptual categories don’t passively mirror reality but actively organize our experience in ways that have proven suitable for the survival and reproduction of our ancestors.)

According to ITP, these encode a medium that conceals objective reality, giving us icons—material objects that have hues, textures, forms, movements, and odors—so we can interact with that hidden reality in precisely the ways necessary for survival and reproduction. The way we perceive the moon and other objects isn't meant to uncover the truth of reality; rather, it's designed to convey what is important in evolutionary terms—fitness payoffs. Physical objects effectively show key information about outcomes that affect our survival and reproduction. They're frameworks of data that we construct and dismantle. We can't adequately convey how a computer functions by discussing desktops and pixels; likewise, we can't explain the objective nature of reality using terms like the space-time continuum and material things.

The Brain as a Prediction Machine

In Surfing Uncertainty, Andy Clark describes the brain as a hierarchical prediction machine that constantly generates top-down expectations about the hidden causes of its sensory inputs. This process involves compressing the vast array of sensory signals into a manageable set of probable causes by minimizing prediction error across multiple levels of processing. The brain uses incoming sensory data primarily as error-correcting feedback to refine its ongoing guesses about the world. This approach suggests that perception is not a passive registration of stimuli but an active process of prediction, where the brain's generative models create a simplified, actionable representation of reality. This framework aligns with Hoffman's idea that our perceptions are not direct reflections of objective reality but are instead constructed icons that help us interact with the world in ways that are evolutionarily advantageous.

Adaptive Dynamics and the Pressures of Evolution

Hoffman asserts that evolutionary pressures shape perceptions to facilitate adaptive behavior rather than reveal objective reality. An illusion occurs when a perception doesn't lead to adaptive behavior. Fitness relies on environmental conditions, the organism's state, and strategy frequencies. Natural selection molds us to see opportunities for survival and how to achieve them, rather than to see the reality of the world.

(Shortform note: Some perception researchers disagree with Hoffman’s definition of an illusion as a perception that doesn’t lead to adaptive behavior. For example, in Eye and Brain, Richard L. Gregory defines an illusion as a discrepancy between the percept and the physical world it is supposed to represent. He explains that perception is a process of forming and testing hypotheses about the external world from sensory information.)

Evidence and Implications of Interface Theory

Hoffman’s interface theory suggests that the way we perceive reality is limited and not a true reflection of objective reality. Evolution shapes how we perceive the world to guide our behaviors toward greater fitness, rather than showing us reality. Our separation of life from non-life is a result of the confines of the way we interact with spacetime and doesn't provide an understanding of reality's true nature.

Enacting Reality

Hoffman’s claim that our perception of reality is limited and shaped by evolution aligns with a tradition of thought that includes phenomenology, enactivism, and embodied cognition. In Mind in Life, Evan Thompson argues that living organisms are autonomous, self-organizing systems that do not passively mirror a pre-given world but actively enact a meaningful environment through their sensorimotor activity. This perspective suggests that our separation of life from non-life is not a neutral reading of reality but a product of our organism-dependent sense-making.

Next, we will discuss the mechanisms and evidence for payoff-tuned perception, followed by the theoretical consequences of interface theory, and the notion of conscious agents.

Mechanisms and Proof for Payoff-Tuned Perception

Hoffman states that evolutionary game theory shows that natural selection prioritizes perceptions that enhance fitness, not truth. This mathematical approach models how organisms evolve strategies to maximize their adaptive success in changing environments. The FBT Theorem, based on evolutionary game theory, shows that natural selection favors perceptions enhancing fitness rather than truth. The theory has been validated by numerous simulations, which demonstrate that truth frequently disappears even when fitness is significantly simpler.

(Shortform note: While evolutionary game theory can show that natural selection favors strategies that increase fitness, it doesn’t necessarily show that fitness and truth are unrelated. In fact, many game-theoretic analyses suggest that fitness and truth are often closely related. For example, McKay and Dennett argue that natural selection generally favors true beliefs, and that false beliefs are only favored in specific circumstances.)

The FBT Theorem indicates that our perceptual language—covering dimensions like time, space, shape, hue, saturation, brightness, texture, taste, sound, smell, and motion—fails to represent the true nature of reality in the absence of observation. It's not only that any particular perception is inaccurate. All our perceptions expressed through this language have to be inaccurate. It's highly likely that the frameworks of fitness payoffs, which influence our perceptions, don't align with the frameworks of objective reality. Objective reality exists, yet it is entirely different from how we perceive things spatially and temporally.

Direct Realism

Many theorists disagree with the idea that every perception in this language must be inaccurate. For example, in Seeing Things as They Are, John R. Searle argues that in ordinary perception, we are directly aware of objects and states of affairs in the world. He explains that the intentional content of perceptual experiences is satisfied by the very objects and features of reality they are about. In other words, when we perceive something, we are directly acquainted with the object itself, not with any internal images or sense-data that stand between us and the world.

Next, we will discuss two mechanisms of perception shaped by rewards, and the evolutionary foundations of how we perceive things.

Mechanisms of Payoff-Tuned Perception

Hoffman claims that we perceive things based on evolutionary advantages rather than actual reality. Fitness payoffs are the benefits that an organism receives from its environment that help it survive and reproduce. Our perceptions serve as an interface that evolved to direct our actions and help us survive long enough to reproduce. Our sensory experiences, akin to a computer desktop, are merely performing their function: not to uncover reality, but to steer effective behaviors. According to the FBT Theorem, the more complex the senses become, the less likely they are to reveal anything true about objective reality.

The Metabolic Cost of Perception

Hoffman’s claim that we perceive things based on evolutionary advantages rather than actual reality is supported by the fact that encoding the fine-grained structure of reality is metabolically expensive. Natural selection favors organisms that economize on energy, so it makes sense that our sensory systems would evolve to encode only those aspects of the environment that provide evolutionary advantages. This perspective aligns with the idea that our perceptions are shaped by evolutionary pressures to maximize fitness payoffs rather than to provide an accurate representation of objective reality.

The Evolutionary Foundations of Our Sensory Framework

Hoffman explains that evolution molds our perceptions to facilitate behaviors that help us adapt, rather than to reveal reality. Our perceptions are not true or ideal. They serve as an adequate medium in a narrow range of forms, such as scent, flavor, hue, contour, auditory sensation, tactile sensation, and feelings. Our interface evolved to be quick, economical, and provide just sufficient details regarding fitness to let us raise children and transmit our genes. The types are random, not the authentic forms of reality. Numerous alternative perceptual styles might be equally effective, or even superior. Visualizing them tangibly is as difficult as picturing an original color. Evolution is still modifying human perceptual systems.

(Shortform note: In Vision Science, Stephen E. Palmer explains that perceptual experiences such as color, pitch, and pain are best understood as elements of an internal representational code used by the nervous system to register and discriminate patterns in the environment. The particular qualitative character of these experiences is determined by the organization of the perceptual system itself and does not mirror the physical properties they are used to represent, even though there is a reliable, lawlike correspondence between them. This supports Hoffman’s claim that “the types are random, not the authentic forms of reality.”)

Mutations giving synesthesia to four percent of people likely play a role, and some of these genetic changes could become widespread; a lot of the experimenting revolves around our color perception. Evolution goes against the unreasonable notion that our senses must be accurate. It liberally investigates countless varieties of sensory frameworks, occasionally finding new ways to steer our ongoing quest for adaptation.

(Shortform note: In 2009, Richard Cytowic and David Eagleman’s Wednesday is Indigo Blue explored the genetic basis of synesthesia and its implications for understanding perception. They argued that synesthesia, which affects about 4% of people, is a heritable trait that offers a unique window into how the brain organizes sensory information. They used synesthesia to challenge the idea that there’s a single “accurate” way to perceive the world, showing that evolution can support multiple coexisting perceptual organizations.)

Implications for Reality, Consciousness, and Knowledge

Theoretical Consequences of Interface Theory

Hoffman's Interface Theory of Perception proposes that objects and spacetime do not exist when unobserved. The spacetime an observer perceives varies from that of other observers, and certain spacetime characteristics might not always align among them. It's a falsehood that objects in the space-time continuum seem to interact causally. Spacetime acts like a unique desktop for each species, with objects as icons on it. Our senses developed to guide how we act and explore in adaptive ways. What we perceive as objects within spacetime isn't the reality of the objects themselves, nor does it represent it.

(Shortform note: In Helgoland, physicist Carlo Rovelli argues that objects and spacetime only exist in relation to observers. He claims that the physical world is nothing more than a network of relations between systems. He explains that physical properties like position, momentum, or energy only exist relative to other physical systems with which they interact. Rovelli’s view differs from Hoffman’s in that he doesn’t see conscious agents as fundamental, but rather the physical relations themselves.)

Hoffman claims that causation is a helpful fabrication. Being able to anticipate outcomes based on causal logic is an indicator of an effectively constructed interface. This is not contradicted by physics. If physicists are right that spacetime is on the verge of becoming obsolete, then its physical objects and the appearance of causality are similarly nearing obsolescence.

(Shortform note: Hoffman’s claim that physics is on the verge of discarding spacetime and causality is misleading. In The Big Picture, physicist Sean Carroll explains that our best current theories describe the universe as a collection of quantum fields living in spacetime and obeying local, orderly dynamical laws. What we call “causation” is the higher-level, yet perfectly legitimate, way we talk about the reliable, temporally directed patterns that arise from those laws.)

Toward a New Beginning: Conscious Agents and Fundamental Theory

Hoffman argues that awareness entities are key to comprehending reality. A conscious agent describes a dynamic being that observes, chooses, and takes action. It has a menu of actions and experiences, which are measurable spaces. It tracks its total number of encounters. It can merge with other conscious entities to create new ones, and these fresh entities can merge to form even higher ones.

Conscious entities have universal computational capabilities. Groups of aware entities are able to accomplish any perceptual or cognitive function, such as learning, memory, problem-solving, and recognizing objects. They present a fresh model for forming ideas in neuroscience of cognition. This model doesn't propose that networks of biological neurons are foundational to cognition. Rather, it considers consciousness foundational and then demonstrates how neurobiology, spacetime, and matter can develop as parts of the perceptual interface of specific conscious entities.

Measurable Spaces

A measurable space is a mathematical concept that helps us assign probabilities to different outcomes in a consistent way. Imagine you have a set of possible outcomes, like the results of rolling a die. A measurable space is a way of organizing these outcomes so that we can talk about the probability of different groups of outcomes happening. For example, we might want to know the probability of rolling an even number or a number greater than three. The measurable space makes sure that we can assign probabilities to these groups in a way that makes sense and follows the rules of probability. In the context of conscious agents, the measurable spaces represent the different actions and experiences that the agent can have, and the structure of these spaces allows us to model the agent's behavior and interactions with the world.