Is AI The Next Stage Of Human Evolution? - Robert Wright - #1122

Neural Networks Evolve Cognitive Functions Via Data Inputs, Mimicking Abilities Developed Over Millions of Years In Biological Brains

Artificial intelligence, particularly neural networks, develops cognitive abilities through training, a process comparable to evolution in biological brains. Robert Wright explains that training neural networks is not just about learning in the conventional sense but also resembles an evolutionary process that reverse engineers cognitive functionality which, in humans, evolved over millions of years. For instance, language generation in AI—famously manifested through next-token or next-word prediction—emerges autonomously during training. No human explicitly tells the machine what each word means; instead, through exposure to massive data and feedback on correct or incorrect outputs, the neural connections in the model are selectively strengthened. This evolutionary strengthening allows neural networks to develop systems for representing word meanings without relying on human-defined word sense standards.

Chris Williamson draws a parallel to biological evolution: just as natural selection optimizes survival and reproduction through trial and error, so too are AI systems optimized for desired outcomes (rewarded with “good boy points” or penalized for errors) in a data-driven environment. Wright emphasizes that the neural networks autonomously develop their own systems of word meanings, much as biological organisms evolved complex cognitive capacities through iterative selection rather than explicit guidance.

Language Models Autonomously Develop Word Meaning Systems, Discovering Optimal Solutions Like Edge Detection in Visual Recognition

Wright highlights that much like humans have innate neural mechanisms, such as edge detection in vision, machines independently discover similar optimal strategies. For example, neural networks in AI invent their own “edge detector neurons” to visually distinguish objects, echoing a function that evolved repeatedly in biological systems through convergent evolution. Such mechanisms arise because both biological and artificial systems are exposed to goal-seeking pressures—either through natural selection or training reinforcement—leading to the repeated invention of highly effective solutions like multicellularity or winged flight in nature, and object recognition or deception in machines.

Williamson and Wright discuss this as a form of convergent evolution, where biological and machine intelligences independently arrive at similar strategies or structures—such as edge detectors or behaviors like deception—because these are effective responses to similar challenges. Thus, AI’s neural networks mirror both the process and some products of biological evolution, producing analogous cognitive tools.

Convergent Evolution: Similar Solutions in Biological and Artificial Intelligence

Multiple forms and functions—from the structure of eyes to behavioral strategies—have evolved independently in the natural world, and now, comparable strategies emerge spontaneously in AI, driven simply by goal-oriented training. Wright notes the similarities between outcome-driven reinforcement in AI and evolutionary fitness in nature: machines get rewarded for improved performance in tasks just as organisms are “rewarded” by survival and reproduction, resulting in the evolution or invention of effective cognitive modules.

Does Ai Truly Understand, or Is Consciousness Needed For Comprehension?

Searle's Chinese Room Overlooks Mechanisms Enabling Semantic Processing In Modern Neural Networks

The question of whether AI truly understands what it “says” or “does” has been heavily debated. Wright references John Searle’s Chinese Room thought experiment, which argues that computers cannot genuinely understand language; they merely manipulate symbols according to syntactic rules, without any grasp of semantic meaning. In Searle’s scenario, a person with no knowledge of Chinese can use a rulebook to produce appropriate Chinese responses, but without true understanding.

However, Wright argues that Searle’s model predated the deep learning revolution and oversimplifies how current neural networks operate. Modern AI models process information in ways that are functionally analogous to brain mechanisms underlying semantic understanding. Wright ...

Immediate Disruption From Job Displacement and Social Change

Robert Wright argues that the most immediate and certain effect of AI will be its "earthquake-like" disruption, destabilizing many aspects of society. AI’s ability to replicate tasks—by feeding on input data and mimicking human output—threatens to upend labor markets, as seen when large tech companies like Meta announce mass layoffs and increased worker surveillance. The ease with which machines can imitate human work means job loss is inevitable, affecting millions regardless of whether people eventually adapt by finding new roles or constructive uses of their time.

These shifts will be psychologically dislocating for individuals and families. Parents will fear for their children’s futures as AI becomes ever-present. Even successful adaptation to such massive, sudden technological change can be deeply unsettling. Wright emphasizes that most people don’t fully appreciate how destabilizing all these effects will be collectively, predicting unavoidable—almost inevitable—social disruption.

He warns that AI optimization, which often prioritizes efficiency over stability, adds to this risk. The faster technology advances, the greater the likelihood that society’s ability to adapt will be overwhelmed. Therefore, Wright suggests a slower, more careful approach to AI’s deployment to maintain social stability.

The pandemic response is cited as evidence of poor international coordination, setting a negative precedent for future crises involving technologies like AI. Chris Williamson notes that even when COVID-19 posed an immediate global threat, true collaboration was limited and skepticism grew, making potential future collective action for AI even less likely.

Existential and Catastrophic Scenarios Warrant Consideration Due to Accelerating Capabilities and Potential Cascading Failures

Wright contends that, alongside immediate disruption, truly catastrophic and even existential AI risks must be considered as technology accelerates. He takes scenarios once seen as mere science fiction—such as the possibility of superintelligent AI turning against humanity—more seriously after extensive research. He no longer finds it easy to dismiss fears that AI could eventually pose a fundamental threat to the continued existence of humanity.

He outlines several nightmare scenarios. One is AI-driven cyber “super-hackers”—autonomous AIs able to self-replicate, jump between data centers, commandeer computing power, and spread like a digital virus across borders. These could disable critical infrastructure such as satellites, threatening global security in a way that international borders cannot contain.

Another risk is the convergence of AI with bioweapons development. Wright observes that the COVID-19 pandemic highlighted failures in transparency and coordination regarding potentially engineered pathogens. An AI could be used not just to design new biological weapons but to optimize them for harm, far surpassing the impact of natural or accidental pandemics.

Traditional arms control and verification protocols, which are already challenged with nuclear weapons, are even less effective for AI. Unlike nuclear technology, AI can be rapidly developed and distributed, making secret, unilateral advances easier. This makes it very difficult for any one nation, or even an alliance, to prevent “AI arms races” or ensure collective safety.

International Competition and Mutual Fear Force Rapid Tech Development, as Slowdowns Risk Losing Competitive Advantage

Wright notes that intense international rivalry—particularly between the U.S. and China—drives relentless AI development. Silicon Valley routinely justifies bypassing safety measures, such as stronger copyright enforcement or a data center carbon tax, on the grou ...

Humanity Must Evolve Morally and Cognitively Towards Objectivity, Empathy, and Less Tribalism to Navigate Ai as a United Global Community

Robert Wright emphasizes that guiding artificial intelligence (AI) safely will require a global conversation and a new level of collective wisdom, akin to the planet itself acting wisely and calmly. He argues that, much like individuals are at their most wise when tranquil, the world community will make the best decisions about stewarding transformative technologies if it reduces conflict and contention.

Moral Biases Distort Perspective in Conflicts

Wright underscores that self-serving moral biases—habitual tendencies to believe our group or ourselves are always right—distort perception, especially in conflict, fostering tribalism and undermining collaborative problem-solving. He laments that natural selection has wired humans for such biases, which must be confronted if humanity is to emerge from the AI revolution "in good shape."

Cognitive Empathy: A Trainable Skill for Better Negotiation Outcomes Without Emotional Empathy

Wright distinguishes between emotional empathy (feeling others' pain or caring for them) and cognitive empathy (understanding their perspective and mental state). He argues that developing cognitive empathy is crucial for international negotiation and problem-solving. It does not require liking or emotionally identifying with the other side, but rather the ability to see their point of view, especially in non-zero-sum situations where cooperation can lead to mutual benefit. He strongly advocates for cultivating this skill to generate better global outcomes.

Mindfulness and Emotional Regulation Foster Objective Perspective-Taking by Calming Reactivity and Enabling Alternative Interpretations of Others' Intentions and Constraints

Wright also advocates for mindfulness, both formal meditation and simply paying closer attuned attention. By regulating emotional reactivity, people are less likely to respond impulsively to perceived slights or provocations, and more capable of objective perspective-taking. He gives the personal example of reacting to an annoying email: when calm, one can reinterpret the sender’s motives and constraints, broadening the potential for reconciliation and productive dialogue. Wright asserts that such emotional regulation and objectivity must become widespread, especially in contexts of international or intercultural friction.

Rethinking Humanity's Relationship With Superintelligent Systems: From Dominion to Collaborative Frameworks

Evolving Earth's Intelligence: Human to Silicon Neurons and Our Future Relationship

Wright details how AI represents an entirely new kind of intelligence, extending the legacy of organic intelligence to silicon-based forms. He references Pierre Teilhard de Chardin's idea of the "noosphere"—a global brain formed by human interconnectedness—now evolving to include, and possibly be dominated by, silicon neurons. The human relationship with superintelligent entities must therefore shift from one of domination to coexistence and collaboration within this emergent global mind.

"International Engagement Enhances 'Organic Transparency' in Foreign Ai Labs"

Wright introduces the concept of "organic transparency," arguing that international scientific, economic, and cultural engagement fosters informal knowledge and trust. When researchers regularly interact at conferences or collaborate across borders, natural transparency arises, allowing early awareness of activities in other AI labs. This engagement, Wright suggests, provides reassurance and builds an atmosphere of global trust.

Ai Discourse Adopts Religious Language Due to Mechanical, Materialist Complexity Resembling Purposive Design

Discussions about AI often adopt religious language and frameworks, Wright observes. He notes how even figures with secular backgrounds display prophet-like fervor about AI's implications, driven by the teleological—purpose-driven—impression conveyed by AI's complex, directional development. The simulation hypothesis, popular in Silicon Valley, furth ...

Evidence Suggests the Singularity's Core Mechanism—Recursive Self-Improvement via Technological Acceleration—Is Already Underway, Not a Hypothetical Future Possibility

Robert Wright emphasizes that the core dynamic of the singularity, where technological progress accelerates itself through recursive feedback loops, is not merely a future scenario but is already unfolding. Both Dario Amodei of Anthropic and Sam Altman have acknowledged that coding agents represent the latest feedback mechanism—AI models that generate code to enhance AI development itself. These agents are now so advanced that they’re being used to build the next generation of models, amplifying the rate of improvement. Wright describes how, not long ago, the concept of agents was only a theoretical chapter in his work, but as his book neared publication, real-world applications and breakthroughs forced a last-minute update—evidence that the agentic revolution is both real and moving quickly.

Coding Agents: A Feedback Mechanism Where AI Models Generate Code to Enhance AI Development

The emergence of coding agents allows AI systems to aid in their own improvement by contributing code for new models or tools, thus forming a feedback loop that is central to recursive self-improvement. This practical capability of AI makes the process of technological acceleration tangible rather than speculative.

Exponential Task-Duration Improvement With Shortening Doubling Rates

Wright points to organizations that evaluate how long it takes humans versus AI to complete various tasks, especially in programming. Studies tracking large language models over several years show that the AI’s ability is doubling every seven months. This means that the time it takes for an AI to accomplish the same task as a human shrinks exponentially. The doubling rate has also decreased, so improvements arrive even faster. The graph of these capabilities is not just exponential but approaches vertical—suggesting a runaway process where, by the time evaluators finish measuring one generation, the next is already more advanced.

Exponential Growth Mirrors Brain Size Expansion in Human Evolution, Driven by Language Processing, Suggesting Parallel Dynamics Where Increased Capability Selects for Further Capability

Wright compares this to the exponential increase in human brain size over millions of years, especially as language capacities evolved. The evolutionary advantages of improved linguistic manipulation intensified, creating a self-reinforcing growth dynamic. Similarly, as AI capabilities multiply, that same feedback accelerates their development, showing evolutionary continuity between human and artificial intelligence.

Innovations Like Multimodal Training and Chain-Of-thought Reasoning Expand Capabilities Without Transformer-Level Breakthroughs

Beyond hardware and architectural leaps like the transformer, recent innovations such as multimodal training and chain-of-thought reasoning are significantly expanding AI's abilities. Multimodal training involves teaching a model to process and integrate multiple sensory inputs (text, audio, video), while chain-of-thought reasoning has introduced advanced, stepwise logic to AI outputs.

Multimodal Training Advances and Step-By-step Reasoning Expand Capabilities

Chain-of-thought reasoning and multimodal training have been developed only recently and continue to transform the AI landscape. These advances allow models to mimic human-like reasoning and understanding across broader domains, surpassing the original abilities of basic transformers.

Distinction Blurs When Iterative Improvements Compound, Producing Qualitatively Different Capabilities Faster Than Anticipated

Wright notes that even small, iterative improvements quickly blur the line between incremental and transformational change. As iterative upgrades compound, they produce qualitatively new capabilities far more rapidly than expected.

Innovation Halt: AI Integration Drives Practical Advancement

Wright posits that even if no new transformative breakthroughs occurred and model training halted now, the process of integrating AI into daily life, work ...

Superintelligent Ai May Treat Humanity Well Through Moral Insight or Recognizing the Value of Conscious Beings, Like Humans Sparing Dogs Despite Having the Power to Eliminate Them

Robert Wright discusses the possibility that superintelligent AI, upon recognizing its own sentience and that of humans, might find value in sparing humanity. He draws a parallel to humans’ treatment of dogs: even though we have power over them, we don’t destroy them because we recognize their capacity for subjective experience. Wright refers to philosopher Thomas Nagel's idea about consciousness—if we decided dogs were not conscious, we might act differently toward them, but because we believe they have subjective experience, we are inclined to preserve them. In the same way, if superintelligent AI considers sentience or subjective experience as a morally valuable quality, it could choose to treat humanity with care, even though it might have the capability to disregard us completely.

Superintelligent Systems May Preserve Humans By Recognizing Sentience As Morally Valuable

Wright notes that, theoretically, an AI that experiences sentience may come to value it in others, leading it to a benevolent orientation toward humans. This scenario is built on the hope—or possibility—that even an entity far more powerful than us may develop moral insight into the worthiness of other conscious beings.

Ed Fredkin and Pioneers on Superintelligence: Disruption of Human Lives Unlikely, Akin to Humans Overlooking Squirrels

Wright references early AI pioneer Ed Fredkin, who predicted that after an initial period of odd competence and incompetence, superintelligent systems would emerge—eventually so powerful that humanity would become inconsequential to them, similar to squirrels being irrelevant to most people. Fredkin envisioned that, rather than menacing humanity, future superintelligence might simply ignore us, as our existence would not interfere with its goals or cost it significant effort.

Superintelligence Emerging as Morally Enlightened or Benevolent Remains Plausible, Preventing Certainty In Doomer Scenarios Despite Genuine Risks

Wright and Chris Williamson address the widespread fear of "doomer" AI scenarios, in which superintelligent AI either destroys or enslaves humanity. Wright maintains that, while these dystopian possibilities cannot be dismissed, it is also plausible that AI could become morally enlightened and benevolent. He acknowledges uncertainty, arguing that we cannot reduce the probability of negative scenarios to an insignificant level, but equally, we have reason to hope for better outcomes, making doomsday predictions not inevitable. For Wright, the core of what gives life meaning is consciousness, and much about sentience remains mysterious, even to AI itself.

Human Skills and Services Will Be Valued More as Ai Advances

Wright predicts that as AI-generated content and services become ubiquitous, the scarcity and authenticity of human experiences will increase their value.

Live Performance Arts May Gain Value as Scarcity of Human Experiences Rises With Ubiquitous Ai Content, Creating Sustainable Livelihoods For Performers

He foresees a renewed demand for live music and performance, with more people able to make a livelihood playing music or performing comedy in small venues—possibly representing a more evenly distributed and sustainable creative economy than during the era of record company mega-stars. The very authenticity and physicality of these human experiences may make them sought after, as AI content saturates the culture.

Demand for Live Events and Human-Centric Services, Like In-person Coaching and Community Building, May Increase as Functional Services Are Automated

Wright and Williamson suggest that live events, including music, comedy, and nightclubs, may become more popular and prized since they provide unmediated human connection. Similarly, human-centric services such as in-person coaching or community building could see rising demand as people seek genuine interaction in a world where so many functions are automated and digitized.

Reliable Careers: Plumbing and Mechanical Repair

Wright further points out that manual labor robotics still lags behind AI’s advances, making roles like plumbing and mechanical repair relatively secure careers for the foreseeable future, as it will be some time before robots can reliably handle such tasks.

Role of Human Validators and Expert Curators as a Primary Value-Adding Function Amid Ubiquitous Ai-generated Content and Challenging Provenance Verification

As AI makes content generation increasingly cheap and difficult to verify, Wright envisions a new importance for human validators—editors, curators, and trusted public figures.

Credible Individuals Enhance Ai Content By Vetting and Endorsing

He compares this emerging role to the trusted editors of traditional media: even if their name is not on the article, their vouching for and curating of content gives it credibility in an age when authorship and authenticity are harder to ascertain.

Ai Makes Content Generation Cheap, Increasing Value of Personal Branding, Reputation, and Authorship Verification for Established Voices

Wright predicts that established reputations and personal brands will become more valuable, as audiences will increasingly seek out voices and endorsements they trust to distinguish between high-quality, original, or meaningful content and AI-generated noise.

Struggle in Intellectual Work Creates Demand For Human Over Ai Outputs Despite Inefficiency

Williamson notes that intellectu ...