Naveen Rao: 4D Computing, AI's Energy Wall & Beating Biology
In this episode of All-In with Chamath, Jason, Sacks & Friedberg, guest Naveen Rao addresses AI's looming energy crisis. He explains that current computing architecture, designed for speed rather than efficiency, requires exponential energy growth that could hit supply limits within three years. With energy costs now representing half the expense of serving AI tokens, and traditional hardware improvements reaching physical limits, Rao argues that fundamental changes are necessary.
Rao presents an alternative: dynamical computing inspired by biological systems. His team has developed a prototype chip that unifies computation and memory at the device level, achieving efficiency gains of roughly 1,000 times compared to conventional GPUs. The episode covers the theoretical foundations of this new architecture, practical demonstrations through both simulated and physical hardware, and plans for commercializing a rack-scale data center system within two years.
This is a preview of the Shortform summary of the Sep 21, 2026 episode of the All-In with Chamath, Jason, Sacks & Friedberg
Sign up for Shortform to access the whole episode summary along with additional materials like counterarguments and context.
1-Page Summary
The AI Energy Crisis: Critical Need For Efficiency
AI is facing an energy crisis as computing demands grow exponentially while energy efficiency improves only incrementally. Naveen Rao reports that Google currently uses 12 gigawatts of continuous power just for AI services, processing 3.2 quadrillion tokens monthly at about 10 joules per token. With global data centers consuming around 100 gigawatts total, Rao warns that at current growth rates, the world could hit practical energy supply limits for AI within three years.
Energy costs now represent about 50% of the expense of serving an AI token, making power consumption a dominant factor in data center economics. The challenge is stark: the AI market is projected to reach $1 trillion by 2030, yet efficiency improvements are linear while demand grows exponentially.
Traditional Computing Architecture Is Inefficient
Modern computers were designed for speed, not energy efficiency. Rao explains that the von Neumann architecture separates memory from computation, forcing constant data movement that consumes most system energy. Today's GPUs move nearly 30 trillion bits per second, while the human cortex moves only 16 billion bits per second—yet the brain accomplishes remarkable computational feats on just 20 watts.
With transistor miniaturization reaching its limits, further efficiency gains through hardware shrinkage are no longer viable. This necessitates a fundamental rethinking of computer design, with biological systems offering compelling examples: a squirrel's brain operates on only 8 milliwatts while performing complex navigation tasks.
Biology-Inspired Dynamical Computing: A New Architecture
Rao presents a new computing paradigm inspired by natural systems, where computation emerges from physical dynamics rather than abstract mathematical operations. He points to examples like flocking birds and synchronizing metronomes, where complex behaviors arise from simple interactions among components. This approach directly connects semiconductor physics to neural network behaviors, bypassing the "lossy abstractions" of traditional digital computing.
The architecture, branded as "4d computing," unifies compute and memory at the device level. Unlike von Neumann systems that separate these functions, each computing element in dynamical systems inherently performs both computation and storage through temporal dynamics coupled with three-dimensional die stacking. Rao also introduces sparsity—maintaining only essential connections between elements—which not only reduces complexity but can actually enhance performance and trainability.
Proof of Concept: 1,000x GPU Power Efficiency
Rao and his team developed Uno, an image generation model based on interconnected oscillators that demonstrates practical generative AI capabilities. This simulation, released as open source, proved that oscillator-based systems could perform real AI tasks.
More impressively, the team fabricated a physical prototype chip in just five months. The resulting hardware achieved groundbreaking efficiency: only 500 nanojoules per image, roughly 1,000 times more power efficient than conventional GPUs. This represents the first public demonstration of functional dynamical computing hardware, offering silicon proof that the theory works in practice. The efficiency gain comes from eliminating the constant data movement between memory and compute units that plagues traditional architectures.
Commercialization Timeline: Two-Year Strategy
Rao describes plans for a rack-scale data center system designed to run large AI models, delivering tokens through standard network interfaces while operating on fundamentally different internal architecture. The team is targeting a full product launch within two years.
The new architecture allows AI model adaptation at the model layer without changing the operations layer, though this requires significant computational effort. Instead of traditional matrix multiplication in hardware, the system reimagines operations as time-varying behaviors and state transformations.
Developing this system requires bridging physicists versed in dynamical systems theory with chip designers. To facilitate this collaboration, Rao's company has built specialized Python libraries that serve as a shared language, allowing researchers to express stochastic, time-varying elements and translate dynamical systems concepts into practical chip design.
1-Page Summary
Additional Materials
Clarifications
- In AI, a "token" is a unit of data, often a word or part of a word, that the model processes to understand and generate language. Tokens are the basic elements AI models analyze sequentially to perform tasks like translation or text generation. The number of tokens processed reflects the workload and energy consumption of AI systems. Efficient token processing is crucial for reducing computational costs and power use.
- "Joules per token" measures the amount of energy used to process a single unit of data, called a token, in AI tasks like language modeling. A token can be a word, part of a word, or a symbol that the AI processes to generate output. Lower joules per token indicate higher energy efficiency, meaning less power is needed for each token processed. This metric helps compare the energy cost of different AI systems or architectures.
- The von Neumann architecture stores data and instructions in the same memory but processes them separately in the CPU. This separation requires frequent data transfers between memory and the processor, creating a bottleneck known as the "von Neumann bottleneck." These transfers consume significant energy and time, limiting overall system efficiency. Modern workloads, especially AI, exacerbate this issue due to massive data movement demands.
- GPUs move vast amounts of data between memory and processing units to perform calculations, causing high energy use. The human cortex processes information with far less data movement by integrating memory and computation within neurons. This integration reduces energy consumption drastically compared to separate memory and compute units in GPUs. Thus, the brain achieves complex tasks efficiently despite lower data transfer rates.
- Transistor miniaturization refers to making transistors—the basic building blocks of chips—smaller to fit more on a chip and improve performance. Physical and quantum effects, like electron leakage and heat dissipation, limit how small transistors can get. As these limits are reached, shrinking transistors no longer yields significant energy efficiency or speed gains. This stagnation forces innovation beyond traditional scaling to improve hardware efficiency.
- Physical dynamics-based computation uses the natural behavior of physical systems—like oscillations or synchronization—to perform calculations, rather than relying on step-by-step arithmetic operations. It leverages continuous changes in physical states over time to represent and process information inherently. This contrasts with traditional digital computing, which manipulates discrete symbols through predefined algorithms. By embedding computation in the physics of the hardware itself, it reduces energy waste from data movement and abstraction layers.
- Flocking birds and synchronizing metronomes are examples of dynamical systems where simple units interact locally to produce coordinated, complex group behavior without central control. These natural phenomena illustrate how computation can emerge from physical interactions and timing rather than explicit step-by-step instructions. In dynamical computing, similar principles apply: computation arises from the evolving states and interactions of hardware elements over time. This contrasts with traditional digital computing, which relies on discrete, sequential operations.
- "4d computing" refers to using time as the fourth dimension alongside the three spatial dimensions in chip design, enabling computation through dynamic physical processes rather than static logic gates. By integrating memory and computation within the same physical elements, data does not need to move back and forth, drastically reducing energy use. This approach leverages temporal changes and interactions in materials to store and process information simultaneously. It contrasts with traditional chips where memory and processing are separate, causing energy-intensive data transfer.
- Temporal dynamics refer to using changes in electrical signals over time to represent and process information, enabling computation through evolving states rather than fixed binary values. 3D die stacking involves layering multiple semiconductor chips vertically to reduce the distance data must travel, improving speed and energy efficiency by integrating memory and compute units closely. Together, these techniques allow the new architecture to perform computation and storage simultaneously within the same physical elements, minimizing energy-intensive data movement. This approach mimics natural systems where information is processed dynamically and spatially compacted.
- Sparsity in computing means using fewer, selective connections between elements rather than connecting everything densely. This reduces energy use and hardware complexity by limiting unnecessary data flow. It also helps improve learning and generalization by focusing on the most relevant interactions. Sparse networks mimic biological brains, which are highly efficient due to their selective connectivity.
- Interconnected oscillators perform AI tasks by using their synchronized timing patterns to represent and process information dynamically. Each oscillator's phase and amplitude interact with others, creating complex collective behaviors that encode data and computations. This temporal coordination replaces traditional digital operations with continuous, time-based transformations. Such systems naturally implement neural network functions through physical dynamics rather than discrete calculations.
- Achieving 500 nanojoules per image means the new chip uses an extremely small amount of energy to generate each image, drastically reducing power consumption. Traditional GPUs typically consume microjoules to millijoules per image, making the new chip about 1,000 times more efficient. This efficiency gain is crucial for scaling AI without unsustainable energy costs. Lower energy use also reduces heat output and cooling needs, improving overall data center sustainability.
- In traditional computers, memory and processing units are separate, so data must constantly travel between them, causing delays and high energy use. This "data movement" consumes more power than the actual computation because transferring bits over physical distances requires significant electrical energy. Eliminating this movement means integrating memory and computation in the same physical location, reducing energy spent on data transfer. Achieving this requires new hardware designs that perform calculations where data is stored, fundamentally changing chip architecture.
- Traditional matrix multiplication involves fixed, discrete calculations on static numerical arrays, producing precise outputs through repeated arithmetic operations. Reimagining operations as time-varying behaviors treats computation as continuous, dynamic processes evolving over time, where information is encoded in changing states rather than fixed values. State transformations refer to how these dynamic systems move between different configurations, using physical or temporal changes to perform computation. This approach leverages natural system dynamics to compute, reducing reliance on rigid, step-by-step arithmetic.
- Physicists understand the principles of dynamical systems and how physical processes can perform computation through time-varying behaviors. Chip designers specialize in creating hardware that implements these principles efficiently at the silicon level. Collaboration is necessary to translate complex physical models into practical, manufacturable chip architectures. This interdisciplinary work ensures the new computing paradigm can be realized as functional hardware.
- Specialized Python libraries enable researchers to model and simulate complex, unpredictable behaviors in dynamical systems using code. They provide tools to represent randomness (stochasticity) and changes over time, which are essential for designing chips that operate on physical dynamics rather than fixed logic. These libraries translate abstract mathematical concepts into practical instructions for hardware engineers. This shared language bridges the gap between theoretical physics and chip fabrication, accelerating development.
Counterarguments
- The claim that AI energy demand will hit practical supply limits within three years is speculative and does not account for potential advances in renewable energy, grid expansion, or improvements in data center energy management.
- While von Neumann architecture has inefficiencies, it remains highly flexible and general-purpose, supporting a vast ecosystem of software and hardware that new paradigms may struggle to match in compatibility and scalability.
- The comparison between biological brains and silicon-based computers overlooks fundamental differences in materials, operational environments, and design constraints, making direct efficiency comparisons potentially misleading.
- The reported 1,000x efficiency gain for the prototype chip is based on a specific image generation task and may not generalize to broader AI workloads or real-world deployment scenarios.
- Transitioning to radically new computing architectures involves significant challenges in software compatibility, developer retraining, and ecosystem support, which could slow adoption regardless of theoretical efficiency gains.
- The energy consumption figures for AI services may not account for ongoing improvements in model optimization, quantization, and algorithmic efficiency that can reduce power requirements without hardware changes.
- The focus on hardware innovation may understate the potential of software-level improvements, such as more efficient algorithms, model pruning, or distributed computing strategies, to address the AI energy crisis.
- The timeline for commercialization of new architectures is ambitious, and unforeseen technical or market challenges could delay or limit widespread adoption.
Get access to the context and additional materials
The Ai Energy Crisis: Critical Need For Efficiency as Data Centers' Massive Power Usage Hits Limits (12 Gw For Google)
AI is on the cusp of an energy crisis as ballooning computing demands far outpace improvements in energy efficiency. With data centers already consuming a sizable fraction of global electricity, the rapid ascent of models like those powering Google and OpenAI highlights urgent engineering and ecological questions.
Computing Demands Grow While Energy Resources Remain Constrained
Naveen Rao reports that Google currently processes 3.2 quadrillion AI tokens per month, using roughly 10 joules per token—an efficient estimate for modern models. This alone translates to a staggering 12 gigawatts of continuous power usage devoted just to Google’s AI services. Globally, data centers collectively consume around 100 gigawatts, with the United States accounting for about 40 gigawatts, representing roughly half the worldwide total. Rao warns that if AI model sizes and usage continue to grow at their current rate, the world could reach practical energy supply ceilings for AI computation within just three years.
The financial impact of this power usage is equally massive. Rao notes that about 50% of the cost of serving an AI token stems from energy consumption—the remainder going to hardware and facility expenses. Power contracts and the need to monetize every watt have become central to data center economics, as energy costs are now a dominant part of maintaining scalable AI services.
The scale of the AI market adds urgency to the efficiency debate. Projections estimate a $1 trillion market for AI by 2030 or sooner, yet energy efficiency in data centers is improving only incrementally—linearly—while computational demand and market size expand exponentially. This creates a growing disparity, mandating substantial innovation if AI growth is to continue sustainably.
Traditional Computing Prioritizes Speed Over Power Optimization, Which Is Inefficient
Modern digital computers were built for speed, not power efficiency. Almost all mainstream systems, from personal laptops to data center GPUs, use the von Neumann architecture, which physically separates memory (storage) from computation (processing). With this design, the movement of information—shuttling bits back and forth between components—consumes the majority of system energy. Rao explains that today’s GPUs can move nearly 30 trillion bits per second between chips and memory; by contrast, the human cortex moves only about 16 billion bits per second.
Historically, as hardware miniaturization progressed in accordance with Moore’s Law, efficiency increased as transistors shrank. However, Rao notes that we’ve reached the limits for miniaturization. Frequency and single-threaded performance have plateaued, and further energy efficiency through smaller transistors is no longer viable. To meet the ever-increasing energy needs, a fundamental rethinking of computer design is now nece ...
Here’s what you’ll find in our full summary
The Ai Energy Crisis: Critical Need For Efficiency as Data Centers' Massive Power Usage Hits Limits (12 Gw For Google)
Additional Materials
Clarifications
- An AI token is a unit of text, like a word or part of a word, that AI models process to understand or generate language. Processing 3.2 quadrillion tokens means handling an enormous volume of text data, reflecting the scale of AI usage. This metric helps quantify the computational workload and energy needed for AI services. It highlights the vast demand on data centers powering AI applications.
- An "AI token" is a unit of text that AI models process, such as a word or part of a word. Tokens are the basic elements that language models analyze to understand and generate language. The size and definition of a token vary depending on the model's design and tokenization method. Counting tokens helps measure the computational workload of AI systems.
- "10 joules per token" means the amount of electrical energy used to process one unit of AI input or output, called a token. A token can be a word, part of a word, or symbol that AI models analyze or generate. Energy per token is measured by tracking the total electricity consumed during AI computation and dividing it by the number of tokens processed. This metric helps quantify and compare the efficiency of AI systems.
- Twelve gigawatts is roughly the output of a dozen large nuclear power plants running continuously. This level of power consumption by a single company’s AI services is unprecedented and strains existing electrical grids. It implies significant environmental impact due to the energy sources used, often fossil fuels. Managing such demand requires massive infrastructure and drives up operational costs.
- Data centers are specialized facilities housing large numbers of servers designed for processing, storing, and managing vast amounts of data. Unlike personal computers or small office servers, data centers provide centralized, scalable computing power and storage for multiple users or services simultaneously. They include infrastructure for power, cooling, and network connectivity to support continuous, high-demand operations. Other computing infrastructures, like edge devices or personal computers, are smaller, less centralized, and typically serve individual or localized tasks.
- The von Neumann architecture is a computer design where the CPU (processor) and memory are separate units connected by a data bus. This separation means data must be constantly transferred back and forth, causing delays and high energy use. Each transfer consumes power because moving data physically requires electrical signals over wires. Modern processors spend significant energy just moving data rather than computing.
- GPUs move data between memory and processors to perform calculations, which requires transferring large volumes of bits rapidly. The "30 trillion bits per second" refers to this high-speed data movement in artificial hardware. In contrast, the human cortex transmits information through neural signals, which are slower but more energy-efficient and specialized. This highlights a fundamental difference: artificial systems prioritize raw data throughput, while biological systems optimize for energy-efficient, complex signal processing.
- Moore’s Law is the observation that the number of transistors on a microchip doubles approximately every two years, leading to exponential growth in computing power. This trend has driven rapid improvements in speed, cost, and energy efficiency for decades. However, physical and material limits, such as quantum effects and heat dissipation, have made further shrinking of transistors increasingly difficult. As a result, transistor miniaturization has slowed, limiting traditional gains in performance and efficiency.
- "Frequency" refers to the speed at which a processor's core executes instructions, measured in hertz (cycles per second). "Single-threaded performance" means how fast one core can complete tasks sequentially, without parallel processing. Limits arise because increasing clock speeds causes excessive heat and power use, and physical constraints prevent further miniaturization of transistors. As a result, improvements in these areas have slowed, pushing the industry toward multi-core and parallel computing instead.
- Floating-point arithmetic is a method computers use to represent and calculate real numbers approximately, enabling complex mathematical operations. Linear algebra involves manipulating vectors and matrices, fundamental for many AI algorithms like neural networks. Physics-based neural computation refers to how biological brains process information through physical interactions of neurons and synapses, not explicit numerical calculations. This approach leverages the brain's structure and dynamics for efficient, parallel processing without relying on traditional arithmetic operations.
- Biological neural circuits process information through networks of neurons that communicate via electrical and chemical signals, enabling parallel and adaptive computation. Unlike artificial systems that use discrete binary operations and separate memory from processing, biological neurons integrate signals continuously and store information within their connections (synapses). This integration allows for energy-efficient, fault-tolerant, and context-sensitive processing. The brain’s architecture supports learning and plasticity, dynamically rewiring ...
Counterarguments
- The estimate that Google alone uses 12 gigawatts continuously for AI services may be an overstatement or based on assumptions that do not reflect actual operational data, as Google does not publicly disclose such specific figures.
- While AI workloads are growing, not all data center energy consumption is attributable to AI; traditional IT services, storage, and networking still account for a significant share.
- Projections about reaching global energy supply limits for AI computation within three years are speculative and do not account for potential breakthroughs in hardware, software optimization, or shifts in energy sourcing.
- The comparison between biological brains and artificial systems overlooks the fact that brains and computers solve fundamentally different types of problems and operate under different constraints and requirements.
- The assertion that energy efficiency improvements are only linear may not account for recent advances in specialized AI hardware (such as TPUs, FPGAs, and neuromorphic chips) that offer significant efficiency gains.
- Data center operators are increasingly investing in renewable energy and advanced cooling technologies, which can mitigate the environmental impact of increased energy use.
- The focus on von Neumann architecture ignores ongoing res ...
Get access to the context and additional materials
Biology-Inspired Dynamical Computing: A New Architecture (4d Computing) Using Dynamical Systems and Neural Principles, Integrating Compute and Memory to Replace Von Neumann Architecture
Naveen Rao presents a new computing architecture inspired by principles observed in natural systems and the brain, aiming to overcome the inefficiencies of traditional computing by leveraging emergent behaviors, physical dynamics, and novel circuit design.
Computation Through Emergent Behavior in Dynamical Systems
Rao explains that computation is ubiquitous in nature. For instance, birds in a flock exhibit complex, coordinated movements through simple individual rules such as observing neighbors, resulting in emergent flocking behaviors. Similarly, ant colonies manifest intelligence from simple, local agent rules. These examples illustrate how complex, system-level behaviors emerge from basic interactions among components.
A practical demonstration of this principle is the synchronization of metronomes. When multiple metronomes are placed on a rigid, movable platform, their interactions with the platform cause them to synchronize in phase over time, regardless of their starting points. Rao notes that as scale increases—potentially hundreds of metronomes—the collective dynamics still drive the system toward distinctive, synchronized patterns or even subgroup synchrony with opposing phases. The intrinsic connections and interactions among units—rather than top-down command—produce computation.
Rao likens this to the brain, which is composed of neurons whose physical operations, rather than abstract mathematical processes such as linear algebra or floating-point math, give rise to intelligence. This observation motivates efforts to build circuits mimicking such dynamical systems using semiconductors.
4d Computing Ends Lossy Abstractions in Architecture
Rao criticizes the layers of abstraction in conventional digital computers: bits representing 1s and 0s are themselves simplifications of more complex physical transistor states. These abstractions enable building up systems but are "lossy"—each step overlooks and discards underlying complexity and, with it, efficiency.
Current neural networks and learning machines sit atop these legacy abstractions, inheriting fundamental inefficiencies. Rao’s approach is to simplify the architectural stack by connecting the physics of semiconductor devices directly to neural network behaviors, bypassing unnecessary abstraction layers. This direct linkage, branded as “4d computing,” uses both the three spatial dimensions of advanced die stacking and the temporal evolution of system states. By harnessing time and space synergistically, the new architecture recaptures the efficiency lost to binary and mathematical compliance.
4d Architecture Unifies Compute and Memory Systems
Traditional computer architectures, including those with CPUs and more parallelized GPUs, adhere to the von Neumann paradigm, separating compute from memory and shuttling information between the two. Even as architectures progress toward finer-grained parallelism, this division remains.
Rao describes an alternative: in dynamical computers, each computing element inherently functions as both computation and storage. Temporal dynamics—how signals evolve over time—are coupled with die stacking in three dimensions to create integrated compute-memory elements. There is no distinct memory interfa ...
Here’s what you’ll find in our full summary
Biology-Inspired Dynamical Computing: A New Architecture (4d Computing) Using Dynamical Systems and Neural Principles, Integrating Compute and Memory to Replace Von Neumann Architecture
Additional Materials
Clarifications
- A dynamical system is a mathematical model describing how a system's state changes over time based on specific rules. In computation, these evolving states can represent information processing without discrete steps or explicit instructions. Such systems leverage continuous interactions and feedback loops to perform complex tasks naturally. This contrasts with traditional digital computing, which relies on fixed, stepwise operations.
- Emergent behavior refers to complex patterns or functions arising from simple interactions among system components without central control. In nature, this means individual agents follow basic rules, yet collectively produce sophisticated outcomes. In computation, it implies that system-wide intelligence or functionality emerges from local, low-level operations. This concept contrasts with top-down design, emphasizing bottom-up self-organization.
- Metronome synchronization illustrates how individual units with simple dynamics can spontaneously coordinate through shared physical interactions, without central control. This emergent synchronization exemplifies how computation can arise from the collective behavior of coupled dynamical systems. It shows that timing and phase relationships encode information and processing, unlike traditional digital logic. This physical phenomenon provides a tangible model for designing computing architectures that exploit natural dynamics rather than abstract algorithms.
- Neurons operate through electrochemical signals and dynamic interactions within networks, not by performing explicit calculations. Their "computation" emerges from physical processes like ion flow, membrane potential changes, and synaptic plasticity. In contrast, abstract mathematical computations use symbolic manipulation and precise arithmetic operations defined by algorithms. Thus, neural processing is inherently analog, distributed, and time-dependent, unlike discrete, step-by-step digital math.
- Conventional digital computing uses multiple abstraction layers to simplify hardware design, such as representing complex transistor states as binary bits. Each abstraction discards detailed physical behaviors, reducing the system's ability to exploit underlying device properties. This simplification leads to inefficiencies because the hardware cannot fully leverage its physical dynamics for computation. Thus, these abstractions are "lossy" as they lose potentially useful information and capabilities inherent in the hardware.
- "4d computing" refers to leveraging three-dimensional chip structures combined with the dimension of time to perform computation. Unlike traditional flat chips, stacking layers vertically allows more components to interact closely, reducing delays and energy use. Time is used as an active computational resource, where the system's state evolves dynamically rather than relying on static binary values. This approach enables more efficient, brain-like processing by integrating memory and computation within the same physical elements over time.
- Advanced die stacking involves layering multiple semiconductor chips vertically within a single package to increase component density. This reduces the physical distance between components, improving speed and energy efficiency by minimizing signal delay and power loss. It also enables more complex, integrated systems by combining different types of chips, such as logic and memory, in close proximity. This vertical integration is crucial for architectures like 4d computing that rely on tight coupling of compute and memory functions.
- The von Neumann architecture is a computer design model where the processor (compute) and memory are separate units connected by a data bus. This separation causes a bottleneck known as the "von Neumann bottleneck," limiting data transfer speed between compute and memory. As a result, processors often wait idly for data, reducing overall efficiency. Modern systems try to mitigate this but cannot fully eliminate the fundamental delay caused by this separation.
- Traditional computers separate memory (where data is stored) and computation (where data is processed), causing delays and energy loss when moving data between them. Integrating computation and memory means each device can both store and process information simultaneously, eliminating this data transfer bottleneck. This is achieved by using physical properties of materials and circuits that naturally hold and manipulate information over time. Such integration enables faster, more efficient computing by closely coupling data storage with processing at the hardware level.
- In a fully connected system, each element must link to every other element, causing the number of connections to grow quadratically as elements increase. This rapid growth demands exponentially more wiring, energy, and processing r ...
Counterarguments
- While emergent behaviors in nature are powerful, translating these principles into reliable, programmable hardware systems remains a significant engineering challenge, as natural systems often tolerate unpredictability and error in ways that digital computers cannot.
- The analogy between biological neural systems and semiconductor circuits may overlook critical differences in materials, energy efficiency, and adaptability; biological systems benefit from evolutionary processes and self-repair mechanisms not present in current hardware.
- The claim that abstraction layers in digital computing are inherently "lossy" and inefficient does not account for the benefits these abstractions provide, such as modularity, error correction, and ease of programming.
- Directly linking physical device dynamics to computation may complicate software development, debugging, and verification, as higher abstraction layers often facilitate these processes.
- The integration of compute and memory at the device level, while potentially efficient, could introduce new bottlenecks or limitations in flexibility, as specialized memory and compute units allow for independent optimization and scaling.
- Sparse connectivity, while reducing complexity, may limit the types of computations or algorithms that can be efficiently implemented, especially those ...
Get access to the context and additional materials
Proof of Concept: Uno Model and Chip Prototype Showing 1,000x GPU Power Efficiency (500 Nanojoules/Image), Proving Theory
Uno: Oscillator Systems Capable of Generative AI Tasks Like Image Generation
Naveen Rao and his team explored whether an interconnected oscillator system, similar to a metronome setup, could perform generative AI tasks. This exploration led to the creation of Uno, an image generation model based on a network of oscillators arranged as a dynamical system. Uno demonstrates that non-traditional computing architectures—distinct from digital logic and memory—can generate recognizable images and extend to tasks like sequence modeling and language models.
The system was scaled up, trained, and able to produce actual images, marking the first demonstration that such a system could be used for practical generative AI tasks. Rao explains that the model’s operation can be analyzed using state-space trajectories, mapping how the phases of all oscillators evolve over time as the system generates conditioned outputs, like an airplane, car, or bird. This analysis helps reveal the underlying system dynamics and computational paths.
Uno and its simulation were released as open source, enabling others to verify and experiment with the oscillator-based approach to computation.
Prototype Chip Achieves Transformative Energy Efficiency, Verified by Silicon Results
Building on the Uno model, the team designed and fabricated the first physical dynamical computer ever built in only five months—having started in earnest in January, taped out the design on June 1, and received the silicon chip for lab testing soon after. The resulting prototype achieved a groundbreaking result: processing an image took only about 500 nanojoules, a figure many orders of magnitude lower than conventional GPUs, which typically consume millijoules per image. This means the prototype is about 1,000 times more power efficient than current GPUs.
Rao emphasizes that this prototype represents the first public demonstration of a functional hardware implementation of dynamical computing, offering direct silicon proof of the core theory. This accomplishment demonstrates that the approach isn't just theoretical—it works in practice and opens avenues for highly efficient computation far closer to biological limits.
Architecture Minimizes Information Movement For Efficiency
A ke ...
Here’s what you’ll find in our full summary
Proof of Concept: Uno Model and Chip Prototype Showing 1,000x GPU Power Efficiency (500 Nanojoules/Image), Proving Theory
Additional Materials
Clarifications
- An oscillator system consists of components that produce repetitive signals or cycles, like a pendulum swinging or a metronome ticking. In computation, these oscillators can represent information through their phase (timing) and amplitude (strength) of oscillations. By coupling many oscillators, their interactions create complex patterns that can encode and process data dynamically. This approach leverages continuous-time dynamics rather than discrete digital states to perform calculations.
- A dynamical system is a mathematical model describing how a system's state evolves over time based on specific rules. In computing, it means using continuous changes in system variables (like oscillator phases) to perform calculations instead of discrete digital steps. This approach leverages natural physical processes to encode and process information dynamically. It contrasts with traditional computing, which relies on fixed logic gates and memory operations.
- State-space trajectories represent the path traced by a system’s variables over time in a multidimensional space. For oscillators, each dimension corresponds to the phase of one oscillator, capturing how all phases change together. This visualization helps understand the system’s dynamic behavior and how it evolves to produce outputs. Tracking these trajectories reveals patterns and stability in the oscillator network’s computations.
- In oscillator systems, the "phase" refers to the position within a cycle of oscillation at a given time. The evolution of these phases over time encodes information through their relative timing and synchronization patterns. This dynamic interaction allows the system to perform computations by representing data and processing it via changes in phase relationships. Tracking phase trajectories reveals how the system transitions between different computational states to generate outputs.
- "Taped out the design" refers to the final step in chip design where the complete layout is sent to the semiconductor foundry for manufacturing. It involves creating a detailed blueprint of the chip's circuits and components in a format suitable for photolithography. This process ensures the design is ready for fabrication on silicon wafers. The term originates from earlier practices of storing design data on magnetic tape.
- A joule is a unit of energy, with a millijoule (mJ) equal to one-thousandth of a joule and a nanojoule (nJ) equal to one-billionth of a joule. This means 1 millijoule is 1,000,000 times larger than 1 nanojoule. Lower energy consumption per image means less power use, which is critical for efficiency and sustainability in computing. Achieving nanojoule-level energy use represents a massive reduction in power compared to traditional GPUs, enabling longer battery life and less heat generation.
- Traditional computers separate memory (where data is stored) and processors (where data is computed). Data must be transferred back and forth over buses or interconnects, which consumes significant electrical energy. This constant movement causes delays and power loss due to charging and discharging of wires and switching circuits. Reducing data movement is crucial for improving energy efficiency in computing systems.
- Traditional computers separate memory (where data is stored) and computation units (where data is processed), requiring energy-intensive data transfers. In the dynamical computer, each computing element simultaneously holds information and performs calculations through its physical state, like an oscillator’s phase. This co-location eliminates the need for constant data movement, drastically reducing energy use. Such integration mimics how neurons in the brain both store and process information locally.
- "Intelligence per watt" measures how much useful computational work, such as AI tasks, a system can perform for each unit of energy consumed. It reflects the efficiency of converting electrical power into effective problem-solving or decision-making ability. This metric is crucial for comparing different computing architectures, especially in AI, wh ...
Counterarguments
- While the Uno prototype demonstrates impressive energy efficiency for image generation, it is not yet clear how well the architecture scales to more complex or larger-scale generative AI tasks compared to established digital systems.
- The current results focus on energy per image, but do not address other important metrics such as throughput, latency, model accuracy, or flexibility across diverse AI workloads.
- The practical integration of oscillator-based dynamical computers into existing AI pipelines and software ecosystems may present significant challenges, potentially limiting near-term adoption.
- The open-source release of Uno and its simulation enables verification, but independent replication and benchmarking against state-of-the-art digital models are necessary to validate claims of practical utility and efficiency.
- While the architecture minimizes information movement, it may introduce new challenges in programmability, debugging, and general-purpose applicability compared to traditional digital computers.
- The compar ...
Get access to the context and additional materials
Commercialization Timeline: Two-year Strategy For Data Center Rack System and Ai Model Adaptation via Model-Layer Porting
Rack-Scale Data Center System For Delivering Tokens Via Standard Network Interfaces
Naveen Rao describes the development of a new data center product built as a complete rack-scale system. This product is designed to run large AI models, offering tokens in and tokens out through a standard network cable. Internally, the system's architecture and operation differ fundamentally from traditional computer infrastructure, particularly GPU-based data center designs. The approach presents a complete alternative to existing solutions, enabling token-based network interface interaction as the main data exchange paradigm.
The team is targeting a full product launch within a two-year timeline, focusing on scaling, integration, and achieving the reliability required for production readiness.
New Architecture Enables Ai Model-Layer Porting Without Operations-Layer Changes
The architecture allows adaptation of AI models at the model layer, eliminating the need to port models at the operations layer. This provides compatibility for existing models, but Rao notes a significant computational effort is involved in making the transition. The process requires the algorithmic redesign of the model layer to fit the new substrate.
Instead of carrying out basic elements like traditional matrix multiplication (“matmole”) in hardware as usual, the new system reimagines matrix multiplication as a series of time-varying behaviors. Each time step translates to multiplying the current state by a transformation matrix—reflecting a shift from static operations to dynamic state transitions. This approach uses state transformation matrices to characterize operations, and the development team must balance the adaptation effort with the performance improvements to justify porting models to the new system.
Bridging Physicists and Chip Engineers Required New Tools and Frameworks
Develo ...
Here’s what you’ll find in our full summary
Commercialization Timeline: Two-year Strategy For Data Center Rack System and Ai Model Adaptation via Model-Layer Porting
Additional Materials
Clarifications
- Token-based network interface interaction means data is exchanged as discrete units called tokens, rather than continuous streams or bulk data blocks. Each token represents a small piece of information or computation step, enabling fine-grained control and synchronization over the network. This contrasts with traditional methods that typically transfer large data packets or memory blocks without such granular semantic meaning. The approach can improve efficiency and scalability in AI model processing by aligning communication closely with model operations.
- Model-layer porting means adapting the high-level structure and parameters of an AI model to a new hardware system without changing the low-level computational steps it performs. Operations-layer changes involve modifying the fundamental mathematical operations or algorithms the model uses, such as how matrix multiplications are executed. By focusing on model-layer porting, existing AI models can run on new hardware architectures with minimal changes to their core computations. This reduces redevelopment effort compared to rewriting the underlying operations for the new system.
- Traditional matrix multiplication treats data as static inputs processed in a single step. Reimagining it as time-varying state transformations means representing computations as sequences of evolving states over discrete time steps. Each step applies a transformation matrix to update the system's state, capturing dynamic behavior rather than fixed operations. This approach aligns with dynamical systems theory, enabling new hardware designs that process information through continuous state evolution.
- State transformation matrices represent how the system's internal state changes over discrete time steps. They encode dynamic behavior by mapping the current state to the next state through matrix multiplication. This approach models computations as evolving processes rather than fixed operations. It enables the hardware to implement time-dependent transformations aligned with dynamical systems theory.
- Traditional GPU-based data centers rely on individual GPUs connected via high-speed interconnects, focusing on parallel processing of fixed operations like matrix multiplication. The proposed rack-scale system integrates compute, memory, and networking tightly within a single rack, optimizing for token-based data exchange over standard network interfaces. This design shifts from static hardware operations to dynamic, time-varying state transformations, enabling new computational paradigms. It aims to improve scalability and efficiency by rethinking how AI workloads are processed at the system level.
- Adapting AI models to a new computational substrate requires redesigning algorithms to fit different hardware behaviors and constraints. Traditional operations like matrix multiplication must be reinterpreted to match the new system’s dynamic state transitions. This often involves significant computational effort to ensure models maintain accuracy and efficiency. Additionally, compatibility challenges arise because existing software frameworks are typically optimized for conventional hardware architectures.
- Dynamical systems theorists focus on mathematical models describing how systems evolve over time, often using abstract concepts and continuous mathematics. Chip designers prioritize practical constraints like hardware efficiency, manufacturability, and digital logic implementation. Their differing languages, goals, and problem-solving approaches create communication barriers. Bridging this gap requires tools that translate theoretical models into hardware-compatible designs.
- The specialized Python libraries are designed to model and simulate stochastic, time-varying beha ...
Counterarguments
- While the architecture enables model adaptation at the model layer, the significant computational effort and algorithmic redesign required may deter widespread adoption, especially for organizations with limited resources or expertise.
- The claim of maintaining compatibility with existing models is limited by the need for substantial adaptation, which could introduce errors or degrade model performance during the porting process.
- The focus on token-based network interface interaction as the primary data exchange paradigm may not align with all existing data center workflows or application requirements, potentially limiting the system's applicability.
- The two-year timeline for full product launch may be optimistic given the complexity of bridging theoretical and engineering disciplines, as well as the need for robust scaling and integration.
- The specialized Python libraries, while facilitating collaboration, may introduce a learning curve for engineers accustomed to established framewo ...
Get access to the context and additional materials
1-Page Summary- The Ai Energy Crisis: Critical Need For Efficiency as Data Centers' Massive Power Usage Hits Limits (12 Gw For Google)
- Biology-Inspired Dynamical Computing: A New Architecture (4d Computing) Using Dynamical Systems and Neural Principles, Integrating Compute and Memory to Replace Von Neumann Architecture
- Proof of Concept: Uno Model and Chip Prototype Showing 1,000x GPU Power Efficiency (500 Nanojoules/Image), Proving Theory
- Commercialization Timeline: Two-year Strategy For Data Center Rack System and Ai Model Adaptation via Model-Layer Porting
Create Summaries for anything on the web
Download the Shortform Chrome extension for your browser
