In this episode of All-In with Chamath, Jason, Sacks & Friedberg, Databricks CEO Ali Ghodsi discusses why most enterprises struggle to move beyond basic AI applications despite the technology's potential. He identifies key barriers preventing meaningful adoption—including lack of organizational context, security concerns, rising costs, and data sovereignty issues—and explains how Databricks' integrated platform addresses these interconnected challenges.
Ghodsi also weighs in on the superintelligence debate, arguing that current AI systems already meet older definitions of artificial general intelligence but lack evidence of the recursive self-improvement needed for superintelligence. The conversation covers practical cybersecurity risks, AI's impact on organizational decision-making and structure, and Databricks' growth strategy. Ghodsi explains the company's decision to remain private and predicts that enterprise AI adoption will unfold gradually over the next 10-15 years rather than through sudden disruption.

Sign up for Shortform to access the whole episode summary along with additional materials like counterarguments and context.
Enterprises are exploring artificial intelligence adoption but face significant barriers that prevent transformative use. Ali Ghodsi of Databricks explains that most organizations use AI primarily as improved search tools or coding assistants that deliver only 20-30% speed increases—not the autonomous automation that could reshape operations.
The fundamental problem, according to Ghodsi, is that AI lacks context within organizations. Meeting transcripts, decisions, and communications remain out of reach due to privacy and liability concerns, leaving AI systems uninformed about actual business activities. Security fears compound this issue, as executives worry that broader AI access will expose companies to cyber threats and insider attacks. Additionally, rising token costs outpace revenue growth, pressuring CFOs to justify AI investments. Finally, using external AI vendors creates risks that proprietary data could leak to competitors.
Databricks addresses these interconnected challenges through products like Genie, which constructs a dynamic knowledge graph capturing organizational structure, communications, meetings, and documents. The platform integrates strict access controls and data repositories to enable secure AI deployment while restricting information flows. Cost optimization through model routing and open-source solutions reduces token expenses, and the platform ensures data sovereignty by keeping all information within enterprise infrastructure.
Ghodsi emphasizes that effective AI deployment requires solving all these problems simultaneously—piecemeal solutions won't unlock transformative value. Organizations must capture internal context, balance agent autonomy with security, ensure revenue growth matches computational costs, and avoid vendor lock-in by controlling their own data and models.
Ali Ghodsi and Jason Calacanis explore the current state of AI capabilities and whether concerns about superintelligence are warranted. Ghodsi argues that present-day AI systems already meet decade-old academic definitions of Artificial General Intelligence, referencing UC Berkeley's AMP Lab benchmarks from 2009-2010. He contends that even if progress stopped now, today's intelligence could transform global economics if implemented thoughtfully.
Ghodsi defines superintelligence following Bostrom's 2014 work as a system capable of millennia of human thought in minutes. However, he argues that none of the essential preconditions for recursive self-improvement are met today: training remains slow and resource-intensive, exponentially increasing hardware is required for marginal improvements, accuracy gains are limited and domain-specific, and no feedback flywheel exists where improvements loop recursively. The apparent absence of such a flywheel is strong evidence against imminent superintelligence.
Ghodsi identifies cyberattacks as the principal near-term risk posed by AI systems. Advanced AI can find vulnerabilities in legacy code much faster than humans, but the solution is engineering: scan and secure new AI-generated code and incrementally patch legacy systems. While concrete risks like ransomware are real dangers, they're addressable and far less sensational than speculative superintelligence scenarios. Much of the existential risk narrative stems from theoretical frameworks rather than real evidence, and Ghodsi urges focus on current real-world risks like misinformation and inappropriate use.
Artificial Intelligence is fundamentally changing how organizations make decisions and structure themselves. Ghodsi describes how executives now query AI directly for performance metrics and meeting summaries without needing human intermediaries. AI-powered dashboards backed by governed data repositories allow leaders to test management theories in real time, examining correlations and causations in employee engagement and performance.
Traditional organizational hierarchies create latency and overhead, but AI's deep knowledge of org charts and relationships allows it to coordinate quickly across departmental silos. By continually learning an organization's structure, AI can identify inefficiencies and suggest realignments. The technology also resolves cultural and operational misalignments at scale by analyzing subgroup deviations from company culture across global offices and languages.
While AI automates context-gathering and accelerates information processes, Ghodsi emphasizes that critical decisions—strategy, budgeting, legal, or marketing approvals—remain firmly in human hands. AI prepares and organizes, but final authority stays with human managers.
Ali Ghodsi emphasizes Databricks' decision to remain private during volatile market transitions, particularly given rapid AI advancements. Public companies face earnings cycles and stock volatility that pressure executives and affect employee morale independently of company fundamentals. To counter these risks while providing liquidity, Databricks conducts annual private tender offers, allowing employees predictable capital access without quarterly earnings pressure.
Founded in 2013 at UC Berkeley, Databricks grew from $1.5 million in revenue in 2015 to a projected $7 billion run rate by 2026, maintaining 70-80% annual growth. This explosive growth resulted from contrarian bets on cloud computing, AI and machine learning, and open-source technology—strategies that were once considered risky but became industry standards. The company is now expanding into security, marketing, and governance markets to address key barriers to enterprise AI adoption.
Ghodsi predicts that broader market AI adoption will happen gradually over 10 to 15 years rather than through abrupt disruption. While digital native companies can rapidly implement AI strategies, Fortune 500 and legacy enterprises face significant friction from regulatory demands, complex security environments, and the challenge of coordinating large workforces. Databricks' strategy specifically targets overcoming these enterprise barriers, acknowledging that genuine economic transformation will unfold over years as organizations slowly adapt and modernize.
1-Page Summary
Enterprises are exploring artificial intelligence (AI) adoption but face numerous challenges that limit its transformative potential. Ali Ghodsi of Databricks explains that most organizations still use AI in basic ways, and that meaningful, wide-scale automation remains elusive due to a complex web of barriers. Databricks, in turn, has built solutions that allow enterprises to overcome these hurdles and realize AI's full promise—if they can address these challenges in tandem.
Ali Ghodsi notes that, in practice, most enterprises use AI as a "glorified Google search" or to marginally improve software coding. The dominant use cases are chatbots and search tools that deliver information already available through conventional search engines, and AI-powered coding agents that deliver incremental speed-ups of only 20-30%. There is little evidence of enterprises running thousands of autonomous agents to automate core work. Even the most advanced organizations still keep humans in the loop because AI agents can "drift off course," returning unreliable or inappropriate results if not closely monitored.
A fundamental reason for limited progress is that AI lacks the "context" within organizations. The most valuable data—meeting transcripts, decisions, communications, and knowledge—remains out of reach for AI systems. Executives frequently refuse to record meetings due to liability and privacy concerns. As a result, Ghodsi likens the problem to having the world's greatest mathematician but keeping them uninformed about the organization's actual activities, rendering them useless for complex problem-solving.
Security and governance obstacles are heightened by fears that broad AI access will expose companies to cyber threats or insider attacks. Organizations are bound by legacy systems, complex regulatory requirements, and a heightened awareness of cybersecurity risks, particularly with generative AI which worsens these concerns. Even when technical solutions allow broader AI access to organizational data, the risk of security breaches remains a top concern for enterprise leaders.
The explosive growth in AI usage brings significant cost increases for compute tokens and cloud resources. Ghodsi observes that for most enterprises, the cost of running AI is rising faster than any measured increase in revenue—a problem for CFOs faced with ballooning operational expenses. Eventually, financial leaders demand justification and cost controls before further expansion of AI deployments.
Finally, there are strategic concerns around data sovereignty and intellectual property (IP). Using external vendors for AI models and storage, such as OpenAI or Anthropic, creates the risk that proprietary organizational data could leak or be incorporated into public AI products—eventually falling into the hands of competitors. As these vendors continuously roll out competing products, enterprises worry about their data and IP being used to erode their future business advantage.
Databricks has developed an integrated platform that addresses the interconnected challenges of AI adoption, focusing on governance, security, cost control, and data sovereignty.
Databricks' Genie product is a core example. Genie constructs a dynamic ontology—a constantly-evolving graph that captures the organizational structure, employee relationships, project status, emails, meetings (with transcripts), decisions, Slack conversations, documents, Excel sheets, and PowerPoint presentations. This rich internal knowledge graph means the AI works with context—not just live search—giving it access to all the fragments of decision-making and expertise that differentiate an enterprise.
The Genie platform integrates strict access controls and repositories for approved data, enabling secure AI deployments. It allows enterprises to restrict flows of information, governing which agents see what data, and preventing data exposure or insider threats. Databricks acts as a governed data repository, providing consumption-based access to organizational assets with full auditing and compliance, significantly reducing cyber and insider risk.
To address the runaway expense of AI compute and tokens, Databricks offers optimization techniques such as model routing (choosing the most economical model for each query) and leveraging open-source solutions to substitute for high-cost proprietary models wherever possible. These strategies reduce token costs while maint ...
Enterprise Ai Adoption Challenges and Databricks' Solutions
Ali Ghodsi and Jason Calacanis explore the current state of AI, the reality versus hype of superintelligence and recursive self-improvement, and the practical risks and safety concerns facing digital society.
Ali Ghodsi argues that present-day AI systems, particularly large language models (LLMs), already meet the decade-old academic benchmark for Artificial General Intelligence (AGI). He references UC Berkeley's AMP Lab definition from 2009-2010, noting that, by those standards, AGI was achieved last year. Consulting peers from the original lab confirms that, while the definition fits, the research community has since shifted the goalposts. Now, attention centers on the quest for superintelligence rather than acknowledging the already transformative capabilities of current models.
Ghodsi contends that, even if progress stopped now, the intelligence present in today’s systems could, if implemented thoughtfully, completely transform global economics and society. However, the fixation among researchers on superintelligence means current AGI-like achievements are overshadowed in public and academic discourse.
Ghodsi defines superintelligence following Bostrom’s 2014 work: a system capable of millennia of human thought in minutes, thinking orders of magnitude faster and producing intellectual outputs like novels or theses instantaneously. The 10% existential risk estimates and notions like "paperclip maximizers" arise from anxiety over this kind of superintelligence potentially escaping human control.
The theoretical leap from current systems to such runaway intelligence hinges on recursive self-improvement (RSI), but Ghodsi argues that none of the essential preconditions for RSI are met today:
Instead of a positive feedback loop, pre-training involves complex manual processes: extensive data cleaning, vendor-sourced skilled labor, laborious domain-specific reinforcement learning, and ever-rising resource costs. Even if all four criteria were met, it wouldn’t guarantee a conscious agent able to outmaneuver humans—it’s just a necessary precondition. The apparent absence of such a flywheel is strong evidence against imminent RSI-driven superintelligence. Ghodsi also emphasizes that even if a system became dangerous, AI operates digitally and can be shut down, limiting harm to that vector.
Ghodsi identifies cyberattacks as the principal near-term risk posed by AI systems. He details that advanced AI can find vulnerabilities in legacy code much faster than humans, creating asymmetric scenarios in which attackers only need a single successful breach while defenders must maintain near-perfect protection. For example, hackers can exploit default credentials in infrastructure like grocery store refrigerators, causing damage but falling far short of superintelligent catastrophe.
The solution, he says, is engineering: scan and secure new AI-generated code to remove vulnerabilities and break the cycle of persistent bugs. Most current attacks target legacy code—finite in scope—which can be incrementally rewritten, patched, and secured. Once this is done, future vulnerabilities should dramatically decrea ...
Ai Safety, Superintelligence, and Recursive Self-Improvement Debate
Artificial Intelligence is fundamentally changing how organizations make decisions and structure themselves. Ai empowers leaders with data, drives cross-functional collaboration, helps resolve cultural misalignments, and automates tasks—while leaving critical final decisions in human hands.
Executives now query Ai directly for performance metrics, customer feedback, and meeting summaries. Ali Ghodsi describes waking up to detailed automated reports from customer meetings—including who attended, who said what, and which features were requested—without needing to ask employees or rely on manual updates. This system replaces the traditional need for executives to obtain information through intermediaries, removing potential communication gaps and bias.
Ai-powered dashboards, backed by governed data repositories, allow leaders to access metrics instantly, track favorability, and delve into details by clicking through authors and related assets. Ghodsi demonstrates how this data-driven approach enables leaders to test management theories in real time; for example, when a CFO suspected that poor work-life balance was causing low engagement scores, Ghodsi used the Ai to investigate. The system showed correlation but not causation and provided concrete examples, like a specific manager with a low work-life score but high action score. This disambiguates which managers drive engagement and results versus those inadvertently creating burnout.
Further, the data powering these dashboards is governed—organizations like Databricks act as repositories, charging for data consumption in models akin to cloud services. As companies deploy more Ai agents and queries, this consumption-based approach accelerates business and encourages adoption.
Traditional organizational charts create latency and overhead, slowing execution due to their rigid, hierarchical structure. Ghodsi points out that real business processes flow horizontally—feedback from customers moves to product management, then to engineering, to marketing, to sales, and cycles back—crossing departmental silos. Ai’s deep knowledge of the org chart, relationships, and responsibilities allows it to quickly coordinate across these boundaries, preventing slowdowns that typically result from human bottlenecks.
By continually learning an organization’s ontology, Ai can identify inefficiencies, such as overly vertical teams with layers of management, and suggest realignments. While adoption remains rare—Ghodsi estimates only a small percentage of global companies currently understand or implement these models—the few that do may gain a sharp competitive edge.
Ai enables organizations to analyze subgroup deviations from company culture by examining meeting and decision patterns, even across global offices and languages. Ghodsi uses Ai to evaluate whether subunits embody the organization’s principles, surfacing potential culture drift or highlighting innovative new norms. Where divergence is found, leadership is notified so that adjustments can be made or lessons learned.
Performance analysis distinguishes between correlation and causation at the management level. The system can reveal, for instance, that certain managers push for results while still engaging with employees versus those whose approach leads to b ...
Organizational Transformation and Ai-driven Decision Making
Ali Ghodsi, CEO of Databricks, emphasizes the decision to remain a private company during volatile market transitions, particularly in the context of rapid advancements in artificial intelligence. He notes that when companies go public, they become susceptible to earnings cycles and frequent stock volatility, which can pressure executives and affect employee morale. Stock price fluctuations often occur independently of company fundamentals, creating a “roller coaster” effect that distracts from long-term company vision and progress. Employees—whose compensation is often heavily equity-based—can feel like they are taking a pay cut if the stock drops, leading to turnover risk and undermining long-term alignment. For example, an employee promised a multimillion-dollar equity value can see that number drop precipitously due to market sentiment rather than business performance, leading to frustration and attrition.
Ghodsi highlights that the current transition period in AI introduces additional uncertainty, making equity compensation especially volatile and demoralizing for both staff and leadership. To counter these risks while still providing liquidity to employees, Databricks conducts private tender offers annually. These tenders offer predictable liquidity without exposing the company or its employees to the pressure of quarterly earnings or the whims of market sentiment. Few employees liquidate more than a small portion (around 5%) of their shares, and the liquidity pool is rarely exhausted, illustrating both their confidence in Databricks’ long-term prospects and a preference for holding equity rather than cashing out at opportunistic moments. This approach provides capital access for employees while supporting Databricks’ overarching, long-term vision.
Founded in 2013 at UC Berkeley as part of the AMP Lab, Databricks grew from $1.5 million in revenue in 2015 to a projected $7 billion run rate by 2026, while maintaining annual growth between 70% and 80%. This explosive growth was fueled by a series of contrarian bets: prioritizing cloud computing, committing early to AI and machine learning, as well as embracing open-source technology. At the time, many analysts considered these strategies risky and even forecasted failure; Databricks' decision to decouple compute and storage, leverage cloud scalability, move all-in on AI use cases, and invest in open source set the company apart from its peers. As these once-controversial approaches became industry standards, Databricks gained significant competitive advantage and momentum, leaving other players struggling to catch up.
The company's capital access as a private entity is exemplified by its ability to make billion-dollar acquisitions, further fueling its strategy and expansion without the burdens of public market scrutiny. The employee base remains confident, with most employees holding onto their equity stakes through multiple liquidity windows, signifying widespread confidence in the company’s trajectory.
Building on its success, Databricks is now expanding into new markets such as security, marketing, and data governance. These expansions are aimed at addressing four key barriers to AI adoption in large enterprises. For instance, Databricks’ new security products focus on protecting against increased attack surfaces associated with the proliferation of AI agents.
The company’s consumption-based revenue models encourage broader AI adoption and platform usage, ensuring that incentives are ...
Databricks' Business Strategy and Market Position
Download the Shortform Chrome extension for your browser
