PDF Summary:The Business Case for AI, by Kavita Ganesan

Implementing AI in your organization isn't just about building sophisticated models—it's about creating real business value and ensuring users embrace the technology. In The Business Case for AI, Kavita Ganesan provides a practical framework for identifying, validating, and deploying AI initiatives that deliver measurable results.

Ganesan introduces the Jumpstart AI method and HI-AI Exploration Model to help you assess your organization's AI readiness and discover high-impact opportunities. You'll learn how to measure AI's return on investment, plan for model deployment from the start, and maintain AI systems after launch. Throughout, Ganesan emphasizes that successful AI requires three pillars: strong model performance, clear business outcomes, and user satisfaction—and she shows you how to evaluate and strengthen each one.

(continued)...

To calculate the ROAI, start by identifying the relevant metrics. The metrics should closely align with the possible advantages and the problem you're aiming to address. Then, set an initial measurement and a target for improvement. This goal might be a flexible objective or a baseline you can modify over time.

The Risks of ROAI

While ROAI can help you assess the impact of AI, it can also introduce risks. In The Tyranny of Metrics, Jerry Z. Muller argues that the most damaging consequence of what he calls “metric fixation” is that organizations come to focus on those aspects of performance that are easily quantified over the short term, tying rewards and punishments to these numbers, and in the process they systematically neglect harder-to-measure forms of value such as learning, innovation, professional judgment, and the development of long-range capabilities. Applying this insight to Ganesan’s discussion of ROAI, we can see that if your ROAI benchmarks are based on narrow, short-term metrics, you may be tempted to cancel AI projects that are crucial for your long-term competitiveness. For example, if your ROAI focuses solely on immediate cost savings, you might abandon an AI initiative that’s essential for developing new capabilities that will be critical in the future.

Identifying and Validating AI Initiatives

Ganesan suggests identifying and validating AI initiatives using the Jumpstart AI method. This four-step process involves: 1) pinpointing your AI-readiness gaps, 2) identifying AI initiatives with significant impact, 3) crafting a temporary AI plan, and 4) monitoring progress, adjusting, and iterating. The Jumpstart AI method offers a structured way to enhance your organization's readiness for AI as you gain AI experience through short-term initiatives.

(Shortform note: The Jumpstart AI method is a form of “validated learning,” a concept popularized by Eric Ries in The Lean Startup. Validated learning is a process of testing hypotheses about your business model through experiments and feedback loops. Each AI initiative you propose is a hypothesis about how AI will impact your business. The goal is to reduce uncertainty about how AI will behave in your specific context.)

First, you assess where your organization is lacking in AI-readiness so you can prepare to address those areas. Next, you'll find possible AI initiatives within your organization to uncover AI opportunities. This will also enhance the AI expertise of your strategic group. Next, in Step 3, you devise a brief-term plan for AI. Specifically, you'll apply both proactive and disruptive methods to fill some AI-readiness gaps and enhance your AI expertise. This stage advances you toward reaching your extended objectives. Lastly, at Step 4, you assess how far you've come and modify your short-term AI plan for the subsequent iteration.

Becoming Ambidextrous

The authors of Lead and Disrupt describe a similar process to the one Ganesan outlines here. They argue that to survive in the face of disruption, companies must become ambidextrous, simultaneously exploiting their existing businesses through continuous improvement and cost reduction while also exploring fundamentally new opportunities through small, protected experiments that challenge current products, processes, and business models. In this context, you can think of “proactive and disruptive methods” as an ambidextrous pairing: Proactive methods deliberately refine and extend current ways of operating, while disruptive methods create contained experiments that intentionally probe radically different ways of working.

Ganesan also recommends using the HI-AI Exploration Model to find promising AI opportunities. This four-step process involves: 1) identifying possible AI initiatives, 2) framing these potential endeavors, 3) verifying them with experts, and 4) scoring them.

The HI-AI Discovery process aids in pinpointing AI opportunities that are feasible, have a strong business impact, and carry identified risks. It assists in eliminating ineffective options and boosts your likelihood of succeeding with every initiative.

(Shortform note: In their 2018 book The AI Advantage, Thomas H. Davenport and Rajeev Ronanki describe a similar process for discovering and scoring AI opportunities. They recommend that companies systematically list potential AI use cases and rate them on dimensions like value and feasibility. This approach, which predates Ganesan’s book, closely mirrors the exploration, expert validation, and scoring steps of the HI-AI Exploration Model and HI-AI Discovery process.)

Tactical Execution & Operationalization

Next, Ganesan explains how to ensure your AI systems are successful after deployment.

Model Deployment & Initial Validation

Ganesan states that deploying a model means integrating it into organizational frameworks. This process officially launches a model into production once it’s achieved success in the development and post-development testing stages. At this point, the model’s accuracy is deemed sufficient, and its known flaws are thoroughly recorded. Once deployed, models typically handle new data by processing it either instantly or almost instantly, or in batch mode, processing data after it accumulates to a certain volume. Your business requirements will determine the most effective way for data to be processed by a model. This phase is primarily guided by data and software engineers working together with business leaders and AI specialists.

(Shortform note: In Competing on Analytics, Thomas H. Davenport and Jeanne G. Harris suggest that the decision to process data instantly or in batch mode should be based on the economic value of reducing decision latency. They argue that organizations should invest in real-time analytics only when the financial benefits of faster decision-making outweigh the costs of implementing and maintaining such systems. This approach ensures that resources are allocated efficiently, focusing on areas where immediate data processing provides a clear competitive advantage.)

Planning to deploy a model should begin as early as possible, preferably in the first phase: Defining and Planning the Problem. While planning a new project, you'll need to discuss your vision for incorporating the model into your product or workflow with your development team. Additionally, you'll define the operational constraints for the model.

(Shortform note: In MLOps, Mark Treveil and Alok Shukla explain that an effective MLOps practice requires you to define a small set of measurable targets for your live system. These targets should be based on your vision for the model’s deployment and the operational constraints you’ve defined. These targets will help you design your system, monitor its performance, and iterate on your models.)

Next, Ganesan explains the importance of testing the model after development, which involves testing a model with authentic data in real-life situations. PDT is crucial because it can reveal performance concerns that weren't apparent during development. It additionally lets you see how the model affects business metrics and compare it to existing solutions. Post-development testing involves an ongoing cycle requiring close collaboration between business leaders and the development team.

(Shortform note: In Designing Machine Learning Systems, Chip Huyen explains that post-development testing often involves a shadow deployment, where the new model runs alongside the current production model. The new model receives a copy of every production request and generates predictions that are logged for later analysis instead of being returned to users. This allows teams to evaluate the model's behavior and resource usage on live traffic without impacting user-facing decisions.)

Ongoing Monitoring & Maintenance

Ganesan stresses the importance of continuously monitoring your model's production results. AI models can deteriorate over time due to issues like changing consumer behavior or data problems. Therefore, you need to detect and resolve these issues promptly. To achieve this, gather insights into the model's effectiveness by seeking user feedback or monitoring clicks and usage patterns.

(Shortform note: While monitoring user feedback, clicks, and usage patterns can provide valuable insights into your AI model's effectiveness, it can also lead to unintended consequences. In Weapons of Math Destruction, Cathy O'Neil argues that algorithms trained on behavioral data and attention metrics can learn to exploit human vulnerabilities, creating feedback loops that serve the interests of the institutions deploying them while undermining individual autonomy and trust.)