mvp test

This article is an excerpt from the Shortform book guide to "Inspired" by Marty Cagan. Shortform has the world's best summaries and analyses of books you should be reading.

Like this article? Sign up for a free trial here.

Do you want to create a technology startup company but don’t know where to start? Once you discover your product, how do you manage its development?

In Marty Cagan’s book Inspired, product management and product discovery are crucial for a startup’s success. Creating a good product idea is only the beginning, then you must prototype, test, and assess risk.

Continue reading for Marty Cagan’s advice on creating a product.

The Product Creating Process

Marty Cagan’s Inspired gives companies and entrepreneurs a two-step plan for creating and sustaining successful technology products: 

  1. Organize and structure effective teams.
  2. Develop products using a flexible “discovery process.”

This article will focus on Inspired’s product management and discovery process.

Inspired also explains how important product managers are in product development, teaches them how to be successful, and explains some of the biggest pitfalls that most tech companies fall into when designing products.

Author Marty Cagan, a Silicon Valley product executive, also details how product designers and engineers should function in a team and how company executives should treat their product teams. Originally published in 2008, the book was updated in 2018 to further address evolving concepts such as lean and agile techniques.

Product Discovery

Many companies start with a product idea, then create a roadmap, which sets specific deadlines for when parts of the product must be finished. However, this strategy doesn’t account for the fact that at the beginning of building a product, it’s unclear what will take a lot of time and what will take little. 

Companies would be better served to adopt a method of product discovery, a process of discovering the optimal product by separating good ideas from bad. It begins with defining a vision. A product vision is the overall goal of the company (or, in a large company, a component of the company). It can be achieved only far in the future—somewhere between two and 10 years. Here are 10 principles for creating a product vision:

  1. Ask why: Your product vision explains why you’re building your product and why you think it will succeed.
  2. Consider the problem rather than the solution: Be more curious about the problem you’re trying to solve than the possibility that a solution could make you rich or noteworthy.
  3. Think big: Again, it’s not a three-month horizon, it’s up to a decade. Don’t be afraid to dream.
  4. Disrupt your own company: If you’re not constantly reinventing products, another company will be and will take your market share or your goals.
  5. Inspire: Your vision should inspire your employees to be missionaries for the company. Build a vision that makes them excited to work for you. 
  6. Consider trends: Similar to point #4, constantly monitor trends in your field, and adjust your plans accordingly. 
  7. Be ready for changes: Even better than watching where the market is heading is predicting the market. Being on top of trends can help a company understand where trends might be going next. This way, you can be proactive rather than reactive. 
  8. Be willing to change the details but not the vision: As we’ve outlined in the past few points, sometimes the details of your plan are going to change. But your overall vision should almost always stay the same. 
  9. Understand that your vision is predicated on faith: If the vision you create isn’t a leap of faith, it’s not ambitious enough. It will likely take a few years to know if what you’re doing is working. This shouldn’t panic you.
  10. Spread the faith: Constantly communicate your vision inside and outside the company. Get people excited about your plan, and it’s more likely to come to fruition. 

The product vision method is better than the roadmap because it provides stakeholders within the company the ability to give their perspective on the product and is generally a more flexible method of building products than setting arbitrary deadlines. 

Moving Through the Product Discovery Chain

Once companies have a clear vision, they can move on to building products. We’ll go through the steps of product discovery to explain how to build products.

Framing

Framing helps you identify underlying issues with your ideas. There are two steps:

  1. Make sure the whole team is aligned and understands what the objective is or what problem you’re trying to solve.
  2. Identify the risks that the company will be taking on during discovery. These could be risks related to value, usability, feasibility, or business viability. The risks can include questions ranging from whether the company has the money to pay for the creation and rollout of the product to whether the product is ethical. 
Planning

Once you’ve established the parameters, the next step is figuring out solutions. The planning process can help to identify significant challenges. A planning technique companies often use is customer discovery. 

This is the process of finding and retaining loyal customers—the lifeblood of any company. The best way to do this is to look for reference customers—people who would pay for a final version of your product and like it enough to recommend it to friends. As few as six reference customers can help your company grow by helping you develop and market your product . Before fully launching, a company should recruit between six and eight of these people. The best reference customers share these characteristics:

  • They are in the company’s target market/audience. 
  • The product helps them solve a significant problem in their life.
  • They are not obsessed with tech.
  • They are willing to help the company and have the time to do so.

Recruit these customers either from existing users who still have a problem they need to have solved, or from surveys that determine whether potential customers could use your product. Once you find these people, treat them like your colleague or partner rather than a test subject. You both need one another equally. 

Prototyping

The next step in the discovery process after framing and planning is prototyping. People often think of prototypes as nearly complete products, but less elaborate prototypes are more useful and less expensive to produce.

While prototypes do take much less time, money, and energy than the final product to create, they still force teams to take the ideas in their heads and put them to work. They are a good balance between putting too much effort and too little effort into an idea. 

There are four basic prototypes:

Feasibility Prototypes

Sometimes engineers aren’t sure whether they can build a tech product—for instance, because of concerns like scale or whether it is possible to successfully code the version of the product they have in mind. The way to answer the question is to build a feasibility prototype. 

In creating a feasibility prototype, engineers typically write just enough code for the project to know that they can complete it. The code doesn’t need to be perfect—it’s unlikely that it will go in the final build. 

The feasibility type is done by the engineers and is for the engineers. The product manager needs to concern herself with it only insofar as she ultimately decides whether to move forward with it. 

User Prototypes

While feasibility prototypes involve writing a minimal amount of working code, user prototypes are simulations of the final product. They’re not actually functional—for example, a customer wouldn’t actually be able to buy anything via a user prototype for an online marketplace like eBay.

The simplest user prototypes often don’t look anything like the final product and serve only as the skeleton. Generally, simple user prototypes are for internal use only—to help the team visualize the product.

In contrast, more complex user prototypes called high-fidelity user prototypes look and feel like the final product. They take longer to create, and they can be used both internally and externally on test subjects. However,  high-fidelity user prototypes aren’t live, and can still be fairly basic—for example, a search function, might generate only a few results, or an algorithm still needs building. 

Live-Data Prototypes

A company pursuing a risky idea, can get real data on whether a product will sell while in the discovery phase by creating a live-data prototype. 

This prototype is a pared-down version of the final product—it’s not scalable, can’t take much traffic, and doesn’t have any SEO or analytics associated with it, but it does function. Test users can use the product and provide qualitative data on how they felt and quantitative data on how they used it and how well it worked.

Remember though, that at this point, the product still has a long way to go—the engineers have likely done less than 10% of the work needed for success. 

Hybrid Prototypes

The final prototype is the hybrid, which is a combination of the first three types. A “Wizard of Oz” prototype is an example of a hybrid prototype—the name comes from what’s behind the curtain. Behind the product’s front-end user experience, an engineer manually performs the tasks that the front-end says it can do. This saves time on automation, and can provide a sense of what people think of the product. 

Most hybrid prototypes are the least scalable of the four types, as they’re meant to be built quickly and provide customer feedback without requiring unnecessary engineering work. 

Addressing Risk

Throughout the product discovery process, companies address four risks:

  1. Value: Does the product have a market niche?
  2. Usability: Can users understand the product?
  3. Feasibility: Can the product reasonably be built?
  4. Business viability: Does the product make sense for our larger business strategy?

Testing for these four risks is meant to once and for all separate the good ideas from the bad. In the following sections, we’ll discuss each of the four areas of risk and how to test for them. 

Value

Determining whether a product has value or a market niche (commercial viability) is one of the most important tests. Customers will generally only buy a new product if it’s significantly more valuable than what they’re currently using. 

There are three ways to test for value:

  1. The fake door demand test: Add your product idea to the company’s already live system without making it functional (when a customer clicks on it they’re taken to a landing page that says the company is looking for test subjects for the idea). Track the number of clicks on the feature to see if there’s demand in the abstract.
  2. The qualitative value test: Show focus groups the product and collect their thoughts. Ask focus groups if they would buy the product right there (but don’t sell it to them). Record and discuss the results as a team immediately.
  3. The quantitative value test: The best type of quantitative test is what’s called an A/B test. The way it works is 99% of users use the “A” version of the product, or the version that’s already publicly available. The other 1% uses the “B” version, or the product that’s in the discovery stage. The company then collects data on the behavior of the two groups. 
Usability

Generally, this type of testing is straightforward. You’re making sure that users understand how to use your product and find it fairly easy to do so.

Recruit a group of test subjects and prepare a usability test. Here are some steps in a good test:

  • Tell the users they are using an early prototype and not the real product. 
  • Ask the users whether they can tell just from the first (landing) website page what your product is/does. This is useful information for the product designer. 
  • Make sure that during the test, the subjects are only using the product and not looking for ways to critique it. The testing environment makes people think they should provide critiques, but you’re just looking to see how they’ll use the product. 
  • Look for whether the user:
    • Used the product with no trouble.
    • Had a little trouble but ultimately figured out the product.
    • Couldn’t figure out the product.
  • Don’t help the test subjects figure anything out, but if you feel the need to talk to them, simply ask what they’re doing with the product. This can give you some insight into how they’re understanding it. 
  • Look for places the users are having trouble or places where they give up. These are the spots in the product to fix. 
Feasibility

The third risk to be explored by testing is feasibility, or whether engineers can actually build a product. Feasibility testing asks engineers to determine whether it is possible for a product to be built either by building a feasibility prototype or by building enough of the product that they are confident they can build the rest. Feasibility testing is also asking whether the current engineers on the team have the skills to do so, whether the company has enough time and resources to build the product, whether the product can be scaled, and even whether the company has the raw materials to build the product if it’s hardware. 

At this stage in discovery, most engineers will decide that the answer to the above is “yes, this is feasible.” Occasionally, though, if engineers are not sure if something is feasible, they need time to investigate it. In this instance, engineers build a feasibility prototype to answer any outstanding questions. As we’ve learned, these prototypes don’t take much time or energy generally, but the product manager still has to decide whether it’s worth the effort. Generally, the answer will be yes, and often, while considering the question of feasibility and dealing directly with the product without other distractions for a day or two, engineers come back not only saying it’s feasible, but offering ideas to make the product better. 

Business Viability

The fourth risk that testing explores is business viability, or whether the product will be profitable. This is the final hurdle before moving on with product creation. The product manager shows departments throughout the company the business model they have created of the product, and these departments in turn analyze the business model for any potential problems. If they don’t see any, they will sign off on the project. There are many stakeholders who need to sign off before a product has business viability—here are some of the most common.

  • Marketing: The marketing department is concerned primarily with driving sales. They will be looking for a product that they can easily explain to customers and one that looks to have a lasting impact and relevance in the marketplace. 
  • Sales: While the marketers are the ones trying to drive people to the product, the sales team are the ones getting pen to paper (or hand to credit card to keyboard) and completing sales. Especially if your company has a direct sales team, they will be interested in whether they have a product that they can get off the shelves and one that they can make good money on. 
  • Customer Success Strategy: Product managers should be aware of a company’s customer success strategy while they are building their product. A customer success strategy is how companies have decided to help their customers through using the product. Some companies use a high touch strategy, meaning that they provide significant support for their customers, while others provide a low touch strategy, which is hands off. The product needs to fit with the company’s strategy.
  • Finance: The finance department will ask whether the company can afford to produce and sell the product. Additionally, they’ll have a good sense of investor expectations—it’s useful to talk through the costs associated with the product (and the benefits) with someone in the finance department.
  • Legal: Especially for larger tech companies working on new projects, project managers have to address legal concerns before they can launch a product. Companies that don’t consider whether there are potential privacy violations, intellectual property violations, or compliance violations may be opening themselves up to legal action that can cost the company money and reputation. 
  • Security: The easiest way for a tech company to enter into a downward spiral is to compromise the security of its users’ information. The product manager needs to be clear that the product is secure before launch.
  • Leadership: Finally, the product manager needs to get approval from leadership. The leadership of the company has to be satisfied that all of the previous business viability conditions have been met before they give their sign-off. Hopefully, they’ve been involved for much of the process, but no matter what, they’re always aware from experience of the risks associated with launching a product. 

Once you’ve completed each of these steps, you can go to work on the product. Don’t be discouraged if many of your ideas fail in this process. The ideas that do pass through the process will likely be successful. 

Inspired: Product Management and Discovery

———End of Preview———

Like what you just read? Read the rest of the world's best book summary and analysis of Marty Cagan's "Inspired" at Shortform.

Here's what you'll find in our full Inspired summary:

  • A two-step plan for creating and sustaining successful technology products
  • Why product managers are so important in product development
  • How to avoid some of the biggest pitfalls that most tech companies fall into

Hannah Aster

Hannah graduated summa cum laude with a degree in English and double minors in Professional Writing and Creative Writing. She grew up reading books like Harry Potter and His Dark Materials and has always carried a passion for fiction. However, Hannah transitioned to non-fiction writing when she started her travel website in 2018 and now enjoys sharing travel guides and trying to inspire others to see the world.

Leave a Reply

Your email address will not be published. Required fields are marked *