How do you test new products? What are the four major risks that need to be addressed?<\/p>\n\n\n\n
A crucial part of the product development and discovery process is new product testing. There are four major risks that need to be addressed during these tests: value, usability, feasibility, and business viability. <\/p>\n\n\n\n
Keep reading to learn how to assess the four risks of new products.<\/p>\n\n\n\n\n\n\n\n
New product testing is the final step in the discovery process\u2014it\u2019s meant to once and for all separate the good ideas from the bad. It should address the four risks of product discovery<\/a>:<\/p>\n\n\n\n The answer to all of these questions must be \u201cyes\u201d before a company moves on to creating the product. A company can\u2019t know whether the product has a market niche, or whether people will buy it, with 100% certainty\u2014which is why some products flop. But tech companies can validate many ideas during the discovery process. Most of these ideas won\u2019t work out, but failures won\u2019t be a large blow if companies use prototypes and focus groups to limit the time and resources spent. Transparency and communication are critical to the testing process as well\u2014a sales executive might realize that an idea doesn\u2019t meet the standard of business viability, whereas an engineer might conclusively say that building a product isn\u2019t feasible. <\/p>\n\n\n\n Determining whether a product has value or a market niche (commercial viability) is one of the most important tests. Customers will generally only buy a new product if it\u2019s significantly more valuable than what they\u2019re currently using. There are three ways to test for value: the fake door demand test, <\/strong>the qualitative value test, <\/strong>and the quantitative value test. <\/strong><\/p>\n\n\n\n The fake door demand test determines whether there\u2019s a demand for your product<\/a>. The test works like this: You add your idea for a product to your company\u2019s already-live system. If it\u2019s a new feature on your company\u2019s app\u2014you\u2019d put the feature link where it would be if you were to launch it. But clicking on it would take the customer to a page recruiting test subjects for the feature. Similarly, for a new product, you\u2019d build a landing page that links the potential customer to a recruiting page. You can track the number of clicks on the feature or product to see if demand exists, and <\/em>you can find some new test subjects. <\/p>\n\n\n\n This type of value testing will show why <\/em>a product has value or doesn\u2019t. If the latter is the case, it will also show what a company can do to fix its value problem (if it\u2019s fixable). You\u2019re not proving <\/em>that your product is good here. Rather, you\u2019re learning quickly. <\/em>This type of testing, according to Cagan, is the most important part of the discovery process. <\/strong>Here\u2019s how it works:<\/p>\n\n\n\n It\u2019s almost certain you\u2019ll get different answers from different test subjects. Figure out why they feel differently from one another. Product managers should be at every <\/em>qualitative test to help understand the differences. Some people are bound to be uninterested in the product. But if most <\/em>test subjects aren\u2019t interested, it might not work. Even if many test subjects are interested, it\u2019s useful to listen to those who aren\u2019t so you can understand what to fix. <\/p>\n\n\n\n The third type of value testing\u2014quantitative testing\u2014is generally done with live-data prototypes, which allow the company to acquire enough data for statistically significant results. While qualitative testing is about learning, <\/em>quantitative testing is about building evidence. <\/em>Generally, larger companies do more quantitative testing because they have more resources to do so and they\u2019re more risk averse. They may also have more site traffic to build their quantitative tests around. <\/p>\n\n\n\n The best type of quantitative test is an A\/B test. <\/strong>In this test, 99% of users use the \u201cA\u201d version of the product, or the version that\u2019s already publicly available. The other 1% uses the \u201cB\u201d version, or the product that\u2019s in the discovery stage. The company then collects data on the behavior of the two groups. <\/p>\n\n\n\n Some companies that don\u2019t have enough traffic for an A\/B test will use an invite only test. <\/em>This is similar, but instead of selecting a random and small group of users, they invite users to use the \u201cB\u201d version. Unfortunately, though, the data isn\u2019t as reliable because subjects opted in to using a new version. <\/p>\n\n\n\n Advanced analytics have changed the way companies can go about their product discovery. They help companies measure progress<\/a> and understand much more about customer behavior, and for this reason, more companies are using them.<\/p>\n\n\n\n (Shortform Note: Michael Lewis\u2019s Moneyball: The Art of Winning an Unfair Game <\/em>explores the differences between qualitative and quantitative analytics through the lens of baseball. Read the Shortform summary here<\/a>.)<\/p>\n\n\n\n1. Value<\/strong><\/h3>\n\n\n\n
Fake Door Demand Test<\/h4>\n\n\n\n
Qualitative Value Test<\/h4>\n\n\n\n
Quantitative Value Test<\/h4>\n\n\n\n
2. Usability<\/strong><\/h3>\n\n\n\n