Designing experiments for disciplined entrepreneurship

Entrepreneurs and managers can consciously design experiments to surface flaws in their business plan and spur revision. An entrepreneurial experiment, as I use the term, is a test designed to reduce uncertainty critical to success before committing additional resources. Common examples include customer research, prototypes, regional service, and beta customers. Based on the results of these experiments, entrepreneurs may decide to cut their losses, revise their working hypothesis and run another experiment, or harvest the value they have created. Below are some examples.

  • Identify the deal killers. Every plan includes countless assumptions. Rather than worrying about all of them, an entrepreneur should identify potential deal killers, variables that could prove fatal. Deal killers vary: In commercial real estate development, title disputes or environmental liabilities could scotch a deal, while a software start-up faces a deal-killer if a deep-pocketed rival has a valid claim on the underlying intellectual property. Deal killers are often discernible early on, and managers and entrepreneurs should try to surface these critical source of uncertainty early. New deal killers may appear as the venture proceeds, while others prove tractable.
  • Know what you are betting on. In turbulent markets, multiple variables influence the an opportunity’s payoff. Entrepreneurs should identify the key drivers of success. One way to do that is to pose the question: What are we betting on here? It could be a better mousetrap, access to a brand or technology that others lack, a critical relationship, or the ability to move quicker than established players. A venture’s big bet is not simply the inverse of its deal killers. In developing a new drug, for example, the big bet is on the compound’s therapeutic benefits, while the deal killer is potential toxicity.
  • Design partial experiments. Partial experiments test a single variable, typically a deal killer or key bet. A real estate developer could test a deal killer by commissioning an Environmental Protection Agency assessment before buying land, while an entrepreneur could run a patent search to ensure her intellectual property rights are secure. Partial experiments can also test the big bet. The founders of Kingsley Management, marketed an automated system to wash cars thoroughly without damaging the exterior finish. They bet that their solution would appeal to consumers across different climates, and not be limited to markets where extreme weather and use of salt on the roads necessitate frequent trips to the car wash. The founders’ used their first round of funding to test consumer adoption in markets with different climates.
  • Design holistic experiments. Holistic experiments test multiple variables and interactions between them on a small scale. Typical examples include a test launch before wider roll-out-recall how Noodles got the first two restaurants right before adding more-or building a prototype for an early customer. Such experiments can reveal unexpected interactions among variables. An MIT professor founded E Ink to commercialize a technology that replaced paper and ink with an electronic display. The start-up introduced a prototype for signs in grocery store aisles to update prices remotely. These experiments revealed an unexpected result-the copper roofs and aisles of metal cans blocked the wireless signals required to refresh the signs.
  • Explicitly consider trade-offs. Entrepreneurs can design experiments that are cheap, fast, or certain but not all three. In designing an experiment, entrepreneurs should consider which dimension is most important in their own situation. For medical devices, certainty about product safety is critical, and entrepreneurs must accept expensive and time-consuming clinical trials. If several companies are pursuing the same opportunity, a well funded start-up might sacrifice certainty and frugality to get a prototype in front of customers quickly. Any entrepreneur bootstrapping a company must look for the cheapest experiment possible, sacrificing both certainty and speed.
  • Stage your experiments. Entrepreneurs can take a page from the venture capital playbook and stage investments by starting with low-cost tests of deal killers or big bets before proceeding to more expensive holistic experiments. The Food and Drug Administration’s human clinical trial process for testing new drugs illustrates the logic of staging experiments. The first and least expensive step tests new compounds for toxicity, while subsequent and more expensive trials test benefits relative to established drugs.
  • Avoid experiment creep. Experiment creep occurs when a test drags on too long, costs too much, or loses clarity about intended results. Those running an experiment often become vested in its success, lose objectivity, and cast findings from their tests in the most favorable terms possible. Inviting outsiders-such as investors or customers-to actively participate in the designing experiments and reviewing their results can mitigate experiment creep.

Leading in turbulent times

This blog is no longer active but it remains open as an archive.

Don Sull is professor of management practice in strategic and international management, and faculty director of executive education at London Business School. This blog is dedicated to helping entrepreneurs, managers, and outside directors to lead more effectively in a turbulent world.

Over the past decade, Prof Sull has studied volatile industries including telecommunications, airlines, fast fashion, and information technology, as well as turbulent countries including Brazil and China, and found specific behaviours that consistently differentiate more, and less, successful firms. His conclusion is that actions, not an individual’s traits, increase the odds of success in turbulent markets, and these actions can be learned.