A losing experiment is a win if it kills a bad idea cheaply
A losing experiment is a win if it kills a bad idea cheaply
The deepest mindset problem in experimentation is treating a losing test as a failure. It is the opposite. An experiment that returns a clean negative has done its job perfectly: it bought you certainty about an idea you would otherwise have argued about for months, and it bought it cheaply. The only real failure is an experiment that teaches you nothing, win or lose.
This reframing changes how a team behaves. When losing is failure, people only test the safe things they are confident will win, which means they only test the things that barely matter. When losing is a successful purchase of certainty, people test the bold, uncertain ideas where the big wins actually live. A healthy programme has a hit rate well below half, because a high hit rate means you are only testing the obvious.
Every result, win or lose, has to leave a durable trace, or the cost was wasted. A losing test that nobody records gets proposed again in six months, re-built, re-run, and re-lost at full price. The lesson from a clean negative is an asset exactly as much as a winning variant is, and it has to be written down with the same care: what we believed, what we changed, what the number was, what we now know.
The trace also has to capture why, not only what. "Pricing above the fold lost" is half a lesson. "Pricing above the fold lost because buyers in this segment want to understand the problem before they see a number" is a lesson that shapes the next ten experiments. Push every result, especially the losses, toward a statement about your buyers rather than about a button. That is the knowledge that compounds.
Celebrate the clean kill the way you celebrate the win. A team that high-fives a definitive negative is a team that will keep taking real swings, because being wrong has been made safe and even valuable. The moment losing feels like a black mark, the testing goes timid, the hit rate climbs, and the programme quietly stops mattering. Protect the appetite to be wrong, it is the most fragile and most valuable thing you have.
INTERVIEW EWOUD: Tell me about a losing experiment that turned out to be one of the most valuable things you ran, because of what it killed or what it taught you about your buyers. What did you believe going in, and what did the loss teach you?