Falsification Mindset: Reducing the Number of Useless Solutions Built
Stop verifying. Start falsifying to build rock-solid solutions.
Stop verifying. Start falsifying to build rock-solid solutions.
Avoiding biases is one of the most complex parts of building products. It’s dangerous to get false positives and ultimately believe we’ve identified a rock-solid solution. And that’s when we slip into a dreadful pitfall; we fall in love with the solution and inevitably build something we love while end-users couldn’t care less about it.
Creating a pointless solution is devasting, and we must avoid it at all costs.
The questions we ask have a massive impact on our outcomes. When we strive to confirm what we think is correct, we may bias people and get false confirmations. However, when we seek to falsify our assumptions, we may uncover what we don’t know and identify hidden opportunities.
We follow a three-step approach to avoid creating useless solutions:
Collect assumptions
Strive for falsification
Adapt according to your learnings
#1 — Collect Assumptions
Whenever you start working on a solution, everyone around you has assumptions; some are transparent, and some are unspoken. For example, you assume end-users are willing to pay for your solution, you also think your solution is what end-users need, and the list will continue.
Naming your assumptions is the first step to avoiding useless solutions. We recommend using the following format:
Assumption: summary of what your assumption is.
Expected Result: the result you wish to observe from end-users.
Importance: how critical the assumption is.
Let me give you an example from e-commerce. You assume satisfied customers are willing to recommend your store to their friends in exchange for a 25€ voucher. And you expect that every third satisfied customer will be interested in joining the referral program. This assumption is critical because this initiative becomes irrelevant if satisfied customers are unwilling to recommend your service.
Hint: use an assumption matrix to facilitate prioritization and decision-making.
#2 — Strive for Falsification
Many product teams strive to validate their assumptions, which is challenging because you can easily fall into confirmation biases. You may already believe your solution is correct, and you search for answers to confirm that, believe me, you will find them. However, if you strive to falsify your assumptions, you will potentially uncover what you don’t know.
The questions you ask are crucial to success, for example:
How do we validate our assumptions? Such questions may encourage you to persuade end-users to generate the answers you want, which is lethal for product teams.
How do we falsify our assumptions? You remain skeptical when you have a falsification mindset until you get objective evidence from end-users. This behavior protects you from being biased and biasing end-users
Considering the previous example of a referral program. Let’s understand the two approaches; validating vs. falsifying.
Validating Assumptions
You’d ask end-users if they would recommend your service in exchange for a 25€ voucher. Surprisingly they would say yes because people are polite. And then, you may show some prototypes and get more confirmation from them. And you become confident with the results. The question is, were they really a confirmation you’re on the right track?
By validating assumptions, you confirm your ideas are correct, limiting your learnings.
Falsifying Assumptions
Instead of naming the solution to end-users, you’d exchange openly with them, trying to learn from their behaviors and interests. You strive to prove yourself wrong. Then, you could ask end-users how they like the platform and ask if their family or friends are also customers. As you learn more, you continue exchanging and striving to get hints that falsify your assumption.
By falsifying assumptions, you remain curious and continuously search for something that resonates with end-users.
#3 — Adapt According to Your Learnings
Valuable learning is the outcome you get from falsifying your assumptions. To succeed, you should adapt your actions accordingly. The more pragmatic you are, the faster you can progress. As you collect evidence from end-users, evaluate each assumption and decide whether to:
Persist: continue the implementation because the evidence was convincing and the expected results were reached.
Improve: the result wasn’t convincing enough, but it gave you signs of potential if you adapt your solution.
Pivot: you recognize the potential, but you need to pivot the solution or audience within a different scenario.
Drop: the evidence clearly showed the assumption was faulty, and it’s worthless investing any additional effort into it.
Gettings Hands-on
We believe experiments should be set up to falsify assumptions because this is rarely done. Most prototypes focus on how the real product would look and then strive for confirmation from users. However, we encourage teams to think of creating prototypes that falsify product ideas or reveal problems that would stay undetected otherwise.
Start by asking yourself: What could potentially go wrong? Then, design a prototype and try to evaluate whether that will happen or not.
Example: You want to prevent users from providing the wrong shipping address. You think the last checkout step (checkout summary) is a candidate to create a shipping problem. To falsify this assumption, you create a prototype and intentionally insert the wrong shipping address. Then, you test with end-users, and if they don’t notice the mistake, you’ve just identified a problem to solve; the shipping address wasn’t prominent enough.
Common Mistakes
Confirmation biases: you lead end-users to give you answers you want to prove your assumptions are correct.
Everything is in the same bucket: all assumptions are equally important, and you don’t know how to prioritize what to do first.
Ignoring assumptions: you start implementing solutions without naming assumptions.
Slow learning: you only verify your assumptions with end-users once you have implemented a minimum part of the solution.
Key Takeaways
Ensure you name your assumptions before you start implementing any solution.
Prioritize your assumptions according to their importance.
Strive to falsify your assumptions as fast as possible.
Use tools to support you while falsifying your assumptions. The faster you learn, the sooner you succeed.
Adapt your actions according to your learnings.