Statistical power
Statistical power is the probability that a test will detect a real effect when one genuinely exists. A test with low power can miss a true winner, you run an experiment, it comes back 'no significant difference', and you wrongly conclude the change did nothing when actually you just didn't have enough data to see it. Convention aims for 80% power.
For a founder running A/B tests on modest traffic, this is the quiet killer. Underpowered tests don't just waste time; they teach you false lessons, you bin a change that worked. Power depends on your sample size, the baseline rate, and how big an effect you're hoping to catch, smaller effects need far more data. Before you run a test, do a quick power calculation to learn how long it needs to run to be conclusive. If the honest answer is 'longer than you can wait', test a bigger, bolder change instead of a tiny tweak.