False Positive: Saved by the Product Review
Why Our Most Loved Feature Never Made It to Launch
We thought we had nailed it.
We’d just wrapped a month-long design sprint aimed at solving a thorny customer onboarding problem: how to give customer admins the confidence that they had correctly configured access rights. We’d done all the right things: x-functional brainstorms, customer interviews, stakeholder meetings, surveys, sketches, prototypes, user testing. We were ready for our product review.
In this edition of Shipping on Fridays (boldly shipped on a Saturday!) I’m going to dive into an aha moment that emerged from a Product Review, which saved my team from building the wrong solution. Enjoy!
👋 Hey, I’m Alex. I write Shipping on Fridays to explore the craft of how great products get built and what we can learn from the people behind them. I publish 1–2x a month, and every post is meant to be fun, useful, and a little unexpected; from design sprints to sailing races to holiday chaos. If you’re into learning, product or design, this is for you.
Our perfect solution
The stakes were high. If you’re setting up a platform that handles sensitive things like payroll, attendance, and personal employee data (think Workday or Rippling), one mistaken permission can have serious consequences. Imagine a junior employee stumbling across everyone’s salaries. 👻
Our solution let customers preview the platform from any user’s perspective. What made us so confident we had it nailed was that customers loved it. In seven customer interviews it consistently ranked as the top concept. Our internal stakeholders also gave it glowing reviews - the Customer Success team couldn’t wait for the launch.
Why’d they love it so much? It turned a customer support hack into a polished, in-product feature. Our solution took something familiar and already helpful and made it even better.
The aha moment that saved us
With this confidence we went to a product review with other PMs, designers and the VP of Product. Everyone was impressed with our positive customer feedback. It seemed like we had a slam dunk.
Then the VP of Product leaned forward and asked. “How does this actually solve the customer problem more elegantly than today?”
The room went quiet.
We’d just spent weeks refining this idea. Customers loved it. Our own team loved it. But now, the question hung in the air—and we didn’t have a crisp answer.
Would this really help customers find misconfigured permissions? Would it save them time? Would they honestly click through dozens of profiles just to verify access? Or were we simply giving them a shinier version of what already existed—a solution that didn’t move our north star metric?
That’s when it hit us: We had a false positive.
We’d created something customers asked for, but not what they truly needed.
Back to the whiteboard we went. This time, we asked ourselves: what would genuinely build confidence? Several weeks and multiple sketches and prototypes later the answer emerged: a clear, digestible summary of who has access to what—across the whole org. Something you could scan in seconds and act on immediately.
Customers and internal teams lit up. We hadn’t just improved the UX—we’d removed a blocker.
💡 Lessons Learned
Be a steel man. Challenge your own solution and try to approach your work from the opposite perspective and poke holes in it.
Get peer feedback early. A fresh perspective can expose hidden flaws, especially when you're too close to the work. Encourage your org to host product reviews with other PMs and designers - you’ll raise the quality of each other’s work.
Lean into hard questions. When you provide peer reviews don’t shy away from the hard questions. The tough questions are often the ones that unlock the real solution.
✅ Final Thoughts
You can do everything “right” in product development—talk to customers, run design sprints, test with users—and still almost ship the wrong thing. That’s why it’s so important to stay humble, stay curious, and stay open to course correction.