← sources

My tests were all green and the product was still broken. I found 5 bugs in one afternoon

source: reddit (reddit:SaaS)published 2026-09-27 11:59ingested 2026-09-28 02:39

retained text

I build a website audit tool called finetooth (finetooth.net). Solo founder, launched this month, no revenue yet. That's the disclosure out of the way.

I finished a big migration a couple of weeks ago. 1,548 tests passing, CI green. I was pretty pleased with myself.

Then I sat down one afternoon and just used the thing. Ran audits on real sites and read the reports the way a client would read them. I wasn't hunting for bugs, I was just reading.

I found five.

Screenshots were being captured and saved to disk but never written to the database. 88 files sitting there, zero rows. That had been happening on every audit since the beginning.

Some pages render fine but never fire the load event, and on those I was silently losing a whole stage of the audit. Privacy, accessibility and mobile checks just weren't there, and nothing complained about it.

On sites behind Cloudflare I was sometimes auditing Cloudflare's holding page and reporting it as the customer's website. Eight findings describing an interstitial. That one hurt, and then I made it worse by breaking the page check with a null pointer while fixing it, which showed up as a check failing in 0ms for no reason at all.

And an old one from before the migration that was quietly throwing away the explanation on partial audits.

Four of those five were obvious the moment I looked at a report. None of them showed up in test output.

All five got a test that week. The one I'm most glad about came two days later, when I built a deliberately broken page and pointed the scanner at it, because three of my findings had never met a page that actually triggers them. They'd only ever been tested against sites that were fine.

Which makes me wonder how many other checks have only ever been tested against sites that pass.

document fetched 2026-09-28 00:29 · canonical https://reddit.com/r/SaaS/comments/1wriewc/my_tests_were_all_green_and_the_product_was_still