The Audit That Found Nothing

Last week, a full bug audit of a production Android codebase I work on came back with one line: no verified bugs found.

That's either the best sentence in engineering or the most worthless one, and the difference has nothing to do with the sentence. It has to do with the process that produced it. "No bugs found" from a process designed to find bugs is a milestone. "No bugs found" from a process designed to finish quickly is a liability with a green checkmark on it.

In the last article I made the case that step one of any app rescue is diagnose before you touch anything. This is the follow-up: what a diagnosis actually looks like when you take it seriously — and why the same audit process had to find (and prove) real bugs in earlier passes before I'd trust the pass that found nothing.

First, agree on what a bug is

Most code audits pad their reports, because most code audits define "bug" as "anything I'd have written differently." Naming complaints, architecture opinions, missing comments — noise that makes a report look thorough while burying the two findings that actually matter.

I use a narrower definition: a bug is behavior that deviates from intent. The code was meant to do X; under some real input or state, it does Y. Style isn't a bug. An architecture you dislike isn't a bug. A query that silently drops rows the user expects to see — that's a bug, and it's the only kind of finding the audit is allowed to report.

That definition sounds obvious. It's also the first thing to check when someone hands you an audit report: if half the findings are opinions, the process wasn't hunting bugs. It was generating volume.

Three specialists beat one generalist

The audit itself is AI-assisted, and this is the part of my practice that's changed the most in the past year. But the design matters more than the AI: I don't ask one agent to "review the code." I run three specialist review agents in parallel, each with a narrow brief:

ANR & threading — main-thread blocking, deadlocks, races, anything that freezes the UI.
Memory & leaks — retained contexts, unclosed resources, listener leaks, allocation storms.
Logic & state — the intent-vs-behavior gap: wrong queries, broken state machines, edge cases at boundaries.

Each agent reads the codebase end to end — this run covered 355 source files across the UI, domain, data, and core layers — but only reports findings inside its lane. Then the findings are cross-validated: every claim gets challenged against the actual code before it's allowed into the report.

Why split it? The same reason you don't ask one doctor to be your cardiologist, neurologist, and radiologist in a single visit. A reviewer with one thing to look for goes deep; a reviewer with everything to look for skims. Attention is a budget, and that's as true for AI agents as it is for humans.

A finding isn't a bug until a test fails

Here's the rule that keeps the whole thing honest: no finding is accepted, and no fix is written, until there's a failing test that reproduces it. Red first. Then the fix. Then the same test goes green and stays in the suite forever as a regression guard.

An earlier pass of this same process on this same codebase is why I trust it. The logic-and-state agent flagged a follow-up-reminder query that filtered on status = 'PENDING'. Sounds fine — until you notice the app also has a snoozed state. Snooze a reminder, let its due time pass, and it appeared in neither the Overdue list nor Today. It just vanished — the exact kind of quiet data loss users churn over without ever filing a complaint about.

The same pass surfaced two siblings: an item due at exactly the current instant showed up in both Overdue and Today at once, and the snooze operation's error path could leave an item stranded in the wrong state if scheduling the alarm threw partway through. Three real bugs, three failing tests written before the fixes, three green tests after. That's not a report you skim — that's a report you can replay.

This discipline matters double with AI in the loop. Review agents are extraordinary at surfacing suspects and — like every reviewer, human or not — capable of being confidently wrong. So the agents get to propose. Evidence gets to decide. A finding that can't produce a failing test doesn't go in the report, no matter how plausible it sounds.

The bugs no static pass can see

Some bug classes don't live in the code at all. They live in the gap between your code and the operating system — and the only way to audit that gap is with a real device in hand.

Reminder apps live or die on one promise: the alarm fires. So after the static passes came the hardware pass, on a physical Sony Android phone. I seeded the database with reminders in known states — three already overdue by 73, 49, and 25 hours, five due in the future — wiped every scheduled alarm to simulate the worst case, and rebooted the phone. Intent: the boot receiver should re-arm the five future reminders and deliberately skip the three stale ones (re-firing a three-day-old reminder at 2 a.m. because the phone restarted is its own kind of bug). Result: exactly that. Then the second experiment: schedule an alarm, force the phone into deep Doze — Android's aggressive sleep mode that silently kills most background work — and verify the reminder still fires. It did.

One detail from that session earns its keep in any Android engineer's notebook: while setting up the reboot test, I stopped the app with adb shell am force-stop — and the boot receiver never ran. Not a bug. Android marks force-stopped apps as stopped and withholds broadcasts from them until the user reopens the app. My test rig had manufactured a failure the product didn't have. If your process can't tell a real bug from a testing artifact, it will send you chasing ghosts — or worse, "fixing" code that was never broken.

Why I believed the report that found nothing

So when pass four came back clean, here's what that sentence actually meant:

The same three specialist agents that had found real, test-proven bugs in earlier passes re-swept all 355 files and found nothing new that survived cross-validation.
Every earlier finding was pinned under a regression test that started red and stays green.
The bug class that static review can't reach — OS-level alarm delivery across reboot and deep Doze — was verified on physical hardware, twice.

"No verified bugs found" from that process is a statement with teeth. The same words from a process with no bug definition, no reproduction discipline, and no device in hand are just a wish, formatted as a deliverable.

What to ask of your next audit

If you're paying for a code audit — or reading one an outgoing team left behind — four questions separate the real ones from the theater:

What counted as a bug? If the report mixes style opinions with defects, the signal is buried.
Can each finding be reproduced? Ask for the failing test, not the paragraph.
Did anything run on a real device? Lifecycle, alarms, Doze, reboots — the OS keeps its sharpest edges off the whiteboard.
Has this process found real bugs before? A hunt that has never drawn blood can't certify that there's none to draw.

That's the work I do: auditing and rescuing mobile codebases — Android, iOS, Flutter — with a process built to find what's actually broken, prove it, and pin it down so it stays fixed. If someone just handed you a suspiciously clean report, or you suspect your app has bugs nobody has managed to catch, it's worth a second pair of eyes. Mine come with three specialist agents and a drawer full of test phones.

Ramiz Raja is a senior mobile developer and technical lead with 12+ years shipping and rescuing production apps on Android, iOS, and Flutter. More at codebyramiz.com.