Android App Audit: How to Verify a Clean Bug Report

Someone hands you an audit that says your app is fine. You don't quite believe it — that's why you're reading this.

You're right not to. A full bug audit of a production Android codebase I work on came back with exactly one line: no verified bugs found. That's either the best sentence in engineering or the most worthless one, and the difference has nothing to do with the sentence. It has to do with the process that produced it.

Here's what's at stake in getting that wrong. The bugs that kill apps commercially are almost never the ones that crash. They're the quiet ones — the reminder that never fires, the row that silently disappears, the alarm that dies overnight. Nobody files a ticket for those. They just leave, your retention curve bends down a month later, and no one on the team ever connects the two. A clean audit report doesn't make those bugs go away. It just tells you to stop looking.

In the last article I made the case that step one of any app rescue is diagnose before you touch anything. This is the follow-up: what a diagnosis actually looks like when you take it seriously.

The reminder that vanished

Start with a user of a reminder app. They snooze a reminder — the most ordinary action in the product. The due time passes while it's snoozed.

It never comes back. Not in Overdue. Not in Today. It's simply gone, and the only person who knows is the user who trusted it and got let down.

Here's where it lived. An earlier pass of this audit process, on this same codebase, flagged a follow-up-reminder query that filtered on status = 'PENDING'. That sounds fine — right up until you notice the app also has a snoozed state. A snoozed item isn't PENDING, so once its due time slid past, it matched no list at all. One clause in one query, and the product quietly broke its single promise.

That gap between what the code was meant to do and what it does under some real state is the only thing I'm willing to call a bug. Not naming complaints. Not architecture I'd have built differently. Not missing comments. Most audits pad their reports because most audits define "bug" as "anything I'd have written differently," and the result is a document that looks thorough while burying the two findings that matter.

So the finding didn't go in the report on the strength of the argument. It went in on the strength of a test that failed. No finding is accepted, and no fix is written, until there's a failing test that reproduces it. Red first. Then the fix. Then the same test goes green and stays in the suite forever as a regression guard.

The same pass surfaced two siblings once we started looking properly: an item due at exactly the current instant showed up in both Overdue and Today at once, and the snooze operation's error path could leave an item stranded in the wrong state if scheduling the alarm threw partway through. Three real bugs. Three failing tests written before the fixes. Three green tests after.

That's not a report you skim. That's a report you can replay — which is the whole difference. A finding that can't produce a failing test doesn't go in, no matter how plausible it sounds.

The bug my own test rig invented

A reminder app lives or dies on one promise: the alarm fires. Which raises the question no static code review can answer — the phone reboots at three in the morning, and does anything survive?

So I picked up a physical Sony Android phone and started setting up a reboot test. First step, stop the app cleanly with adb shell am force-stop. Reboot. Watch the boot receiver re-arm the alarms.

The boot receiver never ran.

For a few minutes that looked like the most serious finding of the entire audit — the exact catastrophic, silent, only-visible-on-real-hardware failure the hardware pass exists to catch. It wasn't a bug at all. Android marks force-stopped apps as stopped and withholds broadcasts from them until the user manually reopens the app. My test rig had manufactured a failure the product didn't have.

So I fixed the rig, not the code, and ran the experiment properly. I seeded the database with reminders in known states — three already overdue by 73, 49, and 25 hours, five due in the future — wiped every scheduled alarm to simulate the worst case, and rebooted the phone. The intent: re-arm the five future reminders, and deliberately skip the three stale ones, because re-firing a three-day-old reminder at 2 a.m. just because the phone restarted is its own kind of bug. Result: exactly that. Then the second experiment — schedule an alarm, force the phone into deep Doze, Android's aggressive sleep mode that silently kills most background work, and verify the reminder still fires. It did.

Two lessons came out of one session, and the cheaper one is the more useful. Some bug classes don't live in your code at all; they live in the gap between your code and the operating system, and the only way to audit that gap is with a real device in hand. (Which device, and how many, is its own problem.) But the sharper lesson is this: a process that can't tell a real bug from a testing artifact will send you chasing ghosts — or worse, "fixing" code that was never broken.

The report that found nothing, and why I believed it

Which brings us back to pass four, and that one-line report.

The audit is AI-assisted, and the design matters far more than that fact does. I don't ask one agent to "review the code." Three specialist review agents run in parallel, each with a deliberately narrow brief:

ANR & threading — main-thread blocking, deadlocks, races, anything that freezes the UI.
Memory & leaks — retained contexts, unclosed resources, listener leaks, allocation storms.
Logic & state — the intent-versus-behavior gap: wrong queries, broken state machines, edge cases at boundaries.

Each one reads the codebase end to end — this run covered 355 source files across the UI, domain, data, and core layers — but reports only inside its lane. Then every claim gets cross-validated against the actual code before it's allowed into the report. It's the reason you don't ask one doctor to be your cardiologist, neurologist, and radiologist in a single visit: a reviewer with one thing to look for goes deep, and a reviewer with everything to look for skims. Attention is a budget, and that's as true for agents as it is for humans.

It matters double with AI in the loop. Review agents are extraordinary at surfacing suspects and — like every reviewer, human or not — perfectly capable of being confidently wrong. The agents get to propose. Evidence gets to decide.

So when pass four came back clean, here's what that sentence actually meant:

The same three specialist lanes that had found real, test-proven bugs in earlier passes re-swept all 355 files and found nothing new that survived cross-validation.
Every earlier finding was pinned under a regression test that started red and stays green.
The bug class static review can't reach — OS-level alarm delivery across reboot and deep Doze — was verified on physical hardware, twice.

"No verified bugs found" from that process is a statement with teeth. The same words from a process with no bug definition, no reproduction discipline, and no device in hand are just a wish, formatted as a deliverable. A hunt that has never drawn blood can't certify there's none to draw.

What to ask of your next audit

If you're paying for a code audit — or reading one an outgoing team left behind — four questions separate the real ones from the theater:

What counted as a bug? If the report mixes style opinions with defects, the signal is buried.
Can each finding be reproduced? Ask for the failing test, not the paragraph.
Did anything run on a real device? Lifecycle, alarms, Doze, reboots — the OS keeps its sharpest edges off the whiteboard.
Has this process found real bugs before? That's the one most people forget to ask, and it's the one that decides whether the other three answers mean anything.

That's the work I do: auditing and rescuing mobile codebases — Android, iOS, Flutter — with a process built to find what's actually broken, prove it, and pin it down so it stays fixed. If someone just handed you a suspiciously clean report, or you suspect your app has bugs nobody has managed to catch, it's worth a second pair of eyes. Mine come with three specialist agents and a drawer full of test phones.

Ramiz Raja is a senior mobile developer and technical lead with 12+ years shipping and rescuing production apps on Android, iOS, and Flutter. More at codebyramiz.com.