What a Scanner Finds, and What It Cannot

Vulnerability scanners are good tools, and they get oversold. Two lists sort out the difference: what a scanner detects reliably, and what it cannot detect at all.

With both in hand you can tell what you are actually buying.

What scanners genuinely do well

Automation beats a person at every one of these.

• Known vulnerabilities in known versions. A scanner identifies software and matches it against published advisories, across thousands of hosts, in an afternoon. No tester matches that scale or that accuracy.
• Missing patches and end-of-life software across a large estate.
• Configuration against a baseline: transport security settings, cipher suites, certificate problems, security headers, default accounts, unnecessary services.
• Vulnerable dependencies. Software composition analysis reads your manifest and flags known-vulnerable libraries. Doing this manually is not realistic on a modern dependency tree.
• Breadth. A scanner checks every host for every known issue without getting bored.
• Repetition. Run it weekly and it catches regressions and newly published vulnerabilities in software you already had.

If you are not scanning continuously you have a gap that no annual test closes: a test describes one day, and new vulnerabilities are published on the others.

What scanners cannot find

These are structural limits. No amount of tuning closes them.

Authorisation flaws

A scanner does not know that invoice 1042 belongs to a different customer than invoice 1041. It has no model of who owns what, so a response containing another customer's data looks exactly like a successful request. Broken access control is consistently among the most common serious findings in web applications, and it is close to invisible to automation.

Business logic flaws

Applying a discount twice. Reaching the final step of a process without the preceding ones. A negative quantity that credits an account. Every request involved is well-formed and individually legitimate, so there is no malformed input to pattern-match. A scanner has no concept of what the workflow is supposed to achieve.

Chained exploitation

Three findings, each moderate on its own, that combine into a full compromise because one yields what the next requires. Scanners report findings as a list of independent items; joining them up is a human step.

Context-adjusted severity

A scanner assigns severity from the technique. The same flaw carries very different risk in a payment system and in an internal room booking tool. Without knowing what the data is and who can reach it, every score is generic. Scanner output ends up alarming and unhelpful.

Anything requiring a sequence

Multi-step workflows, state that depends on prior actions, race conditions between two requests, features only reachable after a specific set-up. Scanners work request by request.

The two error modes

False positives are the visible cost. Raw scanner output contains a large share of findings that are not exploitable in context: a reported flaw in a code path that cannot be reached, an issue already mitigated by a control the scanner cannot see. Triage is real work, and a "report" that is unfiltered scanner output has transferred that work to you.

False negatives cost more, because they are silent. A clean scan reads as good news. All it means is that nothing on the scanner's list of known patterns matched. That says nothing about the authorisation model, the business logic, or anything specific to your application. Most of what a penetration test exists to find sits outside what a clean scan covers.

How to tell what you actually received

Four signals, in order of reliability:

• Findings with no application-specific detail. If every finding could appear verbatim in a report for a different company, a tool wrote it.
• No authorisation findings, and no evidence any were attempted. Either the application is unusually well built, or nobody held two accounts at once.
• Reproduction steps that are tool output. A verified finding shows the request that produced the result and the response that came back. "Scanner detected SQL injection" is a hypothesis.
• Identical severities to the tool's defaults, with no adjustment for what the affected system holds.

Ask at proposal stage: how much of the engagement is manual, and what will the manual portion be spent on? A credible answer names activities: authorisation testing across roles, business logic review of named workflows, chain analysis. An evasive one talks about proprietary technology.

How they fit together

In practice you want both. Scan continuously to catch the newly published vulnerability in software you have run for two years. Test periodically with people to catch the flaw unique to how your application works.

Where the balance sits for a given engagement, and what it does to the price, is covered in what moves the price of a penetration test. What the resulting report should contain is in what is actually inside a VAPT report.