Black, Grey and White Box Testing

Three terms appear in almost every proposal, and they describe one thing: how much information the tester is given before they start.

None of them describes how much access the tester gets. Access is a separate decision: whether the tester holds an account, and how many roles. Scoping conversations mix the two up constantly. You can run a black box test with a user account, told nothing about how the system works but given credentials.

The three

Black box

The tester gets a target and nothing else. No architecture notes, no documentation, no source code. They discover the system as an outsider would.

Grey box

The tester gets partial information: typically credentials, a description of the roles, an API specification, and answers to questions asked during the engagement. They do not get source code.

White box

The tester gets everything available: architecture diagrams, source code, configuration, design documents, and access to the people who built it.

The argument for black box

The case usually made is realism: a real attacker knows nothing about your system, so a test that starts from nothing is more representative.

The premise is right, but the conclusion does not follow. A real attacker also has unlimited time, chooses their own scope, sets their own rules of engagement, and keeps whatever they find to themselves. What you are buying is ten or fifteen days of a specialist's attention, and the question is what those days are best spent on.

In a black box engagement, a meaningful share of them goes on discovery: enumerating what exists, working out the technology, mapping the roles by observation. It is skilled work, and a document you already have would remove most of it. Every day spent there is a day not spent on authorisation flaws.

What black box buys is realism about reconnaissance: the part of an attack that is cheapest for a real adversary and most expensive for you to purchase.

When black box is genuinely the right choice

Three cases, and in each one the question being asked is about discovery itself.

• Validating your external perimeter. What can somebody find from the outside, and does it match your inventory? Here the discovery is the deliverable, and it routinely surfaces hosts nobody knew were exposed.
• Testing detection. If part of the point is whether your monitoring notices, information given to the tester changes the exercise. This shades into red teaming.
• A specific external assurance requirement that names it. Uncommon, but it exists.

Why grey box is the sensible default

For most engagements, whether an application, an API or an internal network, grey box gets the most findings per day. The tester is told what the system is for and who the users are, and spends their time on whether the rules hold.

Grey box is what makes authorisation testing possible. Broken access control is consistently among the most common serious findings, and finding it requires knowing which records are supposed to belong to whom. A tester who has not been told the role model can observe that an endpoint returned data, but not that it should not have.

When white box earns its cost

Source-level access changes what is findable. Some flaws are close to invisible from outside and obvious from inside: a race condition between two operations, a cryptographic mistake in key handling, a permission check that exists but is wrong in a case reached only by a particular sequence.

It is worth it when the assurance requirement is high, when the authorisation model is complex enough that reasoning about it beats probing it, or when the consequence of a missed flaw is severe. The cost is higher, because reading code takes time and the tester needs to understand the language and framework.

Proposals frequently conflate a white box penetration test with a secure code review. The first uses code to inform attacks against a running system; the second is a systematic review of the code itself. Establish which one you are buying before you sign.

What actually happens in practice

Most real engagements are grey box with a black box phase at the start: a few days working from outside to establish what is discoverable, then the information is handed over and the substantive testing begins.

That gets you the discovery finding, the valuable part of black box, without spending the whole engagement on it. If a proposal does not describe something like this, ask for it.

The question to ask a vendor

Not "do you do black box testing", which every vendor answers yes to. Ask instead: given this system, what information would you want, and what would you do differently without it?

A tester who has thought about it will tell you what they would spend the first three days on in each case, and roughly what it costs you in coverage. One who has not will restate the definitions.

The related decision, how many accounts and at what privilege levels, is covered in preparing for your first VAPT. What each choice does to the price is in what moves the price of a penetration test.