Line Coverage Is Not Verification Coverage

Line coverage measures whether code ran. It does not measure whether anyone asserted the code was correct. Those are different numbers, and only one of them survives an audit.

Most verification organizations can report line coverage to two decimal places and cannot answer a simpler question: which parts of the implemented behaviour has nobody written a test for? The reporting tools measure execution because execution is easy to instrument. Intent is not.

From the standard

“Verification and validation (V&V) processes are used to determine whether the development products of a given activity conform to the requirements of that activity and whether the product satisfies its intended use and user needs.”

Why the two numbers diverge

A line executes whenever any test touches it. A test that exercises an initialisation path incidentally marks dozens of lines as covered while asserting nothing about what those lines do.

The result is a familiar and dangerous state: high line coverage, low behavioural coverage, and a team that believes the first number describes the second. It usually surfaces when a defect escapes into a release through code that the coverage report showed as fully exercised.

The third question is where the risk lives

Requirement coverage assumes the requirements are complete. In practice, systems accumulate behaviour that no requirement describes: an interlock added during integration, a timeout adjusted to resolve a field issue, a fallback path introduced to work around a supplier component.

Each is real, each ships, and none appears in a requirements-driven coverage report — because the report can only measure against what was written down. Behaviour that entered the system through a change request rather than a specification is invisible to it.

Computing the gap

Answering the third question is arithmetic once you have the two inputs. Enumerate what exists — every interface element and function in the implementation, across every configuration variant the product ships in. Then subtract everything an existing test exercises. The remainder is the gap.

The difficulty is entirely in the first input. A hand-maintained inventory of interface elements is wrong within a sprint, for the same reason a hand-maintained traceability matrix is. The inventory has to be derived by parsing the implementation itself, so it is regenerated rather than updated.

The second input is available if tests record which elements they exercise at the point they are authored, rather than in a separate document. Where that provenance exists, the subtraction is a query.

The configuration multiplier

In a single-configuration product the gap is a list. In a product that ships across variants — different market regions, optional subsystems, hardware revisions — coverage is not one number but a matrix, and a behaviour verified on one variant may be untested on four others.

This is where manual tracking becomes genuinely impossible rather than merely tedious. The variant count grows multiplicatively while the team grows linearly, and the usual response — verify the most common configuration thoroughly and spot-check the rest — is a defensible engineering trade-off that is very hard to defend to an auditor, because nobody can state which behaviours it left uncovered.

Triage, because the raw list is unusable

The first scan of a mature system returns thousands of gaps. Handed to a team unranked, that output is ignored — correctly, because the list contains diagnostic counters and internal state alongside the interlock nobody tested.

Ranking is what makes it a work queue. Signals participating in safety functions, elements referenced by requirements carrying a higher integrity level, and behaviour that changed recently all rank above a counter that has been stable for three years. Model-assisted triage can propose that ranking; an engineer should confirm it, because the cost of mis-ranking a safety-relevant gap is asymmetric.

What the number is for

Not a target. A coverage percentage adopted as a goal becomes a metric people optimise, and the cheapest way to raise it is to write shallow tests against elements that were never risky.

Its use is directional: knowing where the untested behaviour concentrates, so verification effort goes where the risk is rather than where the last defect happened to be. That is a scheduling input for an engineering lead, not a KPI for a dashboard.