Measuring patch compliance without lying to yourself

Every patch programme eventually gets asked for a number. The number that usually gets given is the count of open vulnerabilities, and it is close to useless.

It moves mostly with what the scanner found this month. Add a new scan target and the number rises while your posture improves. Retire a legacy box and it falls while nothing changed. It measures the scanner, not the work.

Here are four measures that hold up, and why each one is hard to game.

1 · Median time from detection to fix, by severity

Not the mean. A mean breaks on three outliers, and there are always three outliers — the box that needed a hardware replacement, the system in a change freeze, the owner on leave.

Segment it by severity, because a single blended figure hides the only thing worth knowing: whether the critical ones move faster than the rest. If critical and medium have the same median, severity is not driving your queue and something else is.

This is the measure of coordination itself, and I have not seen it on a dashboard anywhere.

A window that was agreed for the 14th and happened on the 14th is a process that works. One that was agreed for the 14th, moved twice and happened on the 3rd of the following month is a process that produced the same end state through three times the effort — and the vulnerability was open for three extra weeks.

Both look identical in a "patched / not patched" report.

3 · Last-patched date per host, as a distribution

Not an average. A histogram: how many hosts were last patched within 30 days, 60, 90, 180, more.

The tail is the finding. An estate where 95% of hosts are current and 5% have not been touched in a year is not 95% healthy — the 5% is where an incident will start, and an average hides it completely.

This also has the property of being hard to argue with. A host either has an install date or it does not.

4 · Windows postponed, with the reason

The count matters less than the reason code. Three reasons are worth separating:

Postponed for a freeze period — a planning failure. Freezes are known months ahead.
Postponed because the owner did not respond — an ownership failure. Go back to the ownership map.
Postponed because of a collision — actually a success. The check caught something.

A single "postponed" count tells you nothing. Split three ways it tells you which of three different problems you have.

Three measures to stop reporting

Open vulnerability count. Covered above.

Percentage compliant. The denominator is doing all the work, and nobody looking at the slide knows what it is. "98% compliant" against a scope that excludes the estate's oldest third is a number that will not survive being asked about.

Number of patches deployed. It rewards volume. An organisation that installs a thousand low-severity updates and leaves four critical ones open reports extremely well on this.

The measure an auditor will actually ask for

Not a metric at all. It is: take one critical vulnerability from six months ago and show me the chain through to the fix.

Detection date, the ranking it was given, the window proposed, the owner's approval, the execution record, the outcome. If producing that takes three systems and a person who remembers, the metrics on the slide were describing something other than the process.

Where to start

Pick two: the median time to fix by severity, and the last-patched distribution. Both come from data you already have, both are hard to game, and between them they answer "are we fast enough" and "is anything being left behind" — which are the only two questions anyone is really asking.