Your audit score measures the audit day, not the risk
A high third-party audit score certifies documentation and a well-run visit, not that a serious hazard is controlled on an ordinary unannounced day.
The score is a sample, not a census
An audit is a sampling exercise. An auditor spends a day, maybe two, on a site that may run 250 shifts a year. They pull a subset of records, interview a handful of people, and walk a route that someone else chose. From that sample they produce a percentage. The percentage feels like a measurement of your safety. It is a measurement of the sample.
Two things inflate that sample without touching a single hazard. The first is documentation completeness: a program with every procedure written, signed, and filed scores well whether or not the procedure is followed at 2 a.m. The second is preparation. Sites know when the auditor is coming. Housekeeping improves, missing guards reappear, and the one supervisor who runs a tight area is scheduled to host the walkthrough. None of that is fraud. It is the entirely rational behavior of people being graded on a scheduled day.
Conformance is not control
Read what an audit is designed to verify. Under ISO 45001:2018, Clause 9.2, an internal audit checks whether the management system “conforms” to the organization’s own requirements and to the requirements of the standard. Conformance is a comparison between your paperwork and a checklist. It is not a measurement of whether the energy on that press is isolated right now.
This is the gap the score hides. A conformance audit answers “does a documented system exist and match the standard.” Verified risk reduction answers a different question: “is this specific serious-injury exposure controlled during ordinary work.” You can pass the first with a serious, uncontrolled exposure sitting one aisle over, because the exposure was not in the sample and the paperwork describing its control was complete.
The evidence on what scores predict
Here is the part that program vendors rarely quote. When researchers went looking for proof that audit scores actually track safety outcomes, they found thin ground. A systematic review of the measurement properties of occupational health and safety management audits reported that the underlying literature was sparse: content validity was demonstrated for only a handful of instruments, inter-rater reliability was formally tested for very few, and construct validity for almost none (Robson et al., systematic review). A separate validation study of an employer-completed safety assessment found the tool was internally consistent, yet its overall scores “were not predictive” of future workers’ compensation claim outcomes (reliability and validity study).
Translate that: high internal consistency means the questions agree with each other. It does not mean the total agrees with reality. An instrument can be precise and still point at the wrong thing.
Why the walkthrough misleads
The walk-around is the part everyone treats as the “real” check. It is also the most sampled and the most stage-managed. An auditor sees the plant on its best-behaved shift, on a route that avoids the areas nobody wants seen. The hierarchy of controls is a useful lens here: an audit mostly verifies the existence of administrative controls, the paperwork tier that NIOSH ranks among the least reliable because it “require[s] significant and ongoing effort by workers and their supervisors.” A physical guard either exists or it does not. A procedure exists on paper and then lives or dies on an ordinary day when no one is watching.
The unannounced-Tuesday test
Take your three highest-scoring audit areas and ask, for each, one question: if an auditor walked in unannounced on your worst-staffed shift, which findings would change? If the honest answer is "housekeeping, guarding, and PPE compliance would look different," your score measured the audit day, not the control. A score that reflects real risk reduction should be nearly identical whether the visit is booked or a surprise.
What to do with the number
Do not throw the audit out. Use it for what it is: a check that a documented system exists and matches a standard. Then stop treating that check as a safety outcome. Pair it with things a prepared site cannot fake on a scheduled day: unannounced verification of your handful of serious-injury and fatality exposures, incident and near-miss data, and direct observation of whether the isolation, the guard, and the permit hold up when the auditor is nowhere near. A conformance score and a controlled hazard are different measurements. When you let one stand in for the other, you have bought a certificate and called it protection.