A Control That Isn't Tested Is a Hope

Category: Operating Models Author: Cody Swidler Tags: control testing, control effectiveness, evidence, attestation, audit readiness, assurance

An internal audit I sat through years ago opened with a simple request: show us the last three executions of the quarterly access review — a control the register rated "Highly Effective." The control owner was confident. The evidence search took two days. What it eventually produced was one completed review, eleven months old, covering two of the seven in-scope systems. The control existed in exactly one place: the spreadsheet that described it.

Nobody had lied. That's the part worth sitting with. The owner sincerely believed the control was operating, the rating had been carried forward at every annual refresh, and each carry-forward felt reasonable because it had been rated Highly Effective the year before. That's how attestation culture works: a rating, once written down, becomes its own evidence. The register stops describing the control environment and starts describing the organization's memory of it.

Attestation Is Not Assurance

Most control monitoring is a survey. Once a year, owners are asked whether their controls are working; owners say yes; the ratings roll forward. The problem isn't dishonesty — it's that the question measures confidence, not operation. An owner who inherited the control eight months ago, whose team reorganized twice since the procedure was written, will still answer yes, because as far as they know it's true, and because answering no invites work.

Testing replaces the question with a demand: show me. Show me the last three executions. Show me the exception that got caught and what happened to it. Show me the alert firing and the ticket it opened. The unit of assurance is the artifact, not the assertion — the same principle that separates a data-driven risk assessment from an opinion survey. And the moment you start demanding artifacts, ratings start moving, almost always downward, toward the truth.

Test Where the Risk Is

The objection is always capacity: nobody can deep-test three hundred controls a year. Correct — and nobody should. Testing effort should follow the risk the control is holding down. The controls standing between you and your Critical residual risks earn full operating-effectiveness testing on a real sample. Mid-tier controls earn a lighter evidence check — one recent artifact, inspected, not attested. The long tail earns rotation, a slice each year. If your controls are mapped to your risks — the connection at the heart of the connected GRC model — this prioritization falls out of the register for free: the risk-weighted list of what to test next is a query, not a planning meeting.

The cadence matters as much as the depth. A control tested once, at audit season, under deadline pressure, tells you what the control looked like that week. Spreading the same testing effort across the year — a few controls a month, owned as routine work — produces the same coverage, catches degradation while it's fresh, and deletes the annual evidence fire drill. Continuous, boring, and scheduled beats heroic and annual every time.

Let the Findings Move the Numbers

Here's the discipline that separates programs that learn from programs that file: when a test fails, the control's effectiveness rating changes that day — not at the next annual refresh. And if residual risk is calculated from control effectiveness, as I've argued it should be in how to build a risk register, the failed test propagates automatically: weaker control, smaller reduction, higher residual, and a risk that may have just crossed an escalation threshold. The test finding stops being a paragraph in an audit report and becomes a moved number that someone accountable has to look at.

Run the loop in reverse, too. When an incident occurs, ask which control should have prevented or caught it, and test that control now — out of cycle, while the failure mode is visible. Incidents are free control tests with perfect realism; most programs waste them by handling the incident and leaving the control rating untouched.

Evidence Is a Supply Chain

The operational failure behind most testing programs isn't the testing — it's the evidence. The artifact exists, somewhere, in a screenshot folder or a departed employee's inbox, and every audit becomes an archaeology project. Treat evidence as a supply chain instead: every control declares what artifact it produces, who owns producing it, on what cadence, and where it lives. Then track currency the way you'd track any other due date — collected, due soon, overdue. Do that and audit prep stops being a season; the evidence an auditor will ask for in March was filed in the ordinary course of business in January. This is the discipline a unified control library makes cheap, because each control's evidence expectation is written once, next to the control, mapped across every framework that will ever ask for it.

The Cultural Shift Is the Point

The mechanics are simple; the shift is cultural. In an attestation culture, a downgraded control rating is an accusation, so ratings never move. In a testing culture, a downgraded rating is a finding — no-fault, evidence-backed, attached to a remediation with an owner and a date. You're not asking owners to confess weakness; you're asking the program to describe reality, because every consumer of the register — the residual calculations, the board report, the audit response — is only as honest as the effectiveness column.

A control that isn't tested is a hope with documentation. Hopes are fine; every program carries some. What's not fine is a register that can't tell you which of its ratings are evidence and which are hopes — because the difference between those two is exactly the information an auditor, an examiner, or an incident will eventually extract, on a day and in a format you don't get to choose.

Cody Swidler is the founder of PivotRisk and a Principal Program Manager, Enterprise Resiliency at Apex Clearing. He has built and scaled GRC, resilience, and risk programs across Microsoft, Twilio, Box, Zayo, and Miro.