All Insights
6 min read

The Document Was Never the Point

A State CISO and caregiver on why policy gaps aren't compliance failures — they're systems design failures hiding in plain sight.

cybersecurity leadershiprisk managementpolicy design
JW

Jason Walker

State CISO, Florida

The discharge instructions were three pages, single-spaced, and completely correct. The medication schedule was laminated and taped to the refrigerator at eye level. The care plan had been reviewed with two nurses, a social worker, and the attending physician. Every box was checked. Every protocol was documented.

And within six weeks, none of it was being followed.

Not because anyone was negligent. Not because the instructions were wrong. But because the system that produced those documents had no model for what daily life actually looks like when you're a family caregiver managing a stroke recovery at home, tired, under-resourced, and making real-time decisions that the laminated sheet never anticipated.

I watched this happen up close. And I have spent years watching the exact same failure mode play out across 35 state agencies and 202,000 devices.

The document was never the point. It was a signal that somebody, somewhere, thought hard about the right answer. But a documented answer and an operational answer are two completely different things.


What Most People Get Wrong

The standard response to a policy gap is to fix the policy. Rewrite the control. Schedule a training. Send a reminder. Close the finding in the tracking spreadsheet.

This treats the symptom as the disease.

When you find a security control that exists on paper and fails in practice, the instinct is to ask: who isn't following the policy? That's the wrong question. The right question is: what is making the policy impossible to follow under real conditions?

These are not the same investigation.

The first question leads you to a compliance response: more audits, more attestations, more documentation of documentation. The second question leads you to a systems design response: remove the friction, change the workflow, or admit that the policy was written for a world that doesn't exist.

Most organizations never get to the second question. They document their way deeper into the problem.


The Core Insight

Policy adherence isn't a behavior problem. It's a friction problem.

When I look at security control failures across our enterprise, the pattern is consistent. The controls that fail aren't failing because people don't care. They're failing because the gap between what the policy requires and what the daily workflow supports is wide enough that adherence takes active, sustained effort against the grain of everything else the person is trying to accomplish.

A state employee processing 200 transactions a day isn't thinking about your access control matrix. They're thinking about the constituent on hold and the supervisor asking for a status update and the system that keeps timing out. Your policy lives in a document. Their reality lives in that moment.

The care plan I watched fail had the same design flaw. It was written by clinicians optimizing for medical outcomes in a controlled environment. It was executed by a family caregiver optimizing for survival in a chaotic one. Nobody designed for the handoff. Nobody asked what happens when the caregiver hasn't slept in three days and the medication window is tight and the instructions say "consult your physician if uncertain."

You don't consult your physician at 2 AM. You make a call and you move on.

Security operations run the same way. Your incident response playbook might be excellent. But if triggering it requires four approvals and a ticket in a system that half your agencies don't have access to, your analysts are going to make the call and move on. And they should. The playbook failed them, not the other way around.


What I See at Scale

Managing cybersecurity across a state enterprise surfaces this problem in ways a single-organization CISO rarely sees. When you're working across 35 agencies with different cultures, different toolsets, different workforce profiles, and different interpretations of the same policy, you stop seeing policy failures as individual compliance issues very fast.

You start seeing them as data.

When the same control fails across 12 agencies in 12 different ways, you don't have 12 compliance problems. You have one systems design problem that is expressing itself 12 different ways. The policy doesn't fit the operational reality of the people being asked to follow it.

Our approach has shifted accordingly. We spend less time asking "are agencies following the policy?" and more time asking "what would have to be true for this policy to be easy to follow?" Those are different analytical tasks. One produces audit findings. The other produces operational improvements.

The FAIR risk quantification work I do in my doctoral research reinforces this. When you model loss exposure, you're not just modeling threat frequency and asset value. You're modeling control effectiveness. And control effectiveness isn't binary. A control either works under real conditions or it doesn't. A control that exists but fails under load isn't a control. It's a liability, because it creates the appearance of risk reduction without delivering it.

A policy your teams can't follow isn't a 50% control. It might be a negative control, because it consumes compliance resources, creates audit noise, and builds the false confidence that the risk has been addressed.


What You Should Do Differently

Start with the friction audit, not the policy review.

For any critical control that shows recurring adherence failures, go find someone actually doing the work and ask them to walk you through their day. Don't ask why they're not following the policy. Ask what's making it hard. Watch what they skip and why. Watch where the workflow breaks down.

You will find one of three things:

  1. The policy asks for something technically impossible in their environment.
  2. The policy asks for something that conflicts with another priority they're being held accountable for.
  3. The policy asks for something that made sense when it was written and no longer maps to how the work actually happens.

None of these are solved by better documentation. All of them are solved by redesigning the system.

The second shift is harder but more important: separate your evidence of compliance from your evidence of effectiveness. These are not the same metric. An agency can be fully compliant on paper and operationally exposed. An agency can have policy gaps and strong operational practice. You need to know which one you're actually looking at.

The discharge instructions were correct. The laminated schedule was legible. The care plan was documented and signed.

And the system still failed, because nobody designed for the reality on the other side of the door.

That's not a caregiving failure. That's not a cybersecurity failure. That's a design failure. And it has the same fix in both domains: get close enough to the actual work to see where the system breaks down, and then fix the system.

The document was never the point. The behavior it needed to produce was the point. Build for that.


Related

  • Vault: [[HOME|Home]]