Testing, auditing, and changing policy

Testing what should be denied

The first safeguard is a set of tests that runs on every change to the policy. Policy tests read best as a table, with one request per row and the decision it should get:

const at = (time: string, managed = true) =>
  ({ time: `2026-06-14T${time}Z`, device: { managed } });

// subject, action, resource, context, expected decision
const cases: PolicyCase[] = [
  [omar, 'photo.download', photo9002,       at('14:05:00'),        'permit'],
  [omar, 'photo.download', photo9002,       at('13:59:59'),        'deny'],
  [omar, 'photo.download', photo9002,       at('14:00:00'),        'permit'],
  [omar, 'photo.download', photo9002,       at('14:05:00', false), 'deny'],
  [omar, 'photo.download', publishedPhoto,  at('14:05:00', false), 'permit'],
  [maya, 'photo.download', photo9002,       at('14:05:00'),        'deny'],
  [lena, 'photo.view',     photo9001,       at('14:05:00'),        'permit'],
  [lena, 'photo.view',     photo7310,       at('14:05:00'),        'deny'],
  [omar, 'photo.approve',  mayasDraft,      at('14:05:00'),        'permit'],
  [omar, 'photo.approve',  omarsDraft,      at('14:05:00'),        'deny'],
  [omar, 'photo.delete',   photo9001,       at('14:05:00'),        'permit'],
  [omar, 'photo.delete',   heldSportsPhoto, at('14:05:00'),        'deny'],
];

for (const [subject, action, resource, context, expected] of cases) {
  test(`${subject.id} ${action} ${resource.id} at ${context.time}`, async () => {
    const result = await decide({ subject, action, resource, context });
    assert.equal(result.decision, expected);
  });
}

Half the rows expect a denial, and that is deliberate. A suite of allowed requests alone would pass against a policy that permits everything, while the mistakes that matter most are requests that should have been refused. Most deny rows also sit next to an allow row that differs in one value. The 13:59:59 and 14:00:00 rows pin down exactly when the embargo ends, which is where a comparison that is off by one would hide, and the managed and unmanaged rows show that the device, and nothing else, made the difference.

The pairs catch a quieter mistake too. The held photo in the last row is in the Sports library, where Omar can otherwise delete photos. Had the test used a held flood photo from the News library, where Omar holds only Viewer, the request would be refused anyway, and the test would keep passing even if someone removed the legal-hold rule. A deny test proves something only when the rule under test is the reason for the denial. Because each decision names the rules that decided it, the suite can check that as well, and fail when some rule never decides any case. For version 15, the News desk adds its own pair: an unpublished photo downloaded on a managed laptop in a session rated low, allowed, and the same download in a session rated high, not permitted.

Trying a change safely

Tests show that the policy does what its authors meant, not how the newsroom actually works. So the newsroom first runs version 15 in shadow mode. For a week, the decision point evaluates both versions for every real request. Version 14 still decides, and wherever version 15 would have decided differently, the request is recorded with both answers. Version 15's obligations are not carried out, its reasons are never shown to anyone, and it is evaluated off the request's critical path so that it adds no delay.

Version 15 would have refused 412 downloads that version 14 allowed. Almost all came from sports editors traveling with the teams, who sign in from hotel and stadium networks that the risk system often rates high. Nobody had thought of them while writing the rule. The desk changed it to ask for a fresh passkey sign-in instead of refusing, the step-up described in the lesson on context, devices, and risk, so a traveling editor can prove who they are and someone holding only a stolen laptop cannot. A second week of shadow mode showed only the differences the desk expected.

Version 15 was then enforced in stages: first in the web app, where the News desk could watch the results closely, and a few days later in the mobile app and the API. Throughout, every decision carried the version that made it, as in Decision and enforcement points, so when an editor complains on Thursday that a download failed, the record says which version refused it. If version 15 turns out to be wrong, rolling back means publishing version 14 again from the administration point. No handler changed when version 15 arrived, so none has to change when it leaves, and the rollback takes minutes rather than a deployment.

Recording decisions

Enforcing access at an API described what a record of an API refusal holds. A record of a policy decision keeps those fields and adds what is needed to understand why the policy decided as it did: the policy version, the rules that decided, the attribute values those rules read, and the reason. Here is the record of Omar's 13:30 attempt on photo 9002:

{
  "time": "2026-06-14T13:30:00Z",
  "request_id": "req-9c41",
  "subject": "user-1187",
  "client": "gazette-web",
  "action": "photo.download",
  "resource": "photo/9002",
  "decision": "deny",
  "reason": "embargoed",
  "policy_version": 14,
  "rules": { "permit": ["editor-unpublished"], "forbid": ["embargo"] },
  "attributes": {
    "subject.roles": ["picture-editor"],
    "resource.status": "approved",
    "resource.embargo_until": "2026-06-14T14:00:00Z",
    "environment.device.managed": true
  }
}

The attribute values are the ones that mattered and nothing more. The deciding rules read Omar's role, the photo's status and embargo, and whether the laptop was managed, so the record holds those. It does not copy Omar's profile or the rest of the photo record, and it never holds the access token, session cookies, or the full request: a record that contains credentials becomes something worth stealing, and one that contains whole requests is mostly noise.

The newsroom records every denial, because refusals are comparatively rare and often the first sign of a problem. It also records every sensitive permit: any download of an unpublished photo, anything involving a held photo, every role change, and every use of emergency access. Recording every thumbnail a sports editor scrolls past would bury those among millions of routine permits. Records are kept for a set period that matches how long investigations and legal questions need them, then deleted. They describe what people looked at and when, which makes them personal data in their own right, so reading them is limited to the people who need to and is itself recorded.

Explaining a denial

The record is for the newsroom. Omar needs a message that says what happened and what to do about it. "This photo is embargoed until 10:00 local time. You can download it after that." is a good message for Omar. It is true, it gives the time the way the newsroom reads it, it says what to do, and it reveals nothing new, because the sports desk's editors can see the photo and work with its embargo every day. "Downloads of unpublished photos need a newsroom-managed device" serves just as well when the laptop is the problem.

Those messages work because Omar can already see photo 9002. Lena, asking for a photo from the News library, cannot, and gets the response that Enforcing access at an API chose for albums a caller cannot see: a 404 that looks exactly like the answer for a photo that does not exist, with no reason and no hint of an embargo.

Everything else stays in the record: the rule names, the policy version, the attribute values, and how close a request came to passing. Shown to the person refused, those details become instructions for getting around the policy. A message explaining that the session's risk was rated too high would invite someone to retry from different networks until the rating dropped, and "denied by rule editor-unpublished" tells a stranger how the policy is organized. The photo API turns reason codes into messages from an approved list, and any reason not on the list gets a plain refusal. Including the request ID in the message lets Omar ask Theo about a confusing refusal, and lets Theo find the record without either of them guessing.

Emergency access

Late on a Saturday night, the Gazette receives a court order to produce the original file of one of the held flood photos by eight the next morning. Raj, on the legal team, can see the photo but cannot download it, because the Legal reviewer role grants only photo.view and photo.place_hold. Theo and the deputy administrator, who could assign a role, cannot be reached, and a role that includes downloads would grant far more than one file for far longer than one night.

Break-glass access is a planned path for this situation: the normal way of getting access has failed or cannot happen in time, and someone needs a specific permission now. The important word is planned. The path is designed, written into the policy, and tested in advance, not improvised at midnight by whoever has access to the database. In the Gazette's policy it is one more rule:

# emergency-download
permit photo.download
  when subject.emergency_grant.photo is resource.id
   and environment.time is before subject.emergency_grant.expires
   and subject.emergency_grant.approved_by is not subject.id

Raj requests a grant for that one photo, for two hours, and writes down the reason, including the court order's reference. Emergency access usually skips approval beforehand and relies on the stated reason, an alert and a review afterwards, as Temporary, privileged, and emergency access describes. The Gazette adds a quick second person, a variant some organizations choose when a few minutes' wait is acceptable: the duty editor on call approves the grant, and the last condition in the rule means nobody can approve their own. Theo and the head of legal are notified the moment it is approved, not in a weekly summary, and every download under the grant is recorded as a sensitive permit. On Monday, they review what was downloaded and whether the emergency was real.

Two signs show whether break-glass access is healthy. It should be rare: if editors reach for it every week, the normal path is broken, and that is where the fix belongs. And it should be noticed every time, because an emergency path that can be used quietly is a back door with a dramatic name.

Monday's review of Raj's grant is a small instance of a habit that matters well beyond emergencies: looking back at who holds access and why, and removing what is no longer needed. Access reviews takes that up as part of identity governance. The decision records that made Monday's review possible are also where an investigation begins when something has gone wrong, which Investigating an account compromise follows.

Every decision in these lessons trusted that the Gazette's role assignments and attributes were right. Keeping them right as people join, change jobs and leave, and proving that they are, is the work of identity governance. Continue to Why access needs governance.