Who Actually Set the Rules Your AI Is Following?

Read the written version →

Subscribe: Spotify · RSS feed

Show notes

This episode draws on Coinmedia's analysis of how AI mandates form — or fail to form — in commercially active organisations. The core observation is that an AI system's effective mandate exists whether or not anyone wrote it down: the system behaves as though decisions about scope, authority, escalation and accountability have been made, even when they have not been made deliberately.

The regulatory reference in this episode concerns the EU AI Act. The high-risk AI obligations discussed — including requirements for documented human oversight, log retention, and operation in accordance with documented instructions — came into full enforcement in August twenty twenty-six. The Act places statutory obligations on deployers of qualifying systems regardless of whether those systems were built in-house or procured as a service. Organisations uncertain about whether their systems meet the high-risk threshold should seek appropriate legal or compliance guidance.

The point about context windows is intended to be accessible rather than technical. The practical implication is that AI systems can produce confident-sounding outputs in situations where they lack the information to distinguish a familiar case from a superficially similar but meaningfully different one. A designed mandate builds in escalation conditions that account for this; an assumed mandate does not.

If any of the questions raised in this episode are live ones for your organisation — particularly around mandate definition, review cadence, or accountability when a system behaves unexpectedly — the Coinmedia contact form is the right starting point. The weekly insight that accompanies this episode goes into additional detail on the structural and proportionality considerations.

Chapters

• 0:08 Cold open
• 0:54 The mandate that nobody wrote down
• 2:34 Why the stakes are higher now than they used to be
• 3:57 What a designed mandate actually contains
• 5:41 The invisible gap inside the system itself
• 7:00 The review cadence — and why once is not enough
• 8:01 Close

Transcript

Cold open

Picture this. An AI system has been running inside a business for about six months. It is doing its job. Nobody is complaining. And then someone — a founder, a department head, maybe a client — notices something unexpected. The system sent a communication it probably should not have sent. Or it filtered something out that it had no business filtering. Or it made a call that nobody authorised it to make.

The first instinct is to ask: why did it do that? But that is not quite the right question. The right question, once you sit with it for a moment, is this: who told it what it was allowed to do in the first place? And in most cases, the honest answer is — nobody did. Not clearly. The episode today is about what happens when that question goes unanswered, and what it looks like to actually answer it.

The mandate that nobody wrote down

Every AI system that operates inside a business has what you might call an effective mandate. That means the set of things it will do, the way it prioritises between competing options, the situations in which it will act without prompting, and the points at which it will pause and wait for a person. Whether or not anyone wrote this down, it exists. The system behaves as though someone made those decisions deliberately.

The problem is that when nobody did write it down, the system fills in the gaps from somewhere else. It inherits from its training. It picks up cues from the way the prompt was phrased. It takes its shape from the first few edge cases that got handled informally. And it carries whatever assumptions were baked in during the initial setup. None of that is necessarily wrong. But none of it was chosen for your context, your clients, or your risk tolerance. It is simply whatever the path of least resistance happened to produce.

That distinction matters more than it might sound. A mandate that someone designed tells the system what to optimise for, what to protect, when to escalate, and what to refuse. A mandate that nobody designed tells the system none of that. It just runs. And here is a useful test: if you cannot point to something — a document, a written record, anything — that answers four basic questions about a given AI system, the mandate is assumed rather than designed. Those four questions are: what is it for, what is it not for, what may it do without a person reviewing the output, and who is accountable when it behaves unexpectedly?

Why the stakes are higher now than they used to be

For a long time, this was a manageable problem. If a system was only drafting subject lines or scoring leads, a little drift from its intended behaviour did not cause much harm. The outputs were easy to check. The consequences of an error were limited.

That situation has changed significantly. AI systems are increasingly being asked to take actions, not just produce drafts for a person to review. Sending communications. Routing enquiries. Filtering candidates. Adjusting pricing. Managing workflows. The gap between what the system does and what a person actually verifies has widened considerably. And that changes the risk profile of an assumed mandate rather dramatically.

There is a regulatory dimension to this as well. The E U AI Act's obligations for high-risk A I systems came into full enforcement in August twenty twenty-six. For organisations operating qualifying systems, that means formally assigning human oversight, retaining logs, and running systems in accordance with documented instructions. The accountability sits with the deployer — meaning your organisation — regardless of whether you built the system yourself or purchased it as part of a platform. Even for systems that do not meet the high-risk threshold, the underlying principle is worth holding onto: if you cannot describe what your A I system is authorised to do, you are not in a position to know whether it is actually doing it.

What a designed mandate actually contains

A mandate does not need to be a lengthy governance document. It is a set of deliberate decisions, recorded in a form that can be reviewed and updated. There are four areas those decisions need to cover.

The first is scope. What tasks is the system designed to handle — and, equally important, what tasks are explicitly outside its remit? A system with no explicit exclusions will attempt to be helpful wherever it can. That sounds fine until it produces an output in an area where it has no reliable knowledge or no authority to act.

The second is authority. Which outputs can the system deliver without a person reviewing them first? Which require confirmation before anything actually happens? And is that line drawn in the right place, given what the consequences of an error would be? Systems that act on behalf of a business — communicating with clients, updating records, triggering workflows — need their authority level stated explicitly, not inherited by default.

The third is escalation. Under what conditions should the system pause and refer the matter to a person? This is not just a safety question. A well-designed escalation condition is also a quality filter. It identifies the situations where the system is operating outside its reliable range and routes those situations to someone who is equipped to handle them.

The fourth is accountability. When the system produces an unexpected outcome, who is responsible for investigating it, deciding how to respond, and updating the system's behaviour going forward? If that accountability is spread across whoever built it, whoever uses it, and whoever commissioned it, nothing will be corrected systematically. Diffuse accountability is effectively no accountability.

The invisible gap inside the system itself

There is a technical point worth understanding here, without needing to go deep into the engineering. Every A I system reasons over a defined window of information at any given moment — the data and instructions it has been given for that specific task. What sits outside that window is, in a practical sense, invisible to the system.

This creates a predictable failure mode. A system can behave well in the situations it was tested on, then produce a poor output in a situation that looks similar but differs in a way that matters. Not because the system is broken. But because the information that would distinguish the two situations was not in the window it was working from.

A well-designed mandate accounts for this. It does not assume the system will always know what it needs to know. It builds in checkpoints — points at which the system is instructed to verify it has sufficient context before proceeding, rather than producing a confident output based on an incomplete picture. Without those checkpoints, a system can generate polished, fluent outputs regardless of how much it actually knows about a given situation. The problem does not show up in any single output. It accumulates quietly across many interactions. The signal is often diffuse: outputs that are technically correct but slightly off, recommendations that miss something contextual, communications that do not quite fit the situation.

The review cadence — and why once is not enough

Here is something that is easy to overlook. A mandate that is defined once and never revisited is only marginally better than no mandate at all. The environment a system operates in keeps changing. The data it receives shifts. The tasks it is asked to handle expand. The regulatory context evolves. And the organisation's own risk appetite may look different six months in than it did at the start.

Sensible practice is to build a review cadence that is proportional to the consequences of the system's outputs. A system that drafts internal summaries might warrant a quarterly check. A system that communicates directly with clients, or that influences commercial decisions, warrants closer and more frequent attention.

The review does not need to be technically deep. The most important questions are organisational, not algorithmic. Is this still doing what we intended? Has its scope drifted since we set it up? Do the people using it understand its limits? And does accountability for its outputs sit with someone who has both the authority and the information to act if something goes wrong? Those are the questions that matter.

Close

So here is the one thing to take from this episode. Every A I system you operate has an effective mandate — a set of rules it is following about what it is allowed to do. The question is not whether that mandate exists. It does. The question is whether someone in your organisation made those decisions deliberately, or whether the system made them by default.

The organisations that handle this well are not necessarily the ones with the most sophisticated A I. They are the ones that treat the mandate as a management decision — something that belongs in the same category as deciding who can approve a contract or sign off on a proposal. Not something that was answered implicitly when the system was set up.

So the question to bring back to your team this week is a simple one. For each A I system currently operating in your business — pick one, start there — can you point to something written down that says what it is authorised to do, what it is not authorised to do, and who is responsible when it behaves unexpectedly? If the answer is no, that is worth a conversation. If you would like to have that conversation with us, you know where to find us.

Each episode is written from that week's Coinmedia insight and voiced with a synthetic model of Zsófia's own voice. The thinking, the editorial line and the approval to publish are human.