When Your AI Acts, Who Actually Owns It?

Read the written version →

Subscribe: Spotify · RSS feed

Show notes

This episode draws on the Coinmedia insight published in October twenty twenty-six on accountability structures for agentic AI systems. The core argument is that governance models designed for deterministic software do not map cleanly onto systems that can act autonomously — initiating communications, modifying records, committing transactions, or delegating to other automated processes — and that the organisations most exposed are those that have not explicitly named human owners for consequential delegations.

The statistic on production deployment obstacles comes from research published in mid-twenty twenty-six, in which more than three quarters of decision-makers reported significant obstacles moving AI agents from pilots into full production, with a majority citing substantial staff time spent reviewing and correcting agent outputs. The EU AI Act reference reflects the act's full enforcement status as of August twenty twenty-six, including its requirements for documented human oversight mechanisms and audit logs in high-risk system categories. The NIST agent standards initiative referenced was launched in early twenty twenty-six and identifies interruptibility — the practical ability to pause, redirect, or stop an autonomous agent — as a foundational property of trustworthy systems.

The four-tier action classification discussed in the episode — read and report, low-stakes reversible, significant or externally visible, and irreversible or high-consequence — is a practical framework emerging from production deployments, not a regulatory standard. Organisations should adapt the tier boundaries to their own risk appetite, sector requirements, and operational context. The framework's value lies in the discipline of drawing the lines at all, rather than in any specific configuration.

If you are working through accountability design for an AI deployment and would like to discuss it, the contact form on the Coinmedia website is the right place to start that conversation. The article behind this episode is linked above.

Chapters

• 0:08 Cold open
• 0:54 Why this is a structural problem, not a careless one
• 2:41 The three places accountability disappears
• 4:14 A mental model: classify actions by what they cost to undo
• 5:43 The regulatory picture as of now
• 7:06 Four questions worth asking this week
• 8:12 Close

Transcript

Cold open

Picture a Monday morning. A director at a mid-sized firm opens their inbox and finds a customer complaint. The customer was contacted at an unusual hour with incorrect information. When the director investigates, they discover the message was sent automatically — by an AI system that had been given responsibility for outreach.

The system did exactly what it was configured to do. No one had told it not to do this particular thing, because no one had anticipated this particular combination of circumstances. The system behaved correctly, by its own logic. The outcome was wrong.

And now comes the question that most organisations deploying AI have not properly answered: who is responsible? Not the vendor. Not 'the technology team'. One named person, with one clear line of accountability. That question is what this episode is about.

Why this is a structural problem, not a careless one

Let's start with why the accountability question is hard. It isn't hard because organisations are negligent. It's hard because the governance models most of us work with were designed for a different kind of software.

When a human employee acts on behalf of an organisation, accountability runs through a familiar structure. Role definitions. Seniority. Approval chains. Professional liability. Those structures evolved over decades and most of us understand them intuitively.

When an AI system acts, those structures often don't map cleanly onto what happened. The system may have been built by one team, deployed by another, configured by a third. The specific action that caused the problem might have been the product of a chain of automated decisions, none of which individually crossed any obvious threshold. There's no single moment where you can point and say: that person made that call.

And these systems can now do consequential things. They can initiate payments, alter configurations, generate and send communications, approve or reject requests, and delegate tasks to other automated systems. The people who authorised those deployments remain legally and operationally accountable for every action those systems take. The systems themselves cannot be accountable. They simply aren't structured to bear it.

Research published in mid-twenty twenty-six found that more than three quarters of decision-makers reported significant obstacles moving AI agents from pilots into full production. A majority pointed to substantial staff time spent reviewing and correcting agent outputs before they were considered reliable enough to proceed. That overhead exists precisely because accountability is unclear. When an output is wrong, someone has to catch it. And that someone needs to have been identified before the problem occurs, not after.

The three places accountability disappears

There are three specific points where accountability tends to evaporate. Each one has a different character, and each has a different remedy.

The first is at the point of delegation. A task is handed to an AI system, and no named human owner remains responsible for the outcome. The system proceeds. If something goes wrong, there's no clear line back to a person. The remedy is straightforward in principle: every consequential delegation needs an owner's name attached to it before the system acts — not after something fails.

The second is at the handoff between systems. When one automated process passes a task to another — or when an AI agent delegates a subtask to a second agent — the point of transfer is where accountability tends to disappear. Each system can plausibly disclaim responsibility for the other's behaviour. The remedy is explicit ownership at every seam. A named human or team responsible for the chain as a whole, with an escalation path that has been tested before it's needed, not assumed to exist.

The third is at the edge of the original brief. A system configured for one purpose encounters a situation its designers didn't anticipate. It does something technically within its permissions but outside the spirit of its mandate. This isn't a malfunction. It's the natural consequence of giving a system authority without defining its limits tightly enough. The remedy here is harder, because it requires thinking carefully about what the system is actually permitted to do — not just what you expect it to do.

A mental model: classify actions by what they cost to undo

So how do you actually build an accountability structure? There's a practical approach that's emerging from organisations running these systems in production. It starts with classifying the actions your system can take.

The idea is to sort actions into tiers based on two things: how reversible they are, and how significant their consequences are. Then you assign different human oversight requirements to each tier.

Not every action needs human approval before it happens. That would eliminate the operational value of the system entirely. But the classification itself forces a useful conversation that most teams haven't had.

Think of it in four rough levels. At the first level: the system reads and reports. It observes, summarises, flags. It takes no action. At the second level: low-stakes reversible actions — things that can be undone without meaningful cost. Those first two levels can generally run without approval gates. At the third level: significant or externally visible actions — sending a communication, modifying a record, initiating a transaction. Those warrant logging and periodic human review. At the fourth level: irreversible or high-consequence actions — deleting data, committing funds, making a public statement, triggering a legal obligation. Those should require explicit human authorisation before the system executes them.

Every organisation will draw these lines differently. What matters isn't the specific tiers — it's the discipline of drawing them at all. A system deployed without this exercise will eventually act at a level its operators didn't intend.

The regulatory picture as of now

This is no longer just an internal governance question. The regulatory context has arrived.

The EU AI Act entered full enforcement in August twenty twenty-six. For systems classified as high-risk, it requires documented human oversight mechanisms with explicit intervention points. Regulated sectors must maintain audit logs and demonstrate that a human could have intervened at defined moments in any consequential automated process.

Separately, a NIST-led initiative on agent standards, launched in early twenty twenty-six, identifies interruptibility as a foundational property of any trustworthy autonomous system. Interruptibility means the practical ability of a human to pause, redirect, or stop an agent. Notice that's both a technical and an organisational requirement at the same time. A system might be architecturally capable of being paused — but if no one knows who has the authority to pause it, or how, that capability is theoretical.

The governance frameworks coming from multiple jurisdictions are converging on the same underlying principle. The organisation that deploys the system carries the accountability. And that accountability must be demonstrated through documented decision logic, named human owners, and tested intervention paths — not through a general claim that humans are 'in the loop'.

Four questions worth asking this week

Before we get to the close, let me give you four questions. These aren't a compliance checklist. They're the questions that tend to surface only after something has gone wrong. They're considerably more useful when asked in advance.

First: if your AI system takes an action that causes a customer complaint tomorrow morning, who is the named person responsible for that system's behaviour? Not the vendor. Not the technology team in aggregate. One name.

Second: for each type of consequential action your system can take, is there a defined approval threshold? Is that threshold documented, or is it tacit knowledge held by whoever configured the system? And what happens when that person is unavailable?

Third: when your system passes a task to another process — automated or human — what context travels with it? Does the recipient know where the task came from, what constraints it was generated under, and what the system was not permitted to do?

Fourth: has anyone tested the stop mechanism? Not whether it exists. Whether the right person can actually invoke it within a reasonable time window when something unexpected occurs.

Close

Here's the one thing I'd ask you to hold onto from this conversation. Accountability design and technical deployment are not two separate phases. They're the same work. A system whose accountability is undefined will eventually act in a way that makes the definition urgent — and urgency is a poor condition for clear thinking.

The organisations that navigate this well share one discipline. They name owners at the point of delegation. They define action tiers before configuration. They document escalation paths and actually test them. And they review the boundary conditions of what the system is permitted to do on a regular schedule — because those boundaries drift as the system is updated, as the business changes, and as the original designers move on.

This isn't an argument against deploying autonomous systems. The operational value of well-designed AI agents is real. It's an argument for treating accountability as part of the deployment, not as an optional addendum to it.

The question to take back to your team this week: is there a place in your current operations where an AI system acts and accountability is genuinely unclear? If there is, that's worth a conversation. It tends to be a more tractable problem than it first appears. If you'd like to talk it through, you know where to find us.

Each episode is written from that week's Coinmedia insight and voiced with a synthetic model of Zsófia's own voice. The thinking, the editorial line and the approval to publish are human.