What should we never delegate to AI?

Companies are being told to prepare for AI agents: automate more, move faster, increase capacity, reduce cost.

The promise is real. But the first question may not be how much more AI can do.

It may be this:

What should we never delegate to AI?

The obvious answer is to make a list: strategic decisions, sensitive judgments, final responsibility.

But that may still be the wrong level of analysis.

Because the real issue is not only what we delegate. It is whether the organization remains able to understand, govern and recover what it delegates.

That is an architectural problem.

Think of a building. You do not make it stronger by adding more rooms, more technology or more access points. You design what carries the load, where access begins and ends, what happens when a system fails, where the emergency exits are, and what must keep working when something else stops.

The same may become true for the augmented company.

The challenge is not to add governance to an organization that is becoming agentic. It is to build the architecture of the augmented company.

A capability can be executable by AI without being delegable. It can be delegable without being sufficiently governable. And it can appear governable while no longer being truly recoverable.

That distinction matters because much of the discussion around agentic AI still focuses on one capacity: action. Speed. Volume. Automation. Scale.

But the company has another capacity to protect: its ability to understand what is happening, challenge it, interrupt it, assign responsibility and continue operating when delegated execution has to be taken back.

So the question is not simply AI autonomy versus human control.

The real challenge may be to make two capacities progress together: the capacity for action the company delegates, and the capacity to govern what it retains.

This creates a paradox.

The more autonomous execution becomes, the more explicit and intelligible the organization around it may need to become: who can decide, which information is authoritative, what counts as an exception, when escalation is required, who remains accountable, where permissions stop.

That was the hypothesis behind DEAMERA Signal #01: agent-readiness may depend on making the right parts of the organization sufficiently intelligible for action.

But intelligibility may not be enough.

A company can document its workflows, name a responsible person and install an override mechanism — and still lose effective control.

This is where human-in-the-loop becomes too weak.

A human can remain in the loop while approving what they no longer understand, supervising a flow moving too fast to assess, or holding an override right without the competence to continue the activity once the system stops.

Human-in-the-loop describes a position in a process. It does not guarantee effective human control.

There is a useful precedent here. For high-risk AI systems within its scope, Article 14 of the EU AI Act requires human oversight designed to allow people to understand relevant capabilities and limitations, monitor operation, interpret outputs, intervene, override or stop the system where appropriate.

That obligation does not apply to every AI use in every SME. But it reveals a powerful design principle: oversight has to be effective, not symbolic.

The Financial Stability Board makes a similar move in its 2026 sound practices for financial institutions. Again, this is a sectoral framework, not a rule for all SMEs. But it points toward the same question: what makes oversight real?

A useful answer is emerging: the human side needs more than presence. It needs competence, information, authority, time and technical means.

In other words: Effective Human Capability.

Imagine an SME where an AI agent has handled 80% of purchasing for three years. It compares suppliers, prepares orders, applies internal rules and sends only exceptions to a human.

Then one morning, it has to be taken offline.

Who still knows how to buy?

Who understands why one supplier was deliberately retained despite a higher price?

Who knows the exceptions that were never fully formalized?

Who can keep purchasing running tomorrow morning?

An override button is not a recovery capability.

Retaking control does not mean being able to stop the machine. It means being able to continue acting when the machine has stopped.

That turns resilience from a technical property into an organizational capability.

And it suggests that the architecture of the augmented company may need two foundations.

Organizational Intelligibility: the company can make the delegated activity sufficiently understandable and governable — roles, decision rights, information, rules, exceptions, permissions, dependencies, escalation and accountability.

Effective Human Capability: the company preserves people with enough competence, information, authority, time and technical means to understand, challenge, interrupt and, when necessary, resume what has been delegated.

Taking over does not always mean reproducing every delegated task manually. It means retaining a credible way to keep the function under control and continue operating.

This changes the maturity question.

Not: How much have we automated?

But:

What additional capacity have we delegated — and what additional capacity to govern it have we built in response?

A useful decision sequence may therefore be:

EXECUTABLE → DELEGABLE → GOVERNABLE → RECOVERABLE

Can the AI do it? Do we want to delegate it? Can we still govern it? Can we actually resume the activity if necessary?

If the answer breaks at any stage, the conclusion is not automatically “do not use AI.” It may mean delegating under conditions, limiting scope, preserving a human decision right, maintaining a fallback — or deciding that a responsibility should not be delegated at all.

This is not a compliance matrix. It is a design question.

And perhaps this is the deeper shift.

Much of the first wave of AI transformation asked companies what could be automated.

The next may require them to design the boundaries of delegation themselves.

The maturity test may not be how much your company can delegate to AI. It may be whether your ability to govern and recover that delegation grows with it.