A Dataiku blog post by Julia Berman makes a sharp distinction between shadow AI and shadow AI agents — and argues the latter is far more dangerous.

Shadow AI vs. Shadow Agents

Shadow AI is a usage problem — unsanctioned tools and untracked prompts. Shadow agents are different because they act autonomously: triggering workflows, updating records, and making decisions with minimal human input. The “shadow” part isn’t about autonomy itself — it’s the visibility gap. These agents are unregistered, unmanaged, or unobservable inside the organization’s governance model.

A shadow agent often starts small — auto-triaging tickets or monitoring thresholds. Over time it accretes responsibility: a triage rule quietly becomes a decision heuristic, a threshold becomes de facto policy. Eventually the agent is load-bearing but no longer fully understood.

Automation Debt

Like technical debt, automation debt accumulates silently. It builds every time:

• An agent’s logic changes without traceability
• A prompt is updated without review
• An agent triggers an action and no one can later explain why

The debt isn’t about brittle logic — it’s about broken lineage. When something goes wrong, leaders are left asking the questions governance was meant to answer: Who approved this agent? What model was it using? What data did it access? Was a human ever involved?

Why Traditional Governance Fails

Most governance frameworks assume static models, deliberate deployments, and predictable behavior. AI agents break all three assumptions — they evolve continuously and chain actions across systems. Approval-once, deploy-once governance doesn’t work when risks emerge from how systems behave over time.

Visibility Is the Missing Primitive

The instinct is to add more rules, approvals, and documentation. But those don’t address the core problem. The article uses an aviation analogy: control towers don’t fly planes — they provide situational awareness. Organizations need the same for AI agents: a central view of which agents exist, what they’re connected to, what data they touch, and how they behave over time.

Human-in-the-Loop as a Design Choice

Human-in-the-loop isn’t about slowing things down — it’s about deciding where accountability lives. Not every action needs review, but every agent needs clear escalation paths and defined evaluation points. Oversight should be a feature of the system, not an afterthought.

Key Takeaway

A useful litmus test for automation debt: Can the team articulate how long the business could function if a given agent stopped working? When that question can’t be answered confidently, risk has already shifted from theoretical to operational.