As enterprises continue to increase their adoption of autonomous AI, many are facing a new challenge where governance frameworks struggle to keep pace with systems that can take independent action rather than simply create content.

So far, much of the focus on generative AI has been on accuracy, hallucinations and productivity. As AI systems gain the ability to act independently, however, the question is becoming less about what AI produces and more about what it is allowed to do. 

AI adoption outpacing governance  

New research from AI governance platform Optro suggests many organisations still have work to do. Although one in three organisations are already using AI in critical resilience workflows, 30% have never tested for potential agentic AI failure.

For marketing leaders, this is not simply a technology issue. AI is already being used across campaign execution, customer communications, and personalisation. Giving those systems more autonomy raises a more practical question for CMOs (Chief Marketing Officers): what happens when an AI agent makes the wrong decision at scale?

The research points to a growing gap between confidence and preparedness. While 58% of business leaders believe their governance controls are evolving alongside AI adoption, only 18% report having dedicated AI risk safeguards in place. 

That gap is reflected in the problems organisations are already encountering. 40% say they have experienced misleading AI outputs in the past year, while 27% have identified AI-related data breaches. Regulatory concerns are also appearing with 26% of organisations having experienced regulatory scrutiny linked to their use of AI.

Human accountability remains essential in the age of AI

Traditional governance models were originally designed around human decision-makers within established approval processes. Autonomous AI introduces a different model, where systems can make recommendations, trigger workflows, and interact with customers with limited human intervention.

For CMOs, CDOs (Chief Digital Officers), and customer experience leaders, this raises strategic questions that look beyond compliance. For instance, which AI systems are interacting directly with customers? Who is accountable when an AI agent crosses marketing, IT, legal and security functions? And how much human oversight is needed as AI agents become more capable?

Many large organisations are responding by bringing marketing, IT, legal, security, and risk teams into AI governance discussions. The challenge is to put clear boundaries around what AI agents can access and do without creating approval processes that make them impractical to use.

Guru Sethupathy, GM of AI Governance at Optro, believes governance models must evolve alongside the technology.

“The reality today is that agentic AI adoption is fast outpacing governance. To harness its potential responsibly, leaders must recognise that governance models designed for static manual processes cannot keep pace with autonomous systems of action.”

Mark Taylor, Director of Information Security Risk Management at Newell Brands, also emphasises the importance of clear accountability as AI capabilities expand.

“The teams that get agentic compliance right will build governance and accountability frameworks for how an agent behaves and what it’s able to touch. Establishing that structure proactively positions organisations to scale AI safely, prevent future incidents, and execute with either humans in the loop or on the loop where appropriate.”

As AI agents move from experimentation into everyday business use, the governance question is becoming harder to separate from how organisations operate. The issue is no longer simply whether an AI system produces an accurate answer, but whether it has the right authority to act in the first place.

As agents take on more responsibility across customer and marketing operations, businesses and marketing leaders will need to decide where those systems can act independently, where humans must remain involved, and who is accountable when they get it wrong.

This is a conversation that will continue across the DMWF community and events as marketing leaders consider not just how quickly AI can be adopted, but how it can be deployed responsibly and at scale.

Marketing Tech News, powered by DMWF Ltd, delivers the latest marketing technology news, insights and industry updates.