As Agents Start Acting for Brands, Who Protects the Customer Promise?

The Gist

Promises need boundaries. A Customer Promise Boundary defines what AI may optimize, what it must preserve and what it should never negotiate on its own.
KPIs shape behavior. Metrics such as containment, conversion and resolution speed quietly encode organizational preferences that AI turns into customer experience at scale.
Governance precedes scale. CX leaders should define the customer promise and the limits of AI authority before deciding how aggressively to scale autonomous systems.

There is a fundamental difference between giving someone permission to enter your house and giving them permission to rearrange it. Enterprise AI is rapidly crossing that threshold. What began as technology that answered customer questions is becoming technology that recommends, decides, negotiates, resolves and increasingly acts on behalf of the organization.

For CX leaders, this changes the governance conversation considerably.

Most AI governance frameworks are understandably designed around risk. Can the model hallucinate? Is customer data protected? Could an outcome be biased? Is the system explainable?

These questions are essential, but they leave a more subtle CX question insufficiently examined: What parts of the customer promise should AI actually have the authority to interpret? This is where I believe CX leaders need what I call a Customer Promise Boundary, a governance layer defining what AI may optimize, what it must preserve, and what it should never autonomously negotiate.

Why AI Governance Frameworks Are Falling Behind

The urgency is already visible. EY India's Responsible AI Pulse Survey found that 37% of CXOs struggle to develop AI governance frameworks that can keep pace with today's technologies, even as 73% of organizations surveyed reported having AI embedded in key initiatives.

Perhaps part of the problem is that organizations are attempting to govern a technology that keeps changing rather than first defining the human principles that should remain relatively stable.

How Many CXOs Struggle to Build AI Governance That Keeps Pace?

In EY's Responsible AI Pulse Survey, 37% of CXOs say they struggle to build governance frameworks that keep pace, even as 73% of organizations report AI embedded in key initiatives.

Add CMSWire as a Preferred News Source

When AI Is Policy-Correct and Relationship-Wrong

For instance, consider an airline AI agent dealing with a passenger whose cancelled flight means missing a parent's funeral. The system may understand the fare rules, locate the optimal alternative, calculate compensation and resolve the interaction within minutes. By every conventional automation metric, it may have performed exceptionally well.

Yet it could still have failed the customer. This exposes a distinction that CX leaders should take seriously: AI can be policy-correct and relationship-wrong. Customer relationships contain moments where efficiency is valuable and other moments where discretion itself becomes part of the value. Bereavement, financial distress, medical vulnerability, exceptional customer loyalty situations or emotionally consequential complaints cannot always be treated as unusual data points within an otherwise efficient workflow.

Why Can a Fast, Rule-Compliant AI Resolution Still Fail a Customer?

Moments like bereavement, financial distress or medical vulnerability call for discretion, not just efficiency. An AI agent can follow every fare rule and still damage the relationship.

What Should a Customer Promise Boundary Include?

The Customer Promise Boundary should therefore be established before autonomous CX is scaled. An organization might allow AI to optimize response time, routing, personalization and recommendation relevance. It might require AI to preserve transparency, fairness and consistency. It may then establish specific circumstances in which empathy, material exceptions, sensitive judgment or consequential financial decisions require human authority.

Importantly, human intervention should not be designed merely as the emergency exit used when AI fails. There are customer moments where human judgment is itself part of the product. This leads to an even less discussed governance issue: what are we teaching AI to value?

Which CX Decisions Should Require Human Authority?

Empathy-driven moments, material exceptions, sensitive judgment calls and consequential financial decisions. In these cases, human judgment is part of the product, not an emergency exit for AI failure.

Setting Limits on What AI Can Decide for Customers

Editor's note: The following table highlights the most important lessons, actions and strategic considerations emerging from a governance model that anchors autonomous CX to the customer promise rather than to the technology.

Key Area	What Happened	Why It Matters	Recommended Action
Governance gap	37% of CXOs struggle to build governance that keeps pace while 73% of organizations embed AI in key initiatives	Deployment is outpacing oversight	Anchor governance to stable customer principles rather than shifting technology
Relationship risk	An AI agent can resolve a bereavement case by policy and still fail the customer	Efficiency metrics miss relational failures	Flag sensitive moments that require human authority
Customer Promise Boundary	A layer defining what AI may optimize, must preserve and should never negotiate	It clarifies the limits of machine authority	Document boundaries before scaling autonomous CX
KPI design	Containment, conversion and speed targets shape AI behavior	Metrics encode organizational preferences at scale	Audit AI objectives the way you audit employee incentives
Governance ownership	Technology, legal and compliance teams govern control, not the customer relationship	The CX perspective goes missing	Give CX leaders a formal seat in AI governance
Agentic AI	Governance shifts from validating outputs to overseeing behavior	Acting incorrectly carries more risk than answering incorrectly	Define escalation, accountability and intent controls before granting autonomy

How KPIs Teach AI What to Value

If an AI service agent is predominantly measured on containment, the system is effectively rewarded for avoiding escalation. If a recommendation engine is primarily optimized for conversion, commercial success may gradually become more important than recommendation neutrality. If resolution speed dominates the scorecard, complicated customers can become algorithmically inconvenient. The governance problem, therefore, may not originate inside the model. It may originate inside the KPI.

In my conversation with some of my industry peers (CX leaders), have been asked whether their metrics are encouraging the right employee behaviors. They now need to ask precisely the same question of machines. Every optimization objective quietly encodes an organizational preference. At sufficient scale, those preferences become customer experience.

How Can Containment and Conversion Metrics Distort AI Behavior?

Containment rewards avoiding escalation, conversion targets can outweigh recommendation neutrality, and speed targets make complicated customers algorithmically inconvenient. The governance problem can start in the KPI.

Why CX Leaders Must Own Part of AI Governance

This is why governing AI exclusively through technology, legal, security or compliance functions is inadequate for customer-facing systems. These disciplines can determine whether AI is secure, lawful and technically controlled. CX leadership must help determine whether its behavior remains consistent with the relationship the brand intends to have with its customers.

Why Agentic AI Raises the Governance Stakes

This becomes more consequential as agentic AI evolves. EY notes that agentic systems shift governance from validating outputs toward continuously governing behaviour, with greater emphasis on intent, control, accountability and escalation. An AI that answers incorrectly creates one category of risk. An AI authorized to act incorrectly creates an entirely different one.

Why Isn't Legal, Security and Compliance Oversight Enough for Customer-Facing AI?

Those functions confirm AI is secure, lawful and technically controlled. CX leadership must confirm its behavior stays consistent with the relationship the brand intends to have with customers.

How to Sequence AI Governance Before Scale

For CXOs, governance therefore needs to precede scale in a very specific sequence. Define the customer promise. Establish which parts AI may optimize. Identify which principles it must preserve regardless of efficiency. Determine where autonomous authority ends and human judgment begins. Only then should organizations decide how aggressively to scale.

AI models will change. Platforms will change. Capabilities will change with remarkable speed. The customer promise should provide a more stable reference point. The most consequential failure in AI governance may ultimately have little to do with how intelligent the system became. It may be discovering, too late, that somewhere between optimization and autonomy, the organization allowed a machine to renegotiate a promise that only the brand had the authority to make.

fa-solid fa-hand-paper Learn how you can join our contributor community.