Every AI Customer Service Failure has the Same Shape

In January, Judith Butterfield had a stroke and spent 2 months in Doncaster Royal Infirmary. The stroke affected the right side of her body and left her speech fragmented.

Her GP surgery had installed an AI reception service to reduce phone queues. She tried to book an appointment 5 times and did not reach a human on any attempt.

The system asked callers not to use speakerphone. Butterfield held the phone in her left hand because her right hand could no longer operate a keypad. When the system asked her to repeat her name, it did not understand her, and the call ended. She told the BBC the experience was “very difficult and upsetting.” A receptionist could have asked her to spell it.

Butterfield, 71, registered with a different practice. Her original surgery has since decommissioned the AI system and recruited more reception staff. The watchdog Healthwatch reports similar problems from other Yorkshire patients, including one case involving a regional accent.

The same failure has now played out at a bank and an airline, with the same underlying shape.

Why do AI customer service systems fail?

AI customer service systems fail for structural reasons rather than technical ones. Two documented cases show what that looks like at scale.

Commonwealth Bank: the queue that did not shrink

In July 2025, Commonwealth Bank of Australia cut 45 customer service roles, saying its new AI voice bot had reduced call volumes by around 2,000 a week. Staff reported the opposite. The Finance Sector Union said volumes were rising, overtime was being offered to cover the load, and team leaders were being pulled onto the phones. The union took the bank to the workplace relations tribunal over its call volume claims.

On 21 August 2025, CBA reversed the redundancies and apologised, saying it “did not adequately consider all relevant business considerations” and that the roles were not redundant.

The work had not disappeared. The bot absorbed the simple calls, and customers with urgent or complicated problems kept calling. Automating the easy contacts changed the composition of the queue rather than its size.

Air Canada: the company owns what the bot says

On the day his grandmother died, Jake Moffatt asked Air Canada’s chatbot about bereavement fares. It told him he could book now and claim the reduced rate within 90 days of the ticket being issued. The airline’s actual policy, on the page the chatbot linked to, does not allow refunds for travel already taken. Moffatt booked at full fare, applied, and was refused.

British Columbia’s Civil Resolution Tribunal found Air Canada liable for negligent misrepresentation and awarded Moffatt C$812.02. The airline had argued its chatbot was a separate legal entity responsible for its own actions. The tribunal rejected that. It also rejected the argument that the correct policy was available elsewhere on the site, asking why a customer should have to check one part of a company’s website against another.

That was not a defence. Whatever your AI says is what your company said.

Deflection rate conceals the problem

Most AI customer service deployments are measured on deflection rate, which is the share of enquiries resolved without a human. Judith Butterfield’s five failed calls each counted as deflected.

Deflection rate does not separate a solved problem from an abandoned one. A deployment can report 80 percent deflection while losing its most frustrated and most vulnerable customers. CBA’s case shows the same distortion one level up: the bot’s deflection figures looked like reduced demand while the queue behind it was growing.

What to measure instead: resolution rate, which is the share of enquiries actually resolved, and escalation rate, which is the share reaching a human. Also track the proportion of conversations that end without either outcome.

What do teams miss when deploying AI in customer service?

Seven failure points account for most of the damage. All seven are design decisions rather than model limitations.

1. The callers the system was not designed for

Voice AI is usually tested on clear, standard-accent speech. Real callers have regional accents, second languages, speech affected by illness or age, background noise and poor reception.

Butterfield’s case involved two assumptions at once: that her speech would be recognised, and that she could use a keypad if it was not. Neither assumption held. The people excluded by assumptions like these are often the ones with the most urgent need.

2. No route to a human

In each of the cases above, the customer could not exit the automated conversation on the terms they needed. An AI assistant that cannot answer a question should hand over on the first failure rather than rephrase the same answer.

The vendor behind the GP reception system states that practices can enable routing straight through to reception on every call. That option existed at Butterfield’s surgery and was not switched on.

3. Emotion as a routing trigger

Distress, anger, urgency, legal language and bereavement appear regularly in support queues. They are also the interactions where a wrong answer costs the most. Moffatt was asking about a bereavement fare on the day his grandmother died, which is exactly the conversation that should have reached a person. Route these on detection rather than after several failed attempts.

4. Scope set too wide

AI support deployments that work handle a small set of high-volume, well-defined requests and escalate everything else with the conversation history attached. Deployments that fail attempt every request type at once and handle all of them poorly.

CBA’s experience adds a second point. Automating the simple contacts changes the composition of what remains. Plan capacity for the queue you will have after deployment, not the one you have now.

5. No memory across the conversation

Some systems treat each message as a separate event. The customer explains the situation, answers a clarifying question, and finds by the third message that the opening has been forgotten. The same happens when a customer moves from chat to phone and has to start again.

Repeating yourself is the point at which most people stop trying. Confirm that context persists across messages and across channels before going live.

6. Answers invented from a contaminated knowledge base

An AI assistant will produce an answer even when the correct one is not available to it. Air Canada’s chatbot described a bereavement policy the airline did not have.

The usual cause is the source material. Internal notes, draft pricing, superseded policies and old support threads get loaded alongside customer-facing content, and the assistant resolves the contradictions at random.

Build a separate knowledge base containing only approved customer-facing information, and check it for contradictions before launch rather than after.

7. The assistant cannot take action

Most support requests need something done rather than something explained. A refund processed. An address changed. An appointment moved. An account unlocked.

An assistant connected only to a knowledge base can describe the process and cannot complete it, so the customer is left with instructions they cannot act on. This produces the most common complaint about AI support, which is being sent a link to a help page the customer has already read.

Decide before deployment which actions the assistant can execute in your systems, and route everything else to someone who can.

Where does AI belong in customer service?

Each failure above has the same shape. A human service existed, it was removed or hidden, and AI became the only way through. A patient needed a doctor. A grieving passenger needed a fare policy. A banking customer needed a problem fixed that the bot could not fix.

Compare that with a visitor on a website at 9pm with a question about whether a product suits their room. That visitor is not in a queue and is not being denied anything. Without an assistant they leave the site, and the business never knows they were there. The alternative to AI in that case is no response at all.

The rule: use AI to add a route, never to remove one.

An example

We built an AI consultant for a major Australian flooring retailer on the add-a-route principle. It answers product and service questions, works out what the visitor needs, and recommends suitable options with clickable links and images shown in the chat. Where relevant it prompts them to book an in-store or in-home consultation, and pre-fills part of the booking form from what the conversation has already covered.

It answers from approved product and service content, holds context so the visitor is not asked the same thing twice, and follows a defined path for each common situation rather than one general instruction set. Anything it cannot resolve goes to a person who follows up directly, which was a requirement at the start of the build rather than an addition to it.

We keep tightening it against real conversations: what visitors actually ask, and where the assistant fell short.

If you want the same approach applied to your own site or support line, we can start by mapping where your customers currently get stuck.  Start with 20 hours free: real consulting work, no cost, before you commit to anything.

Summary

Placement determines the outcome in AI customer service. An AI assistant put in front of an urgent need, with the human alternative removed and success measured by how few callers reach a person, will eventually reproduce Butterfield’s five phone calls or CBA’s reversal.

An AI assistant that opens a route which was previously closed, with a person one step away, handles enquiries that would otherwise go unanswered.

Frequently asked questions