On 8 September Meta launched Muse, its personal AI agent. Not a chatbot that answers questions, the company was clear about that: an agent that acts. It sends email, buys things online, books travel, fills in forms and pays through Stripe. It runs as a standalone app on iOS and Android and sits inside WhatsApp, so a task can be delegated with a text or a voice message.
The market liked it. More than 2.5 million installs in the United States, a fast climb up the download charts, and a share price up more than twenty per cent.
Among the more ambitious functions was the one that made the product feel like the future: Muse could telephone businesses to handle everyday errands, booking an appointment or checking whether something was in stock.
The part where people hung up
Except that on the telephone, people hung up.
They heard a synthetic voice, recognised it, and ended the call, exercising the small residue of taste that the market has not yet worked out how to monetise. Pure artificial intelligence, it turns out, meets resistance from the humans it is pointed at.
Meta’s solution was a feature called “human agent calls”. Some requests were routed to actual call centre operators, external to the company, who made the call while presenting themselves as the digital butler. The user was not told. As far as the person who typed the request was concerned, their AI assistant was on the line.
According to the tests, the tactic worked. Success rates rose to between 95 and 98 per cent.
That number is the whole story, and it is being read backwards everywhere. It does not measure how good the artificial intelligence is. It measures how well a human being performs when paid to impersonate one. It is the Turing test inverted, and the species that passed it was ours.
Not a comedy sketch, a room full of data
The impersonation is funny for about ten seconds. What sits behind it is not.
Somebody delegating a task to their own agent talks to it the way people talk to software, which is to say without editing. That is the entire design promise: stop filling in the form, just say what you need. So the things said out loud include the things nobody would hand to a stranger. Social security numbers. Financial details. Medical information.
In this arrangement, that material could reach a call centre operator outside the company whose existence the user did not know about, in a conversation the user believed was being handled by a machine. The company itself concedes the tests began without proper transparency notices.
The testing began with Meta employees, with an opt out available. But the AI calling feature had been extended to ordinary customers as well, and it is not clear whether calls from external users were also diverted to the call centres. That ambiguity is not a detail. It is the difference between an internal experiment on informed staff and an undisclosed human interception of private requests from the public, and the company that ran the test is the one party in a position to clear it up.
The vocabulary of the apology
The experiment has now been stopped. Senior figures at Meta’s Superintelligence Labs acknowledged that launching the tests without proper transparency notices to customers was a mistake. A company spokesperson said the feature has been temporarily suspended and will only return when the system is ready and the correct legal notices are in place for the user.
A mistake. The word does a great deal of work.
A mistake is the coffee going over the keyboard. What happened here required someone to identify a failure, build a workaround, put external operators on live user requests, let them present themselves as the product, and then record the result as a success rate between 95 and 98 per cent. Whatever the internal paperwork looked like, those are choices, and a number that gets measured is a number somebody wanted. Mistake is the company’s word. Mine is that it worked until people outside found out.
Note also what is promised for the return. Not that human beings will stop presenting themselves as the agent. That the correct legal notices will be added. Read the promise for what it actually says and the thing being repaired is the paperwork, not the arrangement. If that reading is wrong, the company is free to say so in one sentence.
What the demo was for
Strip the story back and the shape is familiar. A company launches an agent that acts in the real world, and the launch is what moves 2.5 million downloads and more than twenty per cent of the share price. On the phone, the capability underperforms. The demonstration survives anyway, because people were put behind the curtain, at a wage, doing the work the product was sold as having automated.
The human fallback was an internal feature, never part of the thing being advertised, and the disclosure was set by the same people who were running the test.
Nothing here required a breakthrough in artificial intelligence. It required a call centre, and a product that had to look finished before it was.
Raffaele Di Marzio
All my “insane” books on cybersecurity and governance are here 👇 https://cyberium.limited/bookshelf.html