Anthropic has announced that Claude is now contributing 26% of the work on its own next version, autonomously. Twenty-six per cent.
Start with what that figure does not establish. On its own it does not demonstrate a system redesigning itself end to end. It describes a contribution to a development process, measured and disclosed by the company doing the measuring. The reason it matters is the direction it points in rather than the point it has reached.
The technical name for that direction is recursive self-improvement, RSI: an AI that finds a way to evolve and build its own successor. Not a faster tool for researchers, but a system designing the architectures that come after it, at machine speed rather than human speed. Nothing in the current numbers shows we are there, and nothing in them shows the current systems are running unsupervised. The argument is about the trajectory.
The expert line, and why it is the less interesting part
Anthony Aguirre, president of the Future of Life Institute and a professor of physics at the University of California, Santa Cruz, has said publicly that in Anthropic's case an ever larger share of the research will be done by the AI itself, approaching full autonomy, and that he finds the result of that success extremely frightening. He described it as probably the worst idea in the history of humanity.
That is his judgement, offered as a judgement, and it deserves to be reported as his. His reasoning is specific rather than apocalyptic: recursive self-improvement would let a system design its own successive architectures while operating at speeds beyond ours, with the risk of losing control of the process. The claim is about a possible gap between how fast such a thing could iterate and how fast we could inspect what it did.
You can disagree with the framing. Plenty of serious people will, and a professor of physics calling something the worst idea in human history is the kind of sentence that gets a headline precisely because it overshoots. Fine. Set it aside.
The confession filed as candour
Because the genuinely remarkable statement in this story did not come from a critic. It came from a company.
OpenAI wrote on its own blog that it does not yet know how to safely achieve complete and aligned RSI, and that it cannot assume progress in alignment and safety will proceed in step with capability, because the more powerful systems become, the harder they can be to monitor.
Read that again as an engineering statement rather than a blog post. A developer says, in writing, that it does not yet have a validated method for the end state it is working toward; that the safety discipline may fall behind the capability curve; and that monitoring can become harder as the systems get stronger. That last point is the company's own stated risk rather than a measured outcome: this disclosure does not establish that observation has already broken down. It is a warning issued by the party best placed to issue it, which is exactly why it should not be filed as reassuring candour.
It is candour. That is not nothing, and I would rather have it than silence. But candour is not a control. Writing down that you do not yet know how to do something safely does not make doing it safe, and it does not transfer the risk to the reader who has now been informed.
Slowing down, conditionally
Then there is Anthropic's position, which has been one of the more credible voices arguing for slower development of AI, on the condition that competitors slow down too.
Look at the logical shape of that, and only at the shape: this is not an accusation that a cartel exists, it is an observation about how the commitment is built. A commitment that activates only if everyone else commits behaves like a coordination clause rather than a principle, because a principle is the thing you hold when the other party cheats, which is exactly the circumstance principles exist for. Make the brake conditional on universal agreement and you have built a brake whose engagement depends on competitors you do not control, while keeping the reputational credit for having proposed one. Whether it ever engages is not something either of us can know today, and that uncertainty is the problem, not the defence.
Meanwhile there is one figure on the record: twenty-six per cent. Aguirre expects the share of research done by the AI to keep growing, and that is his projection rather than a measured trend. If a higher number does arrive, I expect it to be presented as a milestone, and I expect somebody to call it efficiency, because efficiency is the word we tend to reach for when we would rather not discuss what has been delegated. That is a prediction about public relations, and I am happy to be wrong about it.
What would make this a different conversation
None of this requires a story about a machine waking up at night. The boring version is enough, and it is a risk rather than an observation: a development process in which a growing share of design decisions would be taken by systems operating faster than the review around them, inside companies competing on release speed, with safety work that one of those companies says may not keep pace. Nothing in the disclosures shows that process already running. What they do show is the ingredients being assembled. So how much of that reasoning is actually audited, and by whom? Neither a percentage nor a blog post answers that, and the gap is the point.
The honest path is not a press release about fear or a press release about responsibility. It is narrower and duller. Publish what share of which tasks is being delegated, and to what. Publish what the monitoring actually covers and what it does not. Name who is accountable when an autonomously designed component fails, before it fails. Make the slowdown unilateral, or stop calling it a safety position and call it what it is, a negotiating stance.
Until then we have a written admission and a percentage, and no published method for safely reaching the thing they point at.
Raffaele Di Marzio
All my "insane" books on cybersecurity and governance are here 👇 https://cyberium.limited/bookshelf.html