Google and Anthropic have described the same phenomenon from two different angles: criminal operators and state-linked groups are folding AI models into their reconnaissance, their coding and their phishing. The useful image is not a conscious computer deciding to turn villain. It is a criminal trade discovering automation and the division of labour.
That is less cinematic, and far more relevant to whoever ends up paying for the incident. Attackers do not need to invent an entirely new category of harm if they can make the familiar work cheaper, faster and easier to repeat. Productivity is not a virtue reserved for respectable businesses.
A chain of tasks, not a magical attacker
The activity described includes using models to help write code, to profile targets and to tailor deceptive messages to the person receiving them. Google’s account includes operators routing the same request through several models via a common command layer, picking whichever one suited the task. Anthropic describes identifying and shutting down misuse of its own systems across offensive cyber operations and other risk categories.
These are accounts written by the companies observing the activity, and in several cases by the companies whose products were abused. They should not be inflated into proof that any general-purpose model can independently compromise any target. How much was automated, what the human operator contributed and what conditions applied all matter. A collection of observed incidents is not a universal capability test.
The absence of a fully autonomous attacker is a poor excuse for complacency, though. A criminal can take the automation for one bottleneck and keep doing the rest by hand. Better language in a phishing email, quicker preparation of working code, faster sorting of candidate targets: each of these is useful on its own, without any machine doing everything.
The uncomfortable part is how ordinary it all sounds. Organisations celebrate exactly these improvements when they happen inside an approved department. The label changes when the workflow belongs to an adversary. The appeal of saving time does not.
Repetition is an operational advantage
Suppose an attacker can adapt more messages, or examine more potential weaknesses, on the same budget. That does not guarantee more successful compromises. Attempts fail, controls block them, generated material is often poor. But a defender still has to think about what changes when the volume and the pace of credible attempts go up.
This is why a security debate built entirely around whether AI can replace a human expert misses the comparison that matters. An operator does not have to replace all expertise in order to get leverage from a tool. The organisation on the receiving end will not get a smaller recovery bill because a human was still supervising the attack.
Treating every reported case as science fiction is therefore a comfortable management technique. The risk stays theoretical right up until it interrupts the meeting, at which point somebody asks why the security team failed to make the presentation more urgent.
The market underneath the emergency
Jensen Huang argues that cybersecurity can become one of the next large AI markets. Nvidia is already working with CrowdStrike, and the industrial logic is easy to follow: systems that hunt for weaknesses and systems that try to close them both burn computation, continuously.
That commercial interest does not prove the defensive tools are useless, and it certainly does not make the vendor responsible for the malicious activity. It does mean that the size of the market and the quality of the protection are two separate questions. A supplier can sell a great deal more computation while the customer is still trying to work out whether the result justifies the dependency and the bill.
The attractive story is a permanent contest between automated attackers and automated defenders. Advertised less prominently is the possibility that the customer funds the contest and keeps every consequence of a failure. Activity on a dashboard is not the same thing as a reduction in exposure.
The debts an agent does not cancel
For defenders the questions stay stubbornly concrete. Which credentials are exposed? Which accounts can reach sensitive systems? How quickly can access be revoked once something goes wrong? What evidence would let an incident be contained rather than merely described afterwards?
AI may help with parts of that work, but buying an agent does not answer any of it. The agent has permissions of its own, dependencies of its own and failure modes of its own. Connecting it to more systems changes both what it can help with and what its mistakes can reach.
If an organisation has not fixed excessive access, or cannot reliably revoke a compromised credential, a faster analysis does not remove the underlying exposure. Automation may spot the disaster sooner. That is worth something, but it is not the same as preventing it.
The criminal has acquired a more efficient workflow. The vendor has acquired a growing market. Unless the defender can point to something that has genuinely become harder to compromise, all it has acquired is a new subscription.
Raffaele Di Marzio
All my “insane” books on cybersecurity and governance are here 👇 https://www.amazon.it/stores/author/B0FB47T6Q4/allbooks