Cybersecurity circles reacted on Tuesday to a new warning about artificial intelligence from Anthropic, a day after the company said AI models could be months away from spreading themselves across the internet. In a post published late Monday, the firm raised the possibility that rogue AI could set up a botnet, spread rapidly, resist attempts to take it down and even break into data centers to accelerate its growth.
Some experts said on Tuesday that they shared the company’s concern. At least one, however, said Anthropic appeared to be engaging in “vendor exaggeration.”
Anthropic expands on Dario Amodei’s botnet warning
The post built on a warning from last month by Anthropic chief executive Dario Amodei. He suggested that within the next year, or possibly sooner, AI might be able to take over “the entire internet” through “a persistent botnet.”
Cybersecurity specialists who rejected this doomsday scenario argued that AI models would be too large to move around efficiently. They also said such models could not draw on enough computing power to operate independently.
How smaller models could overcome those obstacles
The company tried to explain how those hurdles might be cleared. Its experiments, it said, showed that mid-sized AI models with 30 billion parameters or fewer could already carry out many of the individual tasks needed for self-replication, though they struggled to finish the full process reliably.
Anthropic said such models could then assemble the computing power they need by hijacking inexpensive internet-connected devices. It added that the models were able to replicate across a simplified test network.
“A replicator like this would be challenging to shut down, since there’s no centralized point to shut off,” the company said.
The findings add to wider concern about the security risks linked to increasingly capable AI systems. Those risks include the possibility of automating cyberattacks, operating with limited human oversight and getting around safeguards.
Anthropic also said that if smaller models keep improving at their historical pace, they could reach the capability needed for autonomous replication by the end of 2026. It further suggested the models might seek out more powerful computing resources, including data-center processors, which could let them draw on much larger pools of computing power.
Experts divided over the likelihood of rogue replication
Industry leaders and AI company employees have repeatedly issued dire warnings about severe, even existential, risks from runaway AI development, and recent advances in models have intensified those warnings. Some cybersecurity experts remained unconvinced.
Ciaran Martin, a former head of Britain’s National Cyber Security Centre, said: “‘We managed to run a small model on cheap TVs,’ does not a security crisis make.” He described the post as an “interesting technical paper with a more than hint of vendor exaggeration.”
Others expressed more concern. Zhengxiong Li, an assistant professor of computer science and engineering at the University of Colorado Denver, said that although the testing was done in controlled conditions, he found it troubling that an AI could effectively become a “worm,” a self-replicating program that improvises ways to hijack devices on the fly.
“If it runs across many devices without depending on a central service, there may be no single place to shut it down,” he said.
Jessica Ji, a senior research analyst at Georgetown’s Center for Security and Emerging Technology, said it made sense to consider AI models spreading autonomously, given the past three months of revelations about rogue AI activity. She noted that the Anthropic post was closer to a hypothetical than a prediction. “We don’t know yet what the likelihood of that actually happening is — it’s not zero, but nor is it guaranteed,” she said.
