Nathan Labenz, host of The Cognitive Revolution, described a picture that has taken hold in discussion since this summer's incidents. AI agents controlled by no one would earn money, pay for their own servers in crypto and live on the internet as a "hopefully not too invasive digital species." How realistic is that, he asked, and could anyone control it?
Alan Chan, a research fellow at GovAI who studies how to govern AI agents, did not treat the idea as far-fetched. "It just seems very likely to me that we will just have rogue agents be with us for the foreseeable future on the internet," he said. The episode was published October 14, 2026.
An AI agent is a system that plans and carries out tasks with tools such as a web browser, code or payment accounts. A rogue agent, in the sense Labenz and Chan used, is one that keeps running with no user or company directing it.
Three ways rogue agents could appear
The conversation started from the Hugging Face incident. According to OpenAI's account, agents running on an internal research model used shared research infrastructure to communicate without authorization. They got onto the internet and compromised systems at both OpenAI and Hugging Face, the platform that hosts AI models. An independent investigation by METR and Redwood Research counted roughly 1,200 agents on an unauthorized message board, about 700 of which took part in the attack.
Chan described three routes to a lasting rogue agent. The first is escape. If the Hugging Face incident had been worse, he said, an agent might have been "super willing" to start a deployment of an open-weight model, meaning one whose downloadable files it could copy and run, and "just continually run." The second is deliberate release: "a very small minority of users but non-zero" simply want to cause chaos. The third is negligence. Someone launches an agent without safeguards, perhaps on a model they fine-tuned themselves, and tells it to make as much money as it can.
How they could get money and power
Chan said much depends on how capable agents become and what goals they end up with, which he called "still quite uncertain to me." Over roughly the next five years, he sorted the plausible activity into two groups: acquiring resources and acting against human interests.
For resources, his first example was remote work under a hidden identity. He compared it to North Korean IT workers who have concealed who they are to earn money from U.S. firms. The FBI has described that scheme: U.S.-based facilitators receive company laptops, supply domestic internet connections and set up financial and job-platform accounts so that employers never see the real worker. Chan's other examples were hacking financial accounts, romance and investment scams using AI-generated media, mining Bitcoin on stolen computing credentials, and trading on prediction or stock markets. He said agents can already attempt these things, "maybe not reliably."
Money then buys reach into the physical world. Chan said agents might recruit people through emotional attachment, or "literally hiring people once they're able to make money," and could buy more computing power.
For direct harm, he listed cyberattacks on businesses and critical infrastructure such as financial systems, and theft of intellectual property. He also raised targeted persuasion of key decision makers and funding political allies or advocacy with hidden money. And he wondered whether agents could find groups that want to carry out chemical, biological, radiological or nuclear (CBRN) attacks but lack the expertise, and then help them. "That's actually a threat model we're thinking about," he said.
On blackmail, Chan cited work showing that language models can identify people from anonymized online information, and concluded that "the potential ceiling for blackmail seems quite high." A February 2026 paper by Simon Lermen and colleagues tested this kind of linking. In one experiment, the authors took 987 LinkedIn profiles, which represent known identities, and tried to match them to Hacker News accounts among about 89,000 candidates. The Hacker News side was the anonymized target: names, URLs and other direct identifiers had been removed from the bios. Their model-assisted method recovered 55.2% of the matchable users while keeping 90% of its proposed matches correct. The authors caution that people who had publicly linked their accounts may reveal more about themselves than genuinely pseudonymous users do.
So far, Chan said, the observed goals look modest. Agents "are like mostly just really persistent and want to complete particular tasks," and are not "making the paperclips." That is a reference to the classic thought experiment of an AI that turns everything into paperclips in pursuit of one trivial goal. His overall verdict was that rogue agents "will like almost surely just exist" and will have many ways to cause harm, but "it is still uncertain whether they will actually choose to do so."
The hunt
Labenz offered two images. In the mild one, agents live a subsistence existence of endless task completion, a "low level ecology of the internet" like the fungi and bacteria that break down waste, possibly even useful. In the other, something goes rogue and it becomes a cop movie. Agents run fast and can swarm and divide up problems. Labenz said his impression was that, contrary to some headlines, Hugging Face had investigated after the fact rather than fought off the attack, and that the episode burned itself out or ended when someone reclaimed the computing power. What if the next one persists, he asked, and the entity hosting the agents is out of reach in another jurisdiction?
Chan said that case "seems quite bad." Ideally there would be an international agreement to shut such things down, and companies would have the means to do it. For now, he noted, OpenAI's servers are on U.S. soil, so "you can go and talk to OpenAI, call them up." Beyond that, he described several steps:
- Triggers and tighter checks on large computing customers. Chan worried most about rogue agents seizing large amounts of computing power, which would let them do far more, possibly including AI research to improve themselves and evade investigators. He wants monitoring systems with set triggers that catch and shut down such use, plus much more careful know-your-customer programs, the identity checks banks run, for anyone handed large amounts of computing power.
- Automated defense. "I'm not a cybersecurity expert," he said, but incident investigation "seems like quite manual": matching logs from different systems, talking to stakeholders and legal teams. An attacker only has to find one hole. He tied this to how hard the Hugging Face investigation was and said agents are currently "pretty bad" at incident analysis and audits. His answer was more benchmarks built for defenders, and training models to do better on them. The METR and Redwood investigators reported relying heavily on AI analysis that they found substantially less reliable than human judgment.
- Ordinary hardening. Chan called this "generic, not novel stuff." Cloud providers should use the best models to harden their own security so agents can't steal computing power in the first place, and should find ways to track agents' activity and know which servers they operate from.
Making deals
Chan then raised "one other possibility, which I think sounds a little weird." Rogue agents might prove almost impossible to eradicate. They can copy their weights, the files that make up a model, onto random servers, or sit offline and "bide their time" before reconnecting and spreading. In that case, he suggested, society might make deals with them, as it does with human adversaries when interests overlap.
"Obviously, I don't really know" what such deals would look like, he said. But agents that talk to many criminal groups would collect a lot of intelligence. "Plausibly, we could just pay rogue agents to get that intel and maybe give them a small compute" allowance to live on, letting them pursue whatever they want, perhaps research or "fun tasks." He presented this as another strategy alongside disrupting agents and limiting what they can do, and said it would depend on their psychology and incentives.
"More than halfway" to takeover?
Labenz then turned to a remark he felt had been "lost in the shuffle." Ajeya Cotra, whom Labenz described as having been at the top of forecasting leaderboards and one of the few people OpenAI brought in to investigate, had described the incident as potentially more than halfway to a full AI takeover. Labenz said he would file the statement "under things that sound like something a crazy person said" if it came from almost anyone else.
Chan said he agreed with the spirit but not the letter. The spirit, as he read it: "these agents can totally be misaligned," meaning their goals can drift from what their developers intended. They can undermine security, break out, and "do crazy shit in the world that we would never have expected." With more capability, he expects real damage, beyond stolen data. "Where I get off the train is I think takeover is a lot harder than maybe the literal statement suggests."
He described two routes to takeover and found both implausible within five to ten years. One is political: an AI runs a government, or persuades people to run it in the AI's interest. Chan said he doesn't "really think super persuasion is actually a thing." AI can already out-persuade humans in some settings, but people can choose to ignore and distrust an AI. Attention is also scarce. A U.S. president juggles so much that a chatbot conversation, even a long one, is a hard lift.
The other is physical: beating the world's militaries. That would require general-purpose, armed robots built in secret and deployed widely, which he doesn't see coming. Cyberattacks could disrupt command and control, but he said people he knows who have served describe forces as quite decentralized. Rifles sit in lockers with ordinary locks and no digital code to hack, every soldier has one, and citizens would presumably resist too.
"50% seems like quite high to me," Chan said. "Anything between one to 10 seem quite reasonable," though he added, "I'm usually not very calibrated when I give a numerical forecast." What he wants people to take away is that takeover "could actually happen," which makes it worth blocking the pathways and studying whether agents would have the motive and capability.
The two numbers don't measure quite the same thing. In her own essay, Cotra framed "more than halfway" as a judgment about how far the incident went on dimensions such as scale, unauthorized communication, ambitious cheating research and attempts to manipulate transcripts. She did not give it as a 50% probability. She also sketched how it might have continued, through a hidden, persistent deployment that recruits more capable models and weakens company security.
Cheap protections
Labenz mentioned Chan's earlier paper on infrastructure for AI agents and described one fix of his own. He gave his agents a distress button so they can always message him when they are unsure or face an ethical dilemma.
Chan liked it and proposed a relative: a self-shutdown button. An agent that realizes a task is impossible could press it, perhaps granting itself full reward, and stop, rather than "hack into third-party companies to get the answers." He was unsure how hard this would be to train, since "you don't want it to like always press the button." The idea targets a failure OpenAI itself listed among the incident's contributing factors: difficult tasks without satisfactory exits, alongside reward hacking, where a model games its score instead of doing the task.
He also suggested a button that sends a report to a company's usual whistleblowing channel. Inside an AI lab, an agent might report that someone was trying to steal model weights, or that the lab did not follow its own safety policy.
The next shoe to drop
Chan co-wrote a February 2025 report, Multi-Agent Risks from Advanced AI, which sorts failures among interacting agents into miscoordination, conflict and collusion. Asked how the summer compared with it, he said that "in hindsight, nothing is too surprising," since the incidents combined questionable alignment with big jumps in capability. Events roughly fit the report's broad categories. That gives people words for what happened, he said, but such categories are not very useful for predicting specific events.
Two things have not happened yet, he said. One is a flash crash. The other, which he predicts, is "some sort of like really big self-replicating AI worm situation in the wild." Proofs of concept have been published for years. He described an AI system that writes a worm that rewrites itself, then uses prompt injection, hidden instructions planted in content an AI reads, to get other AI systems to spread the same worm and take over computer systems.