Richard Socher thinks a ban on AI that improves itself would be impossible to enforce without something much worse than the problem it targets. "To really truly enforce no recursive self-improvement," he said, "you would need a totalitarian surveillance state, the likes of which humanity has never seen."
Socher is co-founder and CEO of Recursive, a company built around self-improving AI, and founder and CEO of You.com. He made the argument on the Moonshots with Peter Diamandis podcast, recorded October 2, 2026. He was responding to a proposal from Ro Khanna, a member of Congress, that would restrict the kind of work his company does.
The proposal on the table
Peter Diamandis, the show's host and the founder of XPRIZE, introduced the story. He said Khanna had announced what he called the most comprehensive AI legislation to date, a Human Control Over AI Act. In Diamandis's summary, the bill "includes criminal penalties for the work that you're doing, Richard," and requires independent auditors in every frontier lab, reporting directly to the government. Diamandis also cited a poll by a group he called Common Dream that he said showed 68 percent of voters backing a bill like this.
Khanna's own description is narrower in one respect. A September 29, 2026 campaign message describes planned legislation, not finished bill text. It would restrict recursive self-improvement until safeguards exist. It would also create a federal licensing agency, require containment and shutdown controls, set up secure testing and place independent auditors inside frontier labs. The criminal penalties it describes apply to specified catastrophic harms or to disabling controls, not to self-improvement research as such. The message also mentions liability insurance and international coordination.
Recursive self-improvement, often shortened to RSI, means AI systems that help improve the software, research or hardware used to build the next version of AI. Diamandis put the proposal to Socher alongside an essay Socher had just written, called "Why Doomers Are Wrong."
Why Socher puts the risk of doom at zero
"P(doom)" is shorthand in AI circles for someone's estimated probability that AI causes a catastrophe as bad as human extinction. Socher said his is zero. "There is no realistic scenario where AI wipes out all of humanity," he said.
He said he has debated many AI-risk experts, and that after two or three hours the reasonable ones nearly all drop the extreme scenarios. He set aside what he called the "sci-fi folks", whose stories have AI attacking "from the 15th dimension" seconds after it learns to improve itself. Then he went through the more serious-sounding threats. He said biologists tell him a super virus cannot be built overnight, because automating lab experiments takes a long time. On fears that AI could start a religion that turns people against each other, he pointed to existing religions and other "mind viruses". These already try to make people fight, he said, yet they do not reach all of humanity, because some people fight back.
What remains after that, in Socher's telling, is a scenario where perhaps a hundred million people are harmed or killed. He called that "still bad" but said it can be discussed in concrete terms. Cybersecurity can be improved, including by using AI to defend systems. Existing rules on gain-of-function research, the work of making viruses stronger, can be enforced. People can learn internet literacy and stop trusting AI avatars. "It turns out you should not trust everything you read or see on the internet," he said. "It's been true for 20 years."
Earlier in the conversation he named the pattern he sees in doom stories. Attackers get "near magical abilities to attack, but somehow the defenders in those stories never get near magical abilities to defend." If a story lets someone build a perfectly undetectable super virus with an on-off switch, he said, he could just as well assume "a super magical vaccine" that protects against all of it.
Regulate uses, not the technology
Socher's main argument drew on the internet. The internet carries horrible content, he said, but nobody proposes making it slower or shrinking hard drives so less of that content can be shared or stored. "We actually regulate the applications of the technology."
He said self-improvement can't be handled the same way because of how simple it is to do. Anyone with a GPU, the graphics chip that runs AI models, in their laptop can ask an AI to improve its own harness, the software wrapped around the model that lets it carry out tasks. That, he said, is something you can "literally hack up in like 20 minutes with prompt engineering," and it is already a very small form of recursive self-improvement. Enforcing a ban would therefore take, in his words, "a thought police" that hears everything you say to your private language model on your own laptop. He judged that a much bigger downside than anything AI would do, and called it "kind of scary that more and more Democrats are saying that."
Alex Wissner-Gross, a computer scientist and founder of Reified, gave the idea a name: "basically an AI Stasi." Unlike Socher, he did not treat it as unthinkable. He said he could easily imagine regimes today that would happily build one to stop self-improvement within their borders. Salim Ismail, founder of Open ExO, took the argument further. If people can put an idea into words for an AI and have it carried out, he said, regulation would have to reach into their thoughts. "You cannot regulate this."
Backlash, lawsuits and an old precedent
Dave Blundin, founder and general partner of Link Ventures, read out a Khanna quote accusing "tech lords" of using jargon to exploit the tech illiteracy of the elected class. He called proposals like "stop data centers, ban recursive self-improvement" childish and self-serving. In his view the politicians making them know such bans won't happen, and are positioning themselves to say "I told you so" after a disaster. Blundin said some calamity was likely, probably terrorist-driven, but "tiny compared to the benefits of AI."
Diamandis took the political threat more seriously. He raised the prospect of a Democratic House and said that 70 to 80 percent of Americans, in his words, do not want data centers and fear superintelligence. "It's not logical, but it may very well happen," he said, adding that his podcast exists partly to give people "data driven optimism" to counter such arguments.
Blundin said some of the backlash was self-inflicted. He said Elon Musk was right to call out Anthropic CEO Dario Amodei for telling the world its Mythos model was potentially deadly and then, about 30 days later, saying it would be released. "What do you expect the population's reaction to be?" he asked. In April 2026, Anthropic announced Project Glasswing, which gave selected companies and software maintainers restricted access to Claude Mythos Preview for defensive security work. The announcement described further safeguard development before models with comparable capabilities were deployed more widely.
Ismail found "a glimmer" in all this. Even if a draconian law passed, he said, there was no mechanism to enforce it. Wissner-Gross disagreed: "They could start arresting people." He recalled studying number theory in the 1990s, when, as he described it, cryptography-related math was export-controlled. Non-US people had to leave the room and the blinds came down for basic discussions. "That was just math, but it was being controlled. And it was awful."
Blundin said the bigger danger was neither arrests nor bans but corporate liability. "U.S. lawyers are relentless," he said. If class-action lawsuits target what people do with AI models, he argued, companies like Anthropic, which he said currently gives its best models to everybody in America, would pull back. They would use their models internally and release only products such as drugs or mechanical parts, because they would be liable for everything customers do. That, he said, would halt progress long before arrests and convictions. Wissner-Gross called it a chilling effect that could mean losing "50 years of progress."
The Pessimists Archive
Socher called these outcomes "sad sort of off ramps" in humanity's future, and said fear of new technology is old. He pointed to the Pessimists Archive, a Twitter account that collects historical panics about new inventions. He read a quote it attributes to Pope Alexander VI in 1501, saying printing could "bring about serious evils" and that full control of the press would be necessary. He argued that the wheel, too, was a horrible thing given the accidents, tanks and chariots with archers it made possible. Ismail added his favorite example: stories warning that the telegraph would kill humanity.
Socher said the archive is full of articles predicting that novels, computer games, computers and the internet would kill everyone. "And it's just like, it never does."