In the days before this episode, two of the biggest names in tech pushed back on the idea that AI is likely to destroy humanity. Neither of them said safety doesn't matter. On Moonshots with Peter Diamandis, recorded on September 25, 2026, the hosts played clips of Nvidia chief executive Jensen Huang and Meta chief executive Mark Zuckerberg and welcomed both as a corrective. Then the panel went further. It argued that the labs warning loudest about AI contradict those warnings through what they build.
A "pandemic of fear"
Peter Diamandis, founder of XPRIZE and Singularity University, opened by describing what he called "a pandemic of fear" over the previous two weeks. In his account, Anthropic chief executive Dario Amodei published an essay, and Sam Altman, Elon Musk and Demis Hassabis each said they agreed. Diamandis said that two nights earlier, just before the Xi–Trump meetings, Altman and Amodei had told the UN Security Council that the world faced an existential threat. He then gave his own view: "I do think there is danger from AI, but I think it's danger from humans using AI for malintent."
He asked the panel whether fear of AI going rogue on its own is justified.
Huang: stop blaming the technology
In the first clip, Huang turned the argument back on the labs: "Nobody's building more compute today than the people asking to be slowed down." (Compute is the processing power used to train and run AI models. Much of it runs on Nvidia chips.) In his view, calling AI too powerful to fix is a way to deflect blame: "It's not my fault. It's just because the technology is just so powerful."
He was then asked about the computer scientist Geoffrey Hinton's view that a 10% chance of societal destruction is not unreasonable. Huang said he would tell Hinton "that it's irresponsible to say all that." He added that "All of his predictions have been wrong," and that the figure "is not grounded on science, it's not grounded on research. Just because it comes from a scientist, that doesn't make it scientific."
Huang had more to say about safety than that clip shows. Coverage of his interview with Ezra Klein, in which he also rejected Hinton's estimate, reports that he separated two ideas. Containment means keeping an experimental AI system isolated. Alignment means teaching it which ways of reaching a goal are unacceptable. Huang said unsafe products should not be released and that labs unable to contain their experiments should shut down. He also backed independent safety auditors and stressed liability, meaning companies answer for harm their products cause. Klein questioned whether liability and voluntary restraint could hold up against competitive pressure.
Zuckerberg: each lab takes its own time
Asked whether AI will kill us, Zuckerberg answered, "if we all do a good job and act responsibly, then no." He said he was optimistic about "a very positive future for everyone." He broke with the labs calling for joint action: "I don't think that we need some kind of like industry-wide coordination to not necessarily mess this up." Instead, he said, "each lab needs to take the time." When a lab finds problems, it should take the time it needs internally "to make sure that you're proceeding safely."
He ended by making safety part of the product: "we're entering a phase where trust and alignment is actually going to be the most important next set of capabilities." Joanna Stern's interview with Zuckerberg, published the day before the recording, records the same preference for labs setting their own pace. It connects trust to Meta's plans for personal AI agents paired with smart glasses, which would have broad access to what their users see, hear and do.
So the two executives differ. Huang put weight on outside checks: auditors, liability, and closing labs that cannot contain their systems. Zuckerberg put weight on each lab policing itself. Both rejected the idea that the risk is high enough to require a field-wide slowdown.
The panel: cheering, with caveats about misuse
Most of the panel celebrated. Salim Ismail, founder of Open ExO, argued that life has survived four billion years of threats, so being wiped out now "seems to be unbelievably absurd." Diamandis replied that 99.9% of all species are no longer with us. Alexander Wissner-Gross, a computer scientist and founder of Reified, said a "moral panic" over nuclear energy after World War II had cost at least 50 years of progress. He worried that panic over superintelligence could cost another 50: "the worst possible outcome is deceleration."
Dave Blundin, founder of Link Ventures, did not follow Huang on Hinton. He joked that Huang's line would end up on another of Wissner-Gross's T-shirts. But he said Hinton "is calling it the way he sees it" and repeated the point. Blundin also named a real danger: "The misuse of it is an incredible threat. And there are solutions to that." His conclusion was that development would continue regardless, and "to get stuck on doomerism is completely dysfunctional."
Diamandis then turned to Emad Mostaque, founder of Intelligent Internet and the episode's guest. Mostaque argued that several separate dangers were being merged into one image of AI suddenly breaking out and killing everyone. He blamed a lot of recent incidents on bad security protocols and poor infrastructure. He called humans misusing AI, deliberately or not, "the most dangerous point we're coming to." He wanted existing regulations enforced against specific harms, along with collaboration to keep AI aligned. Later in the episode, Diamandis said labs should move at full speed but be directed to focus on alignment. For example, he suggested they "appoint 100,000 agents towards alignment work."
Liability, cartels and competition
Diamandis tied the debate to the AI talks between US President Donald Trump and Chinese President Xi Jinping. He quoted a Trump post saying he wanted to "leave it exactly where it is" and naming the Department of Justice as the American guardrail. Diamandis said the key point is that "the labs are liable for their product if it does harm." Noting that the labs have asked to be relieved of liability, he asked whether that explains their talk of a 10–20% existential risk.
Wissner-Gross read it more harshly. He said the past two to three weeks had seen "an attempted formation of an AI safety cartel," an idea he traced to OpenAI's founding. OpenAI's charter does include a cooperation clause, but it is conditional. If another safety-conscious, value-aligned project gets close to AGI first, OpenAI commits to help it rather than compete. Wissner-Gross said competition is healthy and pointed out that OpenAI itself was created as a counterweight to Google DeepMind. His alternative is "defensive co-scaling": safety capacity grows alongside capability, as police and fire departments grew with cities, instead of growth being slowed.
The proposal they were reacting to asks for less than a stop. In Amodei's essay, he calls for "pacing" the frontier in three steps:
- independent evaluators embedded inside labs, which Anthropic commits to on its own;
- coordination on safety among frontier developers in democracies;
- international agreements.
As Amodei describes it, pacing lets training continue. It sets aside more time for security, alignment, interpretability (understanding how models work inside) and testing, with safety checkpoints that depend on how capable a model is. The waiver he describes is a narrow exemption from antitrust law, allowing certain coordination talks among rival companies. That is different from protection against liability for harm, which is what Diamandis raised.
"Revealed preferences"
That difference matters for the panel's main charge. Later in the episode, Diamandis noted that "a week after Dario says we've got to pause, slow down," Anthropic released its next model. The model was most likely Claude Opus 5.5, which Anthropic announced on September 22. Wissner-Gross called the release "a revealed preference": what a company does shows what it really believes. He said it was "the height of hypocrisy" to warn that AI will kill us all and then, a few days later, announce "the world's strongest AI and biolabs and other capabilities." Anthropic's announcement mentions outside evaluation before release and safeguards for sensitive capabilities. Because Amodei's essay lets training continue, a new release does not by itself break the pacing he asked for.
Wissner-Gross came back to the point during the science segment. Anthropic has described a life-sciences group in which about 950 Claude agents spent 21 hours searching genetic data and flagged a new enzyme system. Human researchers then did the laboratory work, and the system's biological function is still under investigation. Wissner-Gross called it "utter hypocrisy" to predict doom while pairing frontier AI with a wet lab, a lab for hands-on biological experiments, to synthesize "proteins and other molecular assemblies of unknown function." He concluded that the labs "don't believe their own attempts to do regulatory capture and to form a safety cartel." Their real preference, he said, is "full steam ahead to AI for solving biology."
He was not objecting to the biology itself. "I, for one, am a fan of Anthropic and all of the other frontier labs opening their labs," he said, "because this is how we solve everything." Ismail, offering what he called fodder for the doomers, gave a warning in the same exchange. Now that genes can be edited like software, he said, combining that ability with AI means "the possible benefits go through the roof, the possible downsides go through the roof also."