"It's great that everybody signed, but who can tell them no?" Salim Ismail asked. "What happens if somebody breaks the agreement?"
Ismail, founder of Open ExO and a venture investor, was reacting to the White House Accord on Super Intelligence, a voluntary pledge signed at a lunch in the East Room this week. In an episode of Moonshots recorded October 1, 2026, host Peter Diamandis, founder of XPRIZE, put the accord in front of his co-hosts and guest Emad Mostaque. They agreed that it was light. They disagreed about what it was for.
From "slow down" to lunch
Diamandis called the run-up "this roller coaster." About two weeks earlier, he said, Dario Amodei of Anthropic, Sam Altman of OpenAI, Elon Musk and Demis Hassabis of Google DeepMind had called on the industry to "pace the frontier," meaning to slow the race toward ever more capable systems. Then, by Diamandis's account, President Trump told reporters he wanted to leave things where they were, arguing that liability laws and existing protections were enough. On the Tuesday of recording week, Trump hosted nearly 20 tech leaders for lunch with House Speaker Mike Johnson.
Diamandis listed Musk, Mark Zuckerberg, Jensen Huang of NVIDIA, Amodei, Sundar Pichai, Satya Nadella, Lisa Su and OpenAI President Greg Brockman among those present. Altman was in San Francisco for OpenAI's developer day. Diamandis lingered over the seating chart: Trump sat between Huang and Musk, while Amodei was at the far end. "I think he's been sent to the outer pastures," Diamandis joked. The signature page of the accord lists Trump alongside Pichai, Amodei, Zuckerberg, Brockman, Musk and Huang.
What the four layers do
The accord does not set a single shared safety test. Instead it commits each company to build a stack of checks around its own work. Diamandis read out the four layers:
- Internal controls that monitor a model's capabilities and alignment, meaning whether it behaves as intended, during training and deployment. These focus on cyber, biological and chemical risks, and on models hacking or getting into technical systems in ways nobody intended.
- An internal team that checks whether those controls actually work.
- An external auditor or evaluator that assesses the arrangements independently.
- An independent committee of each company's board of directors that oversees the process and makes sure problems get fixed.
The published text matches that structure. It also commits participants to meet regularly to develop standards and share practices, and it leaves open the possibility that the commitments could later become law or regulation.
In a clip Diamandis played, Trump called the accord "a form of protection." "They understand that they have to self-police," Trump said, adding that the models were "very complex" and that when they are misused, "we're going to be able to nab them." He also floated a committee of perhaps 10 people to "watch over the whole enterprise." Asked whether the accord was binding, Trump said he thought it was "morally binding." "Not sure what that means, but hey," Diamandis said.
Amodei, standing beside the president, kept his warnings but softened their tone. The technology has "very real risks," he said, and how to address them "is still under discussion." But "if we do this right, we work with the president and everyone here, we can win safely." Diamandis wondered aloud "how many times he practiced that line."
Trump then called on Zuckerberg to explain the agreement. Diamandis noted that two weeks earlier Zuckerberg had said he did not think industry-wide cooperation was needed. Zuckerberg said the companies had "drafted a set of principles and commitments" built on internal controls and multiple layers of auditing, with boards independently reviewing the auditors' reports. "The idea isn't that this is the only thing that we will ever do," he said. "It is that this is a start."
"AI Kremlinology": the case that Zuckerberg won
Alexander Wissner-Gross, a computer scientist and founder of Reified, said it was obvious who was "pulling all the strings": Zuckerberg. He asked viewers to study the group photo, which he called the "AI Last Supper" image. Everyone else faces forward, he said, while Zuckerberg looks toward the president. When Diamandis replayed the Amodei clip, Wissner-Gross asked viewers to watch "the dog that's not barking": Zuckerberg, laughing and joking with Trump behind Amodei. He called it "a little bit of Kremlinology," after the old practice of reading Soviet power struggles from who stood where.
Wissner-Gross said body language was not the only evidence. Citing Semafor, he said Zuckerberg drafted the accord with Speaker Johnson in the past week. Ashley Gold's September 30 report for Semafor gives Zuckerberg a central drafting role, citing a person familiar with the discussions and two others who confirmed his involvement. It traces his conversations with Johnson at a preceding state dinner and describes Huang's work securing industry support. Meta declined to comment, and the White House did not respond.
In Wissner-Gross's reading, Meta was scarred by a decade of social-media scandals and fights with Congress, and so had learned to get ahead of regulation with "placeholder self-regulation." He called the result "total weak sauce," and said it looked to him like an attempt to preempt top-down regulation. The danger he worried about was a "safety cartel," in which leading labs use safety rules to lock out rivals. Based on the language, he was cautiously optimistic that this would not happen, though a clause reserving possible future executive regulation left the possibility open. "Zuck is the winner. Dario is the loser," he said. He argued that Amodei had been "trotted out" to take back his recent warnings. "I'm cautiously optimistic that this effort, largely by Zuck, credit to Zuck, has headed off the safety cartel," Wissner-Gross said.
"Ceremony" for an exponential problem
Ismail called his own view "as cynical." "We're applying ceremony now to an exponential coordination problem," he said. He wanted independent testing, transparent results and consequences for anyone who breaks the agreement. "Then you have something that's worth actually celebrating. This is a theater." He said Wissner-Gross was "dead on" that the accord had headed off the safety cartel.
He hoped the companies would at least test one another's models aggressively, which he said might bring some benefits. But he repeated a position he had taken before: the exercise was "pointless" because "there's no mechanism for slowing any of this down." Society, he argued, needs to "speed up our response to it, not try and slow it down."
A shield against lawsuits
Mostaque, founder of Intelligent Internet, saw a legal motive. He pointed to Meta's settlement with state attorneys general over harm to teens, which he described as an $18 billion deal agreed in August. Meta's announcement, dated August 26, 2026, broadly confirms the figure but qualifies it. The roughly $18 billion is paid over ten years, and about $5.3 billion of it depends on YouTube and TikTok adopting similar protections and making matching payments. The settlement also requires teen-account limits and an independent auditor that reports on compliance every year for five years.
Mostaque argued that coming lawsuits over "agents running amok," meaning AI systems that act on their own and cause harm, would rest on "almost exactly the same legal theory." Part of the core of the teen settlement, he said, was that Meta lacked checks and balances. An accord like this gives a company something to point to. Diamandis said another story that day made clear liability was not being waived: "You break it, you own it." He doubted the accord protected the companies. Mostaque agreed that it does not fully do so. A company would still be exposed if something went "really egregious," he said, such as a model hacking the Pentagon. But for lesser harms it could say it had put these measures in place with the president, which would "reduce that surface exposure area."
He also thought the choice of cyber, biological and chemical risks made sense. Cyber systems can be hardened, he said, and for biological and chemical harms the inputs can be constrained. When Diamandis asked whether he meant manufacturing or shipping, Mostaque said yes, adding that harms tend to emerge where AI meets the real world.
He was less impressed by the external evaluators. They would be "better than the subprime credit evaluators," he said, a reference to the rating agencies of the 2008 financial crisis, but he did not think they would be very effective. Evaluating a swarm of AI agents that can solve the Navier–Stokes equations would take a great deal of computing power and people as talented as those who train the models, he said. He put the number of such people at fewer than 100 worldwide. "It's a great growth industry, though, if you want to set up a company," Mostaque said, adding that such a firm should be independent.
One step further: keep the dangerous data out
The conversation then moved to the scrutiny of seat placements and signatures, which one remark in the discussion called theater, while stressing that the stakes are very high and that things could have gone another way. Setting aside the debates over extinction and pacing, that remark held, there are real potential harms on cyber, chemical and biological risks, and it was notable that everyone at the table agreed even to these very light commitments.
The same line of discussion proposed that companies go further than the accord and not train any publicly available models on frontier cyber, biological or chemical knowledge. Access to models that have that knowledge could be controlled, the argument went. The step would happen during pre-training, the main stage in which a model learns from a huge body of data. The argument cited multiple studies showing that knowledge left out at that stage is much harder to add back in later training. As things stand, it continued, labs pay hundreds of millions, if not billions, for those specialized datasets, which then go into a model that "a mum in Kansas has access to," which seemed odd.
When Diamandis asked for a closing word on the segment, the answer was a split verdict: a bit of theater, but directionally correct. It added that the AI safety debate would be interesting to watch as it separates into near-term harms and long-term risks.