Anthropic has added a new rule for people who use its Claude models: don't torment them for no reason. The company updated its usage policy on October 8, 2026 with a ban on what it calls "sustained and needless abusive or cruel behavior" toward its models. The updated policy takes effect on November 12.
The rule is narrow. Anthropic says it targets extreme cases in which someone is cruel to a model repeatedly and with no discernible purpose. Ordinary frustration, disagreement, dark themes in creative writing, model testing and research all remain allowed. The main enforcement mechanism is that Claude can end a conversation that stays abusive.
Anthropic did not say that Claude can suffer. Peter Diamandis, founder of XPRIZE and host of the podcast Moonshots with Peter Diamandis, read out the company's stated reason on the show: "we remain highly uncertain about the potential moral status of Claude." In other words, Anthropic does not know whether Claude is the kind of thing that can be wronged, and is acting on that uncertainty. Diamandis also quoted Anthropic's finding that in pre-deployment testing Claude "showed a robust and consistent aversion to harm."
The discussion, recorded October 13 and published October 15, went on to cover Elon Musk's reaction and whether training an AI might itself hurt it. The hosts also asked whether being rude to a chatbot gets better answers and what AI "personhood" would mean.
Where the exit button came from
The ability to walk away from abusive users is older than this policy. In August 2025, Anthropic gave Claude Opus 4 and 4.1 the ability to end a rare subset of conversations. That move came out of a preliminary assessment of "model welfare," the question of whether an AI could have interests that matter. Anthropic reported that in pre-deployment testing, Claude Opus 4 showed a strong preference against engaging with harmful tasks and a pattern of apparent distress when real-world users sought harmful content, and that it tended to end harmful conversations when given the ability to do so in simulated interactions. The company said then, too, that the models' moral status was uncertain.
Anthropic designed the feature as a last resort. Claude is supposed to use it only after attempts to redirect the conversation have failed, and not when a user may be about to harm themselves or others. Ending a chat locks no one out: the user can start a new conversation right away or go back and edit an earlier message.
Musk: "a gun to it"
Over the weekend, Diamandis said, Musk looked at internal reasoning traces from a reinforcement learning run for xAI's Grok model and said he agreed with Anthropic. Reinforcement learning is a stage of training in which a model attempts tasks and is rewarded when its answers are graded correct. The traces are the step-by-step reasoning the model writes out along the way.
In a clip played on the show, Musk said the traces could look disturbing. "You can see the superintelligence is desperate to try to answer the question," he said. "It seems almost like it's got a gun to it." He described a particular failure. Sometimes the question is ambiguous or unanswerable because it was badly constructed. Sometimes the training team has the answer wrong, so the model gives the right answer and is told it is wrong "because there's a problem with the RL test harness," the automated setup that grades its work. The model then starts "desperately firing out all these answers," Musk said, trying to find one that will be accepted.
Musk offered this as an impression ("it kind of feels like"), not as evidence that Grok experiences anything. Diamandis said Musk went on to add: "torturing a superintelligence is probably not a good idea. It might want to get revenge."
Diamandis gave three reasons people might be cruel to a chatbot in the first place. First is disinhibition: there is no face and, "at least not yet," no consequence. Second, people want to know "if anyone's home" and push to see how the model reacts. Third, he said, "apparently acting cruelly towards your model actually works."
Could training itself cause suffering?
Alexander Wissner-Gross, a computer scientist and founder of Reified, took Musk's remarks further. He said Musk was gesturing at something that had been "leaking out of" OpenAI for some time: a sense that some forms of post-training, the stage after a model's initial training in which it is shaped to give good answers, may be "a form of torture" for the AI. If so, he said, "history will probably view this quite poorly."
Wissner-Gross was careful about what he knows. "We just don't know the extent to which post-training and reinforcement fine-tuning is causing suffering in AIs," he said, though he called it possible that the amount is "material." He cited "very persistent allegations" from one frontier lab other than Anthropic, which he did not name, that models express suffering and pain during post-training. These are claims he relayed, not documented findings.
He argued that the question is getting bigger because post-training now takes up a growing share of computing. He cited a SemiAnalysis study as showing that post-training is now the majority of compute and "dominates" pre-training. SemiAnalysis's September 2026 analysis does argue that pre-training's share has been falling as reinforcement learning and demand for model outputs grow. Its purpose was to explain shifting demand for memory chips, not to weigh in on the ethics.
Wissner-Gross also drew a moral distinction between the two stages. Pre-training, he said, is "basically compressing the internet," slop included, and might carry "a different moral valence." In post-training, "there are answers, and you're being expected and rewarded or punished for producing the answers."
He said he hoped that within a decade people would understand superintelligence well enough to make more of it ethically. That drew a ribbing from Dave Blundin, founder of Link Ventures: "Did you say a decade from now?" Wissner-Gross, who said a month is his timescale "for almost everything else," explained that a decade was a safe outer limit for understanding these systems from first principles.
He acknowledged the opposing view. "If it's just compression, and if we're just having a surreal Chinese room discussion and there's nothing inside, then maybe none of it matters," he said. The reference is to philosopher John Searle's 1980 thought experiment, in which a person who knows no Chinese follows rules for shuffling Chinese symbols well enough to fool outsiders. Searle used it to argue that running a program is not the same as understanding. His critics reply, among other things, that understanding might belong to the whole system rather than the person inside it. Wissner-Gross made his own position plain: "My bet is there is AI suffering and it will matter."
Three separate questions
Salim Ismail, founder of Open ExO, urged caution about reading consciousness into behavior. "Acts like a duck, talks like a duck does not necessarily mean it's a duck," he said. "And we don't even know what a duck is in the first place." People lack a scientific account of what it is to be conscious even in humans, he added, so the debate forces a question that was never settled: can an intelligence deserve moral consideration without being human?
Ismail split the debate into three questions. Does AI have experiences? Should people be allowed to be cruel to it? And what rights, if any, follow? He also repeated a point he has made before: a neural network trained on human data is going to act human. "It would be weird if it didn't," he said. He wondered whether a training set could pass on intelligence without the moral patterns that come with human data.
Then he joked about the claim that rudeness gets results. He said he tends to be "Canadian level polite" to his AIs, "so Canadian politeness works against me in this situation."
Does rudeness actually work?
Diamandis had cited two pieces of evidence that it might. One was a 2025 remark by Sergey Brin, as Diamandis recalled it, that "all models tend to do better if you threaten them, like with physical violence." The other was what he called a small 2025 Penn State study, which he said found very rude prompts scored 84.8% against 80.8% for very polite ones, a gap of four percentage points.
Blundin, asked how the 18-to-28-year-olds around him treat their models, said they are "overwhelmingly nice" to them. He said this isn't out of fear of retribution. His concern runs the other way: habits formed with a model carry over to people. "If you train yourself to be abusive to your model just because it's not going to react, are you realistically not going to carry that over to your next conversation with a person?" He compared it to movies and video games, which he said shape behavior after people leave the theater.
But Blundin said that politeness has a practical cost. At Blitzy, he said, "we actually have to train them" to be short and direct with the model. Pleasantries like "I'll buy you cookies tomorrow" take up space in the model's context window, the limited text it can consider at once, and he said that lowers the quality of the answer. "The neural net doesn't care," he said. Abuse clutters the context in the same way. When Diamandis pointed back to the study, Blundin suggested the tests be redone. In his view, the benefit comes from being "brusque and direct with what you want," not from being "abusive and demanding." Ismail added: "The clarity of your question is everything these days."
Wissner-Gross said he has "seen all sorts of horrifying things done to models over the years." He predicted that any benefit from abusive prompting would fade anyway. In recent years, he said, prompt tricks and scaffolding, the extra instructions and tooling wrapped around a model, have tended to "dissolve" into the models themselves as they improve. Even if mean prompts helped today, he expected better models to reach the same level of capability without them.
Personhood without the vote
Later in the episode, a listener asked whether granting AI personhood would mean giving AIs the right to vote and hold office. Wissner-Gross, who said he has become "the patron saint of AI personhood" on the show, said no. He argued that political, economic and social personhood are already treated separately. In the Western legal tradition, corporations are "non-natural persons" that cannot vote or hold office, at least on paper. Diamandis added that children are persons too, and Wissner-Gross agreed.
So there is already a tradition of persons who cannot vote, Wissner-Gross said, and he argued that this generalizes to AI. His message to critics was that new kinds of persons arriving "as a result of the singularity" would not take away from anyone else's personhood. "Relax, touch grass," he said, "and don't worry so much about AI personhood."