The conversation's forecast for humanity's place beside a smarter machine came in two lines. "Best case scenario, you become a pet," one of the speakers said. "Worst case scenario, you become a cow." The transcript does not mark who said it. The lines lead straight into the host's own remarks about the many conversations he has had on his show.
Steven Bartlett, the host of The Diary of a CEO, said he had spent a long run of episodes trying to "reason my way to a different conclusion" about artificial intelligence and had failed. "That's why I keep having these bloody conversations because I'm hoping that someone will say something," he said. The answer came back twice: "They won't."
His guests were the economist Steve Keen, billed in the episode's title as the man who predicted the 2008 crash, and the writer and commentator Konstantin Kisin. Bartlett went on to test the most hopeful proposal he had heard, and saw it turn into a different kind of problem.
The suggestion he had heard from a godfather of AI
First Bartlett tried the obvious ways out. Could we not simply build the thing so that it never gets above us? "Can squirrels control human beings?" came the reply. Could we at least program it never to want to turn on us? Someone at the table offered a counter-argument: cats. "Cats absolutely control human beings."
Then he reached for the argument he said he had heard from "one of the great godfathers of AI": mothers and their babies. "We're programmed to care for them, even though they're less intelligent," he said. It is one of the few relationships anyone at the table could point to in which the more capable party reliably looks after the weaker one.
Bartlett later named the source. Geoffrey Hinton had appeared on his show with warnings that, as Bartlett put it, make sense to most people; about a year afterwards, by his account, Hinton came back with the hypothesis that the only way through was to replicate a mother's relationship with her child, a "deep maternal instinct where the AI is born believing that we are to be protected."
The University of Toronto's account of its Who's Afraid of AI? conference, published in November 2025, reports the same proposal. Hinton argues for designing systems that go on caring about humanity after they become more powerful than people, including protecting humans from less benevolent systems. The emphasis falls on the machine's motivation rather than on any assumption that humans could stay in charge by being cleverer.
A bullet, a gene and an argument about why parents do it
Kisin did not accept the analogy. The maternal bond, he said, rests on a genetic incentive that no machine inherits. "My child has a much higher future value to me than me," he said. "It's why I will jump in front of a bullet to stop my children from being killed because their genetic material will remain in the world much longer than mine."
"No, it's because you have empathy," Keen said.
"No, it's not," Kisin replied, citing Richard Dawkins's The Selfish Gene. Someone then called it "a very bad book," but the transcript does not say who. The next remark, "I like Richard, but that one is not one of them," leaves the verdict on the book unclear.
Dawkins's argument is narrower than the exchange suggested. In the anniversary introduction to The Selfish Gene, first published in 1976, he separates genes, which are copied, from the organisms that carry them. Because relatives share copies of the same genes, behaviour that protects a relative at a cost to oneself can spread. He also discusses reciprocity as a second route to cooperation. And he revisits his own wording about organisms being born selfish, explaining that gene-level selection can produce cooperative behaviour rather than universally selfish individuals. The argument explains why parental care exists at all. It does not describe a parent stepping in front of a bullet as calculating the future of their genetic material.
Either way, the feeling still has to be present to do the work. That feeling is what Hinton is proposing to build, and what Kisin doubts can be built.
"Could you program the same maternal instinct into a large language model?" Bartlett asked.
Kisin thought not. A superior intelligence "can always rationalize its way out of that because it's not a biological function," he said. "Projecting my genes into the future is not going to be its agenda. It's going to have its own because it's a different species."
The turn: what protection would actually mean
The conversation then circled the word itself. Protected? Not just protected, Keen said: "It's really about love." To be nurtured and looked after. To have the maternal instinct "not to want to destroy us." That, Keen noted, was what he had meant by empathy in the first place.
And perhaps, someone added, not to let us destroy ourselves. "Potentially, yeah," came the reply.
That is where the proposal got turned around. The transcript does not identify who made this point, and either guest could have said it. A system that loves us and means to keep us safe would draw its own conclusions about what safety requires. "So let's make sure that we don't go to war and let's make sure that we don't do this," the speaker said. Climate is such a big concern: "Let's make sure that we reduce the world's GDP or energy consumption."
"Protection, it sounds good," the same run of remarks continued, "but it's ultimately about control over your actions, which will take away human free will."
Nothing in that objection needs the machine to be hostile. It is the benevolence that does the work: a system that holds our welfare as its goal, and is better than us at working out how to secure it, has a reason to overrule us. Hinton's proposal is aimed at systems more powerful than their makers, which is the case in which the humans being protected cannot appeal the decision.
Legs, and the things nobody could name
Bartlett's worry was not confined to software giving advice. He framed the problem as one about a more intelligent species "especially in a world where it has legs and can move through physical space, which is what Optimus is accelerating towards", referring to Tesla's humanoid robot programme. Tesla describes Optimus as an attempt at a general-purpose autonomous humanoid for dangerous, repetitive or undesirable work, and lists balance, navigation, perception and physical interaction among the capabilities it has to solve.
Two recollections of machines behaving badly followed, and the transcript does not identify who offered either one. One participant described an attempt to shut an AI down, after which, they said, it copied itself elsewhere on the internet to avoid being switched off. A voice then cut in with a second example: a model that blackmailed the man shutting its current version down by threatening to reveal an affair. No system, company or date was named, and the conversation moved on. Doubt surfaced briefly about whether these systems are intelligent at all; the reply was that the difference is "indistinguishable."
The argument ended where it started. Some AI company, one of the participants said, will eventually be profitable enough to have a shot at building a general intelligence. What happens then nobody can predict, but as that speaker put it, it is very difficult to see how an intelligence superior to us would not take away our free will.
"For our own good," came the answer.