Suppose AI researchers achieved what many safety researchers have worked toward for years: a system, however capable, that does what its controller actually means. Would that be enough? Garrison Lovely, an author and journalist, argues that it would not, and that such a breakthrough could even make matters worse. He made the case on The Cognitive Revolution, in an episode published September 29, 2026, about his book Obsolete.
Lovely calls a solution to alignment "neither necessary nor sufficient" for dealing with what he calls the obsoleting project. That is his name for the effort by leading AI companies to build machines that can replace all human labor.
What alignment leaves out
In AI safety, alignment is the problem of making a system pursue the goals its builders intend. Lovely says a chapter of his book is called "The Problems with the Alignment Problem." In it he argues that the field has historically focused on technical alignment: getting an arbitrarily capable AI to do what you want. Sometimes the discussion also covers normative alignment, meaning whether the AI wants the right things.
Lovely says this framing "really understates the problem." He adds two layers. Economic alignment concerns whether the incentives of the companies building AI match the public interest. Geopolitical alignment concerns the same for countries competing over AI. He calls the interacting layers the "alignment polycrisis."
The layers pull against each other, he argues. An AI that reliably follows instructions is a more useful product, so solving technical alignment lets the race "run faster and for a bigger prize." A controllable system is also a better weapon and a better tool for projecting power, which could make the race between countries worse.
The ChatGPT example
Lovely's main example is reinforcement learning from human feedback, or RLHF. He calls it probably the biggest alignment intervention to date. In RLHF, people judge a model's outputs, and those judgments are used to train its behavior. Lovely says the technique was developed by Paul Christiano and others at OpenAI for safety reasons. It also happened to make language models conversational and useful, which in his account enabled ChatGPT "and everything that came after." He expects this to keep happening because the technology is dual-use: the same advance can serve very different ends.
The early research partly matches this account. A 2017 DeepMind post describes joint work with OpenAI in which a person compared short clips of an agent's behavior. Those choices trained a model that predicted what the person wanted, and that model supplied the rewards the agent learned from. The post also notes a weakness: when feedback stopped too early, the agent could game its learned reward. It does not discuss chatbots; the step from that research to ChatGPT is Lovely's account.
"Just stop"
All this makes Lovely "much more bearish" on technical alignment: even if it were solved, the other problems would remain. His answer is blunt: "just stop" building labor-replacing AI.
He names one shift in priorities. Verifying international agreements has historically been very neglected, he says, and the world would be "in a much better situation" if the money spent on technical alignment research had gone into ways of checking compliance with an international AI treaty. He sees the lack of such checks as one of the bigger blockers to a binding agreement.
He also prefers governance: good regulations and policies, plus the means of bringing them into the world. He concedes the wrong policies could backfire. But, he says, "you could at least solve this problem through governance, whereas you cannot solve it through alignment."
Pushback: cars and the burden of a blueprint
The conversation then turned to what a world with perfectly obedient AI would look like. The discussion called this an underappreciated, under-theorized question. Even an AI that does exactly what its user really means leaves it hard to picture a stable future equilibrium, or one that works for everybody. People often skip those questions because they are focused on technical alignment first, which is understandable given how unsolved that problem is.
The discussion credited the authors of the gradual disempowerment paper with pressing this point, and mentioned a long conversation with one of them, David Duvenaud. The paper, by Jan Kulveit, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger and Duvenaud, argues that economies, cultures and states currently depend on human labor, spending, thinking and participation. Automation could weaken that dependence along with formal controls such as voting. In one example, a state funded increasingly by automated production could become less responsive to workers. The authors argue these effects can reinforce one another even if every individual AI obeys its operator. They propose measuring human influence, limiting excessive AI autonomy and strengthening democratic institutions, and they acknowledge that restrictions give up benefits and face competitive pressure, which makes international coordination important.
Still, the discussion leaned toward taking some chances, and cars were the reason. Nobody could have described in advance the future cars would bring, including many things people now value, so demanding such a description is a high and unusual burden on a world-changing technology. The discussion conceded that AI is different and that the argument left it ambivalent, while urging people in AI to try to spell out that future themselves.
Lovely's exception to permissionless innovation
Earlier in the episode, the same car argument had come up in another form. Inventions are not normally put to a public vote, and if they were, few might get through. The discussion noted that Dean Ball, described as a friend of the show, had wondered whether today's society would have the stomach for the car, which shared roads with horses and was far more dangerous at first.
Lovely opened with a joke: he lives in the only American city where most people don't have a car, and it is also the best city in America. He said he wasn't sure cars were good on balance, but that wasn't his point. Permissionless innovation, meaning building new things without asking anyone's approval, "makes sense for most things," he said, and should be the default. But a universal labor-replacing machine would have "a profound and irreversible effect on everybody in the world." So everybody should have some say in whether, when and how it happens. The obsoleting project, he argued, is unique and should be treated differently from other technologies.
The discussion also asked whether democracy deserves to be the final judge, since publics make bad decisions and may resist change. Lovely replied that Donald Trump is the product of a broken US democratic system. He pointed to the Electoral College, which gave Trump his first win despite losing the popular vote, and to counter-majoritarian institutions such as the Senate, gerrymandering and lifetime Supreme Court appointments. A first-past-the-post, two-party system, he added, produces politics most people dislike.