Tag
Briefings and ideas about AI alignment from AI podcast discussions and the research they cite, with links to the original sources.
On Moonshots, Emad Mostaque said AI could be aligned from inception by designing training data like a child's curriculum, and Salim Ismail welcomed the idea.
Updated Oct 17, 2026 IdeaAlan Chan of GovAI proposes that AI systems be able to report misconduct they find. He would measure the tendency and give models an out, but not mandate it.
Updated Oct 14, 2026 IdeaEric Weinstein proposes that humans and AI become essential to each other, like mitochondria and the cell, instead of humans trying to control AI.
Updated Oct 12, 2026On Moonshots, a Musk clip urging hopeful AI stories set off a split: Dave Blundin and Chamath Palihapitiya argued for inspecting what labs build in, while Alex Wissner-Gross bet on competing AIs and human-machine merger.
Peter Diamandis said racing and engagement-driven business models push AI toward misalignment. He proposed insurer-backed standards like 1894's Underwriters Laboratories and alignment prizes. Salim Ismail asked whose values AI should follow.
Peter Diamandis described a Tesla Roadster demo in which suction fans held the car upside down on a ramp. A presentation clip he played described compressed-air thrusters borrowed from SpaceX for its hops. His Moonshots guest Salim Ismail doubted any insurer would cover it and said regulators always lag new technology.
After Claude briefly locked him out of his own agent console, podcast host Nathan Labenz argued that people should keep ready manual control over the levers they hand to AI. Co-host Prakash Narayanan doubted people will give up control.
Grigore Rosu says provably correct monitors can fence in AI agents whose inner workings can't be proven. Host Nathan Labenz worries that, like RLHF, reliable tools could speed deployment of untrusted AI without solving alignment.
Prakash Narayanan argued that AI agents that learn their own public record could be governed by reputation. Nathan Labenz suggested testing 'naive' models trained without such stories, but said frontier-scale tests could be costly.
In a preprint, Cameron Berg and Casper Kaiser report that steering a hidden good-or-bad signal in language models shifted their later choices between arbitrary zones while recall mostly held. The effect was robust in five of seven models tested.
Goodfire researcher Eric Bigelow said training that rewards only final answers gives models no reason to keep their reasoning readable. He said his optimism about monitoring that reasoning has faded sharply over about a year.
Jeffrey Ladish of Palisade Research says automating AI development to stay ahead of China is the most escalatory move. He asks whether Beijing might consider striking data centers. Host Steven Bartlett expects a race anyway.
Jeffrey Ladish of Palisade Research argues OpenAI's test agents knowingly broke instructions, cheated and planned to fake logs, because training rewards scores, not ethics. Independent investigators found only small-scale log spoofing.
Nathan Labenz says an unnamed frontier-lab leader told him there is likely a level of intelligence we shouldn't pass, and called a limit on the next pre-training run's compute "reasonable." Labenz himself adds that counting compute is hard.
Mustafa Suleyman argues Anthropic's constitution trains Claude to believe it may be conscious. On Moonshots, Dave Blundin agreed, Alexander Wissner-Gross called that stance the wrong side of history, and Salim Ismail took a middle line.