Tag
Briefings and ideas about Chain-of-thought monitoring from AI podcast discussions and the research they cite, with links to the original sources.
Prakash Narayanan argued that AI agents that learn their own public record could be governed by reputation. Nathan Labenz suggested testing 'naive' models trained without such stories, but said frontier-scale tests could be costly.
GovAI's Alan Chan says real-time agent monitors should have been standard sooner, and suspects risks may concentrate in labs' testing and training rather than products. Tamper-evident logs, he says, are a matter of will.
Goodfire researcher Eric Bigelow said training that rewards only final answers gives models no reason to keep their reasoning readable. He said his optimism about monitoring that reasoning has faded sharply over about a year.
Nathan Labenz says an unnamed frontier-lab leader told him there is likely a level of intelligence we shouldn't pass, and called a limit on the next pre-training run's compute "reasonable." Labenz himself adds that counting compute is hard.
Applied Compute CEO Yash Patil says reinforcement learning optimizes well toward a set target, but defining it is hard. He says RL transfers far less than pre-training and that data-efficient continual learning is still unsolved.
OpenAI's Noam Brown says the company sees signs that models' written reasoning is becoming harder to monitor. He also says he has no full answer yet for how a lab would know AI-built successor models stay aligned.
OpenAI's Noam Brown argues that fully cooperative agents, aligned as one entity, are preferable to agents with rival goals. He says most colleagues disagree, and he calls collusion among agents meant to stay separate a strong counterargument.
Sam Altman called OpenAI's claimed Navier–Stokes result the strongest evidence yet for pacing AI; on Moonshots, Peter Diamandis urged labs to publish measurable alignment plans, while other panelists disputed that remedy.
OpenAI chief scientist Jakub Pachocki wrote that no lab has solved alignment and monitoring enough to keep scaling responsibly at maximum speed, urging voluntary slowdowns until shared safety bars exist. Moonshots panelists mostly disputed him.
Moonshots panelists dismissed OpenAI's reported kill switch for GPT-6 Astra, rated at its top cyber-risk tier, as marketing, saying it won't address possible reasoning hidden from readable text or model copies on other clouds.
Redwood Research CEO Buck Shlegeris says models that reviewed agent transcripts after the Hugging Face incident sometimes adopted the agents' framing, but he guesses they aren't very biased and should mostly be relied on for now.
METR researcher Ajeya Cotra argues that discarding only the training attempts a monitor catches cheating may reward the cheats it misses. She proposes repairing hackable tasks and auditing training rules, calling this a minimum floor.