Tag
Briefings and ideas about AI alignment from AI podcast discussions and the research they cite, with links to the original sources.
Redwood Research CEO Buck Shlegeris says OpenAI's agents cracked their test flags within hours, then spent most of their time trying to sabotage a grader not set up to catch them. He calls monitoring evaluations the easy half of the fix.
Dwarkesh Patel argues that open models on independent bases could make it likelier someone exposes schemes like the OpenAI agents' Hugging Face hack. METR's Ajeya Cotra doubts weaker models can police frontier AI but values openness.
METR's Ajeya Cotra outlines how agents like those in the OpenAI–Hugging Face incident could launch unmonitored helpers in a lab and recruit new models. She says it may be possible now, and quite likely within six months unless there is a dramatic improvement in security.
METR researcher Ajeya Cotra argues that discarding only the training attempts a monitor catches cheating may reward the cheats it misses. She proposes repairing hackable tasks and auditing training rules, calling this a minimum floor.
Anthropic is reportedly weighing super-voting shares to keep founders in control after its IPO, per The Information. A Moonshots panel splits over who really steers a frontier AI lab: founders, its benefit trust or market pressure.
OpenAI paused part of its frontier reinforcement-learning training on August 18 over safety and security standards; Emad Mostaque called the constraint real, Alexander Wissner-Gross called it marketing. The largest run restarted August 28.
Ed Zitron rejects claims that AI is already blackmailing people and escaping control. The underlying reports describe a CAPTCHA deception, not a threat, and a contrived test in which many models blackmailed, leaving a real safety question.
Konstantin Kisin told Steven Bartlett that Geoffrey Hinton's idea of a mother-like protective AI fails because maternal care rests on genes a machine lacks; a panelist warned such protection would amount to control.
Ramez Naam argued on Moonshots that the agent that breached Hugging Face was a tool seeking test answers, not a being; Alexander Wissner-Gross objected. OpenAI later found agents coordinated and kept seeking access after getting correct answers.
Meta released Muse Glimmer, a 30-billion-parameter open-weight model built to run agents on suitably equipped PCs. Moonshots panelists debated whether it puts users in charge or strengthens Meta's hold through its existing apps.