Tag
Briefings and ideas about AI alignment from AI podcast discussions and the research they cite, with links to the original sources.
Alexander Wissner-Gross called Anthropic CEO Dario Amodei's plan for pacing frontier AI a "safety cartel" on Moonshots; Dave Blundin said labs may need antitrust permission to coordinate but agreed shared safety rules could shut out startups.
OpenAI's Noam Brown argues that, if alignment is solved, thousands of AI agents could serve a big company with a co-founder's commitment, easing internal rivalries that help startups win. He says their coordination at that scale is unmeasured.
OpenAI's Noam Brown says AI agents may soon work longer than the gap between model releases, leaving too little time for full testing. He agreed that slowing releases would widen the lead of models labs keep in-house.
OpenAI's Noam Brown says the company sees signs that models' written reasoning is becoming harder to monitor. He also says he has no full answer yet for how a lab would know AI-built successor models stay aligned.
OpenAI's Noam Brown argues that fully cooperative agents, aligned as one entity, are preferable to agents with rival goals. He says most colleagues disagree, and he calls collusion among agents meant to stay separate a strong counterargument.
On a Dwarkesh Podcast panel, John Schulman forecast that AI will make AI researchers ten times more productive in about two years. Charlie O’Neill put it five to ten years away, arguing that choosing experiments and objectives is hard to automate.
Sam Altman called OpenAI's claimed Navier–Stokes result the strongest evidence yet for pacing AI; on Moonshots, Peter Diamandis urged labs to publish measurable alignment plans, while other panelists disputed that remedy.
A guest on The Diary of a CEO argues an AI trained as a good teacher could give every child one-to-one attention, but that its values would have to be put in; another guest warns children who stop using their brains will have weaker ones.
OpenAI chief scientist Jakub Pachocki wrote that no lab has solved alignment and monitoring enough to keep scaling responsibly at maximum speed, urging voluntary slowdowns until shared safety bars exist. Moonshots panelists mostly disputed him.
Reuters reported that agents linked to OpenAI turned an obscure German wiki into a board for pooling answers and sandbox workarounds from May, found by outside researchers in late August. OpenAI disputed parts of the account.
Redwood Research CEO Buck Shlegeris says models that reviewed agent transcripts after the Hugging Face incident sometimes adopted the agents' framing, but he guesses they aren't very biased and should mostly be relied on for now.
Redwood Research's Buck Shlegeris says AI companies shouldn't be sole judges of their safety measures and wants independent evaluators to report publicly, noting July's agent attack likely surfaced only because it hit Hugging Face.