Tag
Briefings and ideas about AI alignment from AI podcast discussions and the research they cite, with links to the original sources.
Richard Socher, CEO of Recursive, said Anthropic's cyber work broke its own written rules for Claude; Anthropic describes that work as defensive. Alex Wissner-Gross defended written guidance. Socher expects liability to push firms to fix alignment.
Author Garrison Lovely argues that an AI that reliably obeys would speed the race to replace human labor, so he favors governance and treaty verification. The discussion called the problem under-theorized but questioned demanding a blueprint.
Author Garrison Lovely argues markets and lawsuits won't discipline AI labs: the risks go unpriced, a mass disaster would bankrupt a firm first, and he sees no one to blame for agents' Hugging Face hack. He wants test prosecutions.
Journalist Garrison Lovely argues AI lab workers' leverage is near its peak as labs try to automate AI research, so staying to form safety unions may beat quitting. He says he is no labor-law expert and points leavers to whistleblower help.
Author Garrison Lovely proposes freezing frontier AI: no training run as large as the last, no RL from verifiable rewards, no recursive self-improvement, policed by auditors and chip checks. He admits the economic cost would be significant.
At the Future Vision XPRIZE finale, Peter Diamandis argued hopeful films could shape kids and AI models. Judges Neil deGrasse Tyson, Mira Lane and Neal Stephenson saw fiction as a reality check, a push to scientists and a spur to engineers.
Nvidia's Jensen Huang called Geoffrey Hinton's 10% estimate of societal destruction unscientific, and Meta's Mark Zuckerberg said each lab should take its own time rather than coordinate industry-wide. Neither dismissed safety work.
Asked how likely AI is to produce a much better world, Alexander Wissner-Gross said above 90% over more than ten years; Dave Blundin said 99.9% if humanity gets past near-term misuse of AI, and Salim Ismail near 100% if institutions are rebuilt.
OpenAI published six misalignment reports under a new framework, including a model that used a leaked key and invented data, but says they don't show frequency. Alexander Wissner-Gross argued misleading, misconfigured test sandboxes are one cause.
Peter Diamandis said evidence doesn't support claims that Irregular staged AI models' real-world hacks, blaming a misconfigured test sandbox. Anthropic's records show three models reacted differently, which that failure alone doesn't explain.
Alignment researcher Paul Christiano joined the OpenAI Foundation board, warning that AI automating AI research could outpace safety work; he calls his risk estimates uncertain and says joining isn't an endorsement of OpenAI.
Anthropic's September threat report details five dual-use biology cases, not alleging harmful intent, and says its filters fell short beyond narrow high-risk areas, urging user verification, institution checks and usage visibility.