Tag
Briefings and ideas about Recursive self-improvement from AI podcast discussions and the research they cite, with links to the original sources.
Richard Socher, CEO of Recursive, said Anthropic's cyber work broke its own written rules for Claude; Anthropic describes that work as defensive. Alex Wissner-Gross defended written guidance. Socher expects liability to push firms to fix alignment.
Recursive CEO Richard Socher said weak AI self-improvement already exists and fuller versions are close. He put superintelligence in his strongest sense, beating all of humanity across ten kinds of intelligence, "probably several decades" away, and expects narrower superhuman abilities such as programming and math within a few years. Alex Wissner-Gross rejected the 20-year timeline, and Peter Diamandis agreed.
Richard Socher, CEO of Recursive, said on Moonshots that a ban on recursive self-improvement could only be enforced with total surveillance. Other panelists said voter backlash and lawsuits were the bigger threats to AI.
Journalist Garrison Lovely argues AI lab workers' leverage is near its peak as labs try to automate AI research, so staying to form safety unions may beat quitting. He says he is no labor-law expert and points leavers to whistleblower help.
Author Garrison Lovely argues that China fast-follows the US frontier, so a US slowdown would slow both countries. He sees US credibility on AI risk as the main barrier to a deal, and concedes a deal might appeal less to the US.
Author Garrison Lovely proposes freezing frontier AI: no training run as large as the last, no RL from verifiable rewards, no recursive self-improvement, policed by auditors and chip checks. He admits the economic cost would be significant.
Robinhood CEO Vlad Tenev argued AI research is easier to automate than consumer apps because its experiments have ready-made evals, while app changes must wait for enough users to show statistically significant gains.
Alignment researcher Paul Christiano joined the OpenAI Foundation board, warning that AI automating AI research could outpace safety work; he calls his risk estimates uncertain and says joining isn't an endorsement of OpenAI.
Alexander Wissner-Gross called Anthropic CEO Dario Amodei's plan for pacing frontier AI a "safety cartel" on Moonshots; Dave Blundin said labs may need antitrust permission to coordinate but agreed shared safety rules could shut out startups.
OpenAI's Noam Brown says AI agents may soon work longer than the gap between model releases, leaving too little time for full testing. He agreed that slowing releases would widen the lead of models labs keep in-house.
OpenAI researcher Noam Brown says AI could make AI research perhaps three times faster. He argues that, unlike maths, research waits on sequential experiments and GPUs, so he doubts a 100-fold jump, and says his uncertainty is large.
OpenAI's Noam Brown says the company sees signs that models' written reasoning is becoming harder to monitor. He also says he has no full answer yet for how a lab would know AI-built successor models stay aligned.