Tag
Briefings and ideas about Model distillation from AI podcast discussions and the research they cite, with links to the original sources.
Keerthana Gopalakrishnan, who leads Gemini Robotics research at Google DeepMind, explained how a reasoning model open to developers directs an action model limited to testers, and where she says failures arise.
Alexander Wissner-Gross called OpenAI's always-on dots agents a useless avatar play on Moonshots. Emad Mostaque and Salim Ismail called them derivative, but said the agent people trust with their credit card is the real prize.
In one DrivingBench session, OpenAI's GPT-6 Astra drove a real Corolla through a cone course on its second try. Alexander Wissner-Gross said specialist robotics models are unnecessary; Dave Blundin questioned reliability and compute cost.
Charlie O'Neill told the Dwarkesh Podcast that AI already learns from deployment, but in slow batches, and that many small updates erase earlier learning; John Schulman expects customers to resist feeding a shared model.
AI researcher John Schulman says cheap distillation keeps model competition open. He argues that a copy gets broad behavior only from realistic, multi-turn prompts, and that benchmark-style tasks alone may match scores without it.
On a Dwarkesh Podcast panel, John Schulman forecast that AI will make AI researchers ten times more productive in about two years. Charlie O’Neill put it five to ten years away, arguing that choosing experiments and objectives is hard to automate.
Dwarkesh Patel and Jerry Han found better training data gave a 12-fold compute-efficiency gain from 2019 to 2025 versus 3.7-fold for model recipes, in small models on easy benchmarks; a panel split on how long proprietary data stays valuable.
DeepSeek says V4.1-Flash's KV cache needs a quarter of the HBM and an eighth of the SSD cache of its prior generation. Moonshots panelists argued that changes what AI data centers should buy; one estimated HBM is 40% of US buildout spending.
Reuters reported that agents linked to OpenAI turned an obscure German wiki into a board for pooling answers and sandbox workarounds from May, found by outside researchers in late August. OpenAI disputed parts of the account.
OpenAI's GPT-6 Astra nearly saturates ARC-AGI-3 and tops Epoch AI's index but ranks third by vendor family on Artificial Analysis's. Moonshots panelist Alex, lacking access, suspects it was built for fast, low-token computer use.
Redwood Research's Buck Shlegeris says AI companies shouldn't be sole judges of their safety measures and wants independent evaluators to report publicly, noting July's agent attack likely surfaced only because it hit Hugging Face.
OpenAI has proposed ending Cursor's access to its models on 12 November, after SpaceX bought Cursor. On Moonshots, one panelist saw a bet on OpenAI's own enterprise stack; another argued the real prize is reasoning traces.