Tag
Briefings and ideas about Language models from AI podcast discussions and the research they cite, with links to the original sources.
Goodfire's Eric Bigelow argues in-context learning covers nearly everything a model does to adapt as it reads. He expects context, memory and harnesses, not per-user fine-tuning, to drive personalization, but hasn't researched continual learning.
Goodfire researcher Eric Bigelow resampled a GPT-3.5 model's reasoning at every token and saw answers collapse at single tokens, some arbitrary. He argues the "decision" happens in sampling; newer reasoning models often fork more smoothly.
Goodfire researcher Eric Bigelow said training that rewards only final answers gives models no reason to keep their reasoning readable. He said his optimism about monitoring that reasoning has faded sharply over about a year.
Google DeepMind's Pushmeet Kohli said scientists need models with calibrated confidence and known limits more than human-readable inner workings. Biohub's Sal Candido said protein models hold knowledge not yet unlocked.
Periodic Labs' Liam Fedus says its records of failed experiments, and of the whole process behind each result, give its AI data that published research lacks. He argues even stronger future models will still need physical experiments.
Liam Fedus of Periodic Labs says rewarding AI for finding a superconductor is too slow and noisy. The startup instead rewards it for identifying what X-ray data show a lab actually made, though ambiguity remains.
Nathan Labenz argues an AI-assembled music track shows models generalizing beyond tasks with checkable answers. Mercor's Edward Hu is more conditional: progress depends on how clearly and cheaply 'better' can be measured.
Applied Compute CEO Yash Patil, whose company trains custom models, says firms now weigh cost against performance instead of chasing the biggest model. He cites OpenAI's Baseten deal, OpenRouter usage and cheap task models like TypeSafe's Jev.
TypeSafe says its Jev model returns typed choices or scores in 70–500 milliseconds in its own tests. On Moonshots, panelists said such models could handle many small organizational decisions and noted open-source versions exist.
Recursive CEO Richard Socher said weak AI self-improvement already exists and fuller versions are close. He put superintelligence in his strongest sense, beating all of humanity across ten kinds of intelligence, "probably several decades" away, and expects narrower superhuman abilities such as programming and math within a few years. Alex Wissner-Gross rejected the 20-year timeline, and Peter Diamandis agreed.
Alex Wissner-Gross said Sonnet 5.5's 70% Terminal-Bench score is no better value per attempt than Opus 5.5; Anthropic's charts put the savings at lower effort settings. He said Gemini 4 Argon stands out mainly on avoiding hallucinations.
Keerthana Gopalakrishnan of Google DeepMind said robot training will probably blend teleoperation, sensor-equipped handheld rigs and human video, each trading scale for precision. On world models, she said the jury is still out.