Tag
Briefings and ideas about Language models from AI podcast discussions and the research they cite, with links to the original sources.
John Schulman expects frontier AI models to keep growing as compute and GPUs expand. Charlie O’Neill expects little parameter growth for a few years, because long reinforcement-learning runs reward cheap inference.
AI researcher John Schulman says cheap distillation keeps model competition open. He argues that a copy gets broad behavior only from realistic, multi-turn prompts, and that benchmark-style tasks alone may match scores without it.
On a Dwarkesh Podcast panel, John Schulman forecast that AI will make AI researchers ten times more productive in about two years. Charlie O’Neill put it five to ten years away, arguing that choosing experiments and objectives is hard to automate.
Dwarkesh Patel and Jerry Han found better training data gave a 12-fold compute-efficiency gain from 2019 to 2025 versus 3.7-fold for model recipes, in small models on easy benchmarks; a panel split on how long proprietary data stays valuable.
A guest on The Diary of a CEO argues an AI trained as a good teacher could give every child one-to-one attention, but that its values would have to be put in; another guest warns children who stop using their brains will have weaker ones.
Tim Sweeney tweeted that Elon Musk's predicted 100x gain in intelligence per gigabyte has come true; Musk says specialist AIs add another 100x. Dave asks how 10,000 agents would work together; Alex says specialists are really sparsification.
Dave Blundin of Link Ventures says one bad coding idea can spread through all 5,000 of his identical AI agents, costing roughly $50,000 in tokens unless he rewinds them. Panelists linked it to a paper on "mind viruses" spreading between agents.
Moonshots panelists debated a relayed claim that 70% of China's AI token use goes to video: Alex Wissner-Gross said US labs chase revenue per token, and Dave Blundin that video sells where enterprises won't trust Chinese models.
Gemini 3.7 Flash topped AA-AnalystAgent at 60% to Claude Opus 5's 54%. Alexander Wissner-Gross argued the score, which counts only questions solved on all five attempts, reflects reliability, not frontier capability.
Ed Zitron said his Bloomberg terminal's AI invented a Microsoft share price and that his editor Matt Hughes, not a hallucination benchmark, earns his trust. The host argued buyers pay for output and AI should be compared with fallible people.
AI critic Ed Zitron argues chatbots help with small tasks but aren't worth trillions, and that subscriptions hide what usage costs. He asks whether users like the host's fiancée would pay the real per-token price.
Ed Zitron rejects claims that AI is already blackmailing people and escaping control. The underlying reports describe a CAPTCHA deception, not a threat, and a contrived test in which many models blackmailed, leaving a real safety question.