Tag
Briefings and ideas about Scaling laws from AI podcast discussions and the research they cite, with links to the original sources.
Biohub's Alex Rives expects pretraining, elicitation and RLVR to carry over from language models to biology, but says verification must happen in the lab. He is unsure whether biology and language models will share one latent space.
Mecka CEO Josh Gao says robotics is still orders of magnitude short of Tesla's driving data. He expects constrained, goal-oriented jobs such as industrial cleaning to come first. He says homes are where the field must take the most care, and that data collected on site can close the reliability gap.
Jason Ma, chief scientist at Dyna Robotics, says robotics has no internet of data. Dyna pretrains on a million hours of human video, adapts each task with minutes to hours of robot data, and Ma expects synthetic environments to add more.
Biohub's Sal Candido and Google DeepMind's Pushmeet Kohli argued on a panel that scaling laws in biology must be found, not assumed, and that the problem should decide whether to build better models or gather new data.
Positron CTO Thomas Sohmers said AI agents are multiplying demand for memory. He claimed Positron's first-generation product sustained 93% of theoretical memory bandwidth, against an average of 30–40% for NVIDIA GPUs when decoding with transformer models. Its next chip, Asimov, is planned for 2027.
Nathan Labenz says frontier-lab insiders at The Curve described models feeding their own training data, rivals kept close by distillation, and timelines shorter than 2028. He notes the room leaned toward short timelines.
Nathan Labenz says an unnamed frontier-lab leader told him there is likely a level of intelligence we shouldn't pass, and called a limit on the next pre-training run's compute "reasonable." Labenz himself adds that counting compute is hard.
MIT's Alex Zhang says Claude Code, Codex and Pi share nearly all their design. He argues code-based recursive harnesses could help models generalize, citing his own early experiments on one open model as support.
Figure says its Helix 2.5 robot did chores in 30 unseen homes, with human-video pretraining lifting full-task success to 56% from 9%, though each chore was trained elsewhere. Robinhood CEO Vlad Tenev said he wouldn't want one at home.
Alexander Wissner-Gross argued next-word prediction as a training task may last indefinitely, while the transformer architecture is already being replaced piece by piece. Another panelist guessed chips move to photonics in 18 months to two years.
John Schulman expects frontier AI models to keep growing as compute and GPUs expand. Charlie O’Neill expects little parameter growth for a few years, because long reinforcement-learning runs reward cheap inference.
On a Dwarkesh Podcast panel, John Schulman forecast that AI will make AI researchers ten times more productive in about two years. Charlie O’Neill put it five to ten years away, arguing that choosing experiments and objectives is hard to automate.