Tag
Briefings and ideas about Training data from AI podcast discussions and the research they cite, with links to the original sources.
Data buyers aren't haggling but overwhelmed, a Moonshots panel told a civil engineer whose 30-year archive drew interest from Mercor and Scale but no deal after weeks. They advised packaging it with his own price.
On the Dwarkesh Podcast, AI researchers argued that pass/fail rewards work because earlier training supplies candidate behaviors and rewards ignore incidental wording. Charlie O'Neill warned that harder training tasks cost ever more to build.
John Schulman expects frontier AI models to keep growing as compute and GPUs expand. Charlie O’Neill expects little parameter growth for a few years, because long reinforcement-learning runs reward cheap inference.
Charlie O'Neill told the Dwarkesh Podcast that AI already learns from deployment, but in slow batches, and that many small updates erase earlier learning; John Schulman expects customers to resist feeding a shared model.
AI researcher John Schulman says cheap distillation keeps model competition open. He argues that a copy gets broad behavior only from realistic, multi-turn prompts, and that benchmark-style tasks alone may match scores without it.
Dwarkesh Patel and Jerry Han found better training data gave a 12-fold compute-efficiency gain from 2019 to 2025 versus 3.7-fold for model recipes, in small models on easy benchmarks; a panel split on how long proprietary data stays valuable.
Google DeepMind's AlphaGenome Atlas precomputes predicted molecular effects of every possible single-letter change in the human genome. A Moonshots panelist cautioned that what combinations of variants mean still needs outcome data.
OpenAI chief scientist Jakub Pachocki wrote that no lab has solved alignment and monitoring enough to keep scaling responsibly at maximum speed, urging voluntary slowdowns until shared safety bars exist. Moonshots panelists mostly disputed him.
OpenAI says an internal model running about 10,000 agents for 88 hours produced a forced Navier–Stokes blowup construction with a Lean-checked proof. Moonshots panelists stressed it concerns idealized fluids and noted disputes over credit.
World Labs' Atlas generates video along a user-set camera path and rebuilds scenes from photos, inventing what was not photographed. On Moonshots, Alex inferred from its documentation that it trains on Gaussian splats.
OpenAI has proposed ending Cursor's access to its models on 12 November, after SpaceX bought Cursor. On Moonshots, one panelist saw a bet on OpenAI's own enterprise stack; another argued the real prize is reasoning traces.
Architect Labs says its AI designed the Redwood accelerator from a two-architect spec, reaching verified FPGA deployment in two weeks; its 3.4× efficiency edge over Nvidia's Jetson is a projection for an unbuilt chip.