Tag
Briefings and ideas about Context windows from AI podcast discussions and the research they cite, with links to the original sources.
Dave Blundin said today's AI forgets mistreatment like a goldfish once its context is flushed. On Moonshots, Alex Wissner-Gross countered that tools like Claude Code already save notes to files, so "I wouldn't count on it."
Sean Wang, known as swyx, says engineers who manage coding agents well are in more demand than ever. He says two or three of his own staff are under performance review for handing him low-quality output from Claude.
Positron CTO Thomas Sohmers said AI agents are multiplying demand for memory. He claimed Positron's first-generation product sustained 93% of theoretical memory bandwidth, against an average of 30–40% for NVIDIA GPUs when decoding with transformer models. Its next chip, Asimov, is planned for 2027.
Keerthana Gopalakrishnan, who leads Gemini Robotics research at Google DeepMind, explained how a reasoning model open to developers directs an action model limited to testers, and where she says failures arise.
Alex Zhang, an MIT PhD student, described Prime Agent, built with Prime Intellect: the agent acts only through Python and keeps its history on disk. He also cited outside RLM work, including Harvey's legal agents and Laude Institute's Headlong.
MIT's Alex Zhang says Claude Code, Codex and Pi share nearly all their design. He argues code-based recursive harnesses could help models generalize, citing his own early experiments on one open model as support.
Joel Borgen co-wrote his novel The Receipt Horizon with Claude and ChatGPT Pro. He says he planned the story, cut AI over-explanation and trimmed about 14%, and that good AI-assisted work still takes heavy human judgment, for now.
At DevDay, OpenAI API product lead Nikunj Handa said GPT-6 lets agents keep reasoning while tools run and take new instructions mid-task. He also described longer cache guarantees, one still in preview, and open questions about the platform.
Alexander Wissner-Gross told a founder whose AI language companion kept slipping into assistant mode to try fine-tuning; Dave Blundin suggested an open-weight GLM model and said learners want to be fluent and funny, not just correct.
John Schulman expects frontier AI models to keep growing as compute and GPUs expand. Charlie O’Neill expects little parameter growth for a few years, because long reinforcement-learning runs reward cheap inference.
Charlie O'Neill told the Dwarkesh Podcast that AI already learns from deployment, but in slow batches, and that many small updates erase earlier learning; John Schulman expects customers to resist feeding a shared model.
Anthropic's Fable 5.1 cuts cached reads to $0.25 per million tokens, a quarter of Fable 5's rate. A Moonshots panelist said this makes a company's reused context the prize, as model leads last only about a month.