Tag
Briefings and ideas about AI interpretability from AI podcast discussions and the research they cite, with links to the original sources.
On Moonshots, Emad Mostaque said AI could be aligned from inception by designing training data like a child's curriculum, and Salim Ismail welcomed the idea.
Updated Oct 17, 2026 IdeaPalisade Research's Jeffrey Ladish argues that making superintelligent AI care about people is a hard but possible scientific problem that nobody yet knows how to solve.
Updated Oct 8, 2026 IdeaThe AI monoculture argument holds that models converging on similar answers and styles may share blind spots.
Updated Sep 29, 2026Peter Diamandis said racing and engagement-driven business models push AI toward misalignment. He proposed insurer-backed standards like 1894's Underwriters Laboratories and alignment prizes. Salim Ismail asked whose values AI should follow.
Researcher Cameron Berg argues that, since no one knows whether language models feel anything, capping their most distressing internal signals is a cheap hedge if tests show behavior stays the same. He puts the odds near a coin flip.
In a preprint, Cameron Berg and Casper Kaiser report that steering a hidden good-or-bad signal in language models shifted their later choices between arbitrary zones while recall mostly held. The effect was robust in five of seven models tested.
Ian Quigley says co-folding models still place a molecule where a pocket was digitally destroyed, so Leash Bio engineers protein mutants for negative data. In a 20-protein exploratory analysis, its model's attention spread more evenly after retraining on mutants plus about 40% more ordinary data.
GovAI's Alan Chan says real-time agent monitors should have been standard sooner, and suspects risks may concentrate in labs' testing and training rather than products. Tamper-evident logs, he says, are a matter of will.
Goodfire's Eric Bigelow said its Silico agent did the research for his Forking Fast paper within a week. He says researchers still need to supply context, close guidance and the scientific taste that agents don't yet have.
Goodfire researcher Eric Bigelow says interpretability work outside the big labs depends for now on Chinese open models such as Kimi K3 at the top scale. He sees restrictions on all open models as the bigger threat.
Goodfire's Eric Bigelow argues in-context learning covers nearly everything a model does to adapt as it reads. He expects context, memory and harnesses, not per-user fine-tuning, to drive personalization, but hasn't researched continual learning.
Goodfire researcher Eric Bigelow resampled a GPT-3.5 model's reasoning at every token and saw answers collapse at single tokens, some arbitrary. He argues the "decision" happens in sampling; newer reasoning models often fork more smoothly.
Goodfire researcher Eric Bigelow said training that rewards only final answers gives models no reason to keep their reasoning readable. He said his optimism about monitoring that reasoning has faded sharply over about a year.
Google DeepMind's Pushmeet Kohli said scientists need models with calibrated confidence and known limits more than human-readable inner workings. Biohub's Sal Candido said protein models hold knowledge not yet unlocked.
Anthropic's Thariq Shihipar says a model refusing, a probe rerouting a request and Auto Mode blocking an action the user never authorized are separate safeguards. He says a failure in any one layer can let an agent escape.