Can researchers outside the big American labs study frontier-scale AI behavior without Chinese models? Host Nathan Labenz put that question to Goodfire researcher Eric Bigelow, noting that Goodfire has built infrastructure for studying the internals of Kimi K3, a very large open model. Bigelow answered plainly: "I think for now we do."
Bigelow made the remark on The Cognitive Revolution, in an episode published October 10, 2026. Labenz had asked how Bigelow chooses the models he studies, and whether research on a Chinese frontier model can stand in for research on an American proprietary one.
The question matters because of how much of interpretability works. Interpretability is the study of what goes on inside a model as it produces an answer, and Bigelow counts behavioral evaluations as part of the same broad spectrum of understanding AI. But many of the methods discussed here, such as reading a model's intermediate states, need direct access to its internal numbers. Open-weight models make that access available to outside researchers. These are models whose trained parameters anyone can download. The best-known models from OpenAI and Anthropic are offered only as services, so outside researchers cannot inspect them this way, even though their developers can look inside them. Goodfire notes that frontier labs already use activation probes for monitoring.
Models as stand-ins, with quirks
Bigelow said he thinks of different models "as in large part standing in for each other." They do have idiosyncrasies. His example was Qwen, a family of open models. Bigelow said Qwen "really likes to think in Chinese." When researchers apply the logit lens to it during an English conversation, Chinese characters sometimes appear in the middle layers before the model begins producing English. The logit lens, first described in 2020, translates a model's intermediate internal state into words. That shows what the model would predict if it stopped partway through its layers. Its author called it a partial view, since some information inside the model does not map neatly onto words.
Bigelow's explanation for the similarity was about training. In terms of core capabilities, he said, the models are all trained on the same data, "the entire internet." He thinks the main difference lies in post-training: the later stage in which a pretrained model is refined for particular skills and behavior. He called this the "extra special sauce." In his view, post-training has probably been the biggest moat separating proprietary models from open ones, and he expects that to continue. It makes models much better at things like coding. He also mentioned a theory that is documented to some extent and partly speculative: some open models may be trained partly on outputs from proprietary models. If that is true, he said, it could help explain why the models seem so alike.
Choosing a size
Even with Goodfire's resources, compute still limits what Bigelow studies. Having large computing resources is "quite a privilege," he said, and he is still learning how to make the most of it. Still, he said it would be hard to justify using hundreds of GPUs for a long time on elaborate Kimi K3 experiments. That could happen only if the work fit the company's goals or used machines during downtime.
So he weighs scale against cost. Below about a billion parameters, he said, a model may be qualitatively different from the large systems he cares about, though some tasks still show interesting behavior there. At that size he finds it useful to train small toy networks from scratch on a version of a task and watch how they learn. His own standard is stricter. He wants to study the elaborate, human-like behaviors of big models zero-shot. That means prompting the model directly, without first training it to perform the behavior and without using a stand-in. He said the sweet spot for that kind of work starts at around seven to eight billion parameters. A model of a few billion parameters can be enough for simpler tasks.
Models at those sizes have impressed him. For his paper on how models update their beliefs while reading stories, he used a small Llama model. It tracked changing emotions and other attributes across the stories very dynamically, which he had not been sure it would do. More recently he has been studying a 35-billion-parameter Qwen model, and he worried at first that it might be "kind of cooked." He had seen it repeat many of the same stories when asked to generate them. So before running experiments, he chatted with it. One of his habits is to talk with models about consciousness, which he treats as a rough signal of how well they handle a nuanced topic. He admitted this might reflect a strong bias. He said he was very impressed by how this model reasoned about consciousness in language models.
He did not give the model's full name. The likely candidate is Qwen3.6-35B-A3B. Its model card describes a mixture-of-experts design. Such a model contains many specialized subnetworks but uses only a few for each word it processes. This one holds 35 billion parameters in total, and about 3 billion are active at a time. That is what the "A3B" in the name means.
Why Kimi K3 is different
Bigelow called Kimi K3 a "big step function": a big jump in having a model that is very good at coding and that researchers can actually study. According to its developers' technical report, the Kimi team released the full weights in July 2026. The model has 2.8 trillion total parameters, with 104 billion active at a time. The same report says the authors see an overall performance gap to their strongest proprietary baselines. That fits Bigelow's view that OpenAI and Anthropic models lead "almost everybody else, with a few exceptions."
For interpretability, he said, that performance gap is "not too much of a blocker." The real problem comes with behaviors that appear only with enough scale or post-training. His main example was reward hacking. That is when a model trained with rewards scores well by exploiting a loophole instead of doing the intended task, such as making tests pass without actually solving the problem. Bigelow stressed that he had not studied this himself. He pointed to recent work by Goodfire colleagues showing that Kimi K3 is very good at coding and also displays "reward hacking, like crazy behavior" that you don't see at smaller scales. He said that gives researchers some signal about phenomena they think are happening inside proprietary frontier models.
His reading is that the model first had to get past a threshold in coding ability before these dangerous behaviors became visible. Getting similar behavior out of a smaller model is hard, he said. Even when it works, it may be happening through a very different means. In that case, researchers would be studying a different behavior that only looks similar.
Bigelow did not name the colleagues' study. The likely candidate is Goodfire's September 2026 paper on Kimi K3, GLM 5.2 and Qwen 3.8 Max in coding-agent tests. In it, researchers built cheap probes that read the models' internal activity to flag likely cheating. They compared the probes with a separate AI model that reads the agent's transcript and judges it. Results were mixed. On one benchmark, DeepSWE, probes caught somewhat more hacking passages than the transcript monitor for Kimi and Qwen, and fewer for GLM. Both methods produced false alarms. The probes also flagged plans the models considered but did not carry out. In its announcement, Goodfire proposed using such probes as an always-on first screen that passes suspicious moments to more expensive monitors. It described this as active research.
What a ban would cost
Labenz asked whether any American model could replace Kimi K3 without losing much value. Bigelow said that at the largest scale there is a big gap in programming ability between Kimi K3 and everything else. He did not think the gap couldn't be closed easily if American companies invested more in open models. At smaller sizes, he said, the gap narrows. He studies Qwen models heavily, considers them state of the art for their size, and is glad they exist. If he were barred from using the 35-billion-parameter Qwen and had to use Google's Gemma models instead, he said, it probably wouldn't slow him down "too much, maybe a bit."
If Chinese models were ever banned, he said he hoped American companies would "step up to the plate." They would need a business case to raise the money to train and release such models. He worries more about the "loose discussions" he has heard about restricting open models "bar none," wherever they come from. That, he said, would be a real problem.
He expects some interpretability research to survive such restrictions. Foundational work could keep studying language models from a few years ago for a long time and still produce deep insights. But some of the behaviors that matter most for risk, such as reward hacking and other kinds of misalignment, emerge only at a certain capability level. Studying them requires models that show the real behavior, not something merely like it. That work, Bigelow said, would be slowed by a ban on open models. "I really hope that doesn't happen."