Pushmeet Kohli of Google DeepMind gave a concrete reason why scientists cannot treat a protein model as a magic box. Suppose AlphaFold had been even more accurate, he said, but its confidence score was "completely uncalibrated." It could give a wrong answer while being "very, very confident," and a researcher could spend "the next one year" on that answer before learning that it was "completely wrong." Kohli asked: "who would trust it?"
Kohli made the argument on a panel with Sal Candido of Biohub, moderated by Brandon Anderson. The Latent Space podcast published the discussion on October 10, 2026. The moderator raised a line attributed to the physicist Richard Feynman, "that which I cannot create, I do not understand." AI now makes it easy to create things without understanding them. A model might design a protein that binds tightly to a target on the first attempt without anyone being able to explain how. Do scientists need models that help people understand biology, or is a reliable black box enough?
Kohli and Candido gave different answers. Candido wants to extract the knowledge stored inside the models. Kohli said what users need most is to know when a model can be trusted.
Candido: the models know more than we do
Candido said people were designing things with "magical black boxes" long before AI existed, so a useful result without a full explanation is nothing new. He still called understanding "really important." As intelligence becomes cheaper and easier to use, he said, AI can be put to work learning more about the world. The hard part is how to "pull that knowledge out of the machine" in a form a person can understand. He suggested that this might be "just my esoteric curiosity."
The moderator then noted that the panel was there to solve translational medicine, which means turning laboratory science into treatments. Candido said that digging deeper shows researchers how to keep making progress. He said "the models actually have a lot more information in them than we know."
He was referring to protein language models. These systems learn from protein sequences, which are strings of amino acids, in roughly the way text models learn from words. Biohub's ESMC model, according to the organization, was trained on about 2.8 billion protein sequences from across life by learning to predict the amino acids that evolution has selected. Researchers already knew that such models learn something about a protein's three-dimensional structure, Candido said. Biohub's interpretability work, which studies what a model has stored internally, has also found information about some protein functions and about protein motions.
Candido said there is "a lot in there still to be unlocked," even in today's models. He offered a reason. To design a protein, a model has compressed "all the information from evolution." He expects that this compression will produce something like a world model: an internal picture of how proteins behave. Biohub uses the same term in its May 2026 announcement of its protein models. For Candido, a model is useful because it produces results and also because "there's certainly something to learn just by looking at what's inside."
Kohli: understand the behavior first
Kohli said he had a different view. Some understanding is necessary, he said, but the essential kind is knowledge of how a model behaves, not how it reaches each answer.
He started by saying AlphaFold "is not perfect." At the time it was about "90 GDT" on the test set. GDT is a 0-to-100 score that measures how many parts of a predicted protein lie close to their experimentally measured positions. Kohli's figure is a round number. In the blind CASP14 assessment, DeepMind reported a median GDT of 92.4 across all targets and 87.0 in the harder free-modeling category.
The confidence measure Kohli has in mind is pLDDT. AlphaFold attaches this score to every residue, or amino-acid position, in a prediction. EMBL-EBI's guide says values above 90 generally indicate an accurately placed backbone and side chains, and values above 70 usually mean the backbone is correct. Values below 50 may signal genuine flexibility or that the model lacked enough information. pLDDT is a prediction of a measurable quantity: the local accuracy of that part of the structure, scored as lDDT-Cα against the experimentally determined structure. "Calibrated" means the predicted score agrees with that measured local accuracy, so a higher pLDDT really does go with a more accurately placed region. The 2021 AlphaFold 2 paper tested this on 10,795 protein chains released after the training cutoff. It found a correlation of 0.76 between predicted confidence and measured local accuracy, which is strong but not perfect.
Kohli said that calibration of the uncertainty measure was "extremely important." In that sense, he said, the team understands, and has made a lot of progress in understanding, how AlphaFold 2 behaves.
"Interpretable by whom?"
Kohli separated that from a second question: why the model gives a particular answer. On that question he agreed with Candido that interpretability matters. He then argued that AlphaFold 2 was interpretable "at the highest level," because it generalized beyond what it was built to do. DeepMind did not discover this before launch. After AlphaFold 2 was released and its weights were made available, he said, people found that it was a strong predictor of disordered proteins, meaning parts of proteins that have no fixed shape.
EMBL-EBI's guide describes the same effect. Low pLDDT scores can help identify intrinsically disordered regions. The guide also gives an example of why Kohli's "behavioral characterization" matters. For a protein called 4E-BP2, AlphaFold 2 confidently predicts a helix resembling the shape the protein takes when bound to a partner, even though the unbound protein is disordered. That bound structure was in the training data. High confidence, in other words, does not always mean what a user might assume.
Kohli then asked: "interpretable by whom?" To a person, working within "the cognitive and computational limitations of the human brain," AlphaFold 2 is not interpretable, he said. A much larger and more sophisticated model might be able to explain it: "We just don't get it." What users of AlphaFold 2 do need, he said, is to know what it can and cannot do. Without that behavioral characterization, he warned, the models could "rather than being helpful, harm us." The discussion summed up his position this way: strict interpretability may not be necessary, but trustworthiness that lets people make actionable decisions is.
A model that explains another model
Kohli went further and called interpretability "in the eyes of the beholder." He described a possibility, not a result. Future frontier models could be given access to AlphaFold's activation layers, the internal signals produced as it computes, and asked to predict what AlphaFold will do. "Maybe they will be able to predict," he said, and "come up with a theory" of how AlphaFold 2 was interpreting its inputs and producing its results.
So far, the published work inside AlphaFold has been led by humans. In the 2021 paper, the team attached separately trained structure readouts to the network's frozen intermediate layers to watch predictions form. For one protein, LmrP, the structure appeared early. For SARS-CoV-2's ORF8 protein, the network went through repeated rearrangements across layers before it settled on an answer.