A common belief about AI is that if you feed a system enough data and computing power, it will eventually win. Sal Candido of Biohub, a nonprofit research organization whose mission is to cure all disease, pushed back on how easily that idea carries over to biology. "One misconception of scaling laws is that, you know, scaling laws are everywhere and they always exist," he said. "I think a lot of the work is actually finding that scaling law."
Candido made the remark on a panel with Pushmeet Kohli of Google DeepMind, moderated by Brandon Anderson. The Latent Space podcast released it on October 10, 2026. The session opened by recasting a famous AI idea, the "bitter lesson," as a question about data: in biology, does the method that scales eventually win?
The lesson being argued over
The bitter lesson comes from a 2019 essay by the AI researcher Rich Sutton. Sutton argued that general methods able to exploit ever more computation, mainly search and learning, have repeatedly beaten approaches built around knowledge that researchers wrote in by hand. His examples included chess, Go, speech recognition and computer vision.
A scaling law describes the payoff Sutton had in mind: a dependable relationship in which adding data, model size or computing power improves results. Candido said finding one is what lets researchers "turn the crank." At that point, he said, the problem becomes an engineering problem, "which is something I like." But the relationship depends on architecture and "very much" on data. A model can only draw on the information present in its training data and generalize beyond it. If the data lacks the right information, Candido said, the model will not gain the capabilities researchers want.
Learning from other people's junk
The discussion turned to whether modelers tend to choose problems where data already exists, rather than the problems that matter most for medicine. Candido admitted it. "I'm lazy, so I'm going to work with what's available," he said.
He described both a good and a bad side to that habit. Conventional machine learning looks for the most pristine examples it can find. Candido said he prefers "rooting around in the back room" through other people's leftovers. His example was a protein language model. Such a model learns from strings of amino acids, the chemical units that make up proteins, much as a chatbot learns from strings of words. Candido said that when his team trains such a model, it uses metagenomic sequences: genetic material read in bulk from mixed environmental samples rather than from carefully studied organisms. He said he could guarantee that much of that data does not even represent a real, whole protein. Even so, he said, including it improves the model at designing real proteins that work and at understanding proteins known to exist. Biohub's protein language model ESMC, released in May 2026, was trained on about 2.8 billion sequences.
The bad side, Candido said, is the temptation to scale up whatever data is easiest to generate. He called it exciting that the Virtual Biology Initiative (VBI) instead starts by asking what data is needed to solve the problem. Biohub launched the initiative in April 2026 as a five-year, $500 million commitment to measurement technologies and coordinated, openly available data. On October 7, 2026, Biohub and its partners expanded it to a commitment of nearly $2 billion, with funding from Google DeepMind among others. Candido said the openness matters: people who only build models lean toward data that exists, and people who only generate data lean toward what is easy to generate. Working together at every step, he said, is how a field figures out what data it needs "and then you can find that scaling law."
Kohli's reading: the problem comes first
Kohli said he was at DeepMind when Sutton was with the lab and thinking about the bitter lesson, and he took the lesson from Sutton's original lecture there to be something conceptual. To him, it was less about whether data is useful than about researchers who treat solutions in "a very religious way," calling themselves modelers or data-generation people. "The problem comes first," Kohli said. If a problem needs modeling effort, do the modeling. If it needs more data, collect data.
He described the old machine-learning routine as broken: researchers took a fixed dataset, split it into training and test data, and only optimized the model. Expertise, he added, is a third ingredient alongside data and modeling.
He gave two cases from DeepMind. For AlphaFold, the system that predicts a protein's three-dimensional shape from its amino-acid sequence, the team worked with existing data. That data came from the Protein Data Bank (PDB), a public archive of structures that scientists determine experimentally from physical samples using methods such as X-ray crystallography and electron microscopy. Kohli said DeepMind had neither the expertise nor the resources to enlarge the PDB by an order of magnitude, and he called the investment that scientists worldwide had put into it invaluable. So the team put its effort into modeling. The AlphaFold 2 paper reports training on PDB structures released by April 30, 2018.
The second case went the other way. DeepMind took the same approach to cell genomics and examined what could be done with cell-by-gene data, measurements of which genes are active in individual cells. After a lot of work, Kohli said, it was clear "the data was not there yet" for the larger goal of building a virtual cell, a computer model that predicts how a cell behaves.
His advice to newcomers was to think of themselves as multidisciplinary, understand why they are working on a problem, and identify their constraints, such as how much data they can generate or what model size they can afford to train. Then, he said, they should try to fail fast and see which approaches could reach the impact they are aiming for.
Handcrafted models and the craft of scaling
The panel then took up AlphaFold 2 itself, described during the session as "a work of art" full of carefully handcrafted features. The question was whether such custom-built solutions still deserve investment, or whether fixed budgets for compute, talent and data should go to scale first.
Kohli said the design was not accidental. It came from scientific intuition from biophysics and biochemistry: amino-acid residues, the individual links in a protein chain, do not act on their own but are influenced by other residues. "So, let's bake that in," he said. Using what the scientific community has learned gives a model "that unfair advantage" and makes it much more data-efficient, because it does not have to rediscover everything. The published architecture reflects this. Its main component repeatedly exchanges information between pairs of residues, and some of its updates push those pairs toward geometry that is consistent across groups of three residues, according to the paper.
Kohli added that curating data is an art in itself. Copying the same data does not get a model anywhere. "It's not just about big data. It's about good data," he said. Knowing which data covers what a problem needs is, in his view, a more interesting and more challenging problem.
Candido agreed, with conditions. With little data, he said, built-in assumptions about the problem, which researchers call inductive bias, help and may be necessary. As data grows, a model can find things researchers did not know. A built-in assumption that is not exactly correct can then hold the model back, which creates a tipping point. Sutton's essay makes a similar observation: human knowledge brings satisfying short-term gains but can complicate methods over time.
Candido also objected to the way the question split craft from scale. Making models bigger and training them on more data and compute takes a lot of algorithmic work, he said, not only infrastructure. Researchers are now modifying the Transformer, the architecture introduced in 2017 that underlies today's language models, into more specialized designs. "I don't think we're in a post transformer world in any way, shape or form," he said, but those designs are being adapted to fit their purpose, even with internet-scale data. At every scale, he said, the question is which next batch of information models need and which architecture gets the most out of it.
Thinking in multiples
Asked near the end how long it would take for AI results to reach the clinic, Kohli said AI is already used in every part of drug discovery. Whether it delivers a 10x or 100x speedup depends on what is being accelerated, he said: optimizing a drug candidate, finding targets, or preclinical and toxicity work. The larger gains, he said, would come only with better models of biology.
Candido said he could not say when a drug made entirely by AI would appear, though he expects rapid progress. He recalled a lesson from earlier in his career at Google: sometimes it is easier to approach a problem by asking what a 10x improvement would take rather than a 10% one. That is not because it is easier, he said, but because it forces a broader view and a return to first principles. Both approaches have value, he said, and Biohub's goal of curing all disease means working out what the 10x approach is.