An AI agent gets a catalog of molecules, a budget of 100 tests and the name of a single protein, and nothing else. "We're not going to tell you anything except the target and go find the best binder you can," said Ian Quigley of Leash Bio, the company that built the task. The agents write out in plain language why they think some molecules stick to the protein and others do not, then spend their budget testing those ideas.
Leash already knows how every molecule in that world behaved. The company runs DNA-encoded library screens: large pooled collections of small molecules, each carrying a DNA barcode. Mix a protein with the pool, keep what bound, then sequence the barcodes, and you learn which molecules stuck. Quigley's documentation of an earlier Leash data release describes that procedure and the control screens used to separate real binding from background.
Because the measurements already exist, the agent's answers come back instantly, and the company can mark its work against the answer key. It knows what the agent found and what the best available molecule was.
Quigley described what that comparison has shown on the Latent Space podcast, in a follow-up conversation with hosts Brandon and RJ published on October 26, 2026. They had first interviewed him earlier in the summer, when the agent sandbox was still an idea the team was playing with.
The flagship task
The problem Leash has spent the past couple of months on is a structure-activity relationship problem, or SAR, a routine part of medicinal chemistry. "I've got some molecules and I know that maybe they bind to this protein that I want them to. How do I change them to make them better?" is how Quigley framed it. Real SAR work also covers drug-like properties, such as whether the body metabolizes the compound. "We're not doing those right now," he said. The task is narrowed to one question: finding molecules that bind better to a single target.
Agents behave differently inside that budget. They have different search policies and different ideas about how to spend their 100 tests. Because nothing has to be synthesized or assayed, the runs are cheap to repeat. "We're up to about 90,000," Quigley said of the rollouts, a rollout being one agent's full attempt at the task from start to finish.
What knowing every outcome buys
Quigley said a preprint was "pretty ready" and would probably appear at about the same time as the episode. Its first claim is that the benchmark tells agents apart: "we create separation of agents across model families and levels."
The second is the one he finds more interesting. An ordinary evaluation records how well an agent did at the job. Leash also holds the results for the entire measured world, including the molecules the agent chose not to test. "And so then you can start to interrogate how good is the agent at making scientific decisions? Because maybe it only got to like the thing that's half as good as the best thing in the entire environment."
That view surfaces specific failures. Quigley listed agents that give up early, agents that "buy molecules that don't agree with their SAR hypotheses," and "ones that keep retesting the same molecules over and over despite having a limited budget." Leash's October 14, 2026 announcement of these environments, which it calls Minos, reported a similar list: repeated tests, unused budgets and poor updating of beliefs. The chart in that post compares cost and performance across 33 targets and roughly six million molecules, with each campaign allowed 100 tests over 20 iterations and each target's mean taken over 10 trials, with 95% confidence intervals.
The explore-and-exploit surprise
The measurement that caught the team off guard concerns when to keep refining a promising molecule and when to walk away. "It turns out that more performant models are much better at abandoning kind of unproductive neighborhoods," Quigley said. "And that was actually a really interesting surprise for us."
Asked why stronger models manage this, he offered no theory. "We don't know that at all yet. All we know is that they're abandoning these crappy neighborhoods more quickly." Stronger models show "a more willingness to jump around overall," he said, but they are particularly quick to leave bad ground, "and then the really poor ones stick around." The weaker ones "just keep running little subtle changes on molecules that aren't ever going to go anywhere," as if reasoning: "oh, these two, they have a low score, but I feel good about them." The behavior is measured across these runs; the mechanism behind it is not.
Reading what the agents said
Alongside the scores, Leash keeps the record of what each agent did and said. The environments are packaged with Harbor, whose documentation defines a task as instructions plus an isolated execution environment plus a verifier that produces a reward, a trial as one agent's attempt, and a trajectory as the conversation and action history. The same documentation calls trajectories one of the most useful tools for understanding and debugging agent behavior.
Quigley's own method for surveying that pile is blunt: take large numbers of transcripts and "just dump them into LMs and just have a look at the different failure modes." What comes back reads like shop talk. "I think that this ring is good. And I think that this particular nitrogen is painful," he said, describing the agents' notes. "And then they're very plausible, you know, medicinal chemistry narratives. So it's pretty fun."
Asked whether humans had been given the same questions, Quigley said medicinal chemists have looked at and evaluated some of the output, "not a ton" of it given how much there is. Their verdict, as he reported it: "yeah, no, those are very plausible thoughts about, you know, how to conduct this particular part of the SAR campaign." That is expert judgment on the agents' reasoning. He did not describe chemists playing the same budgeted game for a score.
The baseline Leash has not run
The discussion turned to why Leash had not run a Bayesian optimization method as a deterministic baseline, meaning a classical statistical search algorithm rather than a language model. Quigley said he thinks the landscape is "too spiky" and "very high dimensional" for that. "We're not really, we don't do that, but we mostly think it's not going to work." Then he flagged his own answer: "I think it's a party line." It is the clearest assumption in the conversation that the company has not tested.
Publishing against an 11-day clock
Quigley recited the current leaders from memory and laughed at himself for having looked at the plot a minute earlier: "It's kind of embarrassing that I forgot already." Frontier models sit at the top, which he said was no big shock. Leash has also run a large number of open-weight families, the kind anyone can download, and found some of them creditable. One earlier frontier release he dismissed on cost as well as capability, calling it "both very expensive and not very performance."
The newest top-tier model was not in the comparison yet, and that gap is a running frustration. "When we talked last, two months ago, we started these rollouts to see what happened," Quigley said. Since then, by his count, a new frontier model has arrived roughly once every 11 days. Each time Leash decides it is ready to ship the public material, "another new stupid model shows up."
He also admitted a stylistic debt. The results plot "kind of looks a little bit like, you know, like the Arc AGI 2 plots where you've got some crappy models," he said. "And that's kind of why we made it like that." ARC-AGI-2 is a reasoning benchmark of a very different kind: according to its official guide, a system sees example pairs of colored grids, has to infer the transformation and then build the output grid for a new input, scored as exactly right or wrong, across 1,000 public training tasks and three evaluation sets of 120 tasks each.
The release Quigley described covers the preprint, a blog post, the environments as Harbor packages, many thousands of transcripts and the data; Leash's dataset page names that release Strelka and routes visitors through a request-access link. "I think that's the big thing," he said. His pitch for all of it is measurement rather than a discovery: if you have scientific agents and want to know how they are behaving, what mistakes they make, how they learn from physical feedback, or whether a new training recipe improves scientific decision making, "we've got the capabilities to measure these things at scale."