Reading Anthropic's Binder Data: Beyond the Hit Rate
Claude's campaign produced 354 measured binders. The public data links better outcomes to deeper iteration and predictor agreement, while the hardest targets expose the limits of a pooled hit rate.
- Evidence type
- News
- Published
- September 2026
A hit rate becomes much more useful when the evidence behind it is public. Anthropic released twenty documented data tables, 113,550 co-folding predictions, raw experimental traces and its campaign prompts. We could examine how the result was reached, so we did.
Of 1,440 ordered designs across 16 targets, 1,320 had interpretable experimental results and 354 bound: 26.8%. The remaining 120 had inconclusive measurements. This is substantial evidence that an agent can carry a computational design campaign through to candidates with measured binding. At Crane Bioworks, we revisited the tables alongside Anthropic's technical report to understand what a research team can learn from the campaign beyond that pooled percentage.
What the panel is
- Designs ordered
- 1,44016 targets, 50 to 120 aa
- Interpretable results
- 1,320mature GDF-8 inconclusive
- Confirmed binders
- 354integrated assessment
- Claude models
- 2Mythos Preview, Opus 4.8
- Contract labs
- 2Adaptyv Bio, Twist Bioscience
- Co-folders re-scored
- 10113,550 predictions
Confirmed means the study's integrated assessment of the available vendor data; it does not require a positive result at both laboratories. Rates use the 1,320 interpretable results.
Autonomy built on existing expertise
Claude researched targets, installed and ran established protein design tools, and interpreted their outputs. The underlying models generated structures and sequences. The achievement is an agent coordinating this work across a campaign, built on methods developed by the protein engineering community.
That could free considerable expert time. It also helps locate the expertise: Anthropic supplied roughly 16,000 words of scientific and operational guidance, developed before the reported campaigns and then frozen. A protocol carries scientific assumptions as well as instructions. Reviewing those assumptions remains consequential when an agent can execute them at scale.
The release lets us look inside that execution. Its most useful signals concern the history of a design and the evidence used to select it.
19% to 65%, along the iteration history
Designs with no recorded optimization rounds bound 19.2% of the time. Those with nine or more bound 64.8%. The middle bins are uneven, but the overall association between iteration depth and binding remains after adjusting for target.
Generator choice tells a less straightforward story. The tools were used on different mixes of targets, and their apparent differences weaken after target adjustment. That makes raw tool leaderboards difficult to interpret. In this panel, iteration history retains an association that the generator comparisons do not clearly establish.
Hit rate by in silico optimization depth
All 1,320 designs with a wet-lab call, binned by predict-and-redesign rounds in the design's own ancestry.
The distinction is worth investigating, without treating the upper bin as a planning number. Promising designs may receive more rounds; target adjustment does not remove that selection. The data supports a controlled test of iteration at a fixed budget, rather than a promise that extra rounds will triple a team's hit rate.
Cost belongs in that test. Anthropic's stated GPU budgets were $10,000 per single-target campaign and $50,000 per multi-target campaign. Allocated budgets do not establish actual expenditure or efficiency. Equally, a cheap set of computationally plausible candidates is not equivalent to measured binders. A useful comparison would include expert time, compute and experiments against the same endpoint.
The signal in predictor agreement
Anthropic had already evaluated an ensemble of three predictors on the Overath et al. benchmark and examined ensembles on its own campaign data. The release lets us extend that analysis across ten predictors. This is a reanalysis of existing experiments, building on an observation the authors made themselves.
The mean interface-confidence score (ipSAE) across ten predictors has the highest observed within-target AUROC: 0.772, compared with 0.734 for RoseTTAFold3, the strongest individual predictor in this comparison. AUROC measures how well a score ranks binders above non-binders; 0.5 is chance. Agreement across predictors carries useful information in this panel.
Within-target AUROC against the confirmed binder call
Maximum confidence over five runs per predictor. AlphaFold-Multimer excludes the SpCas9 ribonucleoprotein, so its evaluation cohort differs.
The extremes make the association tangible. The highest-agreement group had a 53.7% binding rate, compared with 3.9% in the lowest-agreement group. That is about fourteen times the rate in the lowest group, but only twice the panel's 26.8% average. The reference group matters when describing enrichment.
These were already selected designs, and the scores use the best of five prediction runs. Absolute performance may change on an independent panel or with one run per design. Predictors can also share errors. Consensus is encouraging evidence for ranking, while experimental structure, specificity and biological activity remain separate questions.
ESMFold2's results here reinforce our internal findings that it is a valuable addition to our toolbox. In our exploration of nanobody complexes, the selected predictions recovered high-quality interfaces for three of five known complexes. That small study measured structural recovery; Anthropic measured binding. The evidence is complementary. Kudos to the ESMFold2 authors at Biohub for making the model and code available to the community.
Campaigns before model leaderboards
The headline comparison between Mythos Preview and Opus 4.8 is tempting: 26.7% against 22.6% in the multi-target campaigns. Anthropic cautions that model, campaign format and run-to-run variation are confounded. Our exploratory regression pools all campaign formats and shows how much the model estimate changes with adjustment.
It does not establish equal capability or explain away the narrower multi-target comparison. The useful lesson is to examine how a campaign was run before attributing its outcome to the model name.
The model estimate changes with adjustment
Exploratory odds ratios for Opus 4.8 relative to Mythos Preview, pooling all 1,320 designs across campaign formats.
The hard targets remain the interesting ones
The pooled 26.8% conceals a wide spread. TREM2 returned 72 binders from 90 designs. MBP returned none from 90, BBF-14 three from 90, and 15-PGDH one from 30. Mature GDF-8 was a different outcome: aggregation left 120 designs without interpretable measurements. An unsuccessful design campaign and an inconclusive assay call for different follow-up decisions.
The historical competitions add context. Claude's pooled rates were higher on four of six comparable targets; on two, no difference was detected. In the earlier 15-PGDH competition, only one of 106 designs bound. That supports the observation that this target was difficult in both campaigns. It does not establish that Claude matched an expert.
Tool access, budgets and participating teams differed. A stronger predictor can change results independently of who runs it. Anthropic explicitly acknowledges that it did not compare human experts using identical tools and budgets. Nor does failure here establish that MBP or BBF-14 is intrinsically intractable: prior binders exist.
This is where scientific judgement stays central. Computational confidence cannot settle whether the target preparation, assay and molecular design together answer the biological question. A plausible complex also does not establish how the isolated protein behaves. Structural concerns deserve investigation alongside a positive binding measurement, without treating an unusual prediction as proof of an experimental artefact.
What we take into our own work
Iteration history and predictor agreement are useful signals to test further. Target-level outcomes show why a campaign needs more than a pooled hit rate. For antibody and nanobody teams, the miniprotein percentages are not forecasts for their own modalities. The relevance is in evaluating the evidence, including specificity, developability and whether a measured interaction produces the intended biological effect. The release contains no functional validation of these designs.
Anthropic deserves credit for making that evaluation possible. Public sequences, experimental traces and prediction data let other scientists check the conclusions. More releases should look like this one. Open evidence also makes the remaining questions more productive: which observations travel to another campaign, and which depend on this particular panel?
At Crane, we want to connect these analyses to decisions a research team can defend. If you have a completed campaign with measured outcomes, we would welcome a conversation about assessing it together. Testing our assumptions against an independent panel is how we find out whether the analysis would change your next decision.
Sources
- Anthropic (2026). Claude accelerates protein design. anthropic.com/research/Claude-accelerates-protein-design
- Claude Science & Shanehsazzadeh, A. (2026). Autonomous de novo protein binder design with Claude. Anthropic, 18 August 2026. Technical report (PDF)
- Overath, M. D., Rygaard, A., Jacobsen, C. P., Brasas, V., Morell, O., Sormanni, P. & Jenkins, T. P. (2025). Predicting Experimental Success in De Novo Binder Design: A Meta-Analysis of 3,766 Experimentally Characterised Binders. bioRxiv 2025.08.14.670059. biorxiv.org/content/10.1101/2025.08.14.670059v1
- Anthropic (2026). claude-protein-binder-design, v1.0 dataset release, CC BY 4.0. Wet-lab characterization by Adaptyv Bio and Twist Bioscience. huggingface.co/datasets/Anthropic/claude-protein-binder-design
Our exploratory figures use the release tables. Hit rates use binder_final over the 1,320 designs with wet-lab data. Regression models are logistic with target fixed effects. Within-target AUROC pools per-target results, weighted by binder/non-binder pairs. These are descriptive analyses of the selected panel.