Biology Atlas

Map / Concepts

concept · intermediate · partly written

The assay-error ceiling

The noise in measured potency values caps how well any model, and any chemist, can rank close compounds.

Also called: assay noise, cross-lab variability of IC50 and Ki

In short

Two labs measuring the same compound on the same target often disagree by far more than the difference between two analogues. Same-lab repeats are about three to four times tighter. A model that beats the replicate error of its own test labels is probably fitting noise. 12

Reported, not yet replicated

Why the assay moves the number more

A competitive inhibitor competes with the substrate for the same site, so more substrate means more inhibitor is needed to halve the rate: IC50 depends on [S]/Km, while Ki should not. The conversion assumes simple competitive binding and fails for tight binders, slow binders and covalent inhibitors, where IC50 depends on enzyme level or incubation time. 34

Reported, not yet replicated

Units differ between sources more

The sources quote error per measurement or between two measurements, as means, medians or RMS values, in log units or kcal/mol, so a single "0.4 to 0.9" range mixes quantities. The chain that wrote this record found no source stating that range as one figure. 251

Our inference

Plain answer

IC50 is the concentration that halves an activity in one particular assay, so it shifts with enzyme and substrate concentration; for a simple competitive inhibitor Ki = IC50 / (1 + [S]/Km). Public measurements of the same compound against the same target disagree by roughly half a log unit each, so a model scored against such labels cannot honestly beat that noise. 3425

Reported, not yet replicated

Why it matters

Every comparison in lead optimisation, "B is better than A" and every machine-learning score, is a comparison of two measured numbers. The ceiling is how wrong those numbers are before anyone touches them: it sets the smallest difference worth believing. 21

Our inference

Real numbers

Repeat Ki measurements in ChEMBL

Mean error 0.44 pKi, median 0.34, and a maximum achievable R-squared of about 0.81. Abstract read. The authors conclude that models with errors below experimental uncertainty are necessarily overtrained. 2

Reported, not yet replicated

Between assays, in energy units

From 314,535 relative binding free energies of compounds measured in at least two assays, 83.0% agreed within 1 kcal/mol and 2.1% differed by more than 2; the pooled RMSE was about 0.91 kcal/mol. Repeats in the same lab were about 0.23 kcal/mol. The authors work for a vendor of free-energy software. 1

Reported, not yet replicated

Public IC50 pairs

Under minimal curation, 64% of same-compound IC50 pairs differ by more than 0.3 log and 27% by more than 1.0 log; under maximal curation the shares fall to 48% and 13%, at the cost of discarding most of the data. 6

Reported, not yet replicated

A statistical estimate for mixed IC50 data

A per-measurement spread of about 0.6 to 0.7 pIC50 was estimated from ChEMBL pairs; the chain's source check could not reproduce the headline figure exactly, so the range is the safe statement. 5

Reported, not yet replicated

Still open here

Not yet written: Equation.

Linked from

Sources

  1. Ross, Gregory A., Lu, Chao, Scarabelli, Guido, et al.. The maximal and current accuracy of rigorous protein-ligand binding free energy calculations. Communications Chemistry (2023). paper; opened 2026-11-17 all records citing this
  2. Kramer, Christian, Kalliokoski, Tuomo, Gedeck, Peter, et al.. The Experimental Uncertainty of Heterogeneous Public Ki Data. Journal of Medicinal Chemistry (2012). paper; opened 2026-11-17 all records citing this
  3. IC50 (Wikipedia). Wikipedia. web, not peer reviewed; opened 2026-11-17 all records citing this
  4. Assay Guidance Manual section 4, 2008 web archive (NCGC). NCGC via UNT web archive (2008). web, not peer reviewed; opened 2026-11-17 all records citing this
  5. Kalliokoski, T., et al.. Kalliokoski et al. 2013: Comparability of mixed IC50 data (PLoS ONE). PLoS ONE (2013). paper; opened 2026-11-17 all records citing this
  6. Landrum, Gregory A., Riniker, Sereina. Combining IC50 or Ki Values from Different Sources Is a Source of Significant Noise. Journal of Chemical Information and Modeling (2024). paper; opened 2026-11-17 all records citing this
Where it sits on the map
History of this record (1)

This record has been changed 1 time, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.

Status
partly written
Last updated
2026-11-17
Record
content/concepts/assay-error-ceiling.json