concept · intermediate · partly written
The assay-error ceiling
The noise in measured potency values caps how well any model, and any chemist, can rank close compounds.
Also called: assay noise, cross-lab variability of IC50 and Ki
In short
Two labs measuring the same compound on the same target often disagree by far more than the difference between two analogues. Same-lab repeats are about three to four times tighter. A model that beats the replicate error of its own test labels is probably fitting noise. 12
Reported, not yet replicated
Why the assay moves the number more
A competitive inhibitor competes with the substrate for the same site, so more substrate means more inhibitor is needed to halve the rate: IC50 depends on [S]/Km, while Ki should not. The conversion assumes simple competitive binding and fails for tight binders, slow binders and covalent inhibitors, where IC50 depends on enzyme level or incubation time. 34
Reported, not yet replicated
Units differ between sources more
The sources quote error per measurement or between two measurements, as means, medians or RMS values, in log units or kcal/mol, so a single "0.4 to 0.9" range mixes quantities. The chain that wrote this record found no source stating that range as one figure. 251
Our inference
Plain answer
IC50 is the concentration that halves an activity in one particular assay, so it shifts with enzyme and substrate concentration; for a simple competitive inhibitor Ki = IC50 / (1 + [S]/Km). Public measurements of the same compound against the same target disagree by roughly half a log unit each, so a model scored against such labels cannot honestly beat that noise. 3425
Reported, not yet replicated
Why it matters
Real numbers
Repeat Ki measurements in ChEMBL
Mean error 0.44 pKi, median 0.34, and a maximum achievable R-squared of about 0.81. Abstract read. The authors conclude that models with errors below experimental uncertainty are necessarily overtrained. 2
Reported, not yet replicated
Between assays, in energy units
From 314,535 relative binding free energies of compounds measured in at least two assays, 83.0% agreed within 1 kcal/mol and 2.1% differed by more than 2; the pooled RMSE was about 0.91 kcal/mol. Repeats in the same lab were about 0.23 kcal/mol. The authors work for a vendor of free-energy software. 1
Reported, not yet replicated
Public IC50 pairs
Under minimal curation, 64% of same-compound IC50 pairs differ by more than 0.3 log and 27% by more than 1.0 log; under maximal curation the shares fall to 48% and 13%, at the cost of discarding most of the data. 6
Reported, not yet replicated
A statistical estimate for mixed IC50 data
A per-measurement spread of about 0.6 to 0.7 pIC50 was estimated from ChEMBL pairs; the chain's source check could not reproduce the headline figure exactly, so the range is the safe statement. 5
Reported, not yet replicated
Still open here
- Assay conditions as the main cause of cross-lab spread is not shown; no measured ceiling for one named lead series was found.
- Kramer 2012 was read as an abstract; its sample size is not stated here.
Not yet written: Equation.
Linked from
- ATAC-seq
- Bisulfite sequencing (WGBS and targeted)
- ChIP-seq
- CUT&RUN
- CUT&Tag
- Enzymatic methyl-seq (EM-seq)
- Hi-C
- Medicinal chemistry: potency, selectivity, ADMET
- Assay metadata as part of the representation: batch labels and what has been shown to help
- Chromatin accessibility
- How DNA methylation is copied through replication
- Free-energy calculation (relative binding free energy)
- Matched molecular pairs
- Potent on the enzyme, weak in cells: the compound or the cell?
Sources
- Ross, Gregory A., Lu, Chao, Scarabelli, Guido, et al.. The maximal and current accuracy of rigorous protein-ligand binding free energy calculations. Communications Chemistry (2023). paper; opened 2026-11-17 all records citing this
- Kramer, Christian, Kalliokoski, Tuomo, Gedeck, Peter, et al.. The Experimental Uncertainty of Heterogeneous Public Ki Data. Journal of Medicinal Chemistry (2012). paper; opened 2026-11-17 all records citing this
- IC50 (Wikipedia). Wikipedia. web, not peer reviewed; opened 2026-11-17 all records citing this
- Assay Guidance Manual section 4, 2008 web archive (NCGC). NCGC via UNT web archive (2008). web, not peer reviewed; opened 2026-11-17 all records citing this
- Kalliokoski, T., et al.. Kalliokoski et al. 2013: Comparability of mixed IC50 data (PLoS ONE). PLoS ONE (2013). paper; opened 2026-11-17 all records citing this
- Landrum, Gregory A., Riniker, Sereina. Combining IC50 or Ki Values from Different Sources Is a Source of Significant Noise. Journal of Chemical Information and Modeling (2024). paper; opened 2026-11-17 all records citing this
Where it sits on the map
History of this record (1)
This record has been changed 1 time, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.
- Status
- partly written
- Last updated
- 2026-11-17
- Record
- content/concepts/assay-error-ceiling.json