Biology Atlas

Map / Concepts

concept · intermediate · partly written

Models of models

Every biological number is a measurement of a model system standing in for a human, and each layer has its own error; clinical trials are the first real human data, and they are models too.

Also called: models of models, caveat stack

The short answer

Every number is a measurement of a model of a human. Both stand-ins have measured errors for some layers. No source combines them, and the human trial is also a model, through its endpoints and its selected patients. 12

Our inference

His framing (his words)

He said: "these are all models of models. You have a measurement of a model, and the measurement itself is imperfect. Then you have the model of your true model, which is a human, and the model itself is imperfect because you've taken a cell out of its native environment, put it into this somewhat simulatory environment, and you've got problems." 3

Amit's hypothesis

Transcript versus protein: the answer depends on the axis more

Across genes within one human cell line the correlation is moderate: Pearson 0.54 (512 genes, Daoy medulloblastoma line) and Spearman 0.6 in deeply measured HeLa cells. Across samples for one gene it is lower: mean r 0.29 over 12 human tissues, and per-gene medians of 0.31 to 0.48 across cell-line panels and 0.40 in tumours. In yeast, 24 studies give r 0.618 raw and 0.926 after a noise model. The numbers are not comparable without their axis, and one meta-analysis warns that reported correlations differ by Spearman versus Pearson, mean versus median and inclusion rules. See concepts/mrna-protein-correlation for the full table. 456178910

Reported, not yet replicated

Where the studies disagree: mouse NIH3T3 cells more

Schwanhausser 2011 concluded that protein is predominantly controlled at the level of translation. Li, Bickel and Biggin 2014 re-scaled the same data with 61 housekeeping proteins and found mRNA explains at least 56% of protein variance (about 81% by a second route), with measured translation rates spanning 5.8-fold against 115-fold inferred. Jovanovic 2015 in dendritic cells found at steady state mRNA explained 59 to 68% of protein variance, and after LPS 87 to 92% of fold changes. No opened source reconciles them. 111213

Contested

How much of the gap is measurement more

In yeast, most of it: a typical mRNA study explains 39% of the variance of another and a typical protein study 32%, and averaging replicates lifted explained variance from 33% to 72%. In one human cell line the reliability correction moved r only from 0.54 to 0.57. Across human tissues about half of protein variance is attributed to mRNA after correction, which the authors call highly uncertain. Across cancer panels replicate reproducibility explains about three times more of the variance in per-gene correlation than protein-complex membership. Not found: a noise-corrected human figure for one cell type with matched replicates. 1461012

Reported, not yet replicated

Cell lines versus tumours on this measure more

Per-gene correlations in cell-line panels (median 0.31 to 0.48) sit inside the tumour range (0.21 to 0.55 across studies). No opened source shows a cell-line penalty on this measure, and none compares one lineage in lines and tumours. 89107

Reported, not yet replicated

RNA-seq as a model more

RNA-seq counts reads, not molecules. In the SEQC study, sites agreed on up to 95% of differentially expressed genes after filtering, but "none of the technologies we tested could provide reliable absolute quantification", and only 59% of genes titrated correctly. See assays/rna-seq. 14

Reported, not yet replicated

Mass-spectrometry proteomics as a model more

Mass spectrometry counts peptide ions; peptides and then proteins are inferred. In 11 labs, protein CV was 8.3% intra-day, 11.9% inter-day and 22.0% between labs. On 19,800 repository runs, holding protein-group FDR at 1% needed a peptide filter of 0.12%, and DIA entrapment tests put the protein-level false proportion at about 0.8 to 9.1% when tools claim 1%. See assays/mass-spectrometry-proteomics. 151617

Reported, not yet replicated

The model systems, one by one more

Each system keeps something and loses something: cell lines (cheap, uniform; identity and drift problems), primary cells (donor biology; they stop dividing or lose function), organoids (self-organised structure; no immune system, limited vessels), H&E sections (architecture; one thin slice), organ-on-chip (flow and co-culture; one vendor-authored validation) and animals (whole body; different species biology). Each has an assay record: assays/cell-lines, assays/primary-cells, assays/organoids, assays/h-and-e-staining, assays/organ-on-chip, assays/animal-models. 1819220

Reported, not yet replicated

His statement on organoids and H&E (his words)

He said: "Yes, organoids will help, but they still don't simulate the circulatory system. H&E helps, but it's still not at the organ level." 3

Amit's hypothesis

Organoids and the circulatory system: the sourced account more

Blood-vessel organoids form capillary networks within 2 to 3 weeks; vessel-like networks have been engineered into cortical organoids (perfused only after transplant); kidney organoids under flow on chips have perfusable lumens; human cortical organoids transplanted into mice were vascularised by host vessels within 7 to 10 days in 80% of grafts. No opened source shows whole-body circulation (blood cells, lymph, immune trafficking, inter-organ pharmacokinetics); that is a synthesis, not a quoted finding. So the statement holds for the whole body and fails as a flat claim that organoids have no vessels. 2122232425

Reported, not yet replicated

H&E and the organ level: the sourced account more

No opened source measures how much organ-level information an H&E slide loses. What is measured: 115 pathologists reading 240 breast biopsy cases agreed with a consensus reference in 75.3% of 6,900 interpretations (atypia 48%), and a 2025 preprint benchmark found spatial gene expression predicted from H&E poor over all genes. A slide is a thin 2D section read by a fallible human; that it is not the organ is argument rather than measurement here. 2026

Reported, not yet replicated

The caveat stack from human disease to the number on a screen more

Ten layers: (1) patient and disease heterogeneity, (2) sample and cell mixture, (3) choice of model system, (4) model identity, (5) drift and culture conditions, (6) sample preparation, (7) instrument, (8) inference, (9) analysis and statistics, (10) readout as a proxy for a patient outcome. Measured figures: 608 lines on the cross-contamination register, at least 75% inconsistency of strongly active compounds across MCF7 strains, mass-spectrometry CV 8.3 to 22.0%, 0.12% peptide FDR needed for 1% protein-group FDR, yeast mRNA-protein variance explained 38% to 86% depending on noise treatment, and oncology surrogate-to-survival correlations often moderate to low. The ordering and the claim that layers are not additive are our own reasoning; no source combines the layers. 27181516128

Our inference

How well each layer predicts the human more

The published figures count different things. 37% of 76 highly cited animal studies replicated in a human randomised trial and 45% were untested. 5% of 281 therapies reached approval (4% of all 367), and a separate pooled relative risk of 0.86 (95% CI 0.80 to 0.92) is not a share of therapies. 71% of human toxicities were also seen in animals (150 compounds, sensitivity only). 13.8% of Phase 1 entrants reach approval (10.4% in an earlier study), which is clinical-stage attrition, not animal translation. See concepts/animal-to-human-translation. 2923031

Reported, not yet replicated

His statement on in vitro (his words)

He said: "To me, in vitro was basically useless. All it tells you is binding. It's a quick way to kill a project." 3

Amit's hypothesis

In vitro "only tells you binding": the sourced account more

Where he is right: cell lines are misidentified and drift, so a hit may not reproduce. Where the sources disagree with the strong form: a cell assay can kill cells through off-target effects (deleting the nominal target protein did not change efficacy for cancer drugs in clinical testing; the atlas's "ten drugs" count is unverified), each of three cell-based data types predicts 6 to 10% of 270 assays with high accuracy and 21% combined, and in vitro binding is the regulated gate for choosing an animal species. On "a quick way to kill a project", no opened source measures how often in vitro assays kill projects that would have worked. 18323334

Our inference

His statement on clinical trials (his words)

He said: "Anyway, the whole thing looming over all of this is that everything is models of models until you get to clinical trials, and that's real human data." 3

Amit's hypothesis

Trials as models too more

Trial data are real human outcomes read through endpoints and selected populations. Of programmes entering Phase 1, 13.8% reached approval in 2000 to 2015 (406,038 trial entries; oncology 3.4%). A sceptic's review reports oncology surrogate-to-survival correlation as often moderate to low. No opened source quantifies how patient selection distorts effects. Queue item 19 covers this in full (reports/clinical-data-as-it-is). 3128

Reported, not yet replicated

Where money moves more

No primary source gave a figure for money moving between modalities. Found: median capitalised R&D cost per approval of $985.3M (95% CI $683.6M to $1,228.9M) in one 63-drug sample and $2,558M in 2013 dollars in an industry-supplied 106-drug sample; Virtual Cell Challenge 2025 prizes of $100,000, $50,000 and $25,000 plus a $100,000 generalist prize; and the Liver-Chip authors' own modelled saving of over $3 billion a year. NIH states funding announcements "must now also support human-focused approaches" with no dollar figure. 3536371938

Reported, not yet replicated

Evolutionary distance and what an animal preserves (standing question Q1) more

Human and mouse diverged about 80 million years ago; coding regions average 85% identity (60 to 99% by gene) and fewer than 10 of about 4,000 genes studied occur in only one species. Orthology covers "less than half" of the genomes with expanded reproduction and immunity families, and regulation is much more divergent than genes. The best-documented mismatches (inflammation response data, TGN1412) involve the immune system, but no opened source measures how much evolutionary distance explains animal-to-human failure; that link is an inference from two cases. 394041424344

Our inference

Plain answer

A number on a screen is two stand-ins deep: an instrument and its inference measure a model system (a cell line, organoid, tissue slice or animal) that stands in for a human. The sources give measured error for several layers separately (for example noise in mRNA-versus-protein comparisons, mass-spectrometry reproducibility, cell-line drift, animal-to-human translation) but none combines them. 1615182

Reported, not yet replicated

Why it matters

Before trusting a model-system result, a reader needs to know which layer produced it and what error that layer is known to add. The report that wrote this record found measured figures for some layers and none for others, and marks the gaps. 62

Our inference

Equation

steady-state protein level = mRNA level x translation rate / protein removal rate (a simplification of the four rates: mRNA made, mRNA removed, protein made, protein removed)

Real numbers

mRNA-protein correlation, yeast, 24 studies

raw r 0.618 (38% of variance); after a noise model r 0.926 (86%). 1

Reported, not yet replicated

mRNA-protein correlation, one gene across 12 human tissues

mean r 0.29 (R2 0.08), 6,104 genes. 6

Reported, not yet replicated

Mass-spectrometry reproducibility across 11 labs, HEK293

protein CV 8.3% intra-day, 11.9% inter-day, 22.0% between labs. 15

Reported, not yet replicated

Animal-to-human translation

5% of 281 therapies reached approval (4% of all 367); 50% reached any human study, 40% an RCT (45% and 36% across all 367). 2

Reported, not yet replicated

Teach this

In one sentence

Every biological number is a measurement of a model of a human, with a separate error at each layer, and trials are models too. 12

Our inference

The common wrong idea

mRNA level tells you protein level.

It depends on the axis: across genes in one sample it is moderate, for one gene across samples it is weak, and part of the gap is measurement noise.

The one citation to hand over: 6

Still open here

Linked from

Sources

  1. Csárdi G, Franks A, Choi DS, et al.. Accounting for experimental noise reveals that mRNA levels, amplified by post-transcriptional processes, largely determine steady-state protein levels in yeast. PLoS Genetics (2015). paper; opened 2026-11-17 all records citing this
  2. Ineichen BV, Furrer E, Grüninger SL, et al.. Analysis of animal-to-human translation shows that only 5% of animal-tested therapeutic interventions obtain regulatory approval for human applications. PLoS Biol (2024). paper; opened 2026-11-17 all records citing this
  3. Amit's dictation of 2026-11-17: what the atlas must contain. Kept in the atlas repository; not published.
  4. Vogel C, Abreu Rde S, Ko D, et al.. Sequence signatures and mRNA concentration can explain two-thirds of protein abundance variation in a human cell line. Mol Syst Biol (2010). paper; opened 2026-11-17 all records citing this
  5. Nagaraj N, Wisniewski JR, Geiger T, et al.. Deep proteome and transcriptome mapping of a human cancer cell line. Mol Syst Biol (2011). paper; opened 2026-11-17 all records citing this
  6. Franks A, Airoldi E, Slavov N. Post-transcriptional regulation across human tissues. PLoS Comput Biol (2017). paper; opened 2026-11-17 all records citing this
  7. Nusinow DP, Szpyt J, Ghandi M, et al.. Quantitative Proteomics of the Cancer Cell Line Encyclopedia. Cell (2020). paper; opened 2026-11-17 all records citing this
  8. Gonçalves E, Poulos RC, Cai Z, et al.. Pan-cancer proteomic map of 949 human cell lines. Cancer Cell (2022). paper; opened 2026-11-17 all records citing this
  9. Jarnuczak AF, Najgebauer H, Barzine M, et al.. An integrated landscape of protein expression in human cancer. Sci Data (2021). paper; opened 2026-11-17 all records citing this
  10. Upadhya SR, Ryan CJ. Experimental reproducibility limits the correlation between mRNA and protein abundances in tumor proteomic profiles. Cell Rep Methods (2022). paper; opened 2026-11-17 all records citing this
  11. Schwanhäusser B, Busse D, Li N, et al.. Global quantification of mammalian gene expression control. Nature (2011). paper; opened 2026-11-17 all records citing this
  12. Li JJ, Bickel PJ, Biggin MD. System wide analyses have underestimated protein abundances and the importance of transcription in mammals. PeerJ (2014). paper; opened 2026-11-17 all records citing this
  13. Jovanovic M, Rooney MS, Mertins P, et al.. Dynamic profiling of the protein life cycle in response to pathogens. Science (2015). paper; opened 2026-11-17 all records citing this
  14. SEQC/MAQC-III Consortium. A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium. Nat Biotechnol (2014). paper; opened 2026-11-17 all records citing this
  15. Collins BC, Hunter CL, Liu Y, et al.. Multi-laboratory assessment of reproducibility, qualitative and quantitative performance of SWATH-mass spectrometry. Nat Commun (2017). paper; opened 2026-11-17 all records citing this
  16. The M, Samaras P, Kuster B, et al.. Reanalysis of ProteomicsDB Using an Accurate, Sensitive, and Scalable False Discovery Rate Estimation Approach for Protein Groups. Mol Cell Proteomics (2022). paper; opened 2026-11-17 all records citing this
  17. Wen B, Freestone J, Riffle M, et al.. Assessment of false discovery rate control in tandem mass spectrometry analysis using entrapment. Nat Methods (2025). paper; opened 2026-11-17 all records citing this
  18. Ben-David U, Siranosian B, Ha G, et al.. Genetic and transcriptional evolution alters cancer cell line drug response. Nature (2018). paper; opened 2026-11-17 all records citing this
  19. Ewart L, Apostolou A, Briggs SA, et al.. Performance assessment and economic analysis of a human Liver-Chip for predictive toxicology. Commun Med (Lond) (2022). paper; opened 2026-11-17 all records citing this
  20. Elmore JG, Longton GM, Carney PA, et al.. Diagnostic concordance among pathologists interpreting breast biopsy specimens. JAMA (2015). paper; opened 2026-11-17 all records citing this
  21. Wimmer RA, Leopoldi A, Aichinger M, et al.. Human blood vessel organoids as a model of diabetic vasculopathy. Nature (2019). paper; opened 2026-11-17 all records citing this
  22. Cakir B, Xiang Y, Tanaka Y, et al.. Engineering of human brain organoids with a functional vascular-like system. Nat Methods (2019). paper; opened 2026-11-17 all records citing this
  23. Homan KA, Gupta N, Kroll KT, et al.. Flow-enhanced vascularization and maturation of kidney organoids in vitro. Nat Methods (2019). paper; opened 2026-11-17 all records citing this
  24. Mansour AA, Gonçalves JT, Bloyd CW, et al.. An in vivo model of functional and vascularized human brain organoids. Nat Biotechnol (2018). paper; opened 2026-11-17 all records citing this
  25. Takebe T, Sekine K, Enomura M, et al.. Vascularized and functional human liver from an iPSC-derived organ bud transplant. Nature (2013). paper; opened 2026-11-17 all records citing this
  26. Wang C, Chan AS, Fu X, et al.. Benchmarking the translational potential of spatial gene expression prediction from histology. Nat Commun (2025). preprint, not peer reviewed; opened 2026-11-17 all records citing this
  27. Database of Cross-Contaminated or Misidentified Cell Lines (register version 14, 15 Feb 2026). International Cell Line Authentication Committee (2026). database, not peer reviewed; opened 2026-11-17 all records citing this
  28. Olivier T, Haslam A, Ochoa D, et al.. Bedside implications of the use of surrogate endpoints in solid and haematological cancers: implications for our reliance on PFS, DFS, ORR, MRD and more. BMJ Oncol (2024). review; opened 2026-11-17 all records citing this
  29. Hackam DG, Redelmeier DA. Translation of research evidence from animals to humans. JAMA (2006). paper; opened 2026-11-17 all records citing this
  30. Olson H, Betton G, Robinson D, et al.. Concordance of the toxicity of pharmaceuticals in humans and in animals. Regul Toxicol Pharmacol (2000). paper; opened 2026-11-17 all records citing this
  31. Wong, Chi Heem, Siah, Kien Wei, Lo, Andrew W. Estimation of clinical trial success rates and related parameters. Biostatistics (2019). paper; opened 2026-11-17 all records citing this
  32. Lin, Ann, Giuliano, Christopher J., Palladino, Ann, et al.. Off-target toxicity is a common mechanism of action of cancer drugs undergoing clinical trials. Science Translational Medicine (2019). paper; opened 2026-11-17 all records citing this
  33. Moshkov et al.. Predicting compound activity from phenotypic profiles and chemical structures (Moshkov et al. 2023), Europe PMC listing. Europe PMC listing (2023). paper; opened 2026-11-17 all records citing this
  34. ICH S6(R1): Preclinical safety evaluation of biotechnology-derived pharmaceuticals. International Council for Harmonisation. regulatory, not peer reviewed; opened 2026-11-17 all records citing this
  35. Wouters OJ, McKee M, Luyten J. Estimated Research and Development Investment Needed to Bring a New Medicine to Market, 2009-2018. JAMA (2020). paper; opened 2026-11-17 all records citing this
  36. DiMasi JA, Grabowski HG, Hansen RW. Innovation in the pharmaceutical industry: New estimates of R&D costs. J Health Econ (2016). paper; opened 2026-11-17 all records citing this
  37. Virtual Cell Challenge 2025 wrap-up. Arc Institute (2025). news, not peer reviewed; opened 2026-11-17 all records citing this
  38. NIH funding announcements to align with the initiative to prioritize human-based research. NIH Extramural Nexus (2025). organisation-page, not peer reviewed; opened 2026-11-17 all records citing this
  39. Importance of the mouse genome (NHGRI page, updated 2010). NHGRI (2010). organisation-page, not peer reviewed; opened 2026-11-17 all records citing this
  40. Mouse Genome Sequencing Consortium, Waterston RH, Lindblad-Toh K, et al.. Initial sequencing and comparative analysis of the mouse genome. Nature (2002). paper; opened 2026-11-17 all records citing this
  41. Yue F, Cheng Y, Breschi A, et al.. A comparative encyclopedia of DNA elements in the mouse genome. Nature (2014). paper; opened 2026-11-17 all records citing this
  42. Mestas J, Hughes CC. Of mice and not men: differences between mouse and human immunology. J Immunol (2004). review; opened 2026-11-17 all records citing this
  43. Seok J, Warren HS, Cuenca AG, et al.. Genomic responses in mouse models poorly mimic human inflammatory diseases. Proc Natl Acad Sci U S A (2013). paper; opened 2026-11-17 all records citing this
  44. Eastwood D, Findlay L, Poole S, et al.. Monoclonal antibody TGN1412 trial failure explained by species differences in CD28 expression on CD4+ effector memory T-cells. Br J Pharmacol 161:512-526 (2010). paper; opened 2026-11-17 all records citing this
  45. Edfors F, Danielsson F, Hallström BM, et al.. Gene-specific correlation of RNA and protein levels in human cells and tissues. Mol Syst Biol (2016). paper; opened 2026-11-17 all records citing this
Where it sits on the map
History of this record (2)

This record has been changed 2 times, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.

Status
partly written
Last updated
2026-11-17
Record
content/concepts/models-of-models.json