Biology Atlas

Map / Concepts

concept · intermediate · partly written

Does the molecular representation or the data matter more? What chemistry comparisons show

No comparison was found that varies only the representation; near misses suggest evaluation design matters more than representation or algorithm.

His idea, in his words

"You have innovation at the algorithm layer. I find you can innovate at the algorithm layer all you want, but that can only be as powerful as the data representation. The data representation, of course, can only be as powerful as the data, but at least there's more space to innovate there" 1

Amit's hypothesis

What was measured more

Riniker and Landrum fixed the similarity method and varied 12 fingerprints; only the simple baselines differed significantly. Praski and colleagues froze 28 pretrained embeddings under identical classifiers; ECFP count matched all but CheMeleon, though encoders were pretrained on different data. Jiang and colleagues, on 11 datasets and 50 random splits, found descriptor models on average better and cheaper than graph models, with graph models best on larger or multi-task sets; the average neural-network AUC on ToxCast fell from 0.897 to 0.770 once a leak in the original split was removed. Yang and colleagues found their directed message-passing network beat a random forest on Morgan fingerprints significantly on 14 of 15 datasets and worse on one, called random splits a poor proxy for chronological ones, and showed a hybrid beat either alone. Mayr and colleagues found feed-forward networks beat graph convolutions and SMILES networks in almost all feature categories, and the winner still used fingerprint-type features. 23456

Reported, not yet replicated

Where the representation hits a wall more

On activity cliffs, pairs of similar molecules with very different activity, all methods struggled in a study of 24 methods and 30 targets, and descriptor-based machine learning outperformed more complex deep learning. 7

Reported, not yet replicated

A string representation case more

SELFIES guarantees that every string decodes to a valid molecule; that is a validity claim. In the one independent test found with an ECFP plus XGBoost baseline and a structure-distant split, SMILES was better on classification, and another study found no single configuration winning across tasks under MoleculeNet default splits. 8910

Reported, not yet replicated

How to read the disagreement (our inference) more

Jiang, Yang and Mayr give three different answers. Baselines, tuning budgets, feature sets and splits differ, so representation alone does not explain it, and evaluation design acts as a fourth layer between the three he names. 456

Our inference

Cheapest test (our inference) more

One learner on frozen features (forest or ridge); vary only the featuriser (ECFP count, RDKit descriptors, one embedding); identical nested tuning; scaffold split; five to ten public ADMET sets. Compare the spread across featurisers with the spread across splits: if split spread is larger, the data and evaluation layer dominates. 23

Our inference

Still open here

Not yet written: Plain answer, Why it matters, Equation, Real numbers.

Linked from

Sources

  1. Amit's dictation of 2026-11-18: how biology data is represented, and where ontologies can be innovated. Kept in the atlas repository; not published.
  2. Riniker S, Landrum GA. Open-source platform to benchmark fingerprints for ligand-based virtual screening. Journal of Cheminformatics (2013). paper; opened 2026-11-18 all records citing this
  3. Praski M, Adamczyk J, Czech W. Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learning. arXiv (2025). preprint, not peer reviewed; opened 2026-11-18 all records citing this
  4. Jiang D, et al.. Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models. Journal of Cheminformatics (2021). paper; opened 2026-11-18 all records citing this
  5. Yang K, Swanson K, Jin W, et al.. Analyzing Learned Molecular Representations for Property Prediction. Journal of Chemical Information and Modeling (2019). paper; opened 2026-11-18 all records citing this
  6. Mayr A, Klambauer G, et al.. Large-scale comparison of machine learning methods for drug target prediction on ChEMBL. Chemical Science (2018). paper; opened 2026-11-18 all records citing this
  7. van Tilborg, Derek, Alenicheva, Alisa, Grisoni, Francesca. Exposing the Limitations of Molecular Machine Learning with Activity Cliffs. Journal of Chemical Information and Modeling (2022). paper; opened 2026-11-17 all records citing this
  8. Krenn M, Häse F, Nigam A, et al.. Self-referencing embedded strings (SELFIES): A 100% robust molecular string representation. Machine Learning: Science and Technology (2020). paper; opened 2026-11-18 all records citing this
  9. Özçelik R, Grisoni F. A hitchhiker's guide to deep chemical language processing for bioactivity prediction. Digital Discovery (2025). paper; opened 2026-11-18 all records citing this
  10. Fender I, Gut JA, Lemmin T. Beyond performance: how design choices shape chemical language models. Journal of Cheminformatics (2025). paper; opened 2026-11-18 all records citing this
Where it sits on the map
History of this record (5)

This record has been changed 5 times, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.

Status
partly written
Last updated
2026-11-19
Record
content/concepts/representation-only-comparisons-in-chemistry.json