concept · intermediate · partly written
Machine learning and language models in building and mapping ontologies
Language models can propose ontology terms and recognise terms in text at measured, moderate accuracy, and benchmark leakage can halve apparent gains.
Also called: LLM ontology construction, DRAGON-AI
Proposing new terms
DRAGON-AI, a retrieval-augmented GPT-4 system, proposed is-a relations across 10 ontologies with precision 0.894 and recall 0.5; GPT-3.5 scored 0.846 and 0.419; a reasoner baseline had precision 1.0 by definition and recall 0.337. Less confident raters scored the AI definitions as good as human ones. 1
Reported, not yet replicated
Recognising terms in text more
GPT models recognising HPO concepts reached F1 of 0.58 on abstracts and 0.75 on clinical observations, below the existing specialised methods on abstracts. Only the abstract was read. 2
Reported, not yet replicated
Leakage in entity linking more
On a clinical entity-linking benchmark, accuracy fell from 57.47% to 28.06% on one set (CANTEMIST) once training-set overlap was filtered out. The cause is the evaluation split, not the model or the ontology. 3
Reported, not yet replicated
Matching ontologies more
Unsupervised and ML matchers reached F1 of about 0.61 to 0.64 on the OMIM-to-ORDO disease task of the Bio-ML benchmark, and a plain edit-distance baseline matched or beat trained systems on one task. Language-model matcher results from newer systems were not opened. 4
Reported, not yet replicated
Diagnosis ranking more
In rare-disease diagnosis the best of seven language models ranked the correct diagnosis first in 23.6% of 5,213 case reports, against 35.5% for Exomiser. 5
Reported, not yet replicated
Plain answer
Why it matters
A new ontology would be built with these tools, so their measured limits are its starting conditions. 1
Our inference
Still open here
- Who validates language-model-proposed terms when less confident raters accept fluent definitions?
Not yet written: Equation, Real numbers.
Linked from
Sources
- Toro S, Anagnostopoulos AV, Bello SM, Blumberg K, et al.. Dynamic Retrieval Augmented Generation of Ontologies using Artificial Intelligence (DRAGON-AI). Journal of biomedical semantics (2024). paper; opened 2026-11-18 all records citing this
- Groza T, Caufield H, Gration D, Baynam G, et al.. An evaluation of GPT models for phenotype concept recognition. BMC medical informatics and decision making (2024). paper; opened 2026-11-18 all records citing this
- Medical Crossing: a Cross-lingual Evaluation of Clinical Entity Linking.. paper, not peer reviewed; opened 2026-11-18 all records citing this
- Yuan He, Jiaoyan Chen, Hang Dong, Ernesto Jiménez-Ruiz, et al.. Machine Learning-Friendly Biomedical Datasets for Equivalence and Subsumption Ontology Matching. arXiv (2022). preprint, not peer reviewed; opened 2026-11-18 all records citing this
- Reese JT, Chimirri L, Bridges Y, Danis D, et al.. Systematic benchmarking demonstrates large language models have not reached the diagnostic accuracy of traditional rare-disease decision support tools. European journal of human genetics : EJHG (2026). paper; opened 2026-11-18 all records citing this
Where it sits on the map
History of this record (5)
This record has been changed 5 times, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.
- Status
- partly written
- Last updated
- 2026-11-19
- Record
- content/concepts/machine-learning-for-building-and-mapping-ontologies.json