question · intermediate · partly written
Who validates language-model-proposed ontology terms when less confident raters accept fluent definitions?
Less confident raters scored AI-written definitions as good as human ones, which leaves open who checks them.
What is measured more
In one evaluation, less confident raters scored language-model definitions as good as human definitions; GPT-4 reached SubClassOf precision 0.894 and recall 0.5. 1
Reported, not yet replicated
What is missing
What is known
Raters of lower confidence are not a reliable check. 1
Reported, not yet replicated
What is not
How to validate at scale. 1
Reported, not yet replicated
Still open here
- What review process catches fluent but wrong definitions?
Not yet written: Observation, Hypotheses, What would settle it, Who works on it.
Linked from
Sources
- Toro S, Anagnostopoulos AV, Bello SM, Blumberg K, et al.. Dynamic Retrieval Augmented Generation of Ontologies using Artificial Intelligence (DRAGON-AI). Journal of biomedical semantics (2024). paper; opened 2026-11-18 all records citing this
Where it sits on the map
History of this record (5)
This record has been changed 5 times, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.
- Kind
- big-open
- Register
- open-question
- Triage
- needs-research
- Status
- partly written
- Last updated
- 2026-11-19
- Record
- content/questions/who-validates-language-model-proposed-ontology-terms.json