concept · intermediate · partly written
How PaperQA2 turns a paper into something an agent reasons over
PaperQA2 cuts papers into text chunks, has an LLM summarise each against the question, and reasons over those cited summaries; nothing is written back to a store.
Also called: PaperQA, PaperQA2, PaperQA3
The steps
Search by keyword for open-access papers, parse the PDF, cut into windows of about 2,250 tokens, embed, rank, then have an LLM write a 200 to 400 token summary of each chunk in light of the question and score it from 0 to 10. The unit of knowledge is that cited summary, built fresh per question. 1
Reported, not yet replicated
Measured, and by whom more
The builders report 66.0% accuracy and 85.2% precision on LitQA2 (248 questions, a benchmark from the same group), against 67.7% for nine PhD annotators. Removing the summarising step lowered accuracy (t = 9.29). All of this is builder-measured; a chance level was not stated in the text read. 1
Reported, not yet replicated
Entity grounding and what it cannot read more
Nothing opened says gene names are mapped to HGNC or UniProt identifiers; matching is by embedding and keyword. Text-only reading misses table rows and figure panels. Edison's PaperQA3 page (company, 18 Feb 2026) says page-image parsing replaces text-only reading; its figure and PDF-failure numbers come from a vendor-published case study on a prerelease benchmark. 12
Reported, not yet replicated
Plain answer
A paper becomes a set of question-conditioned, cited summaries, and the summariser, not the retriever, limits what later steps can reason over. 1
Reported, not yet replicated
Why it matters
Whatever the summary drops, such as a table row or a negative result in a supplement, is gone for every later step. 1
Our inference
Still open here
- How accurate is gene-name matching inside PaperQA2? Not measured in anything opened.
Not yet written: Equation, Real numbers.
Linked from
Sources
- Skarlinski MD, et al.. Language agents achieve superhuman synthesis of scientific knowledge (PaperQA2). arXiv 2409.13740 (2024). preprint, not peer reviewed; opened 2026-11-18 all records citing this
- Edison literature agent (PaperQA3). Edison Scientific (2026). organisation-page, not peer reviewed; opened 2026-11-18 all records citing this
Where it sits on the map
History of this record (5)
This record has been changed 5 times, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.
- Status
- partly written
- Last updated
- 2026-11-19
- Record
- content/concepts/paperqa-chunk-summary-reading.json