Testing the reliability of an AI-based large language model to extract ecological information from the scientific literature
Problem
Manually extracting ecological data from scientific literature is slow, limiting the creation of large ecological databases.
Why it matters
Large-scale ecological databases at unprecedented speed and scale depend on efficient, accurate data extraction.
What the AI did
An LLM was formally tested against a human reviewer at extracting ecological data from scientific literature, measuring speed and accuracy.
Results
Over 50 times faster than the human reviewer; >90% accuracy on discrete and categorical data; poor on certain quantitative data.
Impact
Evidence supports LLM-assisted literature extraction, but additional quality assurance steps are still required.
Record
| Venue | npj Biodiversity |
|---|---|
| Published | 2024 |
| Work type | article |
| Authors | Andrew V. Gougherty, Hannah L. Clipp |
| Citations | 52 (OpenAlex, at import time) |
| References | 15 |
| Model or method | not stated in source |
| Organization | not stated in source |
| Primary source | https://doi.org/10.1038/s44185-024-00043-9 |
| Open access | gold |
| Keywords | large language models, data extraction, ecology, literature mining, accuracy benchmark |
Importance rating
real but narrower advance
Enables ecological databases at unprecedented speed and scale (50x faster, >90% accuracy on some data), a genuinely useful research capability, though ecological literature extraction is a niche workflow.