Large Language Models, scientific knowledge and factuality: A framework to streamline human expert evaluation.
Journal:
Journal of biomedical informatics
Published Date:
Sep 12, 2024
Abstract
OBJECTIVE: The paper introduces a framework for the evaluation of the encoding of factual scientific knowledge, designed to streamline the manual evaluation process typically conducted by domain experts. Inferring over and extracting information from Large Language Models (LLMs) trained on a large corpus of scientific literature can potentially define a step change in biomedical discovery, reducing the barriers for accessing and integrating existing medical evidence. This work explores the potential of LLMs for dialoguing with biomedical background knowledge, using the context of antibiotic discovery.