A context-augmented large language model for accurate precision oncology medicine recommendations.

Journal: Cancer cell
Published Date:

Abstract

The rapid expansion of molecularly informed therapies in oncology, coupled with evolving regulatory food and drug administration (FDA) approvals, poses a challenge for oncologists seeking to integrate precision oncology medicine into patient care. Large language models (LLMs) have clinical potential, but their reliance on general knowledge limits their ability to provide up-to-date and niche treatment recommendations. Here, we developed a retrieval-augmented generation (RAG)-LLM workflow using the molecular oncology almanac (MOAlmanac) and benchmarked it against an LLM-only approach for biomarker-driven treatment recommendations. Our RAG-LLM achieved up to 95% accuracy on synthetic queries and 93% on real-world queries collected from practicing oncologists. Finally, our study explored several prompting and retrieval strategies to enhance performance. Taken together, this approach may serve as valuable guidance for deploying LLMs to support cancer patients' treatment decisions in precision oncology clinical settings.

Authors

  • Hyeji Jun
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA.
  • Yutaro Tanaka
    Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA; Department of Pediatric Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA.
  • Shreya Johri
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA; Harvard Medical School, Boston, MA 02115, USA.
  • Sabrina Y Camp
    Department of Medical Oncology, Dana-Farber Cancer Institute, Harvard Medical School, Harvard University, Boston, Massachusetts.
  • Erik L Bao
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA.
  • Filipe L F Carvalho
    Department of Urology, MedStar Georgetown University Hospital, Washington, DC, USA.
  • Dan Y Gui
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA; Harvard Medical School, Boston, MA 02115, USA.
  • Alexander C Jordan
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA.
  • Chris Labaki
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, USA; Broad Institute, Cambridge, MA, USA; Harvard Medical School, Boston, MA, USA; Department of Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA.
  • Samantha D Martin
    Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA; Department of Pediatric Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Harvard Medical School, Boston, MA 02115, USA; Boston Children's Hospital, Boston, MA 02115, USA.
  • Matthew Nagy
    Cleveland Clinic Lerner College of Medicine Case Western Reserve University Cleveland Ohio USA.
  • Tess A O'Meara
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA.
  • Theodora Pappa
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA; Brigham and Women's Hospital, Boston, MA 02115, USA.
  • Erica Maria Pimenta
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA.
  • Eddy Saad
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA.
  • David D Yang
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Department of Radiation Oncology, Brigham and Women's Hospital, Boston, MA 02115, USA.
  • Riaz Gillani
    Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA; Department of Pediatric Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Harvard Medical School, Boston, MA 02115, USA; Boston Children's Hospital, Boston, MA 02115, USA.
  • Alok K Tewari
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA; Harvard Medical School, Boston, MA 02115, USA; Brigham and Women's Hospital, Boston, MA 02115, USA.
  • Brendan Reardon
    Department of Medical Oncology, Dana-Farber Cancer Institute, Harvard Medical School, Harvard University, Boston, Massachusetts.
  • Eliezer Van Allen
    Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA 02115, USA; Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA; Harvard Medical School, Boston, MA 02115, USA. Electronic address: [email protected].

Keywords

No keywords available for this article.