Reporting Quality of Mendelian Randomization Studies Based on STROBE-MR: An Investigation Powered by Large Language Models
Journal:
medRxiv
Published Date:
Oct 24, 2026
Abstract
Background: Mendelian randomization (MR) has been widely used to investigate causal relationships between exposures and outcomes. However, concerns have also been raised about the reporting quality of MR studies. STROBE-MR was developed as a reporting guideline for MR studies, but assessments of compliance with STROBE-MR in published MR studies remain limited and rely largely on manual review. With the rapid development of large language models (LLMs), it is now possible to automate STROBE-MR compliance assessment. Methods: We developed a pipeline using OpenAI's GPT-5-mini model for automated assessment of MR reporting quality. Model performance was evaluated on an annotated dataset of 46 eligible MR articles. We applied the pipeline to more than 13,000 MR articles published between 2005 and 2025 to assess compliance with the STROBE-MR guideline and to explore reporting patterns across STROBE-MR items, sections, and time periods. Complete compliance required reporting all applicable components of an item; a sensitivity analysis used a partial compliance criterion requiring at least one applicable component. Results: Our LLM-based pipeline agreed with the human consensus on 89.0\% of the STROBE-MR assessments. On average, 22 of 43 items (51.2\%) met the complete compliance criterion, with substantial variation in reporting rates across STROBE-MR items and sections. The least reported items were found in the "Methods", "Results", and "Other Information" sections. The number of reported items remained stable over time, with slight decreases after 2021. Sensitivity analysis showed that temporal trend patterns differed between complete and partial compliance, particularly in the Methods, Discussion and Other Information sections. Conclusions: This study demonstrates the potential of LLMs to automate reporting quality assessment in MR studies. Our results suggest that reporting quality still needs to be improved, and further efforts are required to promote the use of STROBE-MR. Our findings also suggest potential refinements to the STROBE-MR guideline to support more consistent application.
