Annotating risk factors for heart disease in clinical narratives for diabetic patients.
Journal:
Journal of biomedical informatics
Published Date:
May 21, 2015
Abstract
The 2014 i2b2/UTHealth natural language processing shared task featured a track focused on identifying risk factors for heart disease (specifically, Cardiac Artery Disease) in clinical narratives. For this track, we used a "light" annotation paradigm to annotate a set of 1304 longitudinal medical records describing 296 patients for risk factors and the times they were present. We designed the annotation task for this track with the goal of balancing annotation load and time with quality, so as to generate a gold standard corpus that can benefit a clinically-relevant task. We applied light annotation procedures and determined the gold standard using majority voting. On average, the agreement of annotators with the gold standard was above 0.95, indicating high reliability. The resulting document-level annotations generated for each record in each longitudinal EMR in this corpus provide information that can support studies of progression of heart disease risk factors in the included patients over time. These annotations were used in the Risk Factor track of the 2014 i2b2/UTHealth shared task. Participating systems achieved a mean micro-averaged F1 measure of 0.815 and a maximum F1 measure of 0.928 for identifying these risk factors in patient records.
Authors
Keywords
Aged
Boston
Cohort Studies
Comorbidity
Computer Security
Confidentiality
Coronary Artery Disease
Data Mining
Diabetes Complications
Documentation
Electronic Health Records
Female
Humans
Incidence
Longitudinal Studies
Male
Middle Aged
Narration
Natural Language Processing
New York
Pattern Recognition, Automated
Risk Assessment
Vocabulary, Controlled