The role of fine-grained annotations in supervised recognition of risk factors for heart disease from EHRs.
Journal:
Journal of biomedical informatics
Published Date:
Jun 26, 2015
Abstract
This paper describes a supervised machine learning approach for identifying heart disease risk factors in clinical text, and assessing the impact of annotation granularity and quality on the system's ability to recognize these risk factors. We utilize a series of support vector machine models in conjunction with manually built lexicons to classify triggers specific to each risk factor. The features used for classification were quite simple, utilizing only lexical information and ignoring higher-level linguistic information such as syntax and semantics. Instead, we incorporated high-quality data to train the models by annotating additional information on top of a standard corpus. Despite the relative simplicity of the system, it achieves the highest scores (micro- and macro-F1, and micro- and macro-recall) out of the 20 participants in the 2014 i2b2/UTHealth Shared Task. This system obtains a micro- (macro-) precision of 0.8951 (0.8965), recall of 0.9625 (0.9611), and F1-measure of 0.9276 (0.9277). Additionally, we perform a series of experiments to assess the value of the annotated data we created. These experiments show how manually-labeled negative annotations can improve information extraction performance, demonstrating the importance of high-quality, fine-grained natural language annotations.
Authors
Keywords
Aged
Cohort Studies
Comorbidity
Computer Security
Confidentiality
Coronary Artery Disease
Data Mining
Diabetes Complications
Electronic Health Records
Female
Humans
Incidence
Longitudinal Studies
Male
Maryland
Middle Aged
Narration
Natural Language Processing
Pattern Recognition, Automated
Risk Assessment
Supervised Machine Learning
Vocabulary, Controlled