Clinical evaluation of a natural language processing system for assisting structured diagnosis recording at the point of care: MiADE (Medical Information AI Data Extractor)

Structured recording of key information such as diagnoses is essential for safe, efficient patient care, but is currently done incompletely because it is time consuming for clinicians. We developed a natural language processing system called MiADE integrated with the Epic electronic health record to provide suggestions for structured diagnosis entries at the point of care. To evaluate the usability, usefulness and impact of MiADE, and identify recommendations for systems to improve point of care structured data recording. Mixed methods evaluation of the implementation of MiADE, with surveys, interviews and observed outpatient consultations. The number of structured diagnoses recorded per outpatient encounter was compared before and after MiADE, and completeness of inpatient problem lists was evaluated using the billing diagnoses as a gold standard. 85 clinicians consented to the study and were provided access to MiADE, and 24 used MiADE to receive structured data suggestions during the study period. Baseline survey data and observations showed wide variation in structured data recording despite clinicians considering it to be important. Half of post-implementation survey respondents considered MiADE to be ‘very’ or ‘moderately’ useful. Among outpatient users there was a 36% increase in the number of diagnoses recorded per encounter, but no improvement was seen in the inpatient setting. Natural language processing using MiADE has the potential to improve structured data recording, but further development and better clinician engagement are needed in order to maximise its impact. Despite the benefits of structured recording of healthcare data for individual care and research, much of the key information in electronic health records (such as diagnoses) is only recorded in free text Structured data entry in electronic health record systems can be time-consuming and cumbersome for clinicians Embedding natural language processing within the electronic health record to suggest structured data entries was reported by clinicians to be useful, and increased the recording of outpatient diagnoses Clinician engagement was challenging, and overall usage of structured data remained suboptimal Usability of electronic health record systems needs to improve to enable clinicians to record high quality data without impeding their workflow Natural language processing embedded within electronic health records may improve ease of use for data entry, but high-level clinician buy-in is also needed to influence professional documentation practice