Leveraging BERT for embedding ICD codes from large scale cardiovascular EMR data to understand patient diagnostic patterns.
Journal:
BMC medical informatics and decision making
Published Date:
Aug 11, 2025
Abstract
The integration of electronic medical records (EMRs) with artificial intelligence (AI) is enhancing medical research, particularly in real-world evidence (RWE) studies. Extracting insights from coded medical data, such as ICD-10 codes, is essential for patient characterization. Traditional techniques, such as one-hot encoding (OHE), face limitations, particularly in managing high-dimensional data. In this study, a Bidirectional Encoder Representations from Transformers (BERT) approach is introduced to encode ICD-10 diagnostic codes, significantly improving model performance and reducing dimensionality. Data from 495,269 patients who visited the Cardiology Department at Asan Medical Center between 2000 and 2020 were used. The performance of models trained with OHE and ClinicalBERT embeddings was compared. For predicting major adverse cardiovascular events within one year following percutaneous coronary intervention (PCI) or coronary artery bypass grafting (CABG), the ClinicalBERT (code-embedded) model outperformed OHE. It achieved an AUC of 0.746 compared to 0.719, while also significantly reducing the dimensionality from 2,492 to 128. This method, which integrates diagnostic and medication data, provides valuable insights into patient care, enhancing the precision of predictions and supporting healthcare professionals in making more informed decisions.
