Electrocardiogram Abnormality Classification From Summary Statistics With Patient-Level Validation and Process-Focused Evaluation.

BACKGROUND: Machine learning (ML) applications in clinical medicine are vulnerable to data leakage, particularly temporal leakage from post-diagnostic features and patient-level leakage from improper partitioning, compromising electrocardiogram (ECG) abnormality detection systems. This study addresses these vulnerabilities through patient-level data splitting and systematic evaluation across multiple classification scenarios. METHODS: ECG data from 4419 observations representing 2180 unique patients were analyzed using Random Under-Sampling Boosting (RUSBoost). Three binary classification scenarios were defined: Scenario 1 (Abnormal vs. Normal, excluding borderline), Scenario 2 (Abnormal + Borderline vs. Normal), and Scenario 3 (Abnormal vs. Normal + Borderline). Patient-level stratified partitioning (60:20:20) prevented information leakage. Models were evaluated using accuracy, sensitivity, specificity, F1-score, learning curves, and precision-recall curves across all partitions. RESULTS: Scenario 1 demonstrated optimal generalization with test accuracy of 70.79%, sensitivity of 60.30%, specificity of 86.50%, and F1-score of 0.712, exhibiting monotonic decline across partitions (73.98%→71.50%→70.79%). Scenario 2 achieved 65.26% test accuracy with greater degradation (6.17 percentage points), reflecting increased difficulty when borderline cases were grouped with abnormal findings. Scenario 3 exhibited problematic non-monotonic patterns (71.84%→67.71%→69.36%) with premature convergence, indicating fundamental generalization challenges despite near-balanced classes (1.07:1 ratio). CONCLUSIONS: Aggregate metrics alone inadequately support clinical deployment decisions. Medical AI evaluation must examine learning dynamics, generalization stability, and precision-recall calibration across partitions. Process-focused standards assessing monotonic decline, convergence characteristics, and calibration stability are essential for reliable ECG abnormality detection deployment.