Benchmarking Recording-Level Clinical Speech De-identification for Privacy-Preserving Clinical Audio
Journal:
medRxiv
Published Date:
Oct 17, 2026
Abstract
Transcript-based de-identification protects textual records but leaves sensitive health information (SHI) directly audible in source recordings, limiting trustworthy reuse of clinical audio. However, standardized evaluations of recording-level SHI detection remain limited. We established the SREDH-AICUP 2025 benchmark to evaluate recording-level SHI detection systems and characterize the approaches. All 22 qualifying systems combined automatic speech recognition with downstream SHI detection, whereas we additionally evaluated end-to-end detection from audio. Pipeline baselines outperformed the end-to-end baseline in transcription (MER, 0.123 vs 0.155) and SHI detection (macro-F1, 0.507-0.653 vs 0.422). Synthetic Mandarin speech-text augmentation enabled the end-to-end model to detect Mandarin SHI that was not identified without augmentation (macro-F1, 0.171 vs 0). Foundation models showed category-specific differences in SHI recognition, and type-aware adaptive routing improved macro-F1 from 0.656 to 0.697. These findings establish standardized reference performance, characterize current system designs, and demonstrate how language-specific augmentation and category-aware model integration can improve recording-level SHI detection.
