Generating Human-Readable Labels for SNOMED CT Expressions with LLMs: A Study on Model Performance and Rater Subjectivity.

Post-coordinated SNOMED CT expressions lack human-readable labels, hindering their use in analytics and representation learning. To address this issue, we evaluated labels for 100 SNOMED CT expressions generated by a large language model (LLM), which were blindly rated on semantic equivalence and phrasing quality by two clinical terminologists. While the LLM-generated labels achieved semantic equivalence comparable to official SNOMED CT terms, revealing primarily stylistic rather than conceptual errors, we found negligible interrater reliability between experts when rating novel post-coordinated expressions. This indicates that automated labeling is a viable approach, where the primary challenge is not model performance but the inherent subjectivity of the task itself. This suggests the objective could be reframed towards achieving semantic plausibility at scale, rather than replicating a single, definitive expert consensus.