How Much Does Protein Structure Really Help? A Case Study in Mutation-Induced Stability Prediction

Multimodal neural networks integrating protein language models (PLMs) with structure-derived features are increasingly common for predicting mutation effects, yet fundamental mechanisms remain poorly characterized. In this paper, we articulate and address two key questions: (i) do these architectures exploit mutation-conditioned structural changes, and (ii) does structure provide additive value over PLM-learned sequence embeddings afterall? Using ΔTm (melting temperature shift) prediction as a controlled testbed, a phenotype expected to depend strongly on three-dimensional geometry, we introduce generalizable diagnostic methodologies: systematic channel ablations quantifying each modality’s marginal contribution, and context-radius probing, a novel technique restricting inputs to progressively larger neighborhoods around mutations to spatially localize predictive signal. Across ten independent runs per condition, we find PLM embeddings dominate: removing structure causes minimal performance change, while removing PLMs causes performance collapse. Context-radius probing reveals signal is highly localized; mutation-site-only models recover full-context performance. Critically, comparing wild-type-shared versus mutation-conditioned structural regimes reveals no systematic gain from geometric perturbations, demonstrating that current representations function as static fold priors because downstream featurization attenuates mutation-induced changes. However, structure helps selectively: benefits concentrate in variants with atypical PLM embeddings occupying phenotypically incoherent neighborhoods where sequence-derived priors are locally unreliable. Though focused on a controlled testbed, this work surfaces a key challenge for any protein prediction task where domain knowledge suggests structure should matter: not whether to “add structure,” but how to represent and integrate geometry so that it contributes distinct signal beyond strong sequence priors. We provide architecture-agnostic diagnostics to test and quantify when and how explicit structure delivers that added value.