Why Alzheimer's Diagnostic Models Fail on Anatomy

Medical imaging models for Alzheimer's detection frequently reach high benchmark scores by identifying superficial patterns that do not reflect actual pathology. An analysis of Alzheimer's detection using ADNI-1 data demonstrates that models leveraging ResNet18 and Transformer architectures achieve accuracy figures driven by anatomical misattribution and target leakage. The models prioritize the skull, orbital regions, and artifacts inherent to specific scanner protocols instead of neurodegeneration.

The Mechanism of Shortcut Learning

Scanner-specific artifacts become fixed identifiers if a particular clinical center used a specific machine for a subset of the training data. When the network encounters these features, it assigns them high predictive weights because they remain consistent across the training set. The model is efficiently exploiting unintended correlations that distinguish the classes without looking at the hippocampus or the cortex. The anatomical irrelevance of the classifier's attention was confirmed by comparing its Grad-CAM outputs against FastSurfer segmentation masks. Researchers found that models consistently attended to extracranial noise rather than deep brain structures.

Label Leakage in Multimodal Fusion

Many multimodal architectures use contrastive learning to align image encoders with tabular patient data via the InfoNCE loss function. This function calculates the inner product of the image representation and the diagnostic label vector. During training, the encoder produces feature maps where the dot product with the label vector is maximized. If a non-biological pixel, such as a high-intensity signal in the orbital bone, correlates with a diagnosis, the model treats this pixel as a high-value feature. The loss function forces the encoder to project this noise into the latent space such that its similarity to the label is increased. Because the math prioritizes minimizing the distance to the clinical label, the encoder optimizes its weights to treat that noise as the primary diagnostic signal.

The 73.8% accuracy observed during regional volume alignment demonstrates how leakage-driven bounds skew results. The model treats these target variables as ground truth proxies, forcing the encoder to compress clinical labels into the image representation rather than learning the biological signature of neurodegeneration.

Enforcing Anatomical Constraints

Developers can enforce anatomical constraints by applying a spatial mask to the input that forces the encoder to ignore the skull and extracranial noise. Alternatively, researchers introduce an auxiliary loss term that compares the model's internal attention maps against ground-truth segmentation masks. The loss function calculates the pixel-wise difference between the attention maps and the ground-truth binary masks using Mean Squared Error. During backpropagation, the gradients for voxels identified as non-biological in the segmentation mask are zeroed out. By mathematically nullifying the gradient flow for these extracranial regions, the loss function prevents the model from updating weights based on noise. This forces the encoder to re-allocate its attention to the remaining active, tissue-specific pixels in the brain.

Consider a model trained to detect volume loss in the hippocampus. Without the mask, it may fixate on a brightness artifact near the orbital bone. By adding the segmentation penalty, the training process explicitly prevents the model from using that artifact to lower its loss. This approach requires pre-segmentation overhead but shifts model attention from incidental noise to hippocampal features.

What remains unclear is the degree to which these structural constraints affect the model's ability to generalize to datasets where the noise patterns differ from the training set. If the auxiliary loss forces the model to ignore certain voxels, it risks losing information if those voxels unexpectedly contain valid biomarkers. Balancing segmentation-driven focus with the model's capacity to learn subtle, non-predefined features is the next necessary step for clinical viability.