[Visual prior-guided masked image modeling enhances chest X-ray diagnostic efficacy].
To develop a masked image modeling framework that integrates clinical visual priors to enhance semantic understanding and diagnostic performance on chest X-ray images.
A novel framework VP-MIM was constructed by incorporating clinical visual priors into the MIM process. Eye-tracking data from radiologists were used to distinguish diagnostically relevant from irrelevant regions during the masking phase, enabling a controlled masking strategy. In the reconstruction phase, a pyramid attentive reconstruction module was developed to introduce multi-scale supervision, which was further refined by semantic-aware recalibrated gaze heatmaps to optimize feature learning.
Experiments conducted on the RSNA Pneumonia and ChestXray-14 public datasets showed that under linear evaluation with only 2616 pre-training samples, VP-MIM achieved an AUC of 86.83 on the RSNA Pneumonia single-label classification task and a mean AUC (mAUC) of 72.82 on the ChestXray-14 multi-label classification task. In full fine-tuning experiments, VP-MIM showed strong scalability when the amount of pre-training data increased, reaching an mAUC of 85.49 on ChestXray-14, which verified good scalability and excellent performance of this model in practical diagnostic tasks.
VP-MIM alleviates the limitations of semantic loss and insufficient multi-scale modeling in medical imaging MIM to result in improved diagnostic performance of chest X-ray.
A novel framework VP-MIM was constructed by incorporating clinical visual priors into the MIM process. Eye-tracking data from radiologists were used to distinguish diagnostically relevant from irrelevant regions during the masking phase, enabling a controlled masking strategy. In the reconstruction phase, a pyramid attentive reconstruction module was developed to introduce multi-scale supervision, which was further refined by semantic-aware recalibrated gaze heatmaps to optimize feature learning.
Experiments conducted on the RSNA Pneumonia and ChestXray-14 public datasets showed that under linear evaluation with only 2616 pre-training samples, VP-MIM achieved an AUC of 86.83 on the RSNA Pneumonia single-label classification task and a mean AUC (mAUC) of 72.82 on the ChestXray-14 multi-label classification task. In full fine-tuning experiments, VP-MIM showed strong scalability when the amount of pre-training data increased, reaching an mAUC of 85.49 on ChestXray-14, which verified good scalability and excellent performance of this model in practical diagnostic tasks.
VP-MIM alleviates the limitations of semantic loss and insufficient multi-scale modeling in medical imaging MIM to result in improved diagnostic performance of chest X-ray.