Explainable Multimodal AI Approach to Reduce Unnecessary Breast Biopsies· 减少不必要的乳腺活检的多模态 AI 方法
Recently, there has been rise in multimodal vision-language models for computer-aided diagnosis (CAD), yet their application in breast diagnosis, particularly for predicting biopsy outcomes remains limited. This study presents a multimodal vision-language approach, incorporating Grad-CAM as an Explainable AI (XAI) technique, to predict benign tumors in patients recommended for biopsy, especially those classified as BI-RADS 4 and 5. A novel multimodal framework was devel- oped using Vision Transformer (ViT) for imaging feature extraction and DeBERTa-V3-Large for textual embeddings. The model was trained and evaluated on a comprehensive in-house dataset comprising 1,632 patients, each with four mammography views and a corresponding radiology report. The features of both modalities were extra
多模态 AI 方法结合 Grad-CAM 解释技术预测良性肿瘤减少乳腺活检。
- 核心方法
- 开发基于 Vision Transformer 和 DeBERTa-V3-Large 的多模态框架，提取影像和文本特征，并使用 Grad-CAM 技术增强模型可解释性。
- 适合谁读
- 研究者、工程师、医疗产品开发人员
- 要解决的问题
- 当前计算机辅助诊断中，多模态模型在乳腺诊断特别是预测活检结果的应用较少，且难以解释。
- 关键实验
- 模型在包含1,632名患者的内部数据集上训练和评估，每个患者有四张乳腺X光片和对应的放射学报告。
- 主要贡献
- 提出了一个能够预测良性肿瘤并减少不必要的乳腺活检的多模态 AI 模型，同时通过 XAI 技术提供了可解释性。
- 意义与局限
- 该研究有助于提高乳腺癌早期诊断的准确性和效率，减少患者不必要的医疗干预，但在实际应用中仍需进一步验证和优化。