LoRA-Based Cascaded Multimodal Fusion for Action Recognition in Medical Training Environments· 医疗培训环境中基于LoRA的级联多模态融合
This paper presents a cascaded Low-Rank Adaptation (LoRA)-based multimodal fusion framework for action and activity recognition in healthcare-oriented training environments. The proposed architecture combines parameter-efficient modality-specific adaptation with sequential fusion, enabling modalities to be integrated in stages without retraining previously learned components. Rather than assuming a fixed fusion structure, the framework first integrates more closely related modalities and then incorporates additional heterogeneous modalities, supporting scalable adaptation across datasets with different modality sets.We evaluate the framework on two healthcare-oriented training environment datasets: NurViD and the Nurse Training dataset. Across these datasets, preliminary results suggest that the proposed cascaded fusion strategy improves over individual modality models and provides competitive performance relative to previously reported dataset-specific baselines. Overall, these findings indicate that cascaded LoRA-based fusion is a promising parameter-efficient approach for integrating heterogeneous modalities in medical training action and activity recognition tasks. github: https://github.com/anonymous0-ai/LoRA-Based-Cascaded-Multimodal-Fusion-.git.
一种基于LoRA的级联多模态融合方法,用于医疗培训环境中的动作识别。
- 核心方法
- 采用了LoRA技术实现模态特异性高效参数调整,并通过顺序融合方式逐步整合不同模态数据,避免重复训练。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 医疗培训环境中动作和活动识别的多模态数据整合问题,需高效参数利用。
- 关键实验
- 在NurViD和Nurse Training两个医疗培训数据集上的实验结果表明,该方法优于单一模态模型,且与数据集特定基线相当。
- 主要贡献
- 提出了一种可扩展的多模态融合框架,对于不同模态组合的医疗培训数据集表现出色。
- 意义与局限
- 此方法能够有效整合多模态数据,提高医疗培训环境中的动作识别精度,但其普适性和长期稳定性仍需进一步验证。