Cross-Sign Language Transfer Learning Using Domain Adaptation with Multi-scale Temporal Alignment· 跨手语的多尺度时间对齐域适应
Sign language serves as a vital means of communication for individuals with hearing impairments, yet recognition resources for the over 100 distinct sign languages are severely lacking. In response, we present our work on sign language recognition using transfer learning and the domain adaptation method TA3N, which utilizes the Temporal Relational Network (TRN) module for aligning multi-scale temporal relations. Our findings highlight the superior performance of Domain Adaptation to neural network-based transfer learning, particularly in improving recognition of American Sign Language (ASL). Our research also identifies the effectiveness of aligning shorter-term temporal features between source and target domains. In addition to using RGB, we conducted experiments using Optical Flow mode f
提出一种跨手语识别的域适应方法，改善ASL识别。
- 核心方法
- 利用TA3N域适应方法和TRN模块，对多尺度时间关系进行对齐，以实现从源域到目标域的手语识别迁移。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 超过100种不同的手语缺乏有效的识别资源，导致跨手语识别能力受限。
- 关键实验
- 实验使用RGB模式和光流模式数据，验证了较短期时间特征对齐的有效性。
- 主要贡献
- 证明了域适应方法相较于基于神经网络的迁移学习在跨手语识别任务上表现更优，特别是提高了ASL的识别准确率。
- 意义与局限
- 研究为手语识别资源匮乏提供了有效的解决方案，提升跨手语转换识别的性能，但方法可能不适用于所有手语种类。