Conditional Rank Allocation for Taxonomy-Aware Medical Language Model Adaptation· 条件秩分配的医学语言模型适应
Medical question answering spans specialties and clinical operations that may benefit from different adaptation directions. We propose ARBOR, a parameter-efficient method that selects rank-one components from a shared low-rank basis for each question. An additive gate combines question representations, specialty tags, operation tags, and their interaction; a learned coefficient scales the adapter residual. An illustrative separation under orthogonal, equiprobable subtasks shows how conditional selection can avoid an approximation floor faced by a fixed update with the same active rank. This result motivates the design without asserting a corresponding bound for medical corpora. On Qwen3-8B across CMB, CMExam, MedQA, and MedMCQA, five-seed experiments yield 69.69% mean accuracy across bench
提出条件秩分配方法,改善医学语言模型的适应性。
- 核心方法
- 通过条件秩分配(ARBOR)方法选择每个问题的秩一组件,结合问题表示、专业标签、操作标签及其交互,通过学习的系数调整适配器残差。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 医学问题回答涉及多个专业和临床操作,需要针对不同任务有效适应医学语言模型。
- 关键实验
- 在Qwen3-8B模型上进行了五个种子实验,测试了CMB、CMExam、MedQA和MedMCQA数据集,平均准确率为69.69%。
- 主要贡献
- 提高了医学语言模型在多任务中的适应性和准确性,减少固定更新的近似误差。
- 意义与局限
- 为医学语言模型的多任务适应提供了新的思路,但目前 Haven3-8B 的结果尚未证明在更大规模医学语料上的有效性。