ai.hackcv
论文精选 60arXiv

MedRoundsQA: A Persona and Difficulty Aware Evaluation for Multi-Turn Medical Consultations· MedRoundsQA:多轮医疗咨询的角色与难度感知评估

Medical benchmarks are dominated by single-turn, multiple-choice clinical cases that poorly reflect real consultations. Practically, clinicians elicit evidence interactively and patient communication varies widely. We introduce MedRoundsQA, a multi-turn diagnostic benchmark derived from 1,387 board-exam cases across 17 specialties. Each case is converted into a structured 24-slot clinical record, and then instantiated as controlled doctor-patient dual-agent dialogues under varying patient personas, with the underlying clinical content held fixed. We further classify cases by difficulty using model-based uncertainty to enable easy-to-hard analysis. Evaluations of fifteen LLM doctor agents show that (i) moving from a single-turn diagnosis on the standardized records to multi-turn consultatio

AI 解读论文

多轮医疗咨询评估基准,考虑角色与难度

核心方法
构建MedRoundsQA多轮医疗咨询评估基准,采用1,387个跨17专科的病例,转化为24个结构化临床记录槽位,并根据不同患者角色生成对话;使用模型不确定性对难度进行分类
适合谁读
研究者、工程师、医疗从业者
要解决的问题
现有医疗评估基准集中在单一回合的多选题上,无法真实反映临床实际情况
关键实验
对15个LLM医生代理进行了评估,显示多轮诊断比单回合更能测试真实临床能力
主要贡献
提供了更贴近实际医疗咨询过程的多轮评估基准,及基于模型不确定性的难度分类方法
意义与局限
有助于更好地评估医疗AI系统,提高其在实际应用场景中的可靠性和准确性;但对话的生成和评估标准可能需要进一步完善
领域:cs.AI作者:Youssef Mohamed、Ahmed Heakl、Qinrong Cui
相关推荐
论文精选 65已解读

Simulated Control Evals· 模拟控制评估

Control evaluations measure whether safeguards can prevent an AI model from causing undesi…

领域:cs.AI作者:Jeffrey Olmo、David Zhang、Tyler Tracy
📎 arXiv🕒 10-23 01:26🔗 arxiv.org

本站内容由 LLM 精选聚合,原文版权归 arXiv 所有 · 摘录仅供参考