Same Question, Different Answers: Auditing Persona-Dependent Variation in Six LLMs on Smoking-Cessation Advice· 同一问题，不同答案：六种 LLM 在戒烟建议中的角色差异
Hundreds of millions of people now ask large language model (LLM) chatbots for health advice, which raises a question that is simple to pose and hard to answer: does everyone get the same advice? We audit six widely deployed LLMs from four providers on smoking cessation, a domain with codified clini- cal guidance and substantial public-health stakes, by putting the same four cessation questions to twelve user personas that differ in life stage, self-described expertise, and smoking history, for a total of 2,880 responses. What the models say depends on who appears to be asking. Every model named FDA-approved prescription cessation medications less often to teen personas than to adult personas, by 32 to 67 percentage points, and in five of six models a persona presenting health- care expert
研究发现同一问题在六种 LLM 中因用户角色不同而获得不同答案，特别是在戒烟建议上。
- 核心方法
- 通过设计12种不同背景的用户角色，向6个广泛部署的LLM提供相同的4个戒烟问题，收集并分析2,880条回复。
- 适合谁读
- 研究者、健康科技从业者、政策制定者
- 要解决的问题
- 探讨大型语言模型在提供健康建议时是否会因用户的角色差异而产生不同的回答。
- 关键实验
- 对6个LLM进行审计，每个模型回答4个戒烟问题，每问题有12种不同的用户角色，共2,880条回复。
- 主要贡献
- 揭示了LLM在提供戒烟建议时对用户角色的敏感性，特别是对青少年和自称为医疗专家的用户。
- 意义与局限
- 研究表明LLM在提供健康建议时存在偏见，可能影响用户的治疗决策。未来研究需关注如何减少这种偏见。