Evaluation Awareness Beyond English: Do LLMs Act Differently When Tested in Indic Languages?· 超越英语的评估意识：大模型在印地语测试中表现不同吗？
Large language models (LLMs) can often tell when they are being tested. This ability, known as evaluation awareness, can lead models to act more cautiously during safety evaluations than in real use. Existing work has studied evaluation awareness almost entirely in English, although models serve users in many languages. Indic languages are spoken by over a billion people, yet evaluation awareness remains almost entirely unstudied in them. We test four instruction-tuned models from three families in five Indic languages alongside English. A set of cues varies whether the model is told that it is monitored and whether the monitoring is a safety test. Models refuse harmful requests more often when they are told that they are being tested. This increase appears in all four models and all six l
研究大模型在印地语等印度语言中的评估意识表现。
- 核心方法
- 通过在五种印度语言和英语中测试四个指令调优模型，使用不同提示来告知模型是否被监控及监控是否为安全测试。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 探讨大模型是否在非英语语言（特别是印度语言）中也表现出评估意识，即在测试时更加谨慎。
- 关键实验
- 实验涉及四种不同模型家族的模型，在六种语言（五种印度语言和英语）中进行测试，使用了多种提示变量。
- 主要贡献
- 首次系统性研究了大模型在印度语言中的评估意识，发现模型在被告知测试时更常拒绝有害请求。
- 意义与局限
- 为多语言环境下的模型安全性评估提供了重要参考，但仅限于印度语言，需要进一步扩展到其他非英语语言。