When Robots Mishear Us: Mapping the Safety Risks of Voice-Controlled Embodied AI· 当机器人听错我们：语音控制具身 AI 的安全风险
We investigate whether automatic speech recognition (ASR) errors in user input can lead to unsafe outputs from Embodied AI (EAI) models. We find that ASR errors can lead to harmful instructions being accepted and executed by EAI models, thereby reducing safety. We simulate ASR errors and combine them with existing safety benchmarks (SafeAgentBench and POEX) to evaluate how different errors affect embodied AI safety. We find that some of them preserve semantic structure but increase harmful ambiguity, while others weaken the model refusal behaviour and allow unsafe plans to be generated and executed. We show that in some cases automatic correction of ASR errors can reduce the risk, but this is not always effective. Overall, we show that ASR errors lead to significant safety risks for embodi
研究语音识别错误对具身AI模型安全性的影响。
- 核心方法
- 通过模拟ASR错误并结合现有的安全基准测试来评估不同类型的错误如何影响具身AI的安全性，包括语义结构的保持与有害模棱两可性的增加，以及模型拒绝行为的弱化。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 探讨自动语音识别（ASR）错误是否会导致具身AI模型执行不安全的输出。
- 关键实验
- 使用SafeAgentBench和POEX安全基准测试进行实验，模拟ASR错误并评估其影响。
- 主要贡献
- 首次系统地分析了ASR错误对具身AI安全性的潜在危害，并提出自动纠错在某些情况下可减少风险但非万能。
- 意义与局限
- 强调了语音控制AI系统中ASR错误的严重性，为提高此类系统的安全性提供了重要参考；但研究可能局限于特定的测试场景和模型。