Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbations· 语音与键盘输入对大模型的影响
Human input reaches language models by typing or speaking, and each channel leaves a distinct signature: orthographic noise for keyboards; for voice, disfluency from conventional transcription and restructuring from AI-backed dictation tools. How do they impact an LLM's performance? In this paper we present HIVE (Human Input-Variation Engine), a suite of voice transcription perturbations and QWERTY keyboard perturbations. We use HIVE to evaluate how robust models are to these perturbations. We present seven findings. (i) Voice transcription perturbations lower accuracy across every instruction-tuned model we test, and it is the structure of the transcription rather than its fillers that carries the cost. (ii) QWERTY keyboard perturbations cost less, and a model absorbs a lot of them before
研究语音与键盘输入对大模型性能影响的全面分析。
- 核心方法
- 开发了HIVE工具，包括语音转录扰动和QWERTY键盘扰动，用于评估模型对这些扰动的鲁棒性。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 探讨不同类型的语音和键盘输入扰动如何影响语言模型的性能。
- 关键实验
- 使用HIVE工具对多种指令调优模型进行了评估，展示了在不同扰动条件下的性能变化。
- 主要贡献
- 揭示了语音和键盘输入扰动对模型性能的具体影响，指出转录结构而非填充词是影响语音输入的主要因素。
- 意义与局限
- 该研究为理解用户输入方式如何影响语言模型性能提供了重要 insights，有助于改进模型的鲁棒性和用户体验。然而，研究仅限于特定的扰动类型，可能未涵盖所有实际应用场景。