AISPA: User-Centric System Prompt Auditing for Large Language Model Applications· AISPA：面向用户的大型语言模型系统提示审计
System prompts are instructions configured by developers to govern the behaviors of foundation models in AI applications. They are used throughout commercial AI products, but are rarely disclosed to the public or regulators, creating a serious trust and accountability gap in the wide deployment of AI systems. In this paper, we introduce Artificial Intelligence System Prompt Assurance (AISPA), a user-centric framework for systematically auditing system prompts in AI systems. AISPA examines specific parts of a system prompt and evaluates them along eight dimensions that matter to users. We then use this framework to review 3,249 instructions from system prompts in 88 commercial AI products, classifying each instruction as either protective (of users) or problematic. Our audit surfaces four c
面向用户的大型语言模型系统提示审计框架 AISPA 及其应用。
- 核心方法
- AISPA 框架通过八个维度评估系统提示，识别其对用户的保护性或潜在问题。
- 适合谁读
- 研究者、产品经理、政策制定者
- 要解决的问题
- 大型语言模型在商业应用中的系统提示缺乏透明度和监管，导致信任和责任缺失。
- 关键实验
- 审计了 88 款商业 AI 产品中的 3,249 条系统提示指令，未提供具体实验数据。
- 主要贡献
- 提出了 AISPA 框架，审计了 88 款商业 AI 产品的 3,249 条指令，并发现四大类问题。
- 意义与局限
- 增强了大型语言模型应用的透明度和责任感，为监管和优化提供了依据，但可能因主观判断而受限。