Do "True" and "False" Add Up to One? Complement Consistency in LLM-Agent Opinion Simulations· 大模型代理意见模拟中的互补一致性
In LLM-agent opinion simulations, each agent reports the probability that a claim is true after seeing its own previous probability and the probabilities reported by its neighbors, and that number becomes its state in the next round. If the number describes the agent's position rather than the sentence it was shown, then rewriting the claim as its complement and replacing every probability p with 1 - p should return the complementary answer, so the two reports should sum to one. We test this with three small open models: Qwen3-4B, Llama-3.2-3B and Gemma-3-4B. On 2,304 agent states taken from a running network simulation, Qwen's paired reports sum to 1.18 on average, and on 48 real fact-checking claims from AVeriTeC they again exceed one. A controlled benchmark of 24 worlds, each with exact
探究大模型代理意见模拟中的互补一致性问题。
- 核心方法
- 通过对三个小规模开放模型（Qwen3-4B, Llama-3.2-3B 和 Gemma-3-4B）的实验，测试这些模型在命题与其补命题报告上的互补一致性。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 评估大模型在意见模拟中是否能保持命题与其补命题的报告概率之和为1的一致性。
- 关键实验
- 使用2,304个从网络模拟中提取的代理状态和48个来自AVeriTeC的真实事实核查命题进行测试。
- 主要贡献
- 发现Qwen3-4B在多数情况下未能保持互补一致性，平均报告概率之和超过1，揭示了大模型在逻辑一致性上的局限。
- 意义与局限
- 研究揭示了大模型在逻辑推理和意见表达中可能存在的偏差，对改进模型逻辑一致性和提高其在社会决策中的可靠性具有重要意义。