TIDE: Tree-Induced Fine-Grained Diagnostic Evaluation from Model Behaviors· TIDE: 从模型行为进行树诱导细粒度诊断评估
Benchmark performance is the dominant standard for comparing large language models (LLMs), typically summarized by a single aggregate score, such as accuracy, pass rate, or rubric score. This aggregate score provides limited information about a model's underlying capabilities, yet understanding these capabilities is crucial for model development and evaluation. To this end, we propose Tree-Induced Diagnostic Evaluation (TIDE), which recasts diagnostic evaluation as behavior-driven decision-tree induction to perform fine-grained capability diagnosis. TIDE recursively builds a hierarchical capability tree from benchmark samples and model scores: at each node, an LLM judge proposes candidate splits and performs sample-to-group classification, and behavior-based information gain selects among
从模型行为构建树状诊断评估,深入解析模型能力
- 核心方法
- 通过递归构建层次能力树,使用行为驱动的决策树归纳法进行模型能力的细粒度诊断
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 单一聚合分数无法全面反映大语言模型的真实能力
- 关键实验
- 未提供
- 主要贡献
- 提出TIDE框架,能够更细致地诊断和理解模型的内部能力
- 意义与局限
- 提高了模型评估的透明度和准确性,有助于模型的进一步优化与改进;但目前尚无实验数据支持