ai.hackcv
论文精选 65arXiv

A Dual-Dimensional LLM Framework for Automated Item Incidental Content Similarity Analysis in Large-Scale Assessments· 大规模评估中的自动化题项偶然内容相似度分析框架

The rapid expansion of large-scale assessments and the growing adoption of automatic item generation have intensified concerns about incidental content redundancy, where construct-irrelevant elements such as wording or contextual framing become unintentionally repetitive across items. Traditional similarity metrics like BLEU or cosine similarity, often fail to capture the nuanced structural and semantic layers that drive perceived redundancy simultaneously. This study proposes a dual-dimensional framework for Automated Item Similarity Analysis (AISA) powered by Large Language Models (LLMs), operationalizing similarity through Structured Decomposition and Semantic Relatedness. Psychometric validation indicates that LLM-derived metrics align more closely with indicators of construct-irreleva

AI 解读论文

提出双维度LLM框架以解决大规模评估中的题项偶然内容相似度问题

核心方法
结合结构分解和语义关联两种方法,利用大语言模型构建双维度相似度分析框架
适合谁读
研究者、教育技术开发者、测试评估专家
要解决的问题
大规模评估中自动出题出现的题项结构与语义偶然相似导致的内容冗余问题
关键实验
进行了心理测量验证实验,结果显示LLM推导的度量标准与构念无关指标更紧密对齐
主要贡献
引入了更贴近人类感知的相似度度量标准,提高了大规模评估中题项冗余检测的准确性
意义与局限
提升了大规模评估的质量与效率,但方法依赖于高质量的大语言模型和数据集
领域:cs.AI作者:Jing Huang、Jihong Zhang、Hua-Hua Chang
相关推荐

本站内容由 LLM 精选聚合,原文版权归 arXiv 所有 · 摘录仅供参考