论文精选 75arXiv

QuranicMMLU: A Cognitively-Aware Benchmark for Evaluating Generative AI Solutions on Quranic Linguistic Knowledge· QuranicMMLU：评估生成式 AI 的古兰经语言知识

We introduce QuranicMMLU, a benchmark for evaluating generative AI on Quranic Arabic across multiple dimensions of linguistic complexity. Existing Quranic benchmarks center on general question answering and semantic retrieval, without probing specific linguistic competencies or stratifying by cognitive demand and verse difficulty. We construct a five-pillar Quranic taxonomy spanning Phonology, Morphology, Syntax, Semantics, and Pragmatics, with 31 leaves covering phenomena from tajwīd and root-and-pattern morphology to occasions of revelation and inter-surah coherence. For each leaf we generate questions stratified by Bloom's cognitive level and verse perplexity, then have LLM as a judge to independently answer and score every item and route the annotations to manual review. The resulting

领域：cs.CL作者：Rawan El Ghali、Umm Kulsoom、Anas Madkoor

#cs.CL

📎 发布于 09-19 01:30🔗 阅读原文（arxiv.org）📄 PDF

Low-Bit N-Gram Embeddings for Language Models· 大模型低精度 n-gram 嵌入

Many recent open-weight language models use n-gram embedding tables to improve performance…

领域：cs.CL作者：Stephen Chung、Wenyu Du

📎 arXiv🕒 10-23 01:56🔗 arxiv.org

论文精选 82

Can an Agent Debug a Web Corpus? Agentic Auditing of Slovak FineWeb2· LLM 能否调试网络语料库？

Multilingual pipelines such as FineWeb2 apply one recipe across a thousand languages, leav…

领域：cs.CL作者：Marek Šuppa、Ivan Vykopal、Jaroslav Kopčan

📎 arXiv🕒 10-23 01:52🔗 arxiv.org

论文精选 83

Bilingual Language Models Capture Cross-Linguistic Influence in Second-Language Reading· 双语模型捕捉二语阅读的跨语言影响

While monolingual language model (LM) surprisal is widely established as a predictor of re…

领域：cs.CL作者：Abderrahmane Issam、Irene E. Winther、Stefan L. Frank

📎 arXiv🕒 10-23 01:42🔗 arxiv.org

论文精选 82

How Ground Truth Shapes Retrieval Scores: Evidence-Aware Evaluation of a Retrieval-Augmented Generation System for Breast Lesions· 乳腺病变管理文献中的检索系统评估

Purpose: To describe an evidence-aware retrieval evaluation protocol for a clinical retrie…

领域：cs.CL作者：Miguel Mochizuki Silva、Samuel Ismael、Mateus C. Dias

📎 arXiv🕒 10-23 01:36🔗 arxiv.org

论文精选 80

Would an LLM Match into a Residency? Uncertainty, Reasoning and Image Use of Open Language Models on the Spanish MIR Examination· 大语言模型能通过医学考试吗？

Accuracy on medical examinations is common evidence offered for the clinical use of langua…

领域：cs.CL作者：Miguel Fernandez-de-Retana、Unai Zulaika、Ruben Sanchez-Corcuera

📎 arXiv🕒 10-23 01:23🔗 arxiv.org

本站内容由 LLM 精选聚合，原文版权归 arXiv 所有 · 摘录仅供参考