ARASH: Adaptive Retrieval And Shot Selection for Tabular Prediction
Tabular prediction is a critical task across numerous applications. The recent success of large language models has sparked various approaches for adapting them to the tabular domain. A prevalent strategy involves training or fine-tuning specialized Tabular Foundation Models (TFMs) such as TabPFN. However, TFMs require substantial computational resources, and frequent model retraining is often impractical. In-context learning (ICL), specifically, few-shot prompting, offers a resource-efficient alternative to enhance performance. Yet, identifying the most relevant rows to serve as shots remains a challenge for tabular data. This paper introduces ARASH (Adaptive, query-specific Retrieval And Shot selection), a method that improves TFM efficiency by selecting optimal shots based on local neig
提出ARASH方法，通过自适应选择样本提升表格预测中大模型的效率。
- 核心方法
- ARASH方法结合了自适应查询特定检索和样本选择，通过局部邻域关系为每个查询选择最优的样本。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 表格预测领域存在的问题是使用大模型需要大量计算资源，而少样本提示中选择相关样本又非常困难。
- 关键实验
- 论文通过一系列表格预测任务实验验证了ARASH的有效性，实验结果表明该方法在多种数据集上均能提高预测精度。
- 主要贡献
- 1) 提出了一个新颖的自适应样本选择机制；2) 减少了大模型在表格预测任务上的资源需求；3) 显著提升了少样本提示的性能。
- 意义与局限
- ARASH为提升大模型在表格数据上的应用效率提供了有效方案，具有重要的实践意义。然而，该方法的适用性仍需在更多类型的数据集上进一步验证。