Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks· 无需云端的代理编码：评估开源大模型在数据准备任务中的表现
Large language models (LLMs) and agents are now widely used tools in code development, with data typically sent to third-party cloud-based models. Their adoption in research using personal data is constrained by governance requirements that typically prohibit data transmission to external services. Locally deployable open-weight models offer an alternative since sensitive data never leave the local environment. We introduce an open-source framework for evaluating the efficacy of AI agents powered by open-weight LLMs on one of the most persistent bottlenecks in research on longitudinal population studies: data preparation. The framework comprises: a curated ground-truth dataset (cleaning scripts preparing six sweeps of data from a British cohort study), task definitions encompassing tasks s
评估本地部署的开源大模型在处理纵向研究数据准备任务中的效果。
- 核心方法
- 引入一个开源框架，使用本地部署的开源大模型处理英国队列研究中六个阶段的数据清理脚本，评估模型在数据准备任务中的有效性和实用性。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 治理要求限制了个人数据传输到云端，导致研究人员无法充分利用大语言模型进行数据准备。
- 关键实验
- 使用六个阶段的英国队列研究数据，对比本地部署的开源大模型与云端服务的性能。
- 主要贡献
- 提供了一个评估开源大模型在数据准备任务中的框架，促进本地环境下敏感数据分析的采用。
- 意义与局限
- 意义在于解决了敏感数据的治理问题，推动了本地化 AI 技术的应用；局限性在于实验数据和任务范围有限，可能不适用于所有场景。