标题： 数据集研究：面向需求驱动的数据集发现的代理系统基准测试 显示英文标题

标题： DatasetResearch: Benchmarking Agent Systems for Demand-Driven Dataset Discovery

摘要： 大型语言模型的快速发展从根本上改变了人工智能发展的瓶颈，从计算能力转向数据可用性——无数有价值的数据集仍然隐藏在专业存储库、研究附录和领域平台中。 随着推理能力和深度研究方法的不断发展，一个关键问题浮现：人工智能代理能否超越传统的搜索，系统地发现符合特定用户需求的任何数据集，从而实现真正自主的需求驱动型数据整理？ 我们引入了DatasetResearch，这是首个全面的基准，用于评估人工智能代理从208个跨知识密集型和推理密集型任务的实际需求中发现和综合数据集的能力。 我们的三维评估框架揭示了一个严峻的现实：即使先进的深度研究系统在我们具有挑战性的DatasetResearch-pro子集上也只能获得22%的分数，这暴露了当前能力与完美数据集发现之间的巨大差距。 我们的分析揭示了一个根本性的二分法：搜索代理通过检索广度在知识任务中表现出色，而综合代理则通过结构化生成在推理挑战中占优——但两者在现有分布之外的“边缘情况”中都灾难性地失败。 这些发现建立了数据集发现代理的第一个严格基线，并指明了通往能够查找数字宇宙中任何数据集的人工智能系统的路径。 我们的基准和全面分析为下一代自我改进的人工智能系统奠定了基础，并可在 https://github.com/GAIR-NLP/DatasetResearch 公开获取。 显示英文摘要

摘要： The rapid advancement of large language models has fundamentally shifted the bottleneck in AI development from computational power to data availability-with countless valuable datasets remaining hidden across specialized repositories, research appendices, and domain platforms. As reasoning capabilities and deep research methodologies continue to evolve, a critical question emerges: can AI agents transcend conventional search to systematically discover any dataset that meets specific user requirements, enabling truly autonomous demand-driven data curation? We introduce DatasetResearch, the first comprehensive benchmark evaluating AI agents' ability to discover and synthesize datasets from 208 real-world demands across knowledge-intensive and reasoning-intensive tasks. Our tri-dimensional evaluation framework reveals a stark reality: even advanced deep research systems achieve only 22% score on our challenging DatasetResearch-pro subset, exposing the vast gap between current capabilities and perfect dataset discovery. Our analysis uncovers a fundamental dichotomy-search agents excel at knowledge tasks through retrieval breadth, while synthesis agents dominate reasoning challenges via structured generation-yet both catastrophically fail on "corner cases" outside existing distributions. These findings establish the first rigorous baseline for dataset discovery agents and illuminate the path toward AI systems capable of finding any dataset in the digital universe. Our benchmark and comprehensive analysis provide the foundation for the next generation of self-improving AI systems and are publicly available at https://github.com/GAIR-NLP/DatasetResearch.