标题： 将测试时计算最优缩放推广为可优化的图 显示英文标题

标题： Generalizing Test-time Compute-optimal Scaling as an Optimizable Graph

摘要： 测试时缩放（TTS）通过在推理期间分配额外的计算来提高大型语言模型（LLMs），通常通过并行、顺序或混合缩放实现。 然而，先前的研究通常假设固定的协作架构（例如，拓扑结构）和单模型使用，忽略了最优架构和模型组合可能因任务而异。 因此，我们在固定预算下研究了TTS中搜索计算最优模型组合和架构的新问题。 我们将它形式化为一个多LLM协作图，其中节点编码角色和LLM模型分配，边捕捉信息流。 这个问题具有挑战性，因为（i）组合搜索空间过于庞大，且（ii）特定任务的要求需要定制设计。 为了解决这些问题，我们将问题重新表述为概率图优化，并通过初步实验得出了关于TTS协作图的三个经验见解。 在这些见解的指导下，我们提出了Agent-REINFORCE，这是一种LLM代理增强的框架，通过将采样-梯度更新映射到采样-反馈更新，模仿REINFORCE流程，其中反馈作为文本梯度来更新概率图，并高效地搜索最优多LLM协作图。 实验表明，Agent-REINFORCE在样本效率和搜索性能方面优于传统和基于LLM的基线，并能有效识别在准确性和推理延迟联合目标下的最优图。 显示英文摘要

摘要： Test-Time Scaling (TTS) improves large language models (LLMs) by allocating additional computation during inference, typically through parallel, sequential, or hybrid scaling. However, prior studies often assume fixed collaboration architectures (e.g., topologies) and single-model usage, overlooking that optimal architectures and model combinations can vary across tasks. Therefore, we study the novel problem of searching for compute-optimal model combinations and architectures in TTS under a fixed budget. We formalize it as a multi-LLM collaboration graph, where nodes encode roles and LLM model assignments, and edges capture information flow. This problem is challenging because (i) the combinatorial search space is prohibitively large, and (ii) task-specific requirements demand tailored designs. To address these, we reformulate the problem as probabilistic graph optimization and, through pilot experiments, derive three empirical insights into TTS collaboration graphs. Guided by these insights, we propose Agent-REINFORCE, an LLM-agent-augmented framework that mirrors the REINFORCE pipeline by mapping sampling-gradient-update to sampling-feedback-update, where feedback serves as a textual gradient to update the probabilistic graph and efficiently search for optimal multi-LLM collaboration graphs. Experiments show that Agent-REINFORCE outperforms both traditional and LLM-based baselines in sample efficiency and search performance, and effectively identifies optimal graphs under joint objectives of accuracy and inference latency.