标题： 大型推理嵌入模型：迈向下一代密集检索范式 显示英文标题

标题： Large Reasoning Embedding Models: Towards Next-Generation Dense Retrieval Paradigm

摘要： 在现代电子商务搜索系统中，密集检索已成为不可或缺的组成部分。 通过计算查询与项目（产品）嵌入之间的相似性，它能够从大规模存储库中高效地选择候选产品。 随着大语言模型（LLMs）的突破，主流嵌入模型逐渐从BERT转向LLMs，以实现更精确的文本建模。 然而，这些模型仍然采用直接嵌入方法，嵌入的语义准确性仍显不足。 因此，对比学习被大量用于实现正对之间的紧密语义对齐。 结果，此类模型倾向于捕捉训练数据中的统计共现模式，使其偏向于浅层的词汇和语义匹配。 对于与目标项目存在明显词汇差异的困难查询，性能会显著下降。 在此工作中，我们提出了大型推理嵌入模型（LREM），该模型创新性地将推理过程融入表示学习中。 对于困难查询，LREM首先进行推理以深入理解原始查询，然后生成增强推理的查询嵌入用于检索。 这一推理过程有效弥合了原始查询与目标项目之间的语义差距，显著提高了检索准确性。 具体而言，我们采用两阶段训练过程：第一阶段通过SFT和InfoNCE损失在精心策划的查询-CoT-项目三元组上优化LLM，以建立初步的推理和嵌入能力，第二阶段通过强化学习（RL）进一步精炼推理轨迹。 大量的离线和在线实验验证了LREM的有效性，导致其自2025年8月起在中国最大的电子商务平台上部署。 显示英文摘要

摘要： In modern e-commerce search systems, dense retrieval has become an indispensable component. By computing similarities between query and item (product) embeddings, it efficiently selects candidate products from large-scale repositories. With the breakthroughs in large language models (LLMs), mainstream embedding models have gradually shifted from BERT to LLMs for more accurate text modeling. However, these models still adopt direct-embedding methods, and the semantic accuracy of embeddings remains inadequate. Therefore, contrastive learning is heavily employed to achieve tight semantic alignment between positive pairs. Consequently, such models tend to capture statistical co-occurrence patterns in the training data, biasing them toward shallow lexical and semantic matches. For difficult queries exhibiting notable lexical disparity from target items, the performance degrades significantly. In this work, we propose the Large Reasoning Embedding Model (LREM), which novelly integrates reasoning processes into representation learning. For difficult queries, LREM first conducts reasoning to achieve a deep understanding of the original query, and then produces a reasoning-augmented query embedding for retrieval. This reasoning process effectively bridges the semantic gap between original queries and target items, significantly improving retrieval accuracy. Specifically, we adopt a two-stage training process: the first stage optimizes the LLM on carefully curated Query-CoT-Item triplets with SFT and InfoNCE losses to establish preliminary reasoning and embedding capabilities, and the second stage further refines the reasoning trajectories via reinforcement learning (RL). Extensive offline and online experiments validate the effectiveness of LREM, leading to its deployment on China's largest e-commerce platform since August 2025.