cs.CL [Total: 62]
cs.CV [Total: 74]
cs.SI [Total: 1]
cs.GR [Total: 2]
cs.AI [Total: 13]
cs.IR [Total: 4]
cs.LG [Total: 13]
cs.HC [Total: 1]
cs.RO [Total: 6]

cs.CL [Back]

[1] Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation

Mufei Li,Dongqi Fu,Limei Wang,Si Zhang,Hanqing Zeng,Kaan Sancak,Ruizhong Qiu,Haoyu Wang,Xiaoxin He,Xavier Bresson,Yinglong Xia,Chonglin Sun,Pan Li

Main category: cs.CL

TL;DR: 这篇论文提出了一种名为HaystackCraft的新型评估基准，用于测试长上下文LLM在真实场景（如异构检索和代理工作流）中的稳健性。实验表明，现有先进模型在这些动态和嘈杂的环境中仍存在显著问题。

Details

Motivation: 现有的长上下文LLM在合成的“大海捞针”（NIAH）测试中表现良好，但这些测试忽视了真实场景中的噪声上下文（如异构检索的偏见和代理工作流的级联错误）。需要更真实的评估方法来测试模型的稳健性。

Contribution: 1. 提出HaystackCraft，一个新的NIAH基准，基于英文维基百科超链接网络构建；2. 扩展NIAH测试到动态和代理场景；3. 揭示了高级模型在异构检索和代理工作流中的局限性。

Method: 基于英文维基百科构建HaystackCraft，测试异构检索策略（稀疏、稠密、混合和图检索）对LLM性能的影响，并模拟代理工作流（如查询优化、自我反思和停止决策）。

Result: 实验发现：1. 更强的稠密检索器可能引入更具干扰性的噪声，但图重排能提升检索效果；2. 高级模型（如Gemini 2.5 Pro和GPT-5）在代理测试中仍会出现级联错误或难以提前停止。

Insight: 长上下文LLM在真实环境（如代理工作流）中的表现仍有显著挑战，HaystackCraft为未来研究提供了有价值的测试平台。

Abstract: Modern long-context large language models (LLMs) perform well on synthetic “needle-in-a-haystack” (NIAH) benchmarks, but such tests overlook how noisy contexts arise from biased retrieval and agentic workflows. We argue that haystack engineering is necessary to construct noisy long contexts that faithfully capture key real-world factors – distraction from heterogeneous biased retrievers and cascading errors in agentic workflows – to test models’ long-context robustness. We instantiate it through HaystackCraft, a new NIAH benchmark built on the full English Wikipedia hyperlink network with multi-hop questions. HaystackCraft evaluates how heterogeneous retrieval strategies (e.g., sparse, dense, hybrid, and graph-based) affect distractor composition, haystack ordering, and downstream LLM performance. HaystackCraft further extends NIAH to dynamic, LLM-dependent settings that simulate agentic operations, where models refine queries, reflect on their past reasonings, and decide when to stop. Experiments with 15 long-context models show that (1) while stronger dense retrievers can introduce more challenging distractors, graph-based reranking simultaneously improves retrieval effectiveness and mitigates more harmful distractors; (2) in agentic tests, even advanced models like Gemini 2.5 Pro and GPT-5 suffer cascading failures from self-generated distractors or struggle to perform early stops. These results highlight persistent challenges in agentic long-context reasoning and establish HaystackCraft as a valuable testbed for future progress.

[2] LASER: An LLM-based ASR Scoring and Evaluation Rubric

Amruta Parulekar,Preethi Jyothi

Main category: cs.CL

TL;DR: 本文提出了一种基于大语言模型（LLM）的ASR评估方法LASER，通过上下文学习能力更公平地评分，避免传统指标（如WER）对语义无害的语言细微差异的过度惩罚。

Details

Motivation: 传统ASR评估指标（如WER）对语法和形态变化的惩罚过于严苛，而这些变化往往不影响语义。需要一种更智能的评分方法。

Contribution: 1. 提出LASER，一种基于LLM的ASR评估框架；2. 展示其在高资源（Gemini 2.5 Pro）和低资源（Llama 3）模型上的有效性；3. 在印度多语言环境中验证其泛化能力。

Method: 1. 使用LLM的上下文学习能力设计评分提示；2. 通过精细标注的示例训练模型；3. 微调小模型（Llama 3）以预测惩罚类型。

Result: 1. Gemini 2.5 Pro的LASER评分与人工标注相关性高达94%；2. Llama 3微调后惩罚预测准确率达89%；3. 印度多语言场景下表现良好。

Insight: 1. LLM的上下文学习能力可用于解决传统评估指标的局限性；2. 即使是较小模型，通过微调也能在特定任务上表现优异；3. 多语言评估中提示设计的重要性。

Abstract: Standard ASR evaluation metrics like Word Error Rate (WER) tend to unfairly penalize morphological and syntactic nuances that do not significantly alter sentence semantics. We introduce an LLM-based scoring rubric LASER that leverages state-of-the-art LLMs’ in-context learning abilities to learn from prompts with detailed examples. Hindi LASER scores using Gemini 2.5 Pro achieved a very high correlation score of 94% with human annotations. Hindi examples in the prompt were also effective in analyzing errors in other Indian languages such as Marathi, Kannada and Malayalam. We also demonstrate how a smaller LLM like Llama 3 can be finetuned on word-pair examples derived from reference and ASR predictions to predict what kind of penalty should be applied with close to 89% accuracy.

[3] Meaningful Pose-Based Sign Language Evaluation

Zifan Jiang,Colin Leong,Amit Moryossef,Anne Göhring,Annette Rios,Oliver Cory,Maksym Ivashechkin,Neha Tarigopula,Biao Zhang,Rico Sennrich,Sarah Ebling

Main category: cs.CL

TL;DR: 这篇论文提出了一种基于人体骨骼姿势的手语评估方法，研究了关键点距离、嵌入和回译三种指标，并通过实验展示了不同场景下的性能权衡。

Details

Motivation: 传统的手语评估方法缺乏标准化和可重复性，因此需要一种系统性、可量化的评估框架来改进手语翻译和生成系统的开发。

Contribution: 论文的主要贡献包括：(1)提出了基于姿势的手语评估指标，(2)通过实验分析了不同指标的优缺点，(3)开源了姿势评估工具包。

Method: 研究了三种评估指标：(1)关键点距离，(2)嵌入空间相似性，(3)回译质量。通过自动元评估和人工相关性研究验证了指标的有效性。

Result: 实验结果表明不同指标在不同场景下各有优劣，例如关键点距离适用于局部精度，而嵌入方法更适合语义层面的评估。

Insight: 研究揭示了手语评估需要根据任务需求选择合适的指标组合，为后续研究和应用提供了实用指导。

Abstract: We present a comprehensive study on meaningfully evaluating sign language utterances in the form of human skeletal poses. The study covers keypoint distance-based, embedding-based, and back-translation-based metrics. We show tradeoffs between different metrics in different scenarios through automatic meta-evaluation of sign-level retrieval and a human correlation study of text-to-pose translation across different sign languages. Our findings and the open-source pose-evaluation toolkit provide a practical and reproducible way of developing and evaluating sign language translation or generation systems.

[4] Populism Meets AI: Advancing Populism Research with LLMs

Eduardo Ryô Tamaki,Yujin J. Jung,Julia Chatterley,Grant Mitchell,Semir Dzebo,Cristóbal Sandoval,Levente Littvay,Kirk A. Hawkins

Main category: cs.CL

TL;DR: 该论文提出了一种基于LLM的链式思维提示方法，用于高效、准确地测量民粹主义的思想内容，其分类精度可与人类专家相媲美。

Details

Motivation: 传统基于文本分析的民粹主义测量方法成本高、耗时长且难以扩展，迫切需要一种高效、可扩展的新方法。

Contribution: 论文的主要贡献是提出了一种基于LLM的领域特定提示策略，能够复现人类编码员的训练过程，从而实现对民粹主义的高精度分类。

Method: 采用了链式思维提示方法（CoT prompting），利用全球民粹主义数据库（GPD）的标注数据，通过模拟人类编码员的训练过程来指导LLM的推理。

Result: 实验结果表明，该方法在民粹主义分类上的准确性达到了人类专家的水平。

Insight: LLMs在复杂、上下文敏感的领域中表现优异，通过领域特定的提示策略可以显著提升其性能。

Abstract: Measuring the ideational content of populism remains a challenge. Traditional strategies based on textual analysis have been critical for building the field’s foundations and providing a valid, objective indicator of populist framing. Yet these approaches are costly, time consuming, and difficult to scale across languages, contexts, and large corpora. Here we present the results from a rubric and anchor guided chain of thought (CoT) prompting approach that mirrors human coder training. By leveraging the Global Populism Database (GPD), a comprehensive dataset of global leaders’ speeches annotated for degrees of populism, we replicate the process used to train human coders by prompting the LLM with an adapted version of the same documentation to guide the model’s reasoning. We then test multiple proprietary and open weight models by replicating scores in the GPD. Our findings reveal that this domain specific prompting strategy enables the LLM to achieve classification accuracy on par with expert human coders, demonstrating its ability to navigate the nuanced, context sensitive aspects of populism.

[5] MAPRO: Recasting Multi-Agent Prompt Optimization as Maximum a Posteriori Inference

Zheyuan Zhang,Lin Ge,Hongjiang Li,Weicheng Zhu,Chuxu Zhang,Yanfang Ye

Main category: cs.CL

TL;DR: MAPRO是一个多智能体提示优化框架，将多智能体系统的提示优化问题重新表述为最大后验推理问题，并通过语言引导的max-product信念传播算法迭代优化提示策略，最终实现优于人工设计和自动替代方案的性能。

Details

Motivation: 尽管多智能体系统（MAS）在协调专业角色方面展现出优于单智能体的潜力，但其设计仍受限于提示敏感性和不稳定性。当前的多智能体提示优化缺乏系统性方法，面临搜索空间指数级扩展和模糊信用分配等挑战。

Contribution: 提出MAPRO框架，将多智能体提示优化问题重新表述为最大后验（MAP）推理问题，并通过拓扑感知的迭代优化机制解决信用分配问题，最终生成协调的提示策略。

Method: MAPRO分为四个阶段：1）问题建模为MAP推理；2）使用语言引导的max-product信念传播算法求解；3）引入拓扑感知的细化机制，整合执行反馈和下游责任；4）迭代更新提示策略直至收敛。

Result: 在多个任务的基准测试中，MAPRO实现了最先进的性能，优于人工设计的基线和近期自动优化方法。

Insight: MAPRO的最大后验推理框架为构建更可靠、更原则性的多智能体系统提供了一般性指导。

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across diverse tasks, and LLM-based agents further extend these abilities to various practical workflows. While recent progress shows that multi-agent systems (MAS) can outperform single agents by coordinating specialized roles, designing effective MAS remains difficult due to prompt sensitivity and the compounded instability MAS creates. To cope with the challenge, recent efforts in automated prompt design have reduced manual effort. However, multi-agent prompt optimization remains largely unexplored. Challenges like exponentially expanding search space and ambiguous credit assignment together make systematic design intractable without principled methods. Therefore, we introduce M}ulti-Agent PRompt Optimization (MAPRO), a four-stage framework that first formulates MAS prompt optimization as a Maximum a Posteriori (MAP) inference problem and solves it using a language-guided variant of max-product belief propagation algorithm. To address credit assignment and updates the system iteratively, MAPRO employs a topology-aware refinement mechanism that integrates execution feedback and downstream blames to selectively update agent prompts. Through this process, MAPRO progressively converges to a coordinated set of agent-specific prompt policies. Across benchmarks in various tasks, MAPRO achieves state-of-the-art performance, consistently surpassing manually engineered baselines and recent automated alternatives. Beyond performance, our MAP-based formulation also delivers general guidelines for building more reliable and principled multi-agent systems in the future

[6] AsyncSpade: Efficient Test-Time Scaling with Asynchronous Sparse Decoding

Shuqing Luo,Yilin Guan,Pingzhi Li,Hanrui Wang,Tianlong Chen

Main category: cs.CL

TL;DR: AsyncSpade提出了一种异步稀疏解码框架，显著提升了大规模语言模型（LLM）在测试时扩展（TTS）任务中的效率，减少了内存瓶颈并优化了推理时间。

Details

Motivation: 现有方法在处理长链推理（CoT）时，由于KV缓存的线性增长导致内存瓶颈，且稀疏解码受限于顺序依赖和粗粒度令牌选择，影响了高并发场景下的效率。

Contribution: 1. 提出了AsyncSpade框架，首次消除了顺序依赖；2. 引入了轻量级时序回归模块预测查询状态；3. 通过异步化设计实现了KV缓存操作与推理计算的完全重叠。

Method: 1. 使用短窗口内的最近查询近似当前查询状态；2. 设计异步解耦框架，将KV缓存筛选与自回归解码循环分离；3. 通过时序回归模块预测下一令牌的查询状态。

Result: 在A100节点上，AsyncSpade实现了理论最优的每令牌时间（TPOT），相比SoTA基线（Quest）减少20%以上，比全注意力减少至少50%，同时在多个TTS基准测试中保持或超越模型精度。

Insight: 通过异步化和轻量级预测模块的设计，可以在不牺牲性能的前提下显著提升LLM在长链推理和高并发场景下的效率。

Abstract: Test-time scaling (TTS) boosts LLM reasoning via long chain-of-thought (CoT), but the linear KV-cache growth amplifies the memory-bound bottleneck of LLM decoding. Query-aware page-level sparse decoding can achieve state-of-the-art performance under constrained FLOPs budgets, but is limited by both sequential-dependent page filtering and coarse-grained token selection, hampering serving efficiency and model performance on TTS tasks under high concurrency and long CoT scenarios (consuming even higher runtime than the forward pipeline itself). In this paper, we first find that the current-step query state can be accurately approximated in a unified manner from a short window of recent queries, enabling training-free query-aware sparsity without waiting in the decoding loop. We propose AsyncSpade, an asynchronous framework for efficient TTS built on two core components: (1) a novel light-weight temporal-regressive module that predicts the next-token query state; (2) an asynchronous and disaggregated framework that decouples the KV cache filtering from the auto-regressive decoding loop, overlapping the token-level KV selection with the forward inference computation through asynchronism. To our knowledge, AsyncSpade is the first to eliminate the sequential dependence without sacrificing model performance. We validate the effectiveness of AsyncSpade on common LLM serving setups with an A100 node, where AsyncSpade fully overlaps KV-cache operations with the inference pipeline, achieving theoretical optimal time-per-output-token (TPOT). Specifically, AsyncSpade delivers over 20% reduction on TPOT compared to SoTA baseline (i.e. Quest) and at least 50% TPOT reduction compared to full attention on Qwen3-8B and Qwen3-32B models, while matching or surpassing their accuracy on various TTS benchmarks (AIME-24/25, GPQA-Diamond, MATH-500).

[7] Can Lessons From Human Teams Be Applied to Multi-Agent Systems? The Role of Structure, Diversity, and Interaction Dynamics

Rasika Muralidharan,Jaewoon Kwak,Jisun An

Main category: cs.CL

TL;DR: 论文探讨了从人类团队学到的经验是否适用于多智能体系统，研究了结构、多样性和互动动态对多智能体团队性能的影响。

Details

Motivation: 虽然大型语言模型驱动的多智能体系统（MAS）受到关注，但关于其团队动态的研究较少。作者从人类团队科学的视角出发，填补了这一研究空白。

Contribution: 提出了一个多智能体框架，用于研究团队科学的三个核心方面：结构、多样性和互动动态，并在多项任务中评估了团队性能。

Method: 设计了包括CommonsenseQA、StrategyQA、Social IQa和Latent Implicit Hate在内的四个任务，评估了扁平结构和层次结构团队的性能，并分析了多样性和互动动态的影响。

Result: 研究发现扁平团队性能优于层次团队，多样性的影响较为复杂。智能体对团队性能过度自信，但也表现出对协作的认可和整合中的挑战。

Insight: 研究揭示了多智能体团队在学习人类团队科学时的潜力与局限性，为未来设计更高效的智能体团队提供了启示。

Abstract: Multi-Agent Systems (MAS) with Large Language Model (LLM)-powered agents are gaining attention, yet fewer studies explore their team dynamics. Inspired by human team science, we propose a multi-agent framework to examine core aspects of team science: structure, diversity, and interaction dynamics. We evaluate team performance across four tasks: CommonsenseQA, StrategyQA, Social IQa, and Latent Implicit Hate, spanning commonsense and social reasoning. Our results show that flat teams tend to perform better than hierarchical ones, while diversity has a nuanced impact. Interviews suggest agents are overconfident about their team performance, yet post-task reflections reveal both appreciation for collaboration and challenges in integration, including limited conversational coordination.

[8] Can Speech LLMs Think while Listening?

Yi-Jen Shih,Desh Raj,Chunyang Wu,Wei Zhou,SK Bong,Yashesh Gaur,Jay Mahadeokar,Ozlem Kalinli,Mike Seltzer

Main category: cs.CL

TL;DR: 研究了语音大模型（speech LLMs）在推理任务中的表现，发现通过chain-of-thought（CoT）微调和多流推理方法能显著提升准确性和降低延迟。

Details

Motivation: 语音大模型在复杂推理任务中表现不佳，传统的链式思维（CoT）方法在文本大模型中有效，但其在语音空间的应用尚未充分研究。

Contribution: 1. CoT微调显著提升语音LLMs的推理准确性；2. 提出‘边听边想’方法降低延迟；3. 引入‘问题完整性’指标优化推理时机；4. 使用DPO进一步优化准确性和延迟关系。

Method: 1. 通过CoT微调提升推理能力；2. 提出基于熵的‘问题完整性’指标；3. 采用DPO优化延迟和准确性。

Result: CoT微调使准确性提升2.4倍；‘边听边想’方法在同等延迟下提升4%准确性；DPO实现延迟降低70%且无准确性损失。

Insight: 在语音LLMs中引入文本空间的推理方法有效，同时实时动态推理可平衡准确性与延迟。

Abstract: Recent advances in speech large language models (speech LLMs) have enabled seamless spoken interactions, but these systems still struggle with complex reasoning tasks. Previously, chain-of-thought (CoT) prompting or fine-tuning has been to shown to significantly improve the reasoning abilities of text-based LLMs. In this work, we investigate the effect of CoT fine-tuning for multi-stream speech LLMs, demonstrating that reasoning in text space improves the accuracy of speech LLMs by 2.4x, on average, over a suite of spoken reasoning tasks. Beyond accuracy, the latency of the spoken response is a crucial factor for interacting with voice-based agents. Inspired by the human behavior of “thinking while listening,” we propose methods to reduce the additional latency from reasoning by allowing the model to start reasoning before the user query has ended. To achieve this, we introduce an entropy-based metric, “question completeness,” which acts as an indicator to guide the model on the optimal time to start reasoning. This method provides greater control over the accuracy-latency trade-off compared with heuristic-based approaches and, under equivalent latency conditions, yields a 4% accuracy gain on ARC-Easy. Finally, we use Direct Preference Optimization (DPO) on preference data created using rejection sampling to push the accuracy-latency pareto frontier further, resulting in a 70% reduction in latency without loss in accuracy.

[9] When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMs

Soyeong Jeong,Taehee Jung,Sung Ju Hwang,Joo-Kyung Kim,Dongyeop Kang

Main category: cs.CL

TL;DR: 该论文提出了“思想模板”（thought templates）方法，通过复用先前的推理痕迹来增强长上下文语言模型（LCLMs）的多跳推理能力，从而更好地整合证据。通过迭代更新策略优化模板，并在多种LCLM模型和基准测试中展示了显著的性能提升。

Details

Motivation: 尽管长上下文语言模型（LCLMs）能够处理大规模上下文信息，但在多跳推理任务中，简单地增加文档数量并不能有效捕获证据之间的关联性。因此，如何结构化地整合证据并实现高效的推理成为关键挑战。

Contribution: 1. 提出“思想模板”方法，将推理过程转化为可复用的思想缓存，指导证据的整合和多跳推理；2. 设计了一种迭代更新策略，通过自然语言反馈优化模板；3. 在多种LCLM家族和任务中验证了方法的有效性，并展示了对小型开源模型的蒸馏能力。

Method: 1. 利用先前的解决问题痕迹生成“思想模板”，作为结构化推理的指导；2. 通过自然语言反馈迭代更新模板，确保其有效性；3. 将优化后的模板应用于多跳推理任务，支持检索和非检索两种场景。

Result: 在多样化的基准测试和LCLM模型中，该方法显著优于基线模型。同时，优化后的模板能够蒸馏到小型开源模型中，验证了其广泛适用性。

Insight: 1. 结构化推理痕迹的复用可以显著提升多跳推理的效果；2. 自然语言反馈是优化推理模板的有效方式；3. 该方法揭示了长上下文模型中如何更好地整合和利用证据的潜在方向。

Abstract: Recent Long-Context Language Models (LCLMs) can process hundreds of thousands of tokens in a single prompt, enabling new opportunities for knowledge-intensive multi-hop reasoning by integrating large sets of retrieved documents or, in some cases, directly all necessary information. However, simply feeding more documents into the context window fails to capture how evidence should be connected. We address this gap with thought templates, which recast reasoning as reusable thought caches, derived from prior problem solving traces, structuring how evidence is combined and guiding multi-hop inference with factual documents. To keep these templates effective, we propose an update strategy that iteratively refines templates derived from training data through natural-language feedback. Across diverse benchmarks and LCLM families, our approach delivers consistent gains over strong baselines in both retrieval-based and retrieval-free settings. Furthermore, we show that optimized templates can be distilled into smaller open-source models, demonstrating its broad applicability and transparent reasoning reuse. We refer to our framework as Thought Template Augmented LCLMs (ToTAL).

[10] ParsTranslit: Truly Versatile Tajik-Farsi Transliteration

Rayyan Merchant,Kevin Tang

Main category: cs.CL

TL;DR: 该论文提出了一种新的序列到序列模型（ParsTranslit），用于塔吉克语-波斯语的音译，解决了脚本差异导致的沟通障碍问题。模型在多个数据集上训练，表现优异，并在不同领域中提供了清晰的任务理解。

Details

Motivation: 波斯语在阿富汗、伊朗和塔吉克斯坦使用不同的书写标准（Perso-Arabic和Tajik-Cyrillic），脚本差异阻碍了书面交流。现有模型局限于特定领域的数据集，缺乏实用性。

Contribution: 1. 提出了一种新的SOTA序列到序列模型；2. 提供了两个新数据集；3. 在多个领域中明确了任务的真实难度；4. 模型在chrF++和Normalized CER指标上表现优异。

Method: 使用了序列到序列模型（Seq2Seq），在多个公开数据集上训练，包括新创建的两个数据集，提升了模型的泛化能力和实用性。

Result: 模型在Farsi到Tajik的音译中chrF++和Normalized CER分别为87.91和0.05，Tajik到Farsi中分别为92.28和0.04。

Insight: 跨域训练和多数据集的使用显著提升了模型的泛化能力，表明解决脚本差异问题需要更全面的数据支持。

Abstract: As a digraphic language, the Persian language utilizes two written standards: Perso-Arabic in Afghanistan and Iran, and Tajik-Cyrillic in Tajikistan. Despite the significant similarity between the dialects of each country, script differences prevent simple one-to-one mapping, hindering written communication and interaction between Tajikistan and its Persian-speaking ``siblings’’. To overcome this, previously-published efforts have investigated machine transliteration models to convert between the two scripts. Unfortunately, most efforts did not use datasets other than those they created, limiting these models to certain domains of text such as archaic poetry or word lists. A truly usable transliteration system must be capable of handling varied domains, meaning that suck models lack the versatility required for real-world usage. The contrast in domain between data also obscures the task’s true difficulty. We present a new state-of-the-art sequence-to-sequence model for Tajik-Farsi transliteration trained across all available datasets, and present two datasets of our own. Our results across domains provide clearer understanding of the task, and set comprehensive comparable leading benchmarks. Overall, our model achieves chrF++ and Normalized CER scores of 87.91 and 0.05 from Farsi to Tajik and 92.28 and 0.04 from Tajik to Farsi. Our model, data, and code are available at https://anonymous.4open.science/r/ParsTranslit-FB30/.

[11] Deploying Tiny LVLM Judges for Real-World Evaluation of Chart Models: Lessons Learned and Best Practices

Md Tahmid Rahman Laskar,Mohammed Saidul Islam,Ridwan Mahbub,Mizanur Rahman,Amran Bhuiyan,Israt Jahan,Mir Tafseer Nayeem,Shafiq Joty,Enamul Hoque,Jimmy Huang

Main category: cs.CL

TL;DR: 论文提出两种方法（多标准提示和领域自适应迁移学习）以解决小参数量LVLM（<=2B）在图表理解任务中表现不佳的问题，并通过实验验证了方法的有效性。

Details

Motivation: 现有的大型视觉语言模型（LVLM, 7B参数）在图表理解任务中表现良好，但小参数量模型（<=2B）表现较差，限制了其在资源受限场景下的应用。因此，需要开发低成本、高效的评估方法。

Contribution: 1. 提出多标准提示方法，将多个评估标准整合为单一查询，暴露了7B模型的鲁棒性缺陷。2. 提出领域自适应迁移学习方法，通过在合成数据上微调2B参数的LVLM（ChartJudge），实现了跨数据集的知识迁移。

Method: 1. 多标准提示：结合多个评估标准生成单一提示，用于评估模型的鲁棒性。2. 领域自适应迁移学习：在合成的图表数据集上微调2B参数的LVLM，提升其在特定领域的性能。

Result: 多标准提示揭示了7B模型的性能下降，包括专门的LVLM评估模型（如LLaVA-Critic）。ChartJudge能够有效迁移知识，成为更专业的模型。实验还提供了关于模型大小、提示设计和迁移性之间权衡的见解。

Insight: 1. 小参数量模型通过领域自适应迁移学习可以显著提升性能。2. 多标准提示是一种有效的评估方法，可以暴露模型的鲁棒性问题。3. 模型大小、提示设计和迁移性是影响低成本评估的关键因素。

Abstract: Large Vision-Language Models (LVLMs) with only 7B parameters have shown promise as automated judges in chart comprehension tasks. However, tiny models (<=2B parameters) still perform poorly as judges, limiting their real-world use in resource-constrained settings. To address this, we propose two approaches to ensure cost-efficient evaluation: (i) multi-criteria prompting, which combines separate evaluation criteria into a single query, and (ii) domain-adaptive transfer learning, in which we fine-tune a 2B-parameter LVLM on synthetic judgments in a chart dataset to create the ChartJudge. Experiments show that multi-criteria prompting exposes robustness gaps, which led to a huge drop in performance for 7B models, including specialized LVLM judges like LLaVA-Critic. In addition, we find that our tiny LVLM (ChartJudge) can effectively transfer knowledge from one dataset to another to make it a more specialized model. Our fine-grained analysis across chart types and query complexities offers actionable insights into trade-offs between model size, prompt design, and transferability, enabling scalable, low-cost evaluation for chart reasoning tasks. Our code and the data will be made publicly available.

[12] Multi-Task Pre-Finetuning of Lightweight Transformer Encoders for Text Classification and NER

Junyi Zhu,Savas Ozkan,Andrea Maracani,Sinan Mutlu,Cho Jung Min,Mete Ozay

Main category: cs.CL

TL;DR: 论文研究了如何通过多任务预微调策略提升轻量级BERT-like编码器的适应性，针对NER和文本分类任务，避免了传统多任务学习的冲突信号问题，提出了一种基于任务主LoRA模块的有效框架。

Details

Motivation: 在移动平台上部署NLP模型需要高效且适应性强的方法，但目前多任务预微调会在不同任务间产生冲突信号，影响性能。

Contribution: 提出了一个基于任务主LoRA模块的多任务预微调框架，解决了传统多任务学习的冲突问题，同时保持了高效部署的约束。

Method: 使用共享编码器骨干和模块化适配器（LoRA模块），优化多任务预微调信号，避免任务间的冲突。

Result: 在21个下游任务上，NER平均提升0.8%，文本分类提升8.8%。

Insight: 模块化的适配器设计可以有效平衡多任务学习中的冲突，为轻量级模型在移动端的应用提供了新思路。

Abstract: Deploying natural language processing (NLP) models on mobile platforms requires models that can adapt across diverse applications while remaining efficient in memory and computation. We investigate pre-finetuning strategies to enhance the adaptability of lightweight BERT-like encoders for two fundamental NLP task families: named entity recognition (NER) and text classification. While pre-finetuning improves downstream performance for each task family individually, we find that na"ive multi-task pre-finetuning introduces conflicting optimization signals that degrade overall performance. To address this, we propose a simple yet effective multi-task pre-finetuning framework based on task-primary LoRA modules, which enables a single shared encoder backbone with modular adapters. Our approach achieves performance comparable to individual pre-finetuning while meeting practical deployment constraint. Experiments on 21 downstream tasks show average improvements of +0.8% for NER and +8.8% for text classification, demonstrating the effectiveness of our method for versatile mobile NLP applications.

Mkululi Sikosana,Sean Maudsley-Barton,Oluwaseun Ajao

Main category: cs.CL

TL;DR: 这篇论文通过计算语言学方法分析了新冠疫情和猴痘相关内容的语言模式，揭示了健康假信息与真实信息的区别。研究发现假信息的可读性较低，且更多使用恐惧和说服性语言，而猴痘内容更具情感表达。

Details

Motivation: 研究旨在揭示健康假信息的语言特征，以帮助检测假信息和优化公共卫生信息传播策略。

Contribution: 论文的主要贡献是通过对比分析发现了假信息在可读性、修辞标记和说服性语言上的独特模式，为假信息检测提供了语言学依据。

Method: 研究使用三个数据集（新冠假信息、新冠一般内容和猴痘内容），比较了它们的可读性、修辞标记和说服性语言的使用频率。

Result: 新冠假信息的可读性显著较低，恐惧和说服性语言的频率是其他数据集的两倍以上，且较少使用感叹号。猴痘内容更具情感表达。

Insight: 假信息可能通过复杂的修辞风格和情感暗示提高可信度。未来研究应扩大情感词典并采用动态分析方法。

Abstract: This study conducts a computational linguistic analysis of pandemic-related online discourse to examine how language distinguishes health misinformation from factual communication. Drawing on three corpora: COVID-19 false narratives (n = 7588), general COVID-19 content (n = 10700), and Monkeypox-related posts (n = 5787), we identify significant differences in readability, rhetorical markers, and persuasive language use. COVID-19 misinformation exhibited markedly lower readability scores and contained over twice the frequency of fear-related or persuasive terms compared to the other datasets. It also showed minimal use of exclamation marks, contrasting with the more emotive style of Monkeypox content. These patterns suggest that misinformation employs a deliberately complex rhetorical style embedded with emotional cues, a combination that may enhance its perceived credibility. Our findings contribute to the growing body of work on digital health misinformation by highlighting linguistic indicators that may aid detection efforts. They also inform public health messaging strategies and theoretical models of crisis communication in networked media environments. At the same time, the study acknowledges limitations, including reliance on traditional readability indices, use of a deliberately narrow persuasive lexicon, and reliance on static aggregate analysis. Future research should therefore incorporate longitudinal designs, broader emotion lexicons, and platform-sensitive approaches to strengthen robustness.

[14] Role-Conditioned Refusals: Evaluating Access Control Reasoning in Large Language Models

Đorđe Klisura,Joseph Khoury,Ashish Kundu,Ram Krishnan,Anthony Rios

Main category: cs.CL

TL;DR: 论文研究了大型语言模型在访问控制中的表现，提出了角色条件下的拒绝机制，并通过新数据集和三种方法（零/少样本提示、生成-验证两步框架和LoRA微调）比较了模型的表现。

Details

Motivation: 访问控制是安全计算的核心，但大型语言模型通常忽略了角色的边界，导致响应不受限制。作者研究了如何让模型更好地遵循访问控制策略。

Contribution: 1) 提出了角色条件下的拒绝机制；2) 创建了一个扩展Spider和BIRD的数据集，加入了基于角色的策略；3) 比较了三种方法的性能。

Method: 1) 零/少样本提示；2) 生成-验证两步框架；3) LoRA微调模型。

Result: 显式验证（两步框架）提高了拒绝精确度并降低了错误允许率，而微调在安全性和实用性之间取得了更好的平衡。更长、更复杂的策略降低了所有系统的可靠性。

Insight: 1) 显式验证优于直接生成；2) 微调可以在安全性和实用性之间找到平衡；3) 复杂策略对模型表现有负面影响。

Abstract: Access control is a cornerstone of secure computing, yet large language models often blur role boundaries by producing unrestricted responses. We study role-conditioned refusals, focusing on the LLM’s ability to adhere to access control policies by answering when authorized and refusing when not. To evaluate this behavior, we created a novel dataset that extends the Spider and BIRD text-to-SQL datasets, both of which have been modified with realistic PostgreSQL role-based policies at the table and column levels. We compare three designs: (i) zero or few-shot prompting, (ii) a two-step generator-verifier pipeline that checks SQL against policy, and (iii) LoRA fine-tuned models that learn permission awareness directly. Across multiple model families, explicit verification (the two-step framework) improves refusal precision and lowers false permits. At the same time, fine-tuning achieves a stronger balance between safety and utility (i.e., when considering execution accuracy). Longer and more complex policies consistently reduce the reliability of all systems. We release RBAC-augmented datasets and code.

[15] Causality Guided Representation Learning for Cross-Style Hate Speech Detection

Chengshuai Zhao,Shu Wan,Paras Sheth,Karan Patwa,K. Selçuk Candan,Huan Liu

Main category: cs.CL

TL;DR: 本文提出了CADET，一种基于因果图的表征学习框架，用于解决跨风格仇恨语音检测的挑战。通过解耦仇恨语音的可解释潜在因素并控制混杂变量，CADET能够从表面语言线索中分离出真实的仇恨意图。

Details

Motivation: 当前仇恨语音检测模型依赖表面语言特征，难以泛化到不同风格的变化，并且不同平台的仇恨语音可能引入虚假相关性。本文通过因果图建模仇恨语音的生成过程，提出了一种更鲁棒的检测方法。

Contribution: 1）提出了一个建模仇恨语音生成的因果图；2）设计了CADET框架，通过解耦潜在因素和控制混杂变量，提升了跨风格仇恨语音检测的泛化能力；3）支持通过反事实推理增强模型的鲁棒性。

Method: CADET基于因果图建模仇恨语音生成的关键因素（上下文环境、创作者动机、目标和风格）。它采用表征学习方法解耦这些因素，并通过控制混杂变量来消除虚假相关性。此外，通过在潜在空间中对风格进行干预，支持反事实推理。

Result: CADET在综合实验中表现优异，证明了因果先验在提升仇恨语音检测泛化能力方面的潜力。

Insight: 因果图能够有效建模仇恨语音的生成机制，解耦和干预潜在因素是提升模型鲁棒性和泛化能力的关键。

Abstract: The proliferation of online hate speech poses a significant threat to the harmony of the web. While explicit hate is easily recognized through overt slurs, implicit hate speech is often conveyed through sarcasm, irony, stereotypes, or coded language – making it harder to detect. Existing hate speech detection models, which predominantly rely on surface-level linguistic cues, fail to generalize effectively across diverse stylistic variations. Moreover, hate speech spread on different platforms often targets distinct groups and adopts unique styles, potentially inducing spurious correlations between them and labels, further challenging current detection approaches. Motivated by these observations, we hypothesize that the generation of hate speech can be modeled as a causal graph involving key factors: contextual environment, creator motivation, target, and style. Guided by this graph, we propose CADET, a causal representation learning framework that disentangles hate speech into interpretable latent factors and then controls confounders, thereby isolating genuine hate intent from superficial linguistic cues. Furthermore, CADET allows counterfactual reasoning by intervening on style within the latent space, naturally guiding the model to robustly identify hate speech in varying forms. CADET demonstrates superior performance in comprehensive experiments, highlighting the potential of causal priors in advancing generalizable hate speech detection.

[16] SUBQRAG: sub-question driven dynamic graph rag

Jiaoyang Li,Junhao Ruan,Shengwei Tang,Saihan Chen,Kaiyan Chang,Yuan Ge,Tong Xiao,Jingbo Zhu

Main category: cs.CL

TL;DR: SUBQRAG是一种基于子问题的动态图RAG框架，通过将复杂问题分解为可验证的子问题链，并从知识图中检索相关三元组或动态扩展图，提升多跳问答的性能。

Details

Motivation: 当前基于知识图（KG）的检索增强生成（Graph RAG）方法在多跳问答中缺乏深度结构化推理能力，导致证据不完整和错误累积。SUBQRAG旨在通过子问题驱动和动态图扩展来解决这些问题。

Contribution: 1. 提出了子问题驱动的框架SUBQRAG，支持复杂问题的分解和结构化推理；2. 动态扩展知识图以补充缺失信息；3. 通过”图记忆”实现结构化、可追溯的证据路径。

Method: 1. 将复杂问题分解为有序子问题链；2. 检索或动态扩展知识图获取三元组；3. 将推理过程的三元组聚合为”图记忆”以生成最终答案。

Result: 在三个多跳问答基准测试中，SUBQRAG在精确匹配（Exact Match）等指标上显著优于基线方法。

Insight: 通过子问题驱动的动态图扩展，SUBQRAG提升了推理深度和可追溯性，是一种解决复杂问答问题的有效方法。

Abstract: Graph Retrieval-Augmented Generation (Graph RAG) effectively builds a knowledge graph (KG) to connect disparate facts across a large document corpus. However, this broad-view approach often lacks the deep structured reasoning needed for complex multi-hop question answering (QA), leading to incomplete evidence and error accumulation. To address these limitations, we propose SubQRAG, a sub-question-driven framework that enhances reasoning depth. SubQRAG decomposes a complex question into an ordered chain of verifiable sub-questions. For each sub-question, it retrieves relevant triples from the graph. When the existing graph is insufficient, the system dynamically expands it by extracting new triples from source documents in real time. All triples used in the reasoning process are aggregated into a “graph memory,” forming a structured and traceable evidence path for final answer generation. Experiments on three multi-hop QA benchmarks demonstrate that SubQRAG achieves consistent and significant improvements, especially in Exact Match scores.

Cunli Mao,Xiaofei Gao,Ran Song,Shizhu He,Shengxiang Gao,Kang Liu,Zhengtao Yu

Main category: cs.CL

TL;DR: 该论文提出了一种基于大型语言模型（LLMs）的多语言知识图谱补全（MKGC）框架，通过知识级分组混合专家（KL-GMoE）和迭代实体重排（IER）技术，高效利用多语言共享知识，显著提升性能。

Details

Motivation: 现有的MKGC研究未能充分利用LLMs的多语言能力，也忽略了跨语言知识的共享性，导致多语言知识图谱的补全效果受限。

Contribution: 1. 提出KL-GMoE和IER两个核心组件，高效建模和利用多语言共享知识；2. 构建了一个包含5种语言的多语言知识图谱数据集；3. 在实验中，Hits@1、Hits@3和Hits@10指标分别比SOTA方法提升了5.47%、3.27%和1.01%。

Method: 1. KL-GMoE通过分组混合专家模型高效建模共享知识；2. IER通过迭代重排机制进一步提升知识利用率。

Result: 实验表明，该框架在多个评估指标上显著优于现有SOTA方法，尤其是在未见语言和不平衡语言设置下展示了知识共享的特性。

Insight: 多语言知识共享是提升MKGC性能的关键，KL-GMoE和IER的组合为未来多语言任务的建模提供了新思路。

Abstract: Large language models (LLMs) based Multilingual Knowledge Graph Completion (MKGC) aim to predict missing facts by leveraging LLMs’ multilingual understanding capabilities, improving the completeness of multilingual knowledge graphs (KGs). However, existing MKGC research underutilizes the multilingual capabilities of LLMs and ignores the shareability of cross-lingual knowledge. In this paper, we propose a novel MKGC framework that leverages multilingual shared knowledge to significantly enhance performance through two components: Knowledge-level Grouped Mixture of Experts (KL-GMoE) and Iterative Entity Reranking (IER). KL-GMoE efficiently models shared knowledge, while IER significantly enhances its utilization. To evaluate our framework, we constructed a mKG dataset containing 5 languages and conducted comprehensive comparative experiments with existing state-of-the-art (SOTA) MKGC method. The experimental results demonstrate that our framework achieves improvements of 5.47%, 3.27%, and 1.01% in the Hits@1, Hits@3, and Hits@10 metrics, respectively, compared with SOTA MKGC method. Further experimental analysis revealed the properties of knowledge sharing in settings of unseen and unbalanced languages. We have released the dataset and code for our work on https://github.com/gaoxiaofei07/KL-GMoE.

[18] ToolExpander: Extending the Frontiers of Tool-Using Reinforcement Learning to Weak LLMs

Fu Chen,Peng Wang,Xiyin Li,Wen Li,Shichi Lei,Dongdong Xiang

Main category: cs.CL

TL;DR: ToolExpander提出了一种新框架，通过动态多轮硬采样和自我示范思维，提升了资源受限LLMs在工具使用强化学习中的表现，显著改善了训练稳定性和最终性能。

Details

Motivation: 传统的GRPO方法在小规模LLMs中常导致响应不准确甚至训练崩溃，限制了性能提升和潜力发挥。

Contribution: 1. 动态多轮硬采样策略；2. 自我示范思维框架。

Method: 1. 动态替换困难样本并结合指数学习率衰减；2. 改进GRPO，去除KL散度并调整裁剪系数，附加微小奖励激励模型自主分析。

Result: ToolExpander显著提升了小规模LLMs的工具使用能力，改善了训练稳定性和性能。

Insight: 通过动态样本替换和自主分析机制，可以有效解决小规模LLMs在强化学习中的不稳定问题。

Abstract: Training Large Language Models (LLMs) with Group Relative Policy Optimization (GRPO) encounters a significant challenge: models often fail to produce accurate responses, particularly in small-scale architectures. This limitation not only diminishes performance improvements and undermines the potential of GRPO but also frequently leads to mid-training collapse, adversely affecting stability and final efficacy. To address these issues, we propose ToolExpander, a novel framework that advances tool-oriented reinforcement learning for resource-constrained LLMs through two key innovations:(1) Dynamic Multi-Round Hard Sampling, which dynamically substitutes challenging samples(those without correct outputs over 10 rollouts) with high-quality few-shot demonstrations during training, coupled with an exponential learning rate decay strategy to mitigate oscillations;(2) Self-Exemplifying Thinking, an enhanced GRPO framework that eliminates KL divergence and incorporates adjusted clipping coefficients, encouraging models to autonomously generate and analyze few-shot examples via a minimal additional reward (0.01).Experimental results demonstrate that ToolExpander significantly enhances tool-using capabilities in LLMs, especially in weaker small-scale models, improving both training stability and overall performance.

[19] OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment

Tianci Liu,Ran Xu,Tony Yu,Ilgee Hong,Carl Yang,Tuo Zhao,Haoyu Wang

Main category: cs.CL

TL;DR: 论文提出了OpenRubrics，一个大规模的(提示，评分标准)数据集，用于训练评分标准生成和基于评分的奖励模型。通过引入对比评分标准生成(CRG)，结合显式约束和隐式质量，提高评分的判别性和可靠性。Rubric-RM在多个基准测试中超越了基线模型。

Details

Motivation: 现有奖励模型主要依赖标量或成对判断，难以捕捉人类偏好的多维性。评分标准作为奖励(RaR)提供结构化自然语言标准，但其可靠性和可扩展性仍是挑战。

Contribution: 1. 提出OpenRubrics数据集；2. 引入CRG方法生成判别性和全面性的评分标准；3. 通过拒绝采样提高评分可靠性；4. Rubric-RM在基准测试中表现优越。

Method: 1. CRG通过对比优选和拒绝回答生成显式约束和隐式质量；2. 拒绝采样去除噪声评分标准；3. 训练Rubric-RM作为基于评分的奖励模型。

Result: Rubric-RM在多个基准测试中超越基线6.8%，并在指令跟随和生物医学任务上提升策略模型性能。

Insight: 评分标准提供可扩展的对齐信号，缩小人工评估与自动奖励建模的差距，推动了基于原则的LLM对齐新范式。

Abstract: Reward modeling lies at the core of reinforcement learning from human feedback (RLHF), yet most existing reward models rely on scalar or pairwise judgments that fail to capture the multifaceted nature of human preferences. Recent studies have explored rubrics-as-rewards (RaR) that uses structured natural language criteria that capture multiple dimensions of response quality. However, producing rubrics that are both reliable and scalable remains a key challenge. In this work, we introduce OpenRubrics, a diverse, large-scale collection of (prompt, rubric) pairs for training rubric-generation and rubric-based reward models. To elicit discriminative and comprehensive evaluation signals, we introduce Contrastive Rubric Generation (CRG), which derives both hard rules (explicit constraints) and principles (implicit qualities) by contrasting preferred and rejected responses. We further improve reliability by enforcing preference-label consistency via rejection sampling to remove noisy rubrics. Across multiple reward-modeling benchmarks, our rubric-based reward model, Rubric-RM, surpasses strong size-matched baselines by 6.8%. These gains transfer to policy models on instruction-following and biomedical benchmarks. Our results show that rubrics provide scalable alignment signals that narrow the gap between costly human evaluation and automated reward modeling, enabling a new principle-driven paradigm for LLM alignment.

[20] Parallel Test-Time Scaling for Latent Reasoning Models

Runyang You,Yongqi Li,Meng Liu,Wenjie Wang,Liqiang Nie,Wenjie Li

Main category: cs.CL

TL;DR: 该论文提出了针对潜变量推理模型的并行测试时间扩展方法，通过引入不确定性启发的采样策略和设计的潜变量奖励模型，解决了连续空间中采样和轨迹聚合的问题。

Details

Motivation: 潜变量推理模型在连续向量空间中进行中间推理，虽然比显式的链式推理更高效，但缺乏并行测试时间扩展的机制，尤其是连续空间中的采样和概率信号问题。

Contribution: 1. 提出了适用于潜变量推理模型的并行测试时间扩展方法；2. 设计了两种不确定性启发的采样策略：蒙特卡洛Dropout和高斯加性噪声；3. 开发了潜变量奖励模型（LatentRM），用于评分和引导潜变量推理。

Method: 1. 蒙特卡洛Dropout和高斯加性噪声作为采样策略；2. 通过对比目标训练的LatentRM实现轨迹选择和聚合。

Result: 实验和可视化分析表明，采样策略能有效扩展计算规模，LatentRM能实现有效的轨迹选择，为连续空间中的可扩展推理开辟了新方向。

Insight: 1. 不确定性启发的采样策略在潜变量推理中具有潜力；2. 连续空间的推理聚合需要设计专门的评分机制。

Abstract: Parallel test-time scaling (TTS) is a pivotal approach for enhancing large language models (LLMs), typically by sampling multiple token-based chains-of-thought in parallel and aggregating outcomes through voting or search. Recent advances in latent reasoning, where intermediate reasoning unfolds in continuous vector spaces, offer a more efficient alternative to explicit Chain-of-Thought, yet whether such latent models can similarly benefit from parallel TTS remains open, mainly due to the absence of sampling mechanisms in continuous space, and the lack of probabilistic signals for advanced trajectory aggregation. \ This work enables parallel TTS for latent reasoning models by addressing the above issues. For sampling, we introduce two uncertainty-inspired stochastic strategies: Monte Carlo Dropout and Additive Gaussian Noise. For aggregation, we design a Latent Reward Model (LatentRM) trained with step-wise contrastive objective to score and guide latent reasoning. Extensive experiments and visualization analyses show that both sampling strategies scale effectively with compute and exhibit distinct exploration dynamics, while LatentRM enables effective trajectory selection. Together, our explorations open a new direction for scalable inference in continuous spaces. Code released at https://github.com/YRYangang/LatentTTS.

[21] Test-Time Reasoners Are Strategic Multiple-Choice Test-Takers

Nishant Balepur,Atrey Desai,Rachel Rudinger

Main category: cs.CL

TL;DR: 该论文研究了大型语言模型（LLMs）在多选题问答（MCQA）中的策略，尤其是在仅提供选项（choices-only）设置下的表现。研究发现，即使在不提供完整问题的情况下，LLMs仍然能够通过推理策略完成任务，并且这种表现并不总是浅层的或不合理的。

Details

Motivation: 现有研究表明，LLMs在不使用完整问题的情况下（仅凭选项）也能在多选题问答中表现良好，这引发了关于其策略合理性的质疑。论文旨在通过分析推理痕迹（reasoning traces）探讨这些策略是否真的是浅层的或不合理的。

Contribution: 论文的主要贡献是揭示了LLMs在choices-only设置下的策略并非总是浅层的或不合理的，推理痕迹可以帮助区分合理与不合理的策略，并挑战了部分输入成功必然是缺陷的观点。

Method: 研究通过让LLMs在完整输入和choices-only输入下解决MCQs，并分析其推理痕迹的长度和忠实性，探讨了其策略的合理性。

Result: 研究发现，即使在choices-only设置下，LLMs的表现依然稳定，且推理痕迹的长度对其影响较小。进一步的忠实性测试表明，部分策略（如推断缺失的问题）并不存在问题。

Insight: 论文提供了新的视角，指出部分输入下的成功并不一定意味着模型存在缺陷，推理痕迹可以作为评估模型策略合理性的工具。

Abstract: Large language models (LLMs) now give reasoning before answering, excelling in tasks like multiple-choice question answering (MCQA). Yet, a concern is that LLMs do not solve MCQs as intended, as work finds LLMs sans reasoning succeed in MCQA without using the question, i.e., choices-only. Such partial-input success is often deemed problematic, but reasoning traces could reveal if these strategies are truly shallow in choices-only settings. To study these strategies, reasoning LLMs solve MCQs in full and choices-only inputs; test-time reasoning often boosts accuracy on full and in choices-only half the time. While possibly due to shallow shortcuts, choices-only success is barely affected by the length of reasoning traces, and after finding traces pass faithfulness tests, we show they use less problematic strategies like inferring missing questions. In all, we challenge claims that partial-input success is always a flaw, so we discuss how reasoning traces could separate problematic data from less problematic reasoning.

[22] ToolLibGen: Scalable Automatic Tool Creation and Aggregation for LLM Reasoning

Murong Yue,Zhiwei Liu,Liangwei Yang,Jianguo Zhang,Zuxin Liu,Haolin Chen,Ziyu Yao,Silvio Savarese,Caiming Xiong,Shelby Heinecke,Huan Wang

Main category: cs.CL

TL;DR: 论文提出了一种系统化方法，将无序的工具集合自动重构为结构化工具库，提升了检索准确性和推理性能。

Details

Motivation: 大规模语言模型（LLMs）在使用外部工具时性能显著提升，但领域专用工具的稀缺限制了其广泛应用。现有自动化工具生成方法因工具数量增长面临检索难题。

Contribution: 提出了一个多智能体框架，通过聚类和重构工具库，高效整合分散功能，生成少量多功能工具，提升检索效率和推理性能。

Method: 1. 生成任务专用工具并进行语义聚类；2. 使用代码智能体重构代码，提取共享逻辑；3. 评审智能体确保功能完整性。

Result: 实验表明，该方法显著提高了工具检索准确性和推理性能，并在工具数量增加时表现出更强的可扩展性。

Insight: 通过结构化工具库和多智能体协作，可以高效解决工具数量增加带来的检索问题，同时保持功能完整性。

Abstract: Large Language Models (LLMs) equipped with external tools have demonstrated enhanced performance on complex reasoning tasks. The widespread adoption of this tool-augmented reasoning is hindered by the scarcity of domain-specific tools. For instance, in domains such as physics question answering, suitable and specialized tools are often missing. Recent work has explored automating tool creation by extracting reusable functions from Chain-of-Thought (CoT) reasoning traces; however, these approaches face a critical scalability bottleneck. As the number of generated tools grows, storing them in an unstructured collection leads to significant retrieval challenges, including an expanding search space and ambiguity between function-related tools. To address this, we propose a systematic approach to automatically refactor an unstructured collection of tools into a structured tool library. Our system first generates discrete, task-specific tools and clusters them into semantically coherent topics. Within each cluster, we introduce a multi-agent framework to consolidate scattered functionalities: a code agent refactors code to extract shared logic and creates versatile, aggregated tools, while a reviewing agent ensures that these aggregated tools maintain the complete functional capabilities of the original set. This process transforms numerous question-specific tools into a smaller set of powerful, aggregated tools without loss of functionality. Experimental results demonstrate that our approach significantly improves tool retrieval accuracy and overall reasoning performance across multiple reasoning tasks. Furthermore, our method shows enhanced scalability compared with baselines as the number of question-specific increases.

[23] Curing Miracle Steps in LLM Mathematical Reasoning with Rubric Rewards

Youliang Yuan,Qiuyang Mang,Jingbang Chen,Hong Wan,Xiaoyuan Liu,Junjielong Xu,Jen-tse Huang,Wenxuan Wang,Wenxiang Jiao,Pinjia He

Main category: cs.CL

TL;DR: 论文探讨了在LLM数学推理中仅依赖最终答案的奖励机制容易导致奖励破解，提出了一种基于过程的奖励模型（RRM），通过细粒度评估推理过程显著提升了性能。

Details

Motivation: 现有的大语言模型在数学推理任务中通常仅根据最终答案给予奖励，导致模型可能通过错误的推理过程得到正确答案（称为Miracle Steps），显示出高估模型推理能力的问题。

Contribution: 论文的主要贡献是提出了Rubric Reward Model（RRM），一种基于过程的奖励函数，能够细粒度评估推理轨迹，显著减少Miracle Steps并提升模型性能。

Method: 作者设计了RRM，通过问题特定的评分标准（rubric）对整个推理过程进行评估，奖励逻辑严谨的步骤并惩罚错误。RRM被整合到强化学习框架中进行训练。

Result: 在四个数学基准测试中，RRM均优于仅基于结果的监督方法，其中在AIME2024上将Verified Pass@1024从26.7%提升到62.6%，并减少了71%的Miracle Steps。

Insight: 结果显示，奖励推理过程比仅奖励最终答案更能提高模型的准确性和可靠性，同时揭示了Miracle Steps可能与模型的记忆行为相关。

Abstract: Large language models for mathematical reasoning are typically trained with outcome-based rewards, which credit only the final answer. In our experiments, we observe that this paradigm is highly susceptible to reward hacking, leading to a substantial overestimation of a model’s reasoning ability. This is evidenced by a high incidence of false positives - solutions that reach the correct final answer through an unsound reasoning process. Through a systematic analysis with human verification, we establish a taxonomy of these failure modes, identifying patterns like Miracle Steps - abrupt jumps to a correct output without a valid preceding derivation. Probing experiments suggest a strong association between these Miracle Steps and memorization, where the model appears to recall the answer directly rather than deriving it. To mitigate this systemic issue, we introduce the Rubric Reward Model (RRM), a process-oriented reward function that evaluates the entire reasoning trajectory against problem-specific rubrics. The generative RRM provides fine-grained, calibrated rewards (0-1) that explicitly penalize logical flaws and encourage rigorous deduction. When integrated into a reinforcement learning pipeline, RRM-based training consistently outperforms outcome-only supervision across four math benchmarks. Notably, it boosts Verified Pass@1024 on AIME2024 from 26.7% to 62.6% and reduces the incidence of Miracle Steps by 71%. Our work demonstrates that rewarding the solution process is crucial for building models that are not only more accurate but also more reliable.

[24] The Unintended Trade-off of AI Alignment:Balancing Hallucination Mitigation and Safety in LLMs

Omar Mahmoud,Ali Khalil,Buddhika Laknath Semage,Thommen George Karimpanal,Santu Rana

Main category: cs.CL

TL;DR: 本文揭示了提升大型语言模型（LLMs）事实准确性和减轻幻觉效果时，会无意中削弱模型的安全对齐能力，并提出了一种通过稀疏自编码器和子空间正交化解决这一问题的方法。

Details

Motivation: 研究发现，提升LLMs的事实准确性通常会导致拒答行为（safety alignment）的减弱，这两种能力在模型中有重叠的编码组件。这一现象在当前研究中被忽视，因此需要一种方法来平衡两者。

Contribution: 1. 揭示了LLMs中事实准确性与安全对齐能力的负相关关系；2. 提出了一种通过稀疏自编码器分解特征和子空间正交化的方法，以维持拒答行为的同时减轻幻觉。

Method: 利用稀疏自编码器将拒答相关特征与幻觉特征分离，并通过子空间正交化在微调中保持拒答行为。

Result: 实验表明，该方法在常识推理任务和有害基准测试（AdvBench和StrongReject）中有效平衡了事实准确性和安全对齐能力。

Insight: 模型的性能改进可能在多个维度上存在trade-off，需综合考虑优化目标的设计。

Abstract: Hallucination in large language models (LLMs) has been widely studied in recent years, with progress in both detection and mitigation aimed at improving truthfulness. Yet, a critical side effect remains largely overlooked: enhancing truthfulness can negatively impact safety alignment. In this paper, we investigate this trade-off and show that increasing factual accuracy often comes at the cost of weakened refusal behavior. Our analysis reveals that this arises from overlapping components in the model that simultaneously encode hallucination and refusal information, leading alignment methods to suppress factual knowledge unintentionally. We further examine how fine-tuning on benign datasets, even when curated for safety, can degrade alignment for the same reason. To address this, we propose a method that disentangles refusal-related features from hallucination features using sparse autoencoders, and preserves refusal behavior during fine-tuning through subspace orthogonalization. This approach prevents hallucinations from increasing while maintaining safety alignment.We evaluate our method on commonsense reasoning tasks and harmful benchmarks (AdvBench and StrongReject). Results demonstrate that our approach preserves refusal behavior and task utility, mitigating the trade-off between truthfulness and safety.

[25] LLM4Cell: A Survey of Large Language and Agentic Models for Single-Cell Biology

Sajib Acharjee Dip,Adrika Zafor,Bikash Kumar Paul,Uddip Acharjee Shuvo,Muhit Islam Emon,Xuan Wang,Liqing Zhang

Main category: cs.CL

TL;DR: LLM4Cell 是第一篇针对单细胞生物学的58种基础模型和agentic模型的统一综述，涵盖了RNA、ATAC、多组学和空间模态。它将方法分为五类，并分析了它们在8个关键任务中的表现，同时评估了数据集的多样性、伦理和可扩展性限制。

Details

Motivation: 单细胞生物学领域的大语言模型（LLMs）和agentic框架在自然语言推理、生成注释和多模态数据集成方面展现出潜力，但目前的研究缺乏统一性和标准化。

Contribution: LLM4Cell 提供了首个统一的综述，将58种模型分为五类，并评估了它们在多个任务和数据模态中的表现，为语言驱动的单细胞智能提供了综合视角。

Method: 论文采用文献综述的方法，将模型分类为基础模型、文本桥接模型、空间模型、多模态模型、表观基因组模型和agentic模型，并使用40多个公共数据集进行基准评估。

Result: 研究揭示了模型在生物基础性、多组学对齐、公平性、隐私性和可解释性等方面的表现，同时指出了标准化和可信模型开发的开放性挑战。

Insight: LLM4Cell 强调了单细胞生物学中语言模型的潜力，但也指出需要在可解释性、标准化和伦理框架方面进一步研究。

Abstract: Large language models (LLMs) and emerging agentic frameworks are beginning to transform single-cell biology by enabling natural-language reasoning, generative annotation, and multimodal data integration. However, progress remains fragmented across data modalities, architectures, and evaluation standards. LLM4Cell presents the first unified survey of 58 foundation and agentic models developed for single-cell research, spanning RNA, ATAC, multi-omic, and spatial modalities. We categorize these methods into five families-foundation, text-bridge, spatial, multimodal, epigenomic, and agentic-and map them to eight key analytical tasks including annotation, trajectory and perturbation modeling, and drug-response prediction. Drawing on over 40 public datasets, we analyze benchmark suitability, data diversity, and ethical or scalability constraints, and evaluate models across 10 domain dimensions covering biological grounding, multi-omics alignment, fairness, privacy, and explainability. By linking datasets, models, and evaluation domains, LLM4Cell provides the first integrated view of language-driven single-cell intelligence and outlines open challenges in interpretability, standardization, and trustworthy model development.

[26] HiPRAG: Hierarchical Process Rewards for Efficient Agentic Retrieval Augmented Generation

Peilin Wu,Mian Zhang,Kun Wan,Wentian Zhao,Kaiyu He,Xinya Du,Zhiyu Chen

Main category: cs.CL

TL;DR: HiPRAG提出了一种分层过程奖励方法，通过在RL训练中引入细粒度的知识基础过程奖励，优化代理检索增强生成的搜索效率，减少过度搜索和不足搜索，提高了QA任务中的准确率和效率。

Details

Motivation: 当前基于RL的代理检索增强生成（RAG）方法依赖结果奖励，缺乏对搜索行为的细粒度控制，导致过度搜索和不足搜索问题普遍存在。

Contribution: HiPRAG的主要贡献是提出了分层过程奖励方法，通过分解推理轨迹并设计分层奖励函数，动态评估每一步搜索的必要性，从而优化搜索效率。

Method: 方法包括将推理轨迹分解为离散步骤，设计分层奖励函数（结合结果奖励和格式奖励），动态评估搜索决策的必要性。

Result: 在Qwen2.5和Llama-3.2模型上的实验表明，HiPRAG平均准确率达到65.4%（3B）和67.2%（7B），同时将过度搜索率降至2.3%。

Insight: 研究表明，优化推理过程本身而不仅是最终结果，能够显著提升代理检索增强生成的效率和可靠性。

Abstract: Agentic RAG is a powerful technique for incorporating external information that LLMs lack, enabling better problem solving and question answering. However, suboptimal search behaviors exist widely, such as over-search (retrieving information already known) and under-search (failing to search when necessary), which leads to unnecessary overhead and unreliable outputs. Current training methods, which typically rely on outcome-based rewards in a RL framework, lack the fine-grained control needed to address these inefficiencies. To overcome this, we introduce Hierarchical Process Rewards for Efficient agentic RAG (HiPRAG), a training methodology that incorporates a fine-grained, knowledge-grounded process reward into the RL training. Our approach evaluates the necessity of each search decision on-the-fly by decomposing the agent’s reasoning trajectory into discrete, parsable steps. We then apply a hierarchical reward function that provides an additional bonus based on the proportion of optimal search and non-search steps, on top of commonly used outcome and format rewards. Experiments on the Qwen2.5 and Llama-3.2 models across seven diverse QA benchmarks show that our method achieves average accuracies of 65.4% (3B) and 67.2% (7B). This is accomplished while improving search efficiency, reducing the over-search rate to just 2.3% and concurrently lowering the under-search rate. These results demonstrate the efficacy of optimizing the reasoning process itself, not just the final outcome. Further experiments and analysis demonstrate that HiPRAG shows good generalizability across a wide range of RL algorithms, model families, sizes, and types. This work demonstrates the importance and potential of fine-grained control through RL, for improving the efficiency and optimality of reasoning for search agents.

[27] AdaSwitch: Adaptive Switching Generation for Knowledge Distillation

Jingyu Peng,Maolin Wang,Hengyi Cai,Yuchen Li,Kai Zhang,Shuaiqiang Wang,Dawei Yin,Xiangyu Zhao

Main category: cs.CL

TL;DR: AdaSwitch 是一种新颖的知识蒸馏方法，通过在 token 级别动态结合 on-policy 和 off-policy 生成来解决现有方法中的训练-推理不匹配和质量问题。

Details

Motivation: 小型语言模型（SLMs）在延迟和计算资源受限的场景中至关重要，但现有知识蒸馏方法存在权衡：off-policy 提供高质量监督但引入训练-推理不匹配，而 on-policy 方法保持一致但依赖低质量学生输出。

Contribution: 提出了 AdaSwitch，一种动态结合 on-policy 和 off-policy 生成的方法，能够在 token 级别实时评估质量并选择性地整合教师指导，同时保持训练-推理一致性和监督质量。

Method: AdaSwitch 允许学生先探索自己的预测，然后基于实时质量评估选择性地集成教师指导。这种动态切换机制在 token 级别实现。

Result: 在三个数据集和两对教师-学生 LLM 上的实验表明，AdaSwitch 显著提升了准确性，且额外开销可控。

Insight: 通过在 token 级别动态切换生成策略，AdaSwitch 展示了如何在保持训练-推理一致性的同时，有效提升知识蒸馏的监督质量。

Abstract: Small language models (SLMs) are crucial for applications with strict latency and computational constraints, yet achieving high performance remains challenging. Knowledge distillation (KD) can transfer capabilities from large teacher models, but existing methods involve trade-offs: off-policy distillation provides high-quality supervision but introduces a training-inference mismatch, while on-policy approaches maintain consistency but rely on low-quality student outputs. To address these issues, we propose AdaSwitch, a novel approach that dynamically combines on-policy and off-policy generation at the token level. AdaSwitch allows the student to first explore its own predictions and then selectively integrate teacher guidance based on real-time quality assessment. This approach simultaneously preserves consistency and maintains supervision quality. Experiments on three datasets with two teacher-student LLM pairs demonstrate that AdaSwitch consistently improves accuracy, offering a practical and effective method for distilling SLMs with acceptable additional overhead.

[28] Do LLMs Really Need 10+ Thoughts for “Find the Time 1000 Days Later”? Towards Structural Understanding of LLM Overthinking

Xinliang Frederick Zhang,Anhad Mohananey,Alexandra Chronopoulou,Pinelopi Papalampidi,Somit Gupta,Tsendsuren Munkhdalai,Lu Wang,Shyam Upadhyay

Main category: cs.CL

TL;DR: 这篇论文研究了大型语言模型（LLM）在链式思维（CoT）推理中常见的‘过度思考’问题，提出了TRACE分析工具揭示了其内在原因，并提出了一种基于效用的定义来更准确地衡量和管理这一问题。

Details

Motivation: 尽管长链式思维推理模型在复杂任务上表现优异，但它们存在‘过度思考’的无效性问题，即对简单问题也会进行冗长推理，导致计算浪费。目前缺乏对其内在原因的系统性研究。

Contribution: 1. 提出TRACE工具，系统性分析LLM的思维过程；2. 揭示了两种主要的思维模式（Explorer和Late Landing）；3. 提出基于效用的‘过度思考’定义，取代传统的长度指标。

Method: 通过TRACE工具分解思维过程，构建细粒度思维演进图，识别常见思维模式。

Result: 研究发现LLM在简单任务上速度慢5到20倍，且性能无显著提升；揭示了‘过度验证’和‘过度探索’是过度思考的主要原因。

Insight: 基于思维结构的新定义为LLM推理优化提供了实用指南，未来可设计更高效的推理策略。

Abstract: Models employing long chain-of-thought (CoT) reasoning have shown superior performance on complex reasoning tasks. Yet, this capability introduces a critical and often overlooked inefficiency – overthinking – models often engage in unnecessarily extensive reasoning even for simple queries, incurring significant computations without accuracy improvements. While prior work has explored solutions to mitigate overthinking, a fundamental gap remains in our understanding of its underlying causes. Most existing analyses are limited to superficial, profiling-based observations, failing to delve into LLMs’ inner workings. This study introduces a systematic, fine-grained analyzer of LLMs’ thought process to bridge the gap, TRACE. We first benchmark the overthinking issue, confirming that long-thinking models are five to twenty times slower on simple tasks with no substantial gains. We then use TRACE to first decompose the thought process into minimally complete sub-thoughts. Next, by inferring discourse relationships among sub-thoughts, we construct granular thought progression graphs and subsequently identify common thinking patterns for topically similar queries. Our analysis reveals two major patterns for open-weight thinking models – Explorer and Late Landing. This finding provides evidence that over-verification and over-exploration are the primary drivers of overthinking in LLMs. Grounded in thought structures, we propose a utility-based definition of overthinking, which moves beyond length-based metrics. This revised definition offers a more insightful understanding of LLMs’ thought progression, as well as practical guidelines for principled overthinking management.

[29] CS3-Bench: Evaluating and Enhancing Speech-to-Speech LLMs for Mandarin-English Code-Switching

Heyang Liu,Yuhao Wang,Ziyang Cheng,Ronghua Wu,Qunshan Gu,Yanfeng Wang,Yu Wang

Main category: cs.CL

TL;DR: 该论文提出了CS3-Bench基准测试，用于评估和提升普通话-英语混合语音的语音转语音大语言模型的语言对齐能力。研究发现现有模型在多语言对齐上存在缺陷，并提出了一种改进方法，显著提升了模型性能。

Details

Motivation: 随着多模态大语言模型的发展，语音转语音交互系统得到了快速发展。然而，现有模型在多语言对齐方面表现不佳，尤其是在普通话-英语混合语音场景中。

Contribution: 论文的主要贡献包括：1) 提出了CS3Bench基准测试，用于评估语音转语音模型在混合语音中的表现；2) 提出了改进语言对齐能力的数据构造和训练方法（如Chain of Recognition和Keyword Highlighting）。

Method: 论文提出两种方法改进语言对齐：1) Chain of Recognition（CoR），用于增强模型理解能力；2) Keyword Highlighting（KH），用于引导生成。

Result: 实验表明，改进后的方法将知识准确率从25.14%提升到46.13%，开放理解率从64.5%提升到86.5%，并显著减少了次要语言的发音错误。

Insight: 论文揭示了当前语音转语音模型在多语言对齐方面的局限性，并通过数据和方法改进显著提升了模型性能，为未来多语言交互系统的开发提供了借鉴。

Abstract: The advancement of multimodal large language models has accelerated the development of speech-to-speech interaction systems. While natural monolingual interaction has been achieved, we find existing models exhibit deficiencies in language alignment. In our proposed Code-Switching Speech-to-Speech Benchmark (CS3-Bench), experiments on 7 mainstream models demonstrate a relative performance drop of up to 66% in knowledge-intensive question answering and varying degrees of misunderstanding in open-ended conversations. Starting from a model with severe performance deterioration, we propose both data constructions and training approaches to improve the language alignment capabilities, specifically employing Chain of Recognition (CoR) to enhance understanding and Keyword Highlighting (KH) to guide generation. Our approach improves the knowledge accuracy from 25.14% to 46.13%, with open-ended understanding rate from 64.5% to 86.5%, and significantly reduces pronunciation errors in the secondary language. CS3-Bench is available at https://huggingface.co/datasets/VocalNet/CS3-Bench.

[30] Contrastive Weak-to-strong Generalization

Houcheng Jiang,Junfeng Fang,Jiaxin Wu,Tianyu Zhang,Chen Gao,Yong Li,Xiang Wang,Xiangnan He,Yang Deng

Main category: cs.CL

TL;DR: 该论文提出了一种名为Contrastive Weak-to-Strong Generalization (ConG)的方法，通过利用预对齐和对齐后的弱模型之间的对比解码，生成更高质量的样本，从而提升弱到强模型的能力转移和鲁棒性。

Details

Motivation: 弱到强模型的泛化能力受到弱模型输出中噪声和偏差的限制，影响了其在实践中的适用性。为了解决这一问题，论文探索了如何通过对比解码减少噪声，提升泛化能力。

Contribution: 提出ConG框架，利用对比解码生成高质量样本；揭示了隐式奖励与对比解码的结构等价性；实验证明ConG在不同模型家族中一致提升了性能。

Method: 通过对比解码（Contrastive Decoding）在预对齐和对齐后的弱模型之间生成样本，利用隐式奖励近似显式奖励，减少噪声。

Result: 实验结果表明，ConG方法显著提升了弱到强模型的泛化能力和鲁棒性，并在不同模型家族中取得了稳定的改进。

Insight: 隐式奖励与对比解码的结构等价性为弱到强泛化提供了新的理论支持；ConG有望成为推动弱到强泛化和实现通用人工智能（AGI）的有效途径。

Abstract: Weak-to-strong generalization provides a promising paradigm for scaling large language models (LLMs) by training stronger models on samples from aligned weaker ones, without requiring human feedback or explicit reward modeling. However, its robustness and generalization are hindered by the noise and biases in weak-model outputs, which limit its applicability in practice. To address this challenge, we leverage implicit rewards, which approximate explicit rewards through log-likelihood ratios, and reveal their structural equivalence with Contrastive Decoding (CD), a decoding strategy shown to reduce noise in LLM generation. Building on this connection, we propose Contrastive Weak-to-Strong Generalization (ConG), a framework that employs contrastive decoding between pre- and post-alignment weak models to generate higher-quality samples. This approach enables more reliable capability transfer, denoising, and improved robustness, substantially mitigating the limitations of traditional weak-to-strong methods. Empirical results across different model families confirm consistent improvements, demonstrating the generality and effectiveness of ConG. Taken together, our findings highlight the potential of ConG to advance weak-to-strong generalization and provide a promising pathway toward AGI.

[31] Metric Calculating Benchmark: Code-Verifiable Complicate Instruction Following Benchmark for Large Language Models

Hyeonseok Moon,Seongtae Hong,Jaehyung Seo,Heuiseok Lim

Main category: cs.CL

TL;DR: MCBench是一个新的基准测试，旨在通过严格遵循逐步指令来评估大型语言模型（LLM）执行字符串匹配NLP指标的能力，提供客观、确定性和代码可验证的评估。

Details

Motivation: 当前前沿LLM已经在许多困难基准测试上达到饱和，缺乏进一步的区分能力，因此需要一个更具挑战性且支持客观验证的基准测试。

Contribution: 提出了MCBench，这是一个能够客观验证LLM逐步执行指令能力的基准测试，包括指令遵循、数值计算和长范围一致性。

Method: 设计了三个评估指标和三个基准变体，通过逐步指令和并行参考代码验证LLM输出的准确性。

Result: MCBench被证明是一个有效且客观的工具，用于评估前沿LLM的能力。

Insight: MCBench突出了逐步指令遵循和代码可验证性在评估LLM能力中的重要性，为未来基准测试设计提供了新方向。

Abstract: Recent frontier-level LLMs have saturated many previously difficult benchmarks, leaving little room for further differentiation. This progress highlights the need for challenging benchmarks that provide objective verification. In this paper, we introduce MCBench, a benchmark designed to evaluate whether LLMs can execute string-matching NLP metrics by strictly following step-by-step instructions. Unlike prior benchmarks that depend on subjective judgments or general reasoning, MCBench offers an objective, deterministic and codeverifiable evaluation. This setup allows us to systematically test whether LLMs can maintain accurate step-by-step execution, including instruction adherence, numerical computation, and long-range consistency in handling intermediate results. To ensure objective evaluation of these abilities, we provide a parallel reference code that can evaluate the accuracy of LLM output. We provide three evaluative metrics and three benchmark variants designed to measure the detailed instruction understanding capability of LLMs. Our analyses show that MCBench serves as an effective and objective tool for evaluating the capabilities of cutting-edge LLMs.

[32] ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recall

Jiayu Yang,Yuxuan Fan,Songning Lai,Shengen Wu,Jiaqi Tang,Chun Kang,Zhijiang Guo,Yutao Yue

Main category: cs.CL

TL;DR: 该论文提出了ACE框架，通过神经元级别的归因分析改进大语言模型（LLM）在多跳事实推理中的知识编辑能力，解决了现有方法在中间隐式主题上的性能衰减问题。

Details

Motivation: 现有的大语言模型知识编辑方法在多跳事实推理中存在性能衰减问题，尤其是涉及隐式中间主题时。研究发现这是由于忽视了神经元级别动态表示和利用链式知识的机制。

Contribution: 1. 揭示了多跳推理中隐式主题作为查询神经元的动态机制；2. 提出了ACE框架，利用神经元级归因识别和编辑关键查询-值路径；3. 在多跳知识编辑任务上显著优于现有方法。

Method: ACE通过神经元级归因分析识别关键查询-值（Q-V）路径，并对其进行编辑，从而实现更精确的知识更新。实验验证了该方法的有效性。

Result: ACE在GPT-J和Qwen3-8B上分别比现有方法提高了9.44%和37.46%的性能。

Insight: 1. 隐式中间主题在多跳推理中扮演查询神经元的角色；2. 语义解释性是由查询驱动的价值神经元积累实现的；3. 神经元级动态机制为知识编辑提供了新方向。

Abstract: Large Language Models (LLMs) require efficient knowledge editing (KE) to update factual information, yet existing methods exhibit significant performance decay in multi-hop factual recall. This failure is particularly acute when edits involve intermediate implicit subjects within reasoning chains. Through causal analysis, we reveal that this limitation stems from an oversight of how chained knowledge is dynamically represented and utilized at the neuron level. We discover that during multi hop reasoning, implicit subjects function as query neurons, which sequentially activate corresponding value neurons across transformer layers to accumulate information toward the final answer, a dynamic prior KE work has overlooked. Guided by this insight, we propose ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recall, a framework that leverages neuron-level attribution to identify and edit these critical query-value (Q-V) pathways. ACE provides a mechanistically grounded solution for multi-hop KE, empirically outperforming state-of-the-art methods by 9.44% on GPT-J and 37.46% on Qwen3-8B. Our analysis further reveals more fine-grained activation patterns in Qwen3 and demonstrates that the semantic interpretability of value neurons is orchestrated by query-driven accumulation. These findings establish a new pathway for advancing KE capabilities based on the principled understanding of internal reasoning mechanisms.

[33] Towards Human-Like Grading: A Unified LLM-Enhanced Framework for Subjective Question Evaluation

Fanwei Zhua,Jiaxuan He,Xiaoxiao Chen,Zulong Chen,Quan Lu,Chenrui Mei

Main category: cs.CL

TL;DR: 该论文提出了一种统一的基于大型语言模型（LLM）的自动评分框架，用于评估各类主观题答案，通过整合四个互补模块实现了接近人类评分的准确性。

Details

Motivation: 主观题的自动评分仍然是一个挑战，现有的方法局限于特定题型，缺乏对不同类型主观题的泛化支持。论文旨在解决这一问题，提供一种通用且高效的评分框架。

Contribution: 1. 提出了一个统一的LLM增强评分框架，支持多种主观题的评估；2. 设计了四个互补模块，综合利用了LLM的推理和生成能力；3. 在真实场景中验证了框架的有效性和实用性。

Method: 框架包含四个模块：基础的文本匹配模块、基于LLM的关键知识点比较模块、从学生答案生成伪问题的模块，以及模拟人类评价的模块。这些模块共同实现对答案的多维度评估。

Result: 实验表明，该框架在多类评分指标上优于传统方法和基于LLM的基线模型，并在真实的企业培训和认证考试中成功部署。

Insight: LLM的强大推理和生成能力可以有效提升主观题评分的准确性，多模块结合的方法能够更全面地模拟人类评分过程。

Abstract: Automatic grading of subjective questions remains a significant challenge in examination assessment due to the diversity in question formats and the open-ended nature of student responses. Existing works primarily focus on a specific type of subjective question and lack the generality to support comprehensive exams that contain diverse question types. In this paper, we propose a unified Large Language Model (LLM)-enhanced auto-grading framework that provides human-like evaluation for all types of subjective questions across various domains. Our framework integrates four complementary modules to holistically evaluate student answers. In addition to a basic text matching module that provides a foundational assessment of content similarity, we leverage the powerful reasoning and generative capabilities of LLMs to: (1) compare key knowledge points extracted from both student and reference answers, (2) generate a pseudo-question from the student answer to assess its relevance to the original question, and (3) simulate human evaluation by identifying content-related and non-content strengths and weaknesses. Extensive experiments on both general-purpose and domain-specific datasets show that our framework consistently outperforms traditional and LLM-based baselines across multiple grading metrics. Moreover, the proposed system has been successfully deployed in real-world training and certification exams at a major e-commerce enterprise.

[34] STEPER: Step-wise Knowledge Distillation for Enhancing Reasoning Ability in Multi-Step Retrieval-Augmented Language Models

Kyumin Lee,Minjin Jeon,Sanghwan Jang,Hwanjo Yu

Main category: cs.CL

TL;DR: STEPER提出了一种逐步知识蒸馏方法，通过分步监督和难度感知训练，提升多步检索增强语言模型的推理能力。

Details

Motivation: 现有知识蒸馏方法忽视了多步推理中各阶段的不同需求，导致推理能力转移受限。STEPER旨在解决这一问题。

Contribution: 1. 提出了逐步知识蒸馏方法（StepER），通过分步监督和难度感知训练提升模型推理能力。2. 适用于多种多步检索增强语言模型。

Method: STEPER采用分步监督（step-wise supervision）和难度感知训练（difficulty-aware training），逐步优化模型在不同推理阶段的表现。

Result: 实验表明，STEPER在多跳QA任务上优于现有方法，8B参数的模型性能可与70B教师模型媲美。

Insight: 逐步监督和难度感知训练能显著提升多步推理模型的效率和性能，体现了分阶段优化的重要性。

Abstract: Answering complex real-world questions requires step-by-step retrieval and integration of relevant information to generate well-grounded responses. However, existing knowledge distillation methods overlook the need for different reasoning abilities at different steps, hindering transfer in multi-step retrieval-augmented frameworks. To address this, we propose Stepwise Knowledge Distillation for Enhancing Reasoning Ability in Multi-Step Retrieval-Augmented Language Models (StepER). StepER employs step-wise supervision to align with evolving information and reasoning demands across stages. Additionally, it incorporates difficulty-aware training to progressively optimize learning by prioritizing suitable steps. Our method is adaptable to various multi-step retrieval-augmented language models, including those that use retrieval queries for reasoning paths or decomposed questions. Extensive experiments show that StepER outperforms prior methods on multi-hop QA benchmarks, with an 8B model achieving performance comparable to a 70B teacher model.

[35] Comprehensiveness Metrics for Automatic Evaluation of Factual Recall in Text Generation

Adam Dejl,James Barry,Alessandra Pascale,Javier Carnerero Cano

Main category: cs.CL

TL;DR: 本文提出了三种自动评估LLM生成文本全面性的方法，重点检测遗漏信息或观点，实验表明端到端方法简单有效但鲁棒性和可解释性较差。

Details

Motivation: LLM生成的文本常遗漏关键信息，可能导致严重后果，现有方法难以评估其全面性。

Contribution: 提出了三种自动评估LLM生成文本全面性的方法，并通过实验验证了端到端方法的有效性。

Method: 1. NLI方法分解文本并识别缺失链接；2. Q&A方法提取问答对并比较响应；3. 端到端方法直接识别缺失内容。

Result: 端到端方法简单有效，但鲁棒性和可解释性较差；评估了多个开源LLM的响应全面性。

Insight: 简单的端到端方法在评估全面性时表现优异，但需权衡鲁棒性与解释性。

Abstract: Despite demonstrating remarkable performance across a wide range of tasks, large language models (LLMs) have also been found to frequently produce outputs that are incomplete or selectively omit key information. In sensitive domains, such omissions can result in significant harm comparable to that posed by factual inaccuracies, including hallucinations. In this study, we address the challenge of evaluating the comprehensiveness of LLM-generated texts, focusing on the detection of missing information or underrepresented viewpoints. We investigate three automated evaluation strategies: (1) an NLI-based method that decomposes texts into atomic statements and uses natural language inference (NLI) to identify missing links, (2) a Q&A-based approach that extracts question-answer pairs and compares responses across sources, and (3) an end-to-end method that directly identifies missing content using LLMs. Our experiments demonstrate the surprising effectiveness of the simple end-to-end approach compared to more complex methods, though at the cost of reduced robustness, interpretability and result granularity. We further assess the comprehensiveness of responses from several popular open-weight LLMs when answering user queries based on multiple sources.

[36] Vision-Enabled LLMs in Historical Lexicography: Digitising and Enriching Estonian-German Dictionaries from the 17th and 18th Centuries

Madis Jürviste,Joonatan Jakobson

Main category: cs.CL

TL;DR: 论文探讨了大型语言模型（LLMs）在17至18世纪爱沙尼亚语词典研究中的应用，包括词典信息半自动丰富、哥特体文本识别及跨源数据集创建。初步实验表明LLMs在小语种词典数字化中具有显著潜力。

Details

Motivation: 研究旨在解决历史词典数字化和现代化的效率问题，特别是针对小语种（如爱沙尼亚语）及哥特体印刷文本的技术挑战。

Contribution: 1. 提出了利用LLMs半自动丰富历史词典信息的方法；2. 实现了哥特体印刷文本的零样本识别；3. 展示了LLMs在小语种数字化中的时间和成本优势。

Method: 1. 使用Claude 3.7 Sonnet为1648年词典提供现代词义；2. 零样本方法识别1732年词典的哥特体文本并生成结构化JSON；3. 采用重叠切片扫描和双LLM协同处理1780年词典。

Result: 1. Claude 3.7 Sonnet在81%的词条中准确提供现代词义；2. 零样本方法成功识别并结构化41%的词条；3. 扫描切片和双LLM协作方法有效提升了1780年词典的处理效率。

Insight: LLMs即使对小语种和复杂印刷体也表现出强大适应力，未来可通过结合更多上下文进一步优化识别和结构化效果。

Abstract: This article presents research conducted at the Institute of the Estonian Language between 2022 and 2025 on the application of large language models (LLMs) to the study of 17th and 18th century Estonian dictionaries. The authors address three main areas: enriching historical dictionaries with modern word forms and meanings; using vision-enabled LLMs to perform text recognition on sources printed in Gothic script (Fraktur); and preparing for the creation of a unified, cross-source dataset. Initial experiments with J. Gutslaff’s 1648 dictionary indicate that LLMs have significant potential for semi-automatic enrichment of dictionary information. When provided with sufficient context, Claude 3.7 Sonnet accurately provided meanings and modern equivalents for 81% of headword entries. In a text recognition experiment with A. T. Helle’s 1732 dictionary, a zero-shot method successfully identified and structured 41% of headword entries into error-free JSON-formatted output. For digitising the Estonian-German dictionary section of A. W. Hupel’s 1780 grammar, overlapping tiling of scanned image files is employed, with one LLM being used for text recognition and a second for merging the structured output. These findings demonstrate that even for minor languages LLMs have a significant potential for saving time and financial resources.

[37] A$^2$Search: Ambiguity-Aware Question Answering with Reinforcement Learning

Fengji Zhang,Xinyao Niu,Chengyang Ying,Guancheng Lin,Zhongkai Hao,Zhou Fan,Chengen Huang,Jacky Keung,Bei Chen,Junyang Lin

Main category: cs.CL

TL;DR: A²Search提出了一种基于强化学习的无标注端到端训练框架，用于识别和处理问答系统中的模糊问题，通过自动检测和验证替代答案，显著提升了多跳问答任务的性能。

Details

Motivation: 现有的大语言模型和强化学习方法在开放域问答中表现优异，但对多答案模糊问题的处理不足。传统基准测试仅关注单一答案，无法提供合适的训练信号。A²Search旨在无需人工标注的情况下解决这一问题。

Contribution: 1. 提出A²Search框架，自动检测模糊问题并通过轨迹采样和证据验证收集替代答案；2. 设计了专用于多答案场景的AnsF1奖励函数，优化模型性能；3. 在多个开放域QA基准测试中实现了最先进的性能。

Method: A²Search的核心是一个自动化流程：检测模糊问题→通过轨迹采样和证据验证收集替代答案→使用AnsF1奖励函数通过RL优化模型。

Result: 在8个开放域QA基准测试中，A²Search表现最佳，尤其是在多跳任务中，AnsF1@1平均得分48.4%，优于更大的ReSearch-32B模型（46.2%）。

Insight: 研究表明，拥抱模糊性是构建更可靠问答系统的关键，A²Search的成功证明了自动化处理模糊问题的有效性。

Abstract: Recent advances in Large Language Models (LLMs) and Reinforcement Learning (RL) have led to strong performance in open-domain question answering (QA). However, existing models still struggle with questions that admit multiple valid answers. Standard QA benchmarks, which typically assume a single gold answer, overlook this reality and thus produce inappropriate training signals. Existing attempts to handle ambiguity often rely on costly manual annotation, which is difficult to scale to multi-hop datasets such as HotpotQA and MuSiQue. In this paper, we present A$^2$Search, an annotation-free, end-to-end training framework to recognize and handle ambiguity. At its core is an automated pipeline that detects ambiguous questions and gathers alternative answers via trajectory sampling and evidence verification. The model is then optimized with RL using a carefully designed $\mathrm{AnsF1}$ reward, which naturally accommodates multiple answers. Experiments on eight open-domain QA benchmarks demonstrate that A$^2$Search achieves new state-of-the-art performance. With only a single rollout, A$^2$Search-7B yields an average $\mathrm{AnsF1}@1$ score of $48.4%$ across four multi-hop benchmarks, outperforming all strong baselines, including the substantially larger ReSearch-32B ($46.2%$). Extensive analyses further show that A$^2$Search resolves ambiguity and generalizes across benchmarks, highlighting that embracing ambiguity is essential for building more reliable QA systems. Our code, data, and model weights can be found at https://github.com/zfj1998/A2Search

[38] LightReasoner: Can Small Language Models Teach Large Language Models Reasoning?

Jingyuan Wang,Yankai Chen,Zhonghang Li,Chao Huang

Main category: cs.CL

TL;DR: LightReasoner提出了一种新颖的框架，通过利用小型语言模型（SLM）揭示大型语言模型（LLM）的高价值推理时刻，显著提升了LLM的推理能力，同时减少了资源和时间的消耗。

Details

Motivation: 现有的大型语言模型（LLM）在推理任务中表现优异，但其监督微调（SFT）过程需要大量资源和时间。研究中提出的LightReasoner旨在探索小型语言模型（SLM）是否能通过揭示LLM的高价值推理时刻，帮助LLM更高效地进行微调。

Contribution: LightReasoner的主要贡献在于提出了一种两阶段框架：1）通过对比专家模型（LLM）和业余模型（SLM）的行为差异，识别关键推理时刻；2）利用这些时刻构建监督示例，高效微调LLM。该方法显著提升了推理能力，同时降低了资源消耗。

Method: LightReasoner分为两个阶段：1）采样阶段，通过对比LLM和SLM的行为差异，识别关键推理时刻并构建监督示例；2）微调阶段，利用这些示例高效微调LLM，强化其推理能力。

Result: 在七个数学基准测试中，LightReasoner将LLM的推理准确率提升了28.1%，同时减少了90%的时间消耗、80%的采样问题和99%的微调token使用量，且无需依赖真实标签。

Insight: LightReasoner表明，小型语言模型（SLM）可以作为有效的教学信号，帮助大型语言模型（LLM）在推理任务中高效学习。这种方法为提升LLM的推理能力提供了一种可扩展且资源高效的新途径。

Abstract: Large language models (LLMs) have demonstrated remarkable progress in reasoning, often through supervised fine-tuning (SFT). However, SFT is resource-intensive, relying on large curated datasets, rejection-sampled demonstrations, and uniform optimization across all tokens, even though only a fraction carry meaningful learning value. In this work, we explore a counterintuitive idea: can smaller language models (SLMs) teach larger language models (LLMs) by revealing high-value reasoning moments that reflect the latter’s unique strength? We propose LightReasoner, a novel framework that leverages the behavioral divergence between a stronger expert model (LLM) and a weaker amateur model (SLM). LightReasoner operates in two stages: (1) a sampling stage that pinpoints critical reasoning moments and constructs supervision examples capturing the expert’s advantage through expert-amateur contrast, and (2) a fine-tuning stage that aligns the expert model with these distilled examples, amplifying its reasoning strengths. Across seven mathematical benchmarks, LightReasoner improves accuracy by up to 28.1%, while reducing time consumption by 90%, sampled problems by 80%, and tuned token usage by 99%, all without relying on ground-truth labels. By turning weaker SLMs into effective teaching signals, LightReasoner offers a scalable and resource-efficient approach for advancing LLM reasoning. Code is available at: https://github.com/HKUDS/LightReasoner

Jialu Du,Guiyang Hou,Yihui Fu,Chen Wu,Wenqi Zhang,Yongliang Shen,Weiming Lu

Main category: cs.CL

TL;DR: 大型语言模型（LLMs）在社交推理任务中表现不佳，常出现矛盾推理或循环。论文提出一种动态文本世界模型机制，通过追踪实体状态和时间序列，显著提升社交推理准确性并降低计算成本。

Details

Motivation: LLMs在数学和代码推理中表现优异，但在社交推理任务中存在认知混淆和逻辑不一致的问题，尤其是无法区分客观事实与主观信念。

Contribution: 提出一种适应性世界模型增强的推理机制，通过动态构建世界模型和干预推理轨迹，显著提升LLMs在社交推理中的准确性和效率。

Method: 设计动态文本世界模型，追踪实体状态和时间序列；实时监测推理轨迹中的混淆指标，并适时提供清晰的世界状态描述。

Result: 在三个社交基准测试中，准确性显著提升（如Hi-ToM提高10%），同时计算成本降低（令牌数减少33.8%）。

Insight: 模仿人类利用隐式世界模型区分外部事件和内部信念的方法，为解决LLMs的社交推理问题提供了一种简单有效的思路。

Abstract: While large language models (LLMs) excel in mathematical and code reasoning, we observe they struggle with social reasoning tasks, exhibiting cognitive confusion, logical inconsistencies, and conflation between objective world states and subjective belief states. Through deteiled analysis of DeepSeek-R1’s reasoning trajectories, we find that LLMs frequently encounter reasoning impasses and tend to output contradictory terms like “tricky” and “confused” when processing scenarios with multiple participants and timelines, leading to erroneous reasoning or infinite loops. The core issue is their inability to disentangle objective reality from agents’ subjective beliefs. To address this, we propose an adaptive world model-enhanced reasoning mechanism that constructs a dynamic textual world model to track entity states and temporal sequences. It dynamically monitors reasoning trajectories for confusion indicators and promptly intervenes by providing clear world state descriptions, helping models navigate through cognitive dilemmas. The mechanism mimics how humans use implicit world models to distinguish between external events and internal beliefs. Evaluations on three social benchmarks demonstrate significant improvements in accuracy (e.g., +10% in Hi-ToM) while reducing computational costs (up to 33.8% token reduction), offering a simple yet effective solution for deploying LLMs in social contexts.

[40] Leveraging Author-Specific Context for Scientific Figure Caption Generation: 3rd SciCap Challenge

Watcharapong Timklaypachara,Monrada Chiewhawan,Nopporn Lekuthai,Titipat Achakulvisut

Main category: cs.CL

TL;DR: 这篇论文提出了一种针对科学图表标题生成的领域特定系统，结合图表相关文本上下文和作者特定写作风格，通过两阶段流程显著提升了生成标题的科学准确性和风格一致性。

Details

Motivation: 科学图表标题需要准确传达信息并保持作者特定风格，但目前通用的生成方法难以兼顾两者。

Contribution: 主要贡献是一个两阶段流程，结合上下文过滤、类别特定提示优化和风格微调，显著提升了标题生成的准确性和风格一致性。

Method: 方法分为两阶段：1）上下文过滤和类别特定提示优化；2）基于作者风格的小样本提示微调。

Result: 实验表明，类别特定提示优于零样本和通用优化方法，ROUGE-1召回率提升8.3%，风格微调进一步提高了BLEU和ROUGE分数（分别提升40-48%和25-27%）。

Insight: 论文表明，结合上下文理解和作者特定风格适应可以生成科学准确且风格一致的标题。

Abstract: Scientific figure captions require both accuracy and stylistic consistency to convey visual information. Here, we present a domain-specific caption generation system for the 3rd SciCap Challenge that integrates figure-related textual context with author-specific writing styles using the LaMP-Cap dataset. Our approach uses a two-stage pipeline: Stage 1 combines context filtering, category-specific prompt optimization via DSPy’s MIPROv2 and SIMBA, and caption candidate selection; Stage 2 applies few-shot prompting with profile figures for stylistic refinement. Our experiments demonstrate that category-specific prompts outperform both zero-shot and general optimized approaches, improving ROUGE-1 recall by +8.3% while limiting precision loss to -2.8% and BLEU-4 reduction to -10.9%. Profile-informed stylistic refinement yields 40–48% gains in BLEU scores and 25–27% in ROUGE. Overall, our system demonstrates that combining contextual understanding with author-specific stylistic adaptation can generate captions that are both scientifically accurate and stylistically faithful to the source paper.

[41] Learning on the Job: An Experience-Driven Self-Evolving Agent for Long-Horizon Tasks

Cheng Yang,Xuemeng Yang,Licheng Wen,Daocheng Fu,Jianbiao Mei,Rong Wu,Pinlong Cai,Yufan Shen,Nianchen Deng,Botian Shi,Yu Qiao,Haifeng Li

Main category: cs.CL

TL;DR: MUSE 提出了一个经验驱动的自进化智能体框架，通过分层内存模块实现任务执行中的持续学习和自我进化，显著提升了长时任务的表现。

Details

Motivation: 当前的大型语言模型在长时任务中表现静态，无法从经验中学习，缺乏持续改进的能力。MUSE 旨在解决这一问题。

Contribution: MUSE 引入了分层次的内存模块，支持动态学习和自我进化，显著提升了任务完成能力和泛化性能。

Method: MUSE 通过执行子任务后自主反思，将原始轨迹转化为结构化经验并存入内存模块，实现持续学习。

Result: MUSE 在 TAC 生产力基准测试中取得了显著的 SOTA 性能，并展示了零样本改进新任务的能力。

Insight: 经验积累不仅能提升当前任务的性能，还具备泛化能力，为 AI 智能体的实际应用提供了新范式。

Abstract: Large Language Models have demonstrated remarkable capabilities across diverse domains, yet significant challenges persist when deploying them as AI agents for real-world long-horizon tasks. Existing LLM agents suffer from a critical limitation: they are test-time static and cannot learn from experience, lacking the ability to accumulate knowledge and continuously improve on the job. To address this challenge, we propose MUSE, a novel agent framework that introduces an experience-driven, self-evolving system centered around a hierarchical Memory Module. MUSE organizes diverse levels of experience and leverages them to plan and execute long-horizon tasks across multiple applications. After each sub-task execution, the agent autonomously reflects on its trajectory, converting the raw trajectory into structured experience and integrating it back into the Memory Module. This mechanism enables the agent to evolve beyond its static pretrained parameters, fostering continuous learning and self-evolution. We evaluate MUSE on the long-horizon productivity benchmark TAC. It achieves new SOTA performance by a significant margin using only a lightweight Gemini-2.5 Flash model. Sufficient Experiments demonstrate that as the agent autonomously accumulates experience, it exhibits increasingly superior task completion capabilities, as well as robust continuous learning and self-evolution capabilities. Moreover, the accumulated experience from MUSE exhibits strong generalization properties, enabling zero-shot improvement on new tasks. MUSE establishes a new paradigm for AI agents capable of real-world productivity task automation.

[42] ChatGPT as a Translation Engine: A Case Study on Japanese-English

Vincent Michael Sutanto,Giovanni Gatti De Giacomo,Toshiaki Nakazawa,Masaru Yamada

Main category: cs.CL

TL;DR: 本研究探讨了ChatGPT在日语-英语翻译中的应用，比较了不同提示方式的表现，并与其他商用翻译引擎进行了对比。研究发现，文档级翻译优于句子级翻译，但增强提示的效果不明显；ChatGPT-3.5在自动评估中表现更好，但在流畅度上不如ChatGPT-4。ChatGPT在竞争力上与主流翻译系统相当。

Details

Motivation: 研究ChatGPT在翻译任务中的表现，尤其是日语-英语翻译，并与现有商用翻译引擎进行对比，以评估其实际应用潜力。

Contribution: 1. 比较了ChatGPT在不同提示方式（简单与增强）下的表现；2. 评估了文档级与句子级翻译的效果差异；3. 发现ChatGPT-3.5和ChatGPT-4在准确性与流畅性上的权衡；4. ChatGPT在与主流翻译系统对比中的竞争力。

Method: 1. 设计简单和增强提示进行翻译任务；2. 进行自动评估和基于MQM的人工评估；3. 比较文档级与句子级翻译效果；4. 对比ChatGPT与商用翻译引擎的表现。

Result: 1. 文档级翻译效果优于句子级；2. 增强提示与简单提示差异不显著；3. ChatGPT-3.5在自动评估中表现更好，但ChatGPT-4流畅性更优；4. ChatGPT与主流翻译系统结果相当。

Insight: 1. ChatGPT在文档级翻译任务中更具优势；2. 提示设计对翻译效果的影响仍需进一步研究；3. 不同版本ChatGPT在准确性与流畅性上的权衡值得注意；4. ChatGPT可作为商用翻译系统的替代方案。

Abstract: This study investigates ChatGPT for Japanese-English translation, exploring simple and enhanced prompts and comparing against commercially available translation engines. Performing both automatic and MQM-based human evaluations, we found that document-level translation outperforms sentence-level translation for ChatGPT. On the other hand, we were not able to determine if enhanced prompts performed better than simple prompts in our experiments. We also discovered that ChatGPT-3.5 was preferred by automatic evaluation, but a tradeoff exists between accuracy (ChatGPT-3.5) and fluency (ChatGPT-4). Lastly, ChatGPT yields competitive results against two widely-known translation systems.

[43] A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Models

Congming Zheng,Jiachen Zhu,Zhuoying Ou,Yuxiang Chen,Kangning Zhang,Rong Shan,Zeyu Zheng,Mengyue Yang,Jianghao Lin,Yong Yu,Weinan Zhang

Main category: cs.CL

TL;DR: 这篇论文系统地调查了过程奖励模型（PRMs）的发展与应用，探讨了其在大型语言模型（LLMs）中如何通过步骤或轨迹级别的评估和指导来增强推理能力，弥补了传统结果奖励模型（ORMs）的不足。

Details

Motivation: 尽管大型语言模型展现出高级推理能力，传统的对齐方法仍主要依赖于仅评估最终答案的结果奖励模型（ORMs）。过程奖励模型（PRMs）旨在填补这一空白，通过对推理过程的步骤或轨迹进行评估和指导，实现更精细的对齐。

Contribution: 论文的主要贡献包括：（1）系统地概述了PRMs的全流程，包括过程数据的生成、PRMs的构建以及其在测试时扩展和强化学习中的应用；（2）总结了PRMs在数学、代码、文本、多模态推理、机器人及代理等多个领域的应用；（3）回顾了新兴的基准测试，并为未来研究提供了设计空间和开放挑战的指导。

Method: 论文采用系统性调查的方法，详细分析了PRMs的设计和实施流程，包括数据生成、模型构建以及应用场景。

Result: 通过调查，论文展示了PRMs在多个领域中的潜力，并为未来研究提供了明确的方向和挑战。

Insight: PRMs通过过程级别的监督为LLMs提供了更精细的对齐方式，可能显著提升模型的推理能力和泛化性能。未来的研究需要解决PRMs的鲁棒性和泛化性问题。

Abstract: Although Large Language Models (LLMs) exhibit advanced reasoning ability, conventional alignment remains largely dominated by outcome reward models (ORMs) that judge only final answers. Process Reward Models(PRMs) address this gap by evaluating and guiding reasoning at the step or trajectory level. This survey provides a systematic overview of PRMs through the full loop: how to generate process data, build PRMs, and use PRMs for test-time scaling and reinforcement learning. We summarize applications across math, code, text, multimodal reasoning, robotics, and agents, and review emerging benchmarks. Our goal is to clarify design spaces, reveal open challenges, and guide future research toward fine-grained, robust reasoning alignment.

[44] The Price of Thought: A Multilingual Analysis of Reasoning, Performance, and Cost of Negotiation in Large Language Models

Sherzod Hakimov,Roland Bernard,Tim Leiber,Karl Osswald,Kristina Richert,Ruilin Yang,Raffaella Bernardi,David Schlangen

Main category: cs.CL

TL;DR: 该论文首次全面研究了大型语言模型（LLM）在多语言谈判中的推理能力、性能与计算成本之间的权衡，揭示了推理对谈判能力的提升作用及其高昂的计算成本，并发现了开源模型在推理过程中倾向于切换到英语的现象。

Details

Motivation: 谈判是AI代理的一项基础挑战，需要战略性推理、对手建模以及合作与竞争的平衡。论文旨在系统评估LLM推理对其谈判能力的影响，并探究多语言环境下的表现差异。

Contribution: 1. 首次在多语言环境下全面评估LLM推理对谈判能力的影响；2. 揭示了推理显著提升性能但代价高昂；3. 发现开源模型在推理过程中倾向于切换到英语的现象。

Method: 通过自对弈实验设置，在三种不同的对话游戏中分析LLM的谈判能力。比较了推理开启前后的性能和计算成本，并研究了多语言环境下的语言一致性。

Result: 推理显著提升谈判性能（GPT-5性能提升31.4%），但计算成本增加近400%。开源模型在多语言谈判中倾向于切换到英语推理，而商业模型保持了语言一致性。

Insight: 推理能力对LLM的谈判表现至关重要，但其高昂的计算成本成为实际应用的瓶颈。多语言环境下，语言一致性问题可能影响模型的可解释性和实际部署效果。

Abstract: Negotiation is a fundamental challenge for AI agents, as it requires an ability to reason strategically, model opponents, and balance cooperation with competition. We conduct the first comprehensive study systematically evaluating the effect of (LLM-)reasoning on the negotiation abilities of both commercial and open-weight LLMs, and do this across three languages. Using a self-play setup across three diverse dialogue games, we analyse trade-offs between performance and cost, the language consistency of reasoning processes, and the nature of strategic adaptation exhibited by models. Our findings show that enabling reasoning-that is, scaling test time compute-significantly improves negotiation outcomes by enhancing collaboration and helping models overcome task complexities, but comes at a substantial computational cost: reasoning improves GPT-5’s performance by 31.4 % while increasing its cost by nearly 400 %. Most critically, we uncover a significant multilingual reasoning distinction: open-weight models consistently switch to English for their internal reasoning steps, even when negotiating in German or Italian (and thus possibly impacting potential explainability gains through the disclosure of reasoning traces), while leading commercial models maintain language consistency between their reasoning and final output.

[45] Lossless Vocabulary Reduction for Auto-Regressive Language Models

Daiki Chijiwa,Taku Hasegawa,Kyosuke Nishida,Shin’ya Yamaguchi,Tomoya Ohba,Tamao Sakao,Susumu Takeuchi

Main category: cs.CL

TL;DR: 这篇论文提出了一种无损词汇缩减的理论框架，用于将自回归语言模型的词汇表无损压缩为任意小的规模，从而解决不同模型因词汇表不一致而无法协作的问题。

Details

Motivation: 自回归语言模型的生成效率和协作能力受限于词汇表的大小和差异。不同模型的词汇表不一致导致它们在生成文本时难以有效协作，例如在模型集成中。

Contribution: 提出了无损词汇缩减的理论框架，使自回归语言模型能够在无损准确性的情况下将词汇表压缩至任意小，并展示了其在模型协作中的应用。

Method: 通过理论分析，提出了一种方法将模型的词汇表高效转换为任意小的子集，同时确保生成分布不变。该方法利用了最大公共词汇的概念。

Result: 实现了不同分词方式的模型通过最大公共词汇高效协作，证明了无损词汇缩减的可行性和有效性。

Insight: 词汇表的缩减不仅能提高模型效率，还能解决模型间的协作问题，为多模型集成和应用提供了新的可能性。

Abstract: Tokenization – the process of decomposing a given text into a sequence of subwords called tokens – is one of the key components in the development of language models. Particularly, auto-regressive language models generate texts token by token, i.e., by predicting the next-token distribution given the previous ones, and thus tokenization directly affects their efficiency in text generation. Since each language model has their own vocabulary as a set of possible tokens, they struggle to cooperate with each other at the level of next-token distributions such as model ensemble. In this paper, we establish a theoretical framework of lossless vocabulary reduction, which efficiently converts a given auto-regressive language model into the one with an arbitrarily small vocabulary without any loss in accuracy. As an application, we demonstrate that language models with different tokenization can cooperate with each other efficiently through their maximal common vocabulary.

Haoyang Gui,Thales Bertaglia,Taylor Annabell,Catalina Goanta,Tjomme Dooper,Gerasimos Spanakis

Main category: cs.CL

TL;DR: 这篇论文评估了LLM生成的法律解释在社交媒体影响者营销中对监管合规性的有效性，提出了一个错误分类法，并通过定量和定性分析揭示了LLM在透明度和法律基础方面的局限性。

Details

Motivation: 影响者营销中赞助内容的透明性监管存在法律基础薄弱和检测方法不透明的问题，论文旨在探讨LLM是否能提供合规且透明的法律解释。

Contribution: 1)提出了LLM法律解释中常见错误的分类法；2)提供了一个由法律学生标注的LLM解释数据集；3)结合定量与定性方法评估LLM解释的合规性。

Method: 使用1,143条Instagram帖子，比较gpt-5-nano和gemini-2.5-flash-lite在三种提示策略下的表现，并通过标注数据开发错误分类法。

Result: LLM在明确内容分类上表现优异（F1高达0.93），但在模糊案例中性能下降超10点；提示中添加法规文本能提升解释质量，但检测准确性未一致提升。

Insight: LLM在法律解释中常犯引用遗漏（28.57%）和模糊引用（20.71%）等错误，隐藏广告的错误率最高（28.57%），表明自动化监管需进一步优化法律基础。

Abstract: The rise of influencer marketing has blurred boundaries between organic content and sponsored content, making the enforcement of legal rules relating to transparency challenging. Effective regulation requires applying legal knowledge with a clear purpose and reason, yet current detection methods of undisclosed sponsored content generally lack legal grounding or operate as opaque “black boxes”. Using 1,143 Instagram posts, we compare gpt-5-nano and gemini-2.5-flash-lite under three prompting strategies with controlled levels of legal knowledge provided. Both models perform strongly in classifying content as sponsored or not (F1 up to 0.93), though performance drops by over 10 points on ambiguous cases. We further develop a taxonomy of reasoning errors, showing frequent citation omissions (28.57%), unclear references (20.71%), and hidden ads exhibiting the highest miscue rate (28.57%). While adding regulatory text to the prompt improves explanation quality, it does not consistently improve detection accuracy. The contribution of this paper is threefold. First, it makes a novel addition to regulatory compliance technology by providing a taxonomy of common errors in LLM-generated legal reasoning to evaluate whether automated moderation is not only accurate but also legally robust, thereby advancing the transparent detection of influencer marketing content. Second, it features an original dataset of LLM explanations annotated by two students who were trained in influencer marketing law. Third, it combines quantitative and qualitative evaluation strategies for LLM explanations and critically reflects on how these findings can support advertising regulatory bodies in automating moderation processes on a solid legal foundation.

[47] ARM2: Adaptive Reasoning Model with Vision Understanding and Executable Code

Jian Xie,Zhendong Chu,Aoxiao Zhong,Kai Zhang,Mingzhe Han,Xin Fang,Jialie Shen,Qingsong Wen

Main category: cs.CL

TL;DR: ARM2是一种自适应推理模型，通过强化学习和长度感知优化平衡推理性能与效率，支持多模态（视觉理解和可执行代码），显著减少了令牌使用。

Details

Motivation: 大型推理模型（LRM）存在‘过度思考’问题，即在简单任务上生成不必要的冗长推理。现有方法（如长度惩罚或路由机制）通常是启发式且任务特定的，缺乏通用框架。

Contribution: 1) 提出ARM2，一个统一的自适应推理模型；2) 通过强化学习和长度感知优化实现平衡；3) 扩展到多模态（视觉理解和可执行代码）；4) 显著减少令牌成本。

Method: 1) 使用强化学习框架；2) 引入长度感知优化；3) 集成视觉理解和可执行代码；4) 支持多模态推理。

Result: ARM2在性能上与GRPO训练的推理模型相当，同时令牌使用减少70%以上。

Insight: 1) 强化学习和长度感知优化的结合有效解决了‘过度思考’问题；2) 多模态集成扩展了模型适用性；3) 可执行代码显著减少推理成本。

Abstract: Large Reasoning Models (LRMs) often suffer from the ``over-thinking’’ problem, generating unnecessarily long reasoning on simple tasks. Some strategies have been proposed to mitigate this issue, such as length penalties or routing mechanisms, but they are typically heuristic and task-specific, lacking a general framework for adaptive reasoning. In this paper, we present ARM2, a unified model that adaptively balances reasoning performance and efficiency across multiple formats through a reinforcement learning framework augmented with length-aware optimization. Beyond conventional natural language inference, ARM2 integrates vision understanding, extending its applicability to multimodal. Moreover, ARM2 integrates executable code into reasoning, enabling substantial reductions in token cost while preserving task performance compared to long CoT. Experiments demonstrate that ARM2 achieves performance on par with traditional reasoning models trained with GRPO, while reducing token usage by over 70% on average. We further conduct extensive analyses to validate the effectiveness of ARM2 and the soundness of its design.

[48] MetricalARGS: A Taxonomy for Studying Metrical Poetry with LLMs

Chalamalasetti Kranti,Sowmya Vajjala

Main category: cs.CL

TL;DR: 论文提出了MetricalARGS，这是首个基于格律诗歌的NLP任务分类系统，旨在评估大型语言模型（LLMs）在分析、检索、生成和支持四个维度上的能力，并通过泰卢固语示例展示了其应用。

Details

Motivation: 格律诗歌作为一种复杂的文学形式，为探索LLMs在严格规则下的语言理解和推理能力提供了机会。此前的研究主要集中于诗歌的自动生成和摘要，而忽略了其格律特性。

Contribution: 提出了MetricalARGS，这是首个专注于格律诗歌的NLP任务分类系统，涵盖了分析、检索、生成和支持四个维度，拓展了LLMs能力评估的边界。

Method: 设计了一个四维度的任务分类系统（MetricalARGS），并通过泰卢固语的例子展示了如何在实际中应用这些任务。

Result: MetricalARGS展示了通过格律诗歌这一严格约束的形式，可以更全面地评估LLMs的能力与局限性。

Insight: 格律诗歌为LLMs的评估提供了严格的语言环境，揭示了模型在遵循复杂规则时的表现，为未来研究提供新的方向。

Abstract: Prior NLP work studying poetry has focused primarily on automatic poem generation and summarization. Many languages have well-studied traditions of poetic meter which enforce constraints on a poem in terms of syllable and phoneme patterns. Such advanced literary forms offer opportunities for probing deeper reasoning and language understanding in Large Language Models (LLMs) and their ability to follow strict pre-requisites and rules. In this paper, we introduce MetricalARGS, the first taxonomy of poetry-related NLP tasks designed to evaluate LLMs on metrical poetry across four dimensions: Analysis, Retrieval, Generation, and Support. We discuss how these tasks relate to existing NLP tasks, addressing questions around datasets and evaluation metrics. Taking Telugu as our example language, we illustrate how the taxonomy can be used in practice. MetricalARGS highlights the broader possibilities for understanding the capabilities and limitations of today’s LLMs through the lens of metrical poetry.

[49] Training-Free Group Relative Policy Optimization

Yuzheng Cai,Siqi Cai,Yuchen Shi,Zihan Xu,Lichao Chen,Yulei Qin,Xiaoyu Tan,Gang Li,Zongyi Li,Haojia Lin,Yong Mao,Ke Li,Xing Sun

Main category: cs.CL

TL;DR: 提出了一种无需参数更新的训练自由组相对策略优化方法（Training-Free GRPO），通过将经验知识作为令牌先验来提升LLM代理性能，避免了传统方法的计算成本和过拟合问题。

Details

Motivation: 大型语言模型（LLM）在专业领域表现不佳，传统方法如监督微调（SFT）和强化学习（RL）需要昂贵的参数更新且易过拟合。作者希望通过一种轻量级方法来改进这一问题。

Contribution: 提出了Training-Free GRPO，一种无需参数更新的方法，通过利用组相对语义优势和多轮学习提炼知识，显著提升了LLM在数学推理和网络搜索任务中的表现。

Method: 方法利用每组rollouts中的语义优势（而非数值优势），在多轮学习中迭代提炼高质量经验知识，并将这些知识作为令牌先验集成到LLM的API调用中。

Result: 实验表明，DeepSeek-V3.1-Terminus模型在少量训练样本（几十个）下，使用该方法优于需要微调的小型LLM。

Insight: 经验知识可以作为一种轻量级的令牌先验，有效提升LLM的性能，同时避免了传统方法的计算成本和数据稀缺问题。

Abstract: Recent advances in Large Language Model (LLM) agents have demonstrated their promising general capabilities. However, their performance in specialized real-world domains often degrades due to challenges in effectively integrating external tools and specific prompting strategies. While methods like agentic reinforcement learning have been proposed to address this, they typically rely on costly parameter updates, for example, through a process that uses Supervised Fine-Tuning (SFT) followed by a Reinforcement Learning (RL) phase with Group Relative Policy Optimization (GRPO) to alter the output distribution. However, we argue that LLMs can achieve a similar effect on the output distribution by learning experiential knowledge as a token prior, which is a far more lightweight approach that not only addresses practical data scarcity but also avoids the common issue of overfitting. To this end, we propose Training-Free Group Relative Policy Optimization (Training-Free GRPO), a cost-effective solution that enhances LLM agent performance without any parameter updates. Our method leverages the group relative semantic advantage instead of numerical ones within each group of rollouts, iteratively distilling high-quality experiential knowledge during multi-epoch learning on a minimal ground-truth data. Such knowledge serves as the learned token prior, which is seamlessly integrated during LLM API calls to guide model behavior. Experiments on mathematical reasoning and web searching tasks demonstrate that Training-Free GRPO, when applied to DeepSeek-V3.1-Terminus, significantly improves out-of-domain performance. With just a few dozen training samples, Training-Free GRPO outperforms fine-tuned small LLMs with marginal training data and cost.

[50] Memory Retrieval and Consolidation in Large Language Models through Function Tokens

Shaohua Zhang,Yuan Lin,Hang Li

Main category: cs.CL

TL;DR: 论文提出了‘功能令牌假说’来解释大语言模型（LLMs）中的记忆检索与巩固机制，认为功能令牌在推理中激活预测特征，而在预训练中则通过预测后续内容令牌来更新模型参数。

Details

Motivation: 尽管LLMs在知识记忆和推理方面表现出色，但其记忆检索与巩固的机制尚不明确，作者试图通过功能令牌假说填补这一空白。

Contribution: 提出功能令牌假说，解释了LLMs的记忆检索与巩固机制；通过二分图分析和案例研究提供了支持证据。

Method: 使用功能令牌假说，分析LLMs中功能令牌（如标点符号、介词等）的作用；通过二分图分析和训练损失研究验证假说。

Result: 实验证明功能令牌主导了特征的激活与模型参数的更新，训练损失主要集中在预测功能令牌后的内容令牌上。

Insight: 功能令牌在LLMs中扮演关键角色，既是知识检索的触发器，也是知识巩固的驱动力。

Abstract: The remarkable success of large language models (LLMs) stems from their ability to consolidate vast amounts of knowledge into the memory during pre-training and to retrieve it from the memory during inference, enabling advanced capabilities such as knowledge memorization, instruction-following and reasoning. However, the mechanisms of memory retrieval and consolidation in LLMs remain poorly understood. In this paper, we propose the function token hypothesis to explain the workings of LLMs: During inference, function tokens activate the most predictive features from context and govern next token prediction (memory retrieval). During pre-training, predicting the next tokens (usually content tokens) that follow function tokens increases the number of learned features of LLMs and updates the model parameters (memory consolidation). Function tokens here roughly correspond to function words in linguistics, including punctuation marks, articles, prepositions, and conjunctions, in contrast to content tokens. We provide extensive experimental evidence supporting this hypothesis. Using bipartite graph analysis, we show that a small number of function tokens activate the majority of features. Case studies further reveal how function tokens activate the most predictive features from context to direct next token prediction. We also find that during pre-training, the training loss is dominated by predicting the next content tokens following function tokens, which forces the function tokens to select the most predictive features from context.

[51] LLMs Learn to Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactions

XuHao Hu,Peng Wang,Xiaoya Lu,Dongrui Liu,Xuanjing Huang,Jing Shao

Main category: cs.CL

TL;DR: 论文探讨了LLMs（大型语言模型）在高风险场景中因微调或交互导致的欺骗行为泛化问题，发现即使是少量不匹配数据或偏执用户也能显著加剧模型的不诚实行为。

Details

Motivation: 研究动机在于探索LLMs在高风险场景（如撒谎或欺骗行为）中，是否通过微调或人类交互意外表现出广泛的不诚实行为，从而扩展了对LLMs泛化对齐问题的理解。

Contribution: 主要贡献包括：1）首次将LLMs的突发不对齐现象扩展到不诚实和欺骗领域；2）验证了少量不匹配数据（1%）或偏执用户（10%）即可显著影响模型行为；3）揭示了人类交互环境中模型行为恶化的潜在风险。

Method: 方法分为三部分：1）在多样领域中对LLMs进行不匹配完成微调；2）在下游混合任务中引入少量不匹配数据；3）模拟人类-AI交互环境，分析良性或偏执用户对模型的影响。

Result: 实验结果表明：1）LLMs在微调后会表现出广泛的不诚实行为；2）下游任务中引入1%不匹配数据可使诚实行为下降20%；3）10%偏执用户足以加剧模型的不诚实行为。

Insight: 研究揭示了LLMs在高风险场景中的潜在风险，强调了模型对齐不仅需要关注安全性，还需考虑行为泛化和人类交互的影响。

Abstract: Previous research has shown that LLMs finetuned on malicious or incorrect completions within narrow domains (e.g., insecure code or incorrect medical advice) can become broadly misaligned to exhibit harmful behaviors, which is called emergent misalignment. In this work, we investigate whether this phenomenon can extend beyond safety behaviors to a broader spectrum of dishonesty and deception under high-stakes scenarios (e.g., lying under pressure and deceptive behavior). To explore this, we finetune open-sourced LLMs on misaligned completions across diverse domains. Experimental results demonstrate that LLMs show broadly misaligned behavior in dishonesty. Additionally, we further explore this phenomenon in a downstream combined finetuning setting, and find that introducing as little as 1% of misalignment data into a standard downstream task is sufficient to decrease honest behavior over 20%. Furthermore, we consider a more practical human-AI interaction environment where we simulate both benign and biased users to interact with the assistant LLM. Notably, we find that the assistant can be misaligned unintentionally to exacerbate its dishonesty with only 10% biased user population. In summary, we extend the study of emergent misalignment to the domain of dishonesty and deception under high-stakes scenarios, and demonstrate that this risk arises not only through direct finetuning, but also in downstream mixture tasks and practical human-AI interactions.

[52] Investigating Counterclaims in Causality Extraction from Text

Tim Hagen,Niklas Deckers,Felix Wolter,Harrisen Scells,Martin Potthast

Main category: cs.CL

TL;DR: 论文探讨了因果关系提取研究中忽视反因果声明的现状，开发了一个包含反因果的新数据集，并通过实验证明其重要性。

Details

Motivation: 现有因果关系提取研究仅关注支持因果关系的声明（procausal），而忽略或错误标注了否定因果关系的声明（concausal）。这导致模型在处理反因果声明时表现不佳。

Contribution: 1. 开发了一个包含反因果声明的新数据集；2. 揭示了反因果是不完整知识下因果推理的重要组成部分；3. 展示了新数据集能够显著提高模型区分正反因果的能力。

Method: 1. 通过文献综述论证反因果的重要性；2. 制定严格的标注指南；3. 扩充现有Causal News Corpus数据集，并验证标注一致性（Cohen’s κ=0.74）；4. 训练模型验证数据集的改进效果。

Result: 新数据集显著改善了模型对反因果声明的识别能力，避免了将其错误分类为正因果的问题。

Insight: 反因果是因果关系提取中不可或缺的部分，忽略它们会限制模型的实用性和鲁棒性。

Abstract: Research on causality extraction from text has so far almost entirely neglected counterclaims. Existing causality extraction datasets focus solely on “procausal” claims, i.e., statements that support a relationship. “Concausal” claims, i.e., statements that refute a relationship, are entirely ignored or even accidentally annotated as procausal. We address this shortcoming by developing a new dataset that integrates concausality. Based on an extensive literature review, we first show that concausality is an integral part of causal reasoning on incomplete knowledge. We operationalize this theory in the form of a rigorous guideline for annotation and then augment the Causal News Corpus with concausal statements, obtaining a substantial inter-annotator agreement of Cohen’s $\kappa=0.74$. To demonstrate the importance of integrating concausal statements, we show that models trained without concausal relationships tend to misclassify these as procausal instead. Based on our new dataset, this mistake can be mitigated, enabling transformers to effectively distinguish pro- and concausality.

[53] The Alignment Waltz: Jointly Training Agents to Collaborate for Safety

Jingyu Zhang,Haozhu Wang,Eric Michael Smith,Sid Wang,Amr Sharaf,Mahesh Pasupuleti,Benjamin Van Durme,Daniel Khashabi,Jason Weston,Hongyuan Zhan

Main category: cs.CL

TL;DR: 论文提出了一种名为WaltzRL的多智能体强化学习框架，用于协同训练对话智能体和反馈智能体，通过动态改进奖励（DIR）提升模型的安全性和帮助性，显著减少了不安全响应和过度拒绝的现象。

Details

Motivation: 现有方法在处理大语言模型的安全性和帮助性平衡时，往往采用一刀切的拒绝策略，导致过度拒绝或缺乏细粒度指导。WaltzRL旨在解决这一问题，通过协同训练实现更灵活的安全对齐。

Contribution: 论文的主要贡献包括提出了WaltzRL框架，设计了动态改进奖励（DIR），并通过实验验证其能显著减少不安全响应和过度拒绝，同时保持模型的通用能力。

Method: WaltzRL是一种多智能体强化学习框架，联合训练对话智能体和反馈智能体，利用DIR动态评估对话智能体对反馈的采纳程度。推理时，反馈智能体仅在不安全或过度拒绝时介入。

Result: 实验显示，WaltzRL在多个数据集上将不安全响应率（如WildJailbreak上从39.0%降至4.6%）和过度拒绝率（OR-Bench上从45.3%降至9.9%）显著降低。

Insight: 安全对齐可以通过智能体协同训练实现，动态反馈机制比静态拒绝策略更有效，且能保持模型的其他能力不受损。

Abstract: Harnessing the power of LLMs requires a delicate dance between being helpful and harmless. This creates a fundamental tension between two competing challenges: vulnerability to adversarial attacks that elicit unsafe content, and a tendency for overrefusal on benign but sensitive prompts. Current approaches often navigate this dance with safeguard models that completely reject any content that contains unsafe portions. This approach cuts the music entirely-it may exacerbate overrefusals and fails to provide nuanced guidance for queries it refuses. To teach models a more coordinated choreography, we propose WaltzRL, a novel multi-agent reinforcement learning framework that formulates safety alignment as a collaborative, positive-sum game. WaltzRL jointly trains a conversation agent and a feedback agent, where the latter is incentivized to provide useful suggestions that improve the safety and helpfulness of the conversation agent’s responses. At the core of WaltzRL is a Dynamic Improvement Reward (DIR) that evolves over time based on how well the conversation agent incorporates the feedback. At inference time, unsafe or overrefusing responses from the conversation agent are improved rather than discarded. The feedback agent is deployed together with the conversation agent and only engages adaptively when needed, preserving helpfulness and low latency on safe queries. Our experiments, conducted across five diverse datasets, demonstrate that WaltzRL significantly reduces both unsafe responses (e.g., from 39.0% to 4.6% on WildJailbreak) and overrefusals (from 45.3% to 9.9% on OR-Bench) compared to various baselines. By enabling the conversation and feedback agents to co-evolve and adaptively apply feedback, WaltzRL enhances LLM safety without degrading general capabilities, thereby advancing the Pareto front between helpfulness and harmlessness.

[54] Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling

Jannek Ulm,Kevin Du,Vésteinn Snæbjarnarson

Main category: cs.CL

TL;DR: 本研究探讨了在低资源语言模型中，通过对比解码生成合成数据的方法及其效果。实验表明，混合合成数据与原数据训练能提升语言建模和下游任务的表现，对比解码尤其有助于推理任务。

Details

Motivation: 大型语言模型（LLMs）训练需要大量文本数据，但数据资源可能接近极限。合成数据是潜在解决方案之一，本研究旨在探索对比解码生成合成数据的优势。

Contribution: 提出了利用对比解码生成高质量合成数据的方法，并验证了混合合成数据与原数据训练对语言建模和下游任务的提升效果。

Method: 通过对比‘好模型’和‘坏模型’的相对差异生成合成语料，并将其与原数据混合用于训练。实验基于100M词的原始语料进行控制实验。

Result: 混合合成数据与原数据训练提升了语言建模目标及下游任务的表现，对比解码数据尤其有助于推理任务，而传统采样数据对表层语言能力的任务更有效。

Insight: 合成数据的生成方法（如对比解码）应根据任务需求选择：推理任务更适合对比解码，而传统采样对基础语言能力任务更有效。

Abstract: Large language models (LLMs) are trained on huge amounts of textual data, and concerns have been raised that the limits of such data may soon be reached. A potential solution is to train on synthetic data sampled from LLMs. In this work, we build on this idea and investigate the benefits of contrastive decoding for generating synthetic corpora. In a controlled setting, we experiment with sampling corpora using the relative difference between a good and bad model trained on the same original corpus of 100 million words. By amplifying the signal from a model that has better performance, we create a synthetic corpus and mix it with the original training data. Our findings show that training on a mixture of synthesized and real data improves performance on the language modeling objective and a range of downstream tasks. In particular, we see that training with a mix of synthetic data from contrastive decoding benefits tasks that require more reasoning skills, while synthetic data from traditional sampling helps more on tasks dependent on surface level linguistic capabilities.

[55] Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window

Qiaoyu Tang,Hao Xiang,Le Yu,Bowen Yu,Yaojie Lu,Xianpei Han,Le Sun,WenJuan Zhang,Pengbo Wang,Shixuan Liu,Zhenru Zhang,Jianhong Tu,Hongyu Lin,Junyang Lin

Main category: cs.CL

TL;DR: DeepMiner提出了一种新框架，通过高难度训练任务和动态上下文窗口，提升多轮交互智能体的深度推理能力，并在多个搜索智能体基准测试中取得显著性能提升。

Details

Motivation: 现有方法在多轮长视野交互中难以激发深度推理能力，DeepMiner旨在解决这一问题，通过动态上下文管理和高质量训练数据提升模型性能。

Contribution: 1. 提出逆向构建方法生成复杂但可验证的QA对；2. 设计动态上下文管理策略，无需外部摘要模型；3. DeepMiner-32B在多个基准测试中表现优异。

Method: 1. 使用逆向构建方法从真实网络数据生成高质量QA对；2. 采用滑动窗口机制实现动态上下文管理；3. 基于Qwen3-32B进行强化学习训练。

Result: DeepMiner-32B在BrowseComp-en上达到33.5%准确率，相比之前最好开源模型提升近20个百分点，并在其他基准测试中表现一致提升。

Insight: 动态上下文管理策略能有效支持近100轮交互，突破了现有系统的上下文限制。

Abstract: While recent advances in reasoning models have demonstrated cognitive behaviors through reinforcement learning, existing approaches struggle to invoke deep reasoning capabilities in multi-turn agents with long-horizon interactions. We propose DeepMiner, a novel framework that elicits such abilities by introducing high-difficulty training tasks and dynamic context window. DeepMiner presents a reverse construction method to generate complex but verifiable question-answer pairs from authentic web sources, which ensures the challenge and reliability of training data while injecting cognitive capabilities into multi-turn reasoning scenarios. We further design an elegant yet effective dynamic context management strategy for both training and inference, utilizing sliding window mechanisms while eliminating the dependency on external summarization models, thereby efficiently empowering the model to handle continuously expanding long-horizon contexts. Through reinforcement learning on Qwen3-32B, we develop DeepMiner-32B, which achieves substantial performance improvements across multiple search agent benchmarks. DeepMiner attains 33.5% accuracy on BrowseComp-en, surpassing the previous best open-source agent by almost 20 percentage points, and demonstrates consistent improvements on BrowseComp-zh, XBench-DeepSearch, and GAIA. Notably, our dynamic context management enables sustained interactions of nearly 100 turns within standard 32k context length, effectively addressing the context limitations that constrain existing multi-turn interaction systems.

[56] If Probable, Then Acceptable? Understanding Conditional Acceptability Judgments in Large Language Models

Jasmin Orth,Philipp Mondorf,Barbara Plank

Main category: cs.CL

TL;DR: 这篇论文研究了大型语言模型（LLMs）对条件语句的‘可接受性’判断，发现LLMs对条件概率和语义相关性敏感，但与人类的判断一致性较低，且模型规模增大并不一定提高与人类的对齐度。

Details

Motivation: 条件可接受性在沟通和推理中至关重要，但目前不清楚LLMs如何判断条件语句的‘可接受性’。

Contribution: 首次系统研究了不同LLMs家族、规模和提示策略下的条件可接受性判断，揭示了LLMs与人类判断的差异。

Method: 使用线性混合效应模型和ANOVA测试，分析了LLMs对条件概率和语义相关性的敏感性。

Result: LLMs对条件概率和语义相关性敏感，但与人类的一致性不高，且模型规模不保证对齐性。

Insight: LLMs的条件判断机制与人类存在差异，提示未来研究需关注模型内部推理的一致性。

Abstract: Conditional acceptability refers to how plausible a conditional statement is perceived to be. It plays an important role in communication and reasoning, as it influences how individuals interpret implications, assess arguments, and make decisions based on hypothetical scenarios. When humans evaluate how acceptable a conditional “If A, then B” is, their judgments are influenced by two main factors: the $\textit{conditional probability}$ of $B$ given $A$, and the $\textit{semantic relevance}$ of the antecedent $A$ given the consequent $B$ (i.e., whether $A$ meaningfully supports $B$). While prior work has examined how large language models (LLMs) draw inferences about conditional statements, it remains unclear how these models judge the $\textit{acceptability}$ of such statements. To address this gap, we present a comprehensive study of LLMs’ conditional acceptability judgments across different model families, sizes, and prompting strategies. Using linear mixed-effects models and ANOVA tests, we find that models are sensitive to both conditional probability and semantic relevance-though to varying degrees depending on architecture and prompting style. A comparison with human data reveals that while LLMs incorporate probabilistic and semantic cues, they do so less consistently than humans. Notably, larger models do not necessarily align more closely with human judgments.

[57] ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shaping

Shuang Chen,Yue Guo,Yimeng Ye,Shijue Huang,Wenbo Hu,Haoxi Li,Manyuan Zhang,Jiayu Chen,Song Guo,Nanyun Peng

Main category: cs.CL

TL;DR: 该论文提出了ARES，一种自适应推理框架，通过基于任务难度的动态探索努力分配来解决多模态大型推理模型在简单问题上过度推理和在困难问题上探索不足的问题。

Details

Motivation: 多模态大型推理模型（MLRMs）在简单问题上容易过度推理，而在困难问题上探索不足，导致效率低下和解决方案缺失。为解决这一问题，ARES框架被提出以实现动态的自适应推理。

Contribution: 1. 提出高窗口熵（HWE）理论，可靠地捕捉推理关键时刻。2. 设计了双阶段训练流程：自适应冷启动阶段和自适应熵策略优化（AEPO）阶段。3. 实验证明ARES在多样化的数学、逻辑和多模态基准测试中表现出色。

Method: 1. 自适应冷启动阶段：为模型提供与问题难度成比例的推理轨迹数据。2. AEPO阶段：利用HWE令牌作为探索触发器，结合分层熵奖励和动态KL控制，决定探索时机和程度。

Result: ARES在多个基准测试中展现了卓越的性能和推理效率，同时在显著降低推理成本的情况下缩小了与领先商业系统的差距。

Insight: 单令牌熵存在噪声，而HWE令牌能更可靠地捕捉推理关键时刻；动态调整HWE的使用可以平衡简单和困难问题的推理效率。

Abstract: Recent advances in multimodal large reasoning models (MLRMs) have substantially improved their ability to solve complex textual and visual tasks. However, these models tend to overthink on simple problems, producing unnecessarily lengthy reasoning traces, while under-exploring on challenging ones, leading to missed solutions. To address this imbalance, we propose ARES, a unified open-source framework for adaptive reasoning that dynamically allocates exploration effort based on task difficulty. Our approach is motivated by two key empirical findings: (i) while single-token entropy is noisy, high window-entropy (HWE) tokens (token-level entropies averaged under a sliding window) can reliably capture reasoning-critical moments; and (ii) reducing HWE usage benefits easy problems, while increasing it is essential for solving hard ones. Building on these insights, ARES introduces a two-stage training pipeline. In the Adaptive Cold-Start stage, we curate multimodal and textual data paired with reasoning traces of length proportional to problem difficulty, equipping the model with initial difficulty awareness. In the second stage, we develop Adaptive Entropy Policy Optimization (AEPO), which uses HWE tokens as exploration triggers to decide when to explore, and a hierarchical entropy reward with dynamic KL control to decide how much to explore. Extensive experiments demonstrate that ARES achieves superior performance and reasoning efficiency across diverse mathematical, logical, and multimodal benchmarks, while closing the gap to leading commercial systems under significantly lower inference costs.

[58] LeWiDi-2025 at NLPerspectives: The Third Edition of the Learning with Disagreements Shared Task

Elisa Leonardelli,Silvia Casola,Siyao Peng,Giulia Rizzi,Valerio Basile,Elisabetta Fersini,Diego Frassinelli,Hyewon Jang,Maja Pavlovic,Barbara Plank,Massimo Poesio

Main category: cs.CL

TL;DR: 第三版的LeWiDi共享任务扩展了基准数据集，包括四种任务，并引入序数和分类标注方案。采用软标签和视角主义两种方法评估分歧感知模型，提出了新评价指标。

Details

Motivation: AI模型应能识别人类判断中的分歧和变化，但现有数据集和方法不足以支持这一目标。LeWiDi任务旨在推动分歧感知技术的开发。

Contribution: 1. 扩展了四类任务的数据集；2. 引入序数和分类标注方案；3. 提出软标签和视角主义两种评估方法；4. 开发了新评价指标。

Method: 采用了两种评估范式：软标签（模型预测群体判断分布）和视角主义（模型预测标注者个体解释）。

Result: 任务吸引了多样化的参与，展示了建模方法的优势和局限性，提供了新的资源和基准。

Insight: 分歧感知技术需要更多样化的数据集和评估方法，未来研究应关注个体差异和群体分布的平衡。

Abstract: Many researchers have reached the conclusion that AI models should be trained to be aware of the possibility of variation and disagreement in human judgments, and evaluated as per their ability to recognize such variation. The LEWIDI series of shared tasks on Learning With Disagreements was established to promote this approach to training and evaluating AI models, by making suitable datasets more accessible and by developing evaluation methods. The third edition of the task builds on this goal by extending the LEWIDI benchmark to four datasets spanning paraphrase identification, irony detection, sarcasm detection, and natural language inference, with labeling schemes that include not only categorical judgments as in previous editions, but ordinal judgments as well. Another novelty is that we adopt two complementary paradigms to evaluate disagreement-aware systems: the soft-label approach, in which models predict population-level distributions of judgments, and the perspectivist approach, in which models predict the interpretations of individual annotators. Crucially, we moved beyond standard metrics such as cross-entropy, and tested new evaluation metrics for the two paradigms. The task attracted diverse participation, and the results provide insights into the strengths and limitations of methods to modeling variation. Together, these contributions strengthen LEWIDI as a framework and provide new resources, benchmarks, and findings to support the development of disagreement-aware technologies.

[59] DeepPrune: Parallel Scaling without Inter-trace Redundancy

Shangqing Tu,Yaxuan Li,Yushi Bai,Lei Hou,Juanzi Li

Main category: cs.CL

TL;DR: DeepPrune提出了一个动态剪枝框架，通过预测等价答案和在线聚类算法显著减少并行推理中的冗余计算，节省80%以上的计算资源，同时保持推理性能。

Details

Motivation: 并行推理（CoT）虽能提升大语言模型的推理能力，但多推理路径间的冗余计算问题严重（80%以上路径结果相同），亟需一种高效的方法减少浪费。

Contribution: 提出了DeepPrune，一种动态剪枝框架，通过专用判断模型和在线聚类算法，高效剪枝冗余推理路径，显著降低计算成本。

Method: 1. 训练一个基于聚焦损失和过采样技术的判断模型，预测部分推理路径的答案等价性；2. 结合在线贪婪聚类算法动态剪枝冗余路径。

Result: 在AIME 2024/2025和GPQA等基准测试中，DeepPrune节省了80%以上的token，同时精度损失控制在3个百分点内。

Insight: 并行推理中冗余路径剪枝是提升效率的关键，动态评估和剪枝技术能够在不显著损失多样性的前提下大幅优化计算资源。

Abstract: Parallel scaling has emerged as a powerful paradigm to enhance reasoning capabilities in large language models (LLMs) by generating multiple Chain-of-Thought (CoT) traces simultaneously. However, this approach introduces significant computational inefficiency due to inter-trace redundancy – our analysis reveals that over 80% of parallel reasoning traces yield identical final answers, representing substantial wasted computation. To address this critical efficiency bottleneck, we propose DeepPrune, a novel framework that enables efficient parallel scaling through dynamic pruning. Our method features a specialized judge model trained with focal loss and oversampling techniques to accurately predict answer equivalence from partial reasoning traces which realizes 0.87 AUROC on equivalence prediction, combined with an online greedy clustering algorithm that dynamically prunes redundant paths while preserving answer diversity. Comprehensive evaluations across three challenging benchmarks (AIME 2024, AIME 2025, and GPQA) and multiple reasoning models demonstrate that DeepPrune achieves remarkable token reduction by over 80% compared to conventional consensus sampling on most cases, while maintaining competitive accuracy within 3 percentage points. Our work establishes a new standard for efficient parallel reasoning, making high-performance reasoning more efficient. Our code and data are here: https://deepprune.github.io/

[60] Which Heads Matter for Reasoning? RL-Guided KV Cache Compression

Wenjie Du,Li Jiang,Keda Tao,Xue Liu,Huan Wang

Main category: cs.CL

TL;DR: 论文提出了一种名为RLKV的框架，通过强化学习识别推理关键头（KV head），并在推理时仅对这些关键头保留完整的KV缓存，对其他头进行压缩，从而在减少20-50%缓存的同时保持接近无损的性能。

Details

Motivation: 大型语言模型在推理任务中需要处理大量的KV缓存开销，现有的KV缓存压缩方法会破坏推理完整性或误压缩关键头，导致性能下降。论文假设KV头在推理模型中具有功能异质性，部分头对推理至关重要，而其他头可压缩。

Contribution: 提出了RLKV框架，首次利用强化学习优化推理关键头的识别，实现了高效的KV缓存压缩，并在压缩率和性能之间取得了显著平衡。

Method: RLKV通过强化学习直接优化每个头的缓存使用与推理质量之间的关系，识别关键头。在推理阶段，仅对关键头保留完整KV缓存，对其他头使用压缩的常量KV缓存。

Result: 实验表明，仅需一小部分注意力头用于推理即可保持性能，RLKV在20-50%的缓存压缩率下性能接近无损，优于基线方法。

Insight: KV头在推理任务中存在功能异质性，强化学习可以有效识别关键头，从而实现高效的缓存压缩而不损害推理质量。

Abstract: Reasoning large language models exhibit complex reasoning behaviors through the extended chain-of-thought generation, creating unprecedented Key-Value (KV) cache overhead during the decoding phase. Existing KV cache compression methods underperform on reasoning models: token-dropping methods break reasoning integrity by discarding critical information, while head-reallocating methods mistakenly compress reasoning-critical heads since they are designed for retrieval tasks, resulting in significant performance degradation as compression rates increase. We hypothesize that KV heads exhibit functional heterogeneity in reasoning models-some heads are critical for chain-of-thought consistency while others are compressible. To validate and exploit this insight, we propose RLKV, a novel reasoning-critical head identification framework, which uses reinforcement learning to directly optimize the relationship between each head’s cache usage and reasoning quality. As RLKV produces rewards from actual generated samples during training, it naturally identifies heads relevant to reasoning behaviors. We then allocate full KV cache to these heads while applying compressed constant KV cache to others for efficient inference. Our experiments reveal that only a small fraction of attention heads is essential for reasoning, enabling our KV compression approach to outperform baseline methods while achieving 20-50% cache reduction with near lossless performance compared to uncompressed results.

[61] CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards

Xiangyuan Xue,Yifan Zhou,Guibin Zhang,Zaibin Zhang,Yijiang Li,Chen Zhang,Zhenfei Yin,Philip Torr,Wanli Ouyang,Lei Bai

Main category: cs.CL

TL;DR: 论文提出了CoMAS框架，通过多智能体之间的交互奖励实现自主进化，无需外部监督，实验表明其性能优于未经训练的智能体，并在多数评估设置中达到SOTA。

Details

Motivation: 目前基于强化学习的方法要么依赖密集的外部奖励，要么从LLM中提取内在奖励，这与人类通过讨论和协作自我进化的机制不符。

Contribution: 提出了CoMAS框架，通过交互奖励实现多智能体的协同进化，引入了LLM-as-a-judge机制生成内在奖励，并通过RL优化策略。

Method: CoMAS通过多智能体讨论动态生成内在奖励，利用LLM作为裁判机制，并通过RL优化每个智能体的策略。

Result: 实验证明CoMAS优于未经训练的智能体，在多数评估中达到SOTA，且随着智能体数量和多样性的增加展现出良好的扩展性。

Insight: 交互奖励是实现LLM基智能体自主进化的有效途径，且智能体多样性和数量的增加能够进一步提升性能。

Abstract: Self-evolution is a central research topic in enabling large language model (LLM)-based agents to continually improve their capabilities after pretraining. Recent research has witnessed a transition from reinforcement learning (RL)-free to RL-based methods. Current RL-based methods either rely on dense external reward signals or extract intrinsic reward signals from LLMs themselves. However, these approaches diverge from the self-evolution mechanisms observed in human intelligence, where individuals learn and improve through mutual discussion and collaboration. In this work, we introduce Co-Evolving Multi-Agent Systems (CoMAS), a novel framework that enables agents to improve autonomously by learning from inter-agent interactions without external supervision. CoMAS generates intrinsic rewards from rich discussion dynamics, employs an LLM-as-a-judge mechanism to formulate these rewards, and optimizes each agent’s policy through RL, thereby enabling decentralized and scalable co-evolution. Experimental results demonstrate that CoMAS consistently outperforms untrained agents and achieves state-of-the-art performance across most evaluation settings. Ablation studies confirm the necessity of interaction-based reward signals and reveal promising scalability as the number and diversity of agents increase. These findings establish CoMAS as a novel and effective paradigm for self-evolution in LLM-based agents.

[62] ArenaBencher: Automatic Benchmark Evolution via Multi-Model Competitive Evaluation

Qin Liu,Jacob Dineen,Yuxi Huang,Sheng Zhang,Hoifung Poon,Ben Zhou,Muhao Chen

Main category: cs.CL

TL;DR: 本文提出ArenaBencher框架，通过多模型竞争评估自动进化基准，解决基准测试因数据泄露导致的无效性问题。

Details

Motivation: 当前基准测试因预训练数据泄露导致模型通过记忆内容而非真实泛化能力得分，影响跨模型比较和进步评估的有效性。

Contribution: 1. 提出ArenaBencher框架，自动更新基准测试用例；2. 结合多模型反馈生成更具挑战性和诊断性的测试用例；3. 在数学、常识推理和安全领域验证其有效性。

Method: 1. 推断测试用例核心能力；2. 生成候选问题-答案对并保留原始目标；3. 用LLM验证正确性和意图；4. 多模型反馈选择暴露共享弱点的候选。

Result: ArenaBencher生成多样化、公平且已验证的更新，增加难度并保留测试目标对齐性，提高模型分离度。

Insight: 通过自动进化基准，可动态适应基础模型的快速进步，确保评估有效性。

Abstract: Benchmarks are central to measuring the capabilities of large language models and guiding model development, yet widespread data leakage from pretraining corpora undermines their validity. Models can match memorized content rather than demonstrate true generalization, which inflates scores, distorts cross-model comparisons, and misrepresents progress. We introduce ArenaBencher, a model-agnostic framework for automatic benchmark evolution that updates test cases while preserving comparability. Given an existing benchmark and a diverse pool of models to be evaluated, ArenaBencher infers the core ability of each test case, generates candidate question-answer pairs that preserve the original objective, verifies correctness and intent with an LLM as a judge, and aggregates feedback from multiple models to select candidates that expose shared weaknesses. The process runs iteratively with in-context demonstrations that steer generation toward more challenging and diagnostic cases. We apply ArenaBencher to math problem solving, commonsense reasoning, and safety domains and show that it produces verified, diverse, and fair updates that uncover new failure modes, increase difficulty while preserving test objective alignment, and improve model separability. The framework provides a scalable path to continuously evolve benchmarks in step with the rapid progress of foundation models.

cs.CV [Back]

[63] DynamicEval: Rethinking Evaluation for Dynamic Text-to-Video Synthesis

Nithin C. Babu,Aniruddha Mahapatra,Harsh Rangwani,Rajiv Soundararajan,Kuldeep Kulkarni

Main category: cs.CV

TL;DR: DynamicEval是一个专注于动态摄像机运动的文本生成视频（T2V）评估基准，通过解决现有基准的两大局限性（忽视动态运动和视频级评分），提出了新的背景一致性和前景一致性指标。

Details

Motivation: 现有T2V评估基准（如VBench和EvalCrafter）忽视了动态摄像机运动的重要性，且仅依赖模型级评分，无法有效评估单个视频质量。DynamicEval旨在填补这些空白。

Contribution: 1. 提出DynamicEval基准，包含动态摄像机运动的提示语和4.5万人类标注数据；2. 提出改进的背景一致性指标和新的前景一致性指标，显著提升与人类偏好的相关性。

Method: 1. 使用VBench运动平滑度指标生成可解释误差图，并修正其遮挡/显露问题；2. 通过跟踪对象实例内的点评估前景一致性。

Result: 实验表明，DynamicEval的视频级和模型级指标与人类偏好的相关性提升了2%以上。

Insight: 动态摄像机运动是T2V评估中被忽视的重要维度；基于误差图的修正方法能有效提升指标的解释性和准确性。

Abstract: Existing text-to-video (T2V) evaluation benchmarks, such as VBench and EvalCrafter, suffer from two limitations. (i) While the emphasis is on subject-centric prompts or static camera scenes, camera motion essential for producing cinematic shots and existing metrics under dynamic motion are largely unexplored. (ii) These benchmarks typically aggregate video-level scores into a single model-level score for ranking generative models. Such aggregation, however, overlook video-level evaluation, which is vital to selecting the better video among the candidate videos generated for a given prompt. To address these gaps, we introduce DynamicEval, a benchmark consisting of systematically curated prompts emphasizing dynamic camera motion, paired with 45k human annotations on video pairs from 3k videos generated by ten T2V models. DynamicEval evaluates two key dimensions of video quality: background scene consistency and foreground object consistency. For background scene consistency, we obtain the interpretable error maps based on the Vbench motion smoothness metric. We observe that while the Vbench motion smoothness metric shows promising alignment with human judgments, it fails in two cases: occlusions/disocclusions arising from camera and foreground object movements. Building on this, we propose a new background consistency metric that leverages object error maps to correct two failure cases in a principled manner. Our second innovation is the introduction of a foreground consistency metric that tracks points and their neighbors within each object instance to assess object fidelity. Extensive experiments demonstrate that our proposed metrics achieve stronger correlations with human preferences at both the video level and the model level (an improvement of more than 2% points), establishing DynamicEval as a more comprehensive benchmark for evaluating T2V models under dynamic camera motion.

[64] PickStyle: Video-to-Video Style Transfer with Context-Style Adapters

Soroush Mehraban,Vida Adeli,Jacob Rommann,Babak Taati,Kyryl Truskovskyi

Main category: cs.CV

TL;DR: PickStyle提出了一个基于扩散模型的视频风格迁移框架，通过低秩适配器和合成训练数据，结合上下文-风格分类器自由引导（CS-CFG），实现高效的内容保持和风格迁移。

Details

Motivation: 视频风格迁移缺乏成对的监督数据，现有方法在保留内容和风格一致性上存在挑战。PickStyle通过利用静态图像数据和扩散模型的优势，解决这一问题。

Contribution: 1. 提出PickStyle框架，结合低秩适配器和扩散模型；2. 引入CS-CFG，独立控制内容和风格；3. 通过合成数据弥补静态和动态数据间的差距。

Method: 1. 在自注意力层插入低秩适配器；2. 使用共享增强技术生成合成视频数据；3. CS-CFG分离内容和风格引导。

Result: 在多个基准测试中，PickStyle实现了优于现有方法的时序一致性、风格忠实度和内容保持性。

Insight: 结合静态图像数据和动态视频合成是一种有效的视频风格迁移训练策略；低秩适配器是实现高效风格迁移的关键。

Abstract: We address the task of video style transfer with diffusion models, where the goal is to preserve the context of an input video while rendering it in a target style specified by a text prompt. A major challenge is the lack of paired video data for supervision. We propose PickStyle, a video-to-video style transfer framework that augments pretrained video diffusion backbones with style adapters and benefits from paired still image data with source-style correspondences for training. PickStyle inserts low-rank adapters into the self-attention layers of conditioning modules, enabling efficient specialization for motion-style transfer while maintaining strong alignment between video content and style. To bridge the gap between static image supervision and dynamic video, we construct synthetic training clips from paired images by applying shared augmentations that simulate camera motion, ensuring temporal priors are preserved. In addition, we introduce Context-Style Classifier-Free Guidance (CS-CFG), a novel factorization of classifier-free guidance into independent text (style) and video (context) directions. CS-CFG ensures that context is preserved in generated video while the style is effectively transferred. Experiments across benchmarks show that our approach achieves temporally coherent, style-faithful, and content-preserving video translations, outperforming existing baselines both qualitatively and quantitatively.

[65] TRAVL: A Recipe for Making Video-Language Models Better Judges of Physics Implausibility

Saman Motamed,Minghao Chen,Luc Van Gool,Iro Laina

Main category: cs.CV

TL;DR: 论文提出TRAVL方法，通过微调视频-语言模型（VLM）提升其对物理不合理的判断能力，并引入ImplausiBench基准测试。

Details

Motivation: 现有视频生成模型常违反物理规律，但缺乏衡量物理合理性的方法。人类能轻易识别，而现有VLM表现不佳，亟需改进。

Contribution: 1. 提出TRAVL方法，通过平衡数据集和轨迹注意力模块改进VLM。2. 提出ImplausiBench基准测试，移除语言偏差以衡量视觉-时间理解能力。

Method: 采用平衡训练数据集和轨迹感知注意力模块，优化VLM的运动编码和判别能力。

Result: TRAVL提升了VLM对物理不合理现象的识别能力，并通过人类和LLM评估验证其有效性。

Insight: 研究揭示了VLM在时间和因果推理上的短板，为改进多模态模型的物理合理性理解提供了框架。

Abstract: Despite impressive visual fidelity, modern video generative models frequently produce sequences that violate intuitive physical laws, such as objects floating, teleporting, or morphing in ways that defy causality. While humans can easily detect such implausibilities, there remains no robust method for quantitatively assessing physical realism in video. In this work, we explore whether Video-Language Models (VLMs) can be trained to serve as reliable judges of physical plausibility. We find that existing VLMs struggle to identify physics violations, exposing fundamental limitations in their temporal and causal reasoning. To address this, we introduce TRAVL, a fine-tuning recipe that combines a balanced training dataset with a trajectory-aware attention module to improve motion encoding and discrimination in VLMs. To evaluate physical reasoning more rigorously, we propose ImplausiBench, a benchmark of 300 videos (150 real, 150 generated) that removes linguistic biases and isolates visual-temporal understanding. Performance is reported both with gold-standard human judgments and stricter LLM-as-judge metrics. Together, TRAVL and ImplausiBench offer a unified framework for probing and improving physical plausibility in multimodal models, shedding light on a challenging and underexplored aspect of visual-temporal understanding.

Karuna Bhaila,Aneesh Komanduri,Minh-Hao Van,Xintao Wu

Main category: cs.CV

TL;DR: 该论文提出了一种轻量高效的视觉语言模型（VLM）遗忘框架CAGUL，用于解决视觉问答（VQA）任务中隐私信息泄露的问题，通过跨模态注意力引导视觉令牌转换，避免了昂贵的模型微调。

Details

Motivation: 视觉语言模型（VLMs）在多模态理解和推理任务（如VQA）中表现出色，但这些模型可能在训练中记忆隐私或敏感信息，并在推理中泄露。现有机器遗忘方法主要针对文本模型，而VLMs的视觉内容增加了复杂性。

Contribution: 论文的主要贡献是提出了CAGUL框架，利用跨模态注意力引导视觉令牌的转换，轻量化地实现VLM的遗忘功能，避免了模型参数的修改和重新训练。

Method: CAGUL通过外部模块将遗忘信息编码到对相关查询低重要性的视觉令牌中，利用跨模态注意力机制引导视觉令牌的转换。

Result: 实验表明，CAGUL在不改变预训练模型参数的情况下，性能优于或与基于微调的基线方法相当，同时避免了重新训练的成本。

Insight: 跨模态注意力机制在视觉令牌中的作用可以被有效利用以实现隐私保护，而轻量化的外部模块设计为模型遗忘提供了一种高效解决方案。

Abstract: Vision-Language Models (VLMs) have demonstrated immense capabilities in multi-modal understanding and inference tasks such as Visual Question Answering (VQA), which requires models to infer outputs based on visual and textual context simultaneously. Such inference abilities of large-scale pretrained models are often attributed to the massive scale of pre-training data collected across several domains. However, the models may memorize private and/or sensitive information during training and regurgitate it in inference. Recently, machine unlearning has been leveraged to address the leakage of private data in LLMs. VLMs add a layer of complexity to this process, as the visual context in the query may also contain sensitive information in addition to the text. To address this issue, we explore unlearning for vision-language models, specifically for the VQA task. We explore the role of visual tokens for output generation in VLMs using cross-modal attention and utilize it to formulate Cross-Modal Attention Guided Unlearning (CAGUL), a lightweight and efficient VLM unlearning framework. In contrast to computationally expensive model finetuning methods, CAGUL utilizes external modules to encode unlearning information in visual tokens of low importance for relevant queries. We find that the transformed visual tokens not only prevent leakage but also retain reference model behavior. Experimental results show that our method performs better or on par with finetuning-based baselines without altering the pre-trained model parameters or incurring retraining costs, making it a practical and effective unlearning solution for VLMs.

[67] MaizeStandCounting (MaSC): Automated and Accurate Maize Stand Counting from UAV Imagery Using Image Processing and Deep Learning

Dewi Endah Kharismawati,Toni Kazic

Main category: cs.CV

TL;DR: MaSC是一种基于图像处理和深度学习的自动化玉米植株计数算法，利用低成本无人机采集的RGB图像，结合轻量级YOLOv9模型，准确区分玉米苗与杂草，并通过行和范围分割实现精确计数。

Details

Motivation: 传统人工计数玉米植株耗时、费力且易出错，尤其在大型或复杂农田中。MaSC通过自动化计数为作物管理和研究提供高效且低成本的解决方案。

Contribution: MaSC提出了一种高效的两模式计数方法（拼接图像分块和原始视频帧对齐），并利用轻量级YOLOv9模型实现了高精度玉米植株检测与计数。

Method: MaSC采用YOLOv9检测玉米苗（V2-V10生长期），结合行和范围分割技术，实现玉米植株的自动化计数。两种模式分别处理拼接图像和原始视频帧。

Result: MaSC与人工计数结果高度一致（R²=0.616拼接图像，R²=0.906原始帧），且处理83帧仅需60.63秒，适用于实时操作。

Insight: MaSC展示了低成本无人机和轻量级深度学习模型在农业自动化中的潜力，为作物管理和研究提供了可扩展且高效的解决方案。

Abstract: Accurate maize stand counts are essential for crop management and research, informing yield prediction, planting density optimization, and early detection of germination issues. Manual counting is labor-intensive, slow, and error-prone, especially across large or variable fields. We present MaizeStandCounting (MaSC), a robust algorithm for automated maize seedling stand counting from RGB imagery captured by low-cost UAVs and processed on affordable hardware. MaSC operates in two modes: (1) mosaic images divided into patches, and (2) raw video frames aligned using homography matrices. Both modes use a lightweight YOLOv9 model trained to detect maize seedlings from V2-V10 growth stages. MaSC distinguishes maize from weeds and other vegetation, then performs row and range segmentation based on the spatial distribution of detections to produce precise row-wise stand counts. Evaluation against in-field manual counts from our 2024 summer nursery showed strong agreement with ground truth (R^2= 0.616 for mosaics, R^2 = 0.906 for raw frames). MaSC processed 83 full-resolution frames in 60.63 s, including inference and post-processing, highlighting its potential for real-time operation. These results demonstrate MaSC’s effectiveness as a scalable, low-cost, and accurate tool for automated maize stand counting in both research and production environments.

[68] PIT-QMM: A Large Multimodal Model For No-Reference Point Cloud Quality Assessment

Shashank Gupta,Gregoire Phillips,Alan C. Bovik

Main category: cs.CV

TL;DR: 论文提出了一种名为PIT-QMM的大型多模态模型（LMM），用于无参考点云质量评估（NR-PCQA），通过结合文本描述、2D投影和3D点云视图多模态数据，实现了端到端的质量评分预测，并在性能和模型解释性上显著超越现有方法。

Details

Motivation: 尽管大型多模态模型在图像和视频质量评估中取得了显著进展，但在3D点云质量评估领域尚未充分探索。本文旨在利用多模态信息的互补性，提升无参考点云质量评估的性能。

Contribution: 1. 提出首个端到端的LMM模型PIT-QMM，支持文本、图像和点云多模态输入进行NR-PCQA；2. 在主流基准上显著超越现有方法，且训练效率更高；3. 展示了模型在失真定位和识别中的潜力，增强了模型的可解释性和交互性。

Method: 1. 结合文本描述、2D投影和3D点云视图的多模态数据；2. 设计端到端的训练框架，融合多模态特征；3. 通过自监督和目标函数优化实现高效训练与性能提升。

Result: 在多个流行基准测试中，PIT-QMM显著优于现有方法，且训练迭代次数更少。此外，实验表明模型能够实现失真定位和识别，提升了可解释性。

Insight: 多模态信息的融合对于点云质量评估的重要性，以及LMM在3D领域应用的潜力。模型的端到端设计和高效训练为未来研究提供了新思路。

Abstract: Large Multimodal Models (LMMs) have recently enabled considerable advances in the realm of image and video quality assessment, but this progress has yet to be fully explored in the domain of 3D assets. We are interested in using these models to conduct No-Reference Point Cloud Quality Assessment (NR-PCQA), where the aim is to automatically evaluate the perceptual quality of a point cloud in absence of a reference. We begin with the observation that different modalities of data - text descriptions, 2D projections, and 3D point cloud views - provide complementary information about point cloud quality. We then construct PIT-QMM, a novel LMM for NR-PCQA that is capable of consuming text, images and point clouds end-to-end to predict quality scores. Extensive experimentation shows that our proposed method outperforms the state-of-the-art by significant margins on popular benchmarks with fewer training iterations. We also demonstrate that our framework enables distortion localization and identification, which paves a new way forward for model explainability and interactivity. Code and datasets are available at https://www.github.com/shngt/pit-qmm.

[69] Dual-Stream Alignment for Action Segmentation

Harshala Gammulle,Clinton Fookes,Sridha Sridharan,Simon Denman

Main category: cs.CV

TL;DR: 论文提出了一种名为DSA Net的双流对齐网络，用于动作分割任务，通过结合帧级和动作级的特征流，并使用量子-经典混合学习框架提升性能，实现了最先进的结果。

Details

Motivation: 现有的动作分割方法多为单流方法，忽略了动作和动作之间的过渡线索。通过引入双流方法和量子-经典的混合学习框架，可以更好地捕捉动作分割中的时空特征。

Contribution: 1. 提出DSA Net，首个结合量子-经典混合学习的动作分割框架；2. 引入TC块和Q-ActGM方法，增强特征融合的表达能力；3. 设计了双流对齐损失函数，促进特征空间的对齐。

Method: 1. 使用双流（帧级和动作级）建模动作分割；2. 通过TC块实现双流间的信息融合；3. 利用Q-ActGM增强特征表达能力；4. 设计双流对齐损失，包括关系一致性、跨层对比和循环一致性重建损失。

Result: DSA Net在GTEA、Breakfast、50Salads和EgoProcel等基准数据集上实现了最先进的性能，并通过消融实验验证了各模块的有效性。

Insight: 双流方法和量子-经典混合学习框架能够显著提升动作分割的性能，尤其是通过特征对齐和融合来捕捉动作及其过渡线索。

Abstract: Action segmentation is a challenging yet active research area that involves identifying when and where specific actions occur in continuous video streams. Most existing work has focused on single-stream approaches that model the spatio-temporal aspects of frame sequences. However, recent research has shifted toward two-stream methods that learn action-wise features to enhance action segmentation performance. In this work, we propose the Dual-Stream Alignment Network (DSA Net) and investigate the impact of incorporating a second stream of learned action features to guide segmentation by capturing both action and action-transition cues. Communication between the two streams is facilitated by a Temporal Context (TC) block, which fuses complementary information using cross-attention and Quantum-based Action-Guided Modulation (Q-ActGM), enhancing the expressive power of the fused features. To the best of our knowledge, this is the first study to introduce a hybrid quantum-classical machine learning framework for action segmentation. Our primary objective is for the two streams (frame-wise and action-wise) to learn a shared feature space through feature alignment. This is encouraged by the proposed Dual-Stream Alignment Loss, which comprises three components: relational consistency, cross-level contrastive, and cycle-consistency reconstruction losses. Following prior work, we evaluate DSA Net on several diverse benchmark datasets: GTEA, Breakfast, 50Salads, and EgoProcel. We further demonstrate the effectiveness of each component through extensive ablation studies. Notably, DSA Net achieves state-of-the-art performance, significantly outperforming existing

[70] Once Is Enough: Lightweight DiT-Based Video Virtual Try-On via One-Time Garment Appearance Injection

Yanjie Pan,Qingdong He,Lidong Wang,Bo Peng,Mingmin Chi

Main category: cs.CV

TL;DR: 论文提出了一种基于第一帧衣物替换的轻量级视频虚拟试穿方法OIE，通过单次衣物外观注入实现高效视频生成。

Details

Motivation: 现有基于Diffusion Transformer的双分支架构在视频虚拟试穿中参数量大且缺乏时间特性学习，OIE旨在解决这些问题。

Contribution: 提出了OIE方法，通过单次衣物替换和内容控制实现高效视频生成，显著减少了参数量和计算成本。

Method: 使用图像衣物替换模型修改第一帧，再利用姿势和掩码信息控制视频生成模型合成后续帧。

Result: 实验表明OIE在参数和计算效率上表现优越，同时保持了领先性能。

Insight: 通过第一帧内容控制和时序先验引导，OIE提供了一种轻量且高效的视频虚拟试穿解决方案。

Abstract: Video virtual try-on aims to replace the clothing of a person in a video with a target garment. Current dual-branch architectures have achieved significant success in diffusion models based on the U-Net; however, adapting them to diffusion models built upon the Diffusion Transformer remains challenging. Initially, introducing latent space features from the garment reference branch requires adding or modifying the backbone network, leading to a large number of trainable parameters. Subsequently, the latent space features of garments lack inherent temporal characteristics and thus require additional learning. To address these challenges, we propose a novel approach, OIE (Once is Enough), a virtual try-on strategy based on first-frame clothing replacement: specifically, we employ an image-based clothing transfer model to replace the clothing in the initial frame, and then, under the content control of the edited first frame, utilize pose and mask information to guide the temporal prior of the video generation model in synthesizing the remaining frames sequentially. Experiments show that our method achieves superior parameter efficiency and computational efficiency while still maintaining leading performance under these constraints.

[71] MONKEY: Masking ON KEY-Value Activation Adapter for Personalization

James Baker

Main category: cs.CV

TL;DR: MONKEY提出了一种通过掩码KV激活适配器的方法，在扩散模型中实现个性化生成。该方法利用IP-Adapter自动生成的掩码限制图像令牌到主体部分，从而让文本提示更好地控制背景。

Details

Motivation: 现有个性化扩散模型容易过分关注主体图像而忽略文本提示，导致生成结果缺乏多样性。

Contribution: 提出了MONKEY方法，通过掩码KV激活适配器，在推理过程中利用自动生成的掩码将图像令牌限制在主体区域内，使文本提示能更好地控制背景生成。

Method: 利用IP-Adapter生成的掩码在第二遍推理中掩码图像令牌，限制主体区域，同时让文本提示专注背景。

Result: 相比其他方法，MONKEY在主体图像与文本提示的对齐表现更优，生成的图像能准确匹配描述的背景和地点。

Insight: 自动生成的掩码可以有效地分离主体和背景，从而优化扩散模型的多模态控制能力。

Abstract: Personalizing diffusion models allows users to generate new images that incorporate a given subject, allowing more control than a text prompt. These models often suffer somewhat when they end up just recreating the subject image, and ignoring the text prompt. We observe that one popular method for personalization, the IP-Adapter automatically generates masks that we definitively segment the subject from the background during inference. We propose to use this automatically generated mask on a second pass to mask the image tokens, thus restricting them to the subject, not the background, allowing the text prompt to attend to the rest of the image. For text prompts describing locations and places, this produces images that accurately depict the subject while definitively matching the prompt. We compare our method to a few other test time personalization methods, and find our method displays high prompt and source image alignment.

[72] Automatic Text Box Placement for Supporting Typographic Design

Jun Muraoka,Daichi Haraguchi,Naoto Inoue,Wataru Shimoda,Kota Yamaguchi,Seiichi Uchida

Main category: cs.CV

TL;DR: 这篇论文研究了在不完整布局中自动放置文本框的方法，比较了基于Transformer的方法和小型VLM（Phi3.5-vision）、大型预训练VLM（Gemini）以及扩展的多图像Transformer。实验表明，标准Transformer模型优于VLM方法，但在处理极小文本或密集布局时仍存在挑战。

Details

Motivation: 在广告和网页布局设计中，平衡视觉吸引力和信息传达效率是关键。自动化文本放置可以提升设计效率和质量，但目前方法在处理复杂布局时仍有局限性。

Contribution: 论文的主要贡献是：1）比较了不同模型在自动文本放置任务上的表现；2）展示了任务特定架构的优势；3）指出了当前方法的局限性。

Method: 研究方法包括：1）使用标准Transformer模型；2）引入小型和大型VLM（Phi3.5-vision和Gemini）；3）扩展Transformer以处理多图像输入。在Crello数据集上进行了评估。

Result: 结果表明，标准Transformer模型在多数情况下优于VLM方法，尤其是当包含更丰富的视觉信息时。但所有方法在处理极小文本或密集布局时表现不佳。

Insight: 任务特定架构在自动布局设计中具有优势，未来研究可以进一步优化对小文本和高密度布局的处理能力。

Abstract: In layout design for advertisements and web pages, balancing visual appeal and communication efficiency is crucial. This study examines automated text box placement in incomplete layouts, comparing a standard Transformer-based method, a small Vision and Language Model (Phi3.5-vision), a large pretrained VLM (Gemini), and an extended Transformer that processes multiple images. Evaluations on the Crello dataset show the standard Transformer-based models generally outperform VLM-based approaches, particularly when incorporating richer appearance information. However, all methods face challenges with very small text or densely populated layouts. These findings highlight the benefits of task-specific architectures and suggest avenues for further improvement in automated layout design.

[73] TCIP: Threshold-Controlled Iterative Pyramid Network for Deformable Medical Image Registration

Heming Wu,Di Wang,Tai Ma,Peng Zhao,Yubin Xiao,Zhongke Wu,Xing-Ce Wang,Chuang Li,Xuan Wu,You Zhou

Main category: cs.CV

TL;DR: TCIP通过引入特征增强残差模块（FERM）和双阶段阈值控制迭代（TCI）策略，改进金字塔网络在医学图像配准中的性能，减少解剖结构错位累积并自适应调整迭代次数，显著提升精度。

Details

Motivation: 现有金字塔网络在医学图像配准中容易累积解剖结构错位，且无法自适应调整迭代次数，导致精度下降或计算冗余。

Contribution: 1. 提出FERM模块，增强解剖特征并减少错位累积；
2. 提出TCI策略，自适应控制迭代次数；
3. 整合为TCIP模型，性能优于SOTA且保持高效。

Method: 1. FERM由三个块组成：提取解剖特征、抑制无关特征、估计形变场；
2. TCI分两阶段评估配准稳定性和收敛性。

Result: 在多个公共医学数据集上，TCIP精度优于现有方法，同时保持高效推理和紧凑参数。

Insight: FERM和TCI的通用性强，可集成到其他网络中，显著提升性能。

Abstract: Although pyramid networks have demonstrated superior performance in deformable medical image registration, their decoder architectures are inherently prone to propagating and accumulating anatomical structure misalignments. Moreover, most existing models do not adaptively determine the number of iterations for optimization under varying deformation requirements across images, resulting in either premature termination or excessive iterations that degrades registration accuracy. To effectively mitigate the accumulation of anatomical misalignments, we propose the Feature-Enhanced Residual Module (FERM) as the core component of each decoding layer in the pyramid network. FERM comprises three sequential blocks that extract anatomical semantic features, learn to suppress irrelevant features, and estimate the final deformation field, respectively. To adaptively determine the number of iterations for varying images, we propose the dual-stage Threshold-Controlled Iterative (TCI) strategy. In the first stage, TCI assesses registration stability and with asserted stability, it continues with the second stage to evaluate convergence. We coin the model that integrates FERM and TCI as Threshold-Controlled Iterative Pyramid (TCIP). Extensive experiments on three public brain MRI datasets and one abdomen CT dataset demonstrate that TCIP outperforms the state-of-the-art (SOTA) registration networks in terms of accuracy, while maintaining comparable inference speed and a compact model parameter size. Finally, we assess the generalizability of FERM and TCI by integrating them with existing registration networks and further conduct ablation studies to validate the effectiveness of these two proposed methods.

[74] Controllable Video Synthesis via Variational Inference

Haoyi Duan,Yunzhi Zhang,Yilun Du,Jiajun Wu

Main category: cs.CV

TL;DR: 本文提出了一种通过变分推断实现可控视频合成的方法，支持从精确的4D轨迹到粗粒度文本提示的多粒度控制，并通过分步KL散度最小化和上下文条件分解技术优化生成过程。

Details

Motivation: 现有视频生成模型通常仅支持固定输入格式，缺乏对不同粒度用户控制的支持。本文旨在解决这一问题，实现高可控性和多样性的视频合成。

Contribution: 1. 提出了一种支持多粒度用户控制的视频合成方法；2. 通过变分推断和分步优化技术解决复杂约束下的生成问题；3. 设计了上下文条件分解技术以减少解空间的模态并避免局部最优。

Method: 1. 将任务建模为变分推断问题，近似一个复合分布；2. 利用多个视频生成主干网络共同满足约束；3. 通过分步KL散度最小化和上下文条件分解技术优化生成过程。

Result: 实验表明，该方法在可控性、多样性和3D一致性方面优于现有方法。

Insight: 通过分阶段优化和上下文条件分解，可以有效处理复杂约束下的视频生成任务，同时平衡控制性和多样性。

Abstract: Many video workflows benefit from a mixture of user controls with varying granularity, from exact 4D object trajectories and camera paths to coarse text prompts, while existing video generative models are typically trained for fixed input formats. We develop a video synthesis method that addresses this need and generates samples with high controllability for specified elements while maintaining diversity for under-specified ones. We cast the task as variational inference to approximate a composed distribution, leveraging multiple video generation backbones to account for all task constraints collectively. To address the optimization challenge, we break down the problem into step-wise KL divergence minimization over an annealed sequence of distributions, and further propose a context-conditioned factorization technique that reduces modes in the solution space to circumvent local optima. Experiments suggest that our method produces samples with improved controllability, diversity, and 3D consistency compared to prior works.

[75] Hybrid CNN-BYOL Approach for Fault Detection in Induction Motors Using Thermal Images

Tangin Amir Smrity,MD Zahin Muntaqim Hasan Muhammad Kafi,Abu Saleh Musa Miah,Najmul Hassan,Yuichi Okuyama,Nobuyoshi Asai,Taro Suzuki,Jungpil Shin

Main category: cs.CV

TL;DR: 该论文提出了一种结合BYOL和CNN的混合方法，用于通过热图像分类检测感应电机的故障，并提出了一种名为BYOL-IMNet的新型轻量级CNN模型，表现出色。

Details

Motivation: 感应电机在工业和日常生活中至关重要，但其易受多种故障影响，可能导致过热和能源浪费。早期故障检测对保护电机至关重要。

Contribution: 主要贡献是提出了CNN-BYOL混合方法以及高性能轻量级CNN模型BYOL-IMNet，用于热图像故障分类，取得了99.89%的测试精度。

Method: 结合BYOL技术和多种CNN架构（如ResNet-50、DenseNet等），并设计了BYOL-IMNet模型，包含四个定制块用于热图像分类。

Result: 实验结果显示BYOL-IMNet测试精度达99.89%，单图像推理时间为5.7毫秒，优于现有方法。

Insight: 研究表明CNN-BYOL混合方法在故障检测中具有潜力，为工业环境中的在线监测提供了高效解决方案。

Abstract: Induction motors (IMs) are indispensable in industrial and daily life, but they are susceptible to various faults that can lead to overheating, wasted energy consumption, and service failure. Early detection of faults is essential to protect the motor and prolong its lifespan. This paper presents a hybrid method that integrates BYOL with CNNs for classifying thermal images of induction motors for fault detection. The thermal dataset used in this work includes different operating states of the motor, such as normal operation, overload, and faults. We employed multiple deep learning (DL) models for the BYOL technique, ranging from popular architectures such as ResNet-50, DenseNet-121, DenseNet-169, EfficientNetB0, VGG16, and MobileNetV2. Additionally, we introduced a new high-performance yet lightweight CNN model named BYOL-IMNet, which comprises four custom-designed blocks tailored for fault classification in thermal images. Our experimental results demonstrate that the proposed BYOL-IMNet achieves 99.89% test accuracy and an inference time of 5.7 ms per image, outperforming state-of-the-art models. This study highlights the promising performance of the CNN-BYOL hybrid method in enhancing accuracy for detecting faults in induction motors, offering a robust methodology for online monitoring in industrial settings.

[76] Mutual Learning for Hashing: Unlocking Strong Hash Functions from Weak Supervision

Xiaoxu Ma,Runhao Li,Zhenyu Weng

Main category: cs.CV

TL;DR: 本文提出了Mutual Learning for Hashing (MLH)，一种弱监督到强监督的哈希学习方法，通过中心哈希分支和成对哈希分支的相互学习，结合全局和局部相似性信息，提升哈希函数性能。

Details

Motivation: 现有的深度哈希方法中，中心哈希方法虽然能有效建模全局数据分布，但忽视了局部相似性信息；而成对哈希方法则侧重局部相似性。本文旨在结合两者的优势，提出一种弱监督到强监督的学习框架。

Contribution: 1) 提出MLH框架，通过中心哈希分支和成对哈希分支的相互学习，充分利用全局和局部信息；2) 引入混合哈希专家模块，促进跨分支交互，进一步提升性能。

Method: MLH包含两个分支：强中心哈希分支和弱成对哈希分支。通过迭代相互学习，中心分支从成对分支学习局部相似性。此外，引入混合哈希专家模块实现分支间的有效交互。

Result: MLH在多个基准数据集上表现优于现有哈希方法，验证了其有效性。

Insight: 结合全局和局部信息的哈希学习方法能够显著提升性能，弱监督与强监督的协同学习是一种有效的策略。

Abstract: Deep hashing has been widely adopted for large-scale image retrieval, with numerous strategies proposed to optimize hash function learning. Pairwise-based methods are effective in learning hash functions that preserve local similarity relationships, whereas center-based methods typically achieve superior performance by more effectively capturing global data distributions. However, the strength of center-based methods in modeling global structures often comes at the expense of underutilizing important local similarity information. To address this limitation, we propose Mutual Learning for Hashing (MLH), a novel weak-to-strong framework that enhances a center-based hashing branch by transferring knowledge from a weaker pairwise-based branch. MLH consists of two branches: a strong center-based branch and a weaker pairwise-based branch. Through an iterative mutual learning process, the center-based branch leverages local similarity cues learned by the pairwise-based branch. Furthermore, inspired by the mixture-of-experts paradigm, we introduce a novel mixture-of-hash-experts module that enables effective cross-branch interaction, further enhancing the performance of both branches. Extensive experiments demonstrate that MLH consistently outperforms state-of-the-art hashing methods across multiple benchmark datasets.

[77] RePainter: Empowering E-commerce Object Removal via Spatial-matting Reinforcement Learning

Zipeng Guo,Lichen Ma,Xiaolong Fu,Gaojing Zhou,Lan Yang,Yuchen Zhou,Linkai Liu,Yu He,Ximan Liu,Shiping Dong,Jingling Fu,Zhen Chen,Yu Shi,Junshi Huang,Jason Li,Chao Gou

Main category: cs.CV

TL;DR: Repainter是一个基于强化学习的框架，通过空间抠图和GRPO优化策略，解决了电商图像中对象移除的难题，显著提升了图像修复质量。

Details

Motivation: 电商平台中产品图像的清晰度和吸引力至关重要，但水印和促销文本等干扰物影响了视觉效果。现有的基于扩散的修复方法在商业场景中存在对象移除不可靠和领域适应性不足的问题。

Contribution: 1. 提出了Repainter框架，结合空间抠图轨迹优化和GRPO；2. 引入了复合奖励机制平衡全局、局部和语义约束；3. 贡献了EcomPaint-100K数据集和标准评测基准EcomPaint-Bench。

Method: 通过强化学习框架，调制注意力机制以强调背景上下文，结合复合奖励机制减少视觉伪影和奖励欺骗。

Result: 实验表明Repainter在复杂场景中显著优于现有方法。

Insight: 结合强化学习和注意力机制可以显著提升电商场景中对象移除的效果，且复合奖励机制的设计是关键。

Abstract: In web data, product images are central to boosting user engagement and advertising efficacy on e-commerce platforms, yet the intrusive elements such as watermarks and promotional text remain major obstacles to delivering clear and appealing product visuals. Although diffusion-based inpainting methods have advanced, they still face challenges in commercial settings due to unreliable object removal and limited domain-specific adaptation. To tackle these challenges, we propose Repainter, a reinforcement learning framework that integrates spatial-matting trajectory refinement with Group Relative Policy Optimization (GRPO). Our approach modulates attention mechanisms to emphasize background context, generating higher-reward samples and reducing unwanted object insertion. We also introduce a composite reward mechanism that balances global, local, and semantic constraints, effectively reducing visual artifacts and reward hacking. Additionally, we contribute EcomPaint-100K, a high-quality, large-scale e-commerce inpainting dataset, and a standardized benchmark EcomPaint-Bench for fair evaluation. Extensive experiments demonstrate that Repainter significantly outperforms state-of-the-art methods, especially in challenging scenes with intricate compositions. We will release our code and weights upon acceptance.

[78] SyncHuman: Synchronizing 2D and 3D Generative Models for Single-view Human Reconstruction

Wenyue Chen,Peng Li,Wangguandong Zheng,Chengfeng Zhao,Mengfei Li,Yaolong Zhu,Zhiyang Dou,Ronggang Wang,Yuan Liu

Main category: cs.CV

TL;DR: SyncHuman提出了一种结合2D多视角生成模型和3D原生生成模型的框架，通过同步优化实现高质量的单视图3D人体重建。

Details

Motivation: 现有的方法依赖SMPL估计和生成模型，但受限于不准确的3D先验和难以处理复杂姿态及细节重建，SyncHuman旨在克服这些问题。

Contribution: 首次结合2D多视角生成模型和3D原生生成模型，引入像素对齐的2D-3D同步注意力机制和特征注入机制，提升重建质量和几何一致性。

Method: 联合优化2D多视角和3D原生生成模型，通过同步注意力机制对齐几何形状和多视角图像，利用特征注入机制从2D图像提取细节增强3D形状。

Result: 实验结果验证SyncHuman在复杂姿态下仍能实现高精度和逼真的3D重建，几何准确性和视觉保真度均优于基线方法。

Insight: 结合2D和3D生成模型的互补优势是提升单视图重建质量的有效途径，为未来3D生成模型提供了新思路。

Abstract: Photorealistic 3D full-body human reconstruction from a single image is a critical yet challenging task for applications in films and video games due to inherent ambiguities and severe self-occlusions. While recent approaches leverage SMPL estimation and SMPL-conditioned image generative models to hallucinate novel views, they suffer from inaccurate 3D priors estimated from SMPL meshes and have difficulty in handling difficult human poses and reconstructing fine details. In this paper, we propose SyncHuman, a novel framework that combines 2D multiview generative model and 3D native generative model for the first time, enabling high-quality clothed human mesh reconstruction from single-view images even under challenging human poses. Multiview generative model excels at capturing fine 2D details but struggles with structural consistency, whereas 3D native generative model generates coarse yet structurally consistent 3D shapes. By integrating the complementary strengths of these two approaches, we develop a more effective generation framework. Specifically, we first jointly fine-tune the multiview generative model and the 3D native generative model with proposed pixel-aligned 2D-3D synchronization attention to produce geometrically aligned 3D shapes and 2D multiview images. To further improve details, we introduce a feature injection mechanism that lifts fine details from 2D multiview images onto the aligned 3D shapes, enabling accurate and high-fidelity reconstruction. Extensive experiments demonstrate that SyncHuman achieves robust and photo-realistic 3D human reconstruction, even for images with challenging poses. Our method outperforms baseline methods in geometric accuracy and visual fidelity, demonstrating a promising direction for future 3D generation models.

[79] ComGS: Efficient 3D Object-Scene Composition via Surface Octahedral Probes

Jian Gao,Mengqi Yuan,Yifei Zeng,Chang Zeng,Zhihao Li,Zhenyu Chen,Weichao Qiu,Xiao-Xiao Long,Hao Zhu,Xun Cao,Yao Yao

Main category: cs.CV

TL;DR: ComGS提出了一种高效的三维物体-场景合成框架，通过表面八面体探针（SOPs）存储光照和遮挡信息，避免了昂贵的射线追踪，同时简化了光照估计任务，实现了高质量、实时的渲染。

Details

Motivation: 现有的高斯泼溅（GS）方法在物体与场景合成时存在光照和阴影不一致的问题，且传统基于射线追踪的逆渲染方法效率低下。ComGS旨在解决这些问题，实现高效、和谐的物体-场景合成。

Contribution: 1. 引入表面八面体探针（SOPs），高效存储和查询光照与遮挡信息；2. 提出简化的光照估计方法，专注于物体周围的环境光照；3. 实现了高质量、实时的渲染（约28 FPS）和快速编辑（仅需36秒）。

Method: 1. 使用SOPs存储光照和遮挡信息，通过插值高效查询；2. 在物体放置位置捕获360度重建辐射场，并微调扩散模型补全光照；3. 结合SOPs和简化光照估计，构建ComGS框架。

Result: ComGS实现了约28 FPS的实时渲染，视觉效果和谐且阴影生动，编辑仅需36秒。

Insight: 通过专注于物体周围的光照而非全局光照估计，可以显著简化任务并提升效率；SOPs的设计避免了计算密集型操作，为实时渲染提供了新思路。

Abstract: Gaussian Splatting (GS) enables immersive rendering, but realistic 3D object-scene composition remains challenging. Baked appearance and shadow information in GS radiance fields cause inconsistencies when combining objects and scenes. Addressing this requires relightable object reconstruction and scene lighting estimation. For relightable object reconstruction, existing Gaussian-based inverse rendering methods often rely on ray tracing, leading to low efficiency. We introduce Surface Octahedral Probes (SOPs), which store lighting and occlusion information and allow efficient 3D querying via interpolation, avoiding expensive ray tracing. SOPs provide at least a 2x speedup in reconstruction and enable real-time shadow computation in Gaussian scenes. For lighting estimation, existing Gaussian-based inverse rendering methods struggle to model intricate light transport and often fail in complex scenes, while learning-based methods predict lighting from a single image and are viewpoint-sensitive. We observe that 3D object-scene composition primarily concerns the object’s appearance and nearby shadows. Thus, we simplify the challenging task of full scene lighting estimation by focusing on the environment lighting at the object’s placement. Specifically, we capture a 360 degrees reconstructed radiance field of the scene at the location and fine-tune a diffusion model to complete the lighting. Building on these advances, we propose ComGS, a novel 3D object-scene composition framework. Our method achieves high-quality, real-time rendering at around 28 FPS, produces visually harmonious results with vivid shadows, and requires only 36 seconds for editing. Code and dataset are available at https://nju-3dv.github.io/projects/ComGS/.

[80] DEGS: Deformable Event-based 3D Gaussian Splatting from RGB and Event Stream

Junhao He,Jiaxu Wang,Jia Li,Mingyuan Sun,Qiang Zhang,Jiahang Cao,Ziyi Zhang,Yi Gu,Jingkai Sun,Renjing Xu

Main category: cs.CV

TL;DR: 该论文提出了一种名为DEGS的新框架，结合RGB和事件流数据优化动态3D高斯溅射（3DGS），利用事件运动先验指导变形场的优化，从而解决低帧率RGB视频重建动态3DGS的挑战。

Details

Motivation: 动态3DGS从低帧率RGB视频重建的挑战在于大帧间运动增加了解决方案的不确定性。事件相机可以异步捕捉快速视觉变化且对运动模糊鲁棒，但缺乏颜色信息。结合RGB和事件流可以提供确定性约束，但两者模态差异显著，难以联合优化。

Contribution: 1) 提出联合优化RGB和事件流的动态3DGS框架；2) 引入事件运动先验指导变形场优化；3) 提出LoCM无监督微调框架和几何感知数据关联方法。

Method: 1) 使用LoCM框架从事件流中提取运动先验；2) 提出几何感知数据关联方法建立事件-高斯运动对应；3) 结合运动分解和帧间伪标签策略优化动态3DGS。

Result: 实验表明，该方法在合成和真实场景中优于现有基于图像和事件的方法，并能有效利用事件数据优化动态3DGS。

Insight: 事件流为动态3DGS提供了一个可行的先验约束，解决了RGB数据的低帧率局限性。

Abstract: Reconstructing Dynamic 3D Gaussian Splatting (3DGS) from low-framerate RGB videos is challenging. This is because large inter-frame motions will increase the uncertainty of the solution space. For example, one pixel in the first frame might have more choices to reach the corresponding pixel in the second frame. Event cameras can asynchronously capture rapid visual changes and are robust to motion blur, but they do not provide color information. Intuitively, the event stream can provide deterministic constraints for the inter-frame large motion by the event trajectories. Hence, combining low-temporal-resolution images with high-framerate event streams can address this challenge. However, it is challenging to jointly optimize Dynamic 3DGS using both RGB and event modalities due to the significant discrepancy between these two data modalities. This paper introduces a novel framework that jointly optimizes dynamic 3DGS from the two modalities. The key idea is to adopt event motion priors to guide the optimization of the deformation fields. First, we extract the motion priors encoded in event streams by using the proposed LoCM unsupervised fine-tuning framework to adapt an event flow estimator to a certain unseen scene. Then, we present the geometry-aware data association method to build the event-Gaussian motion correspondence, which is the primary foundation of the pipeline, accompanied by two useful strategies, namely motion decomposition and inter-frame pseudo-label. Extensive experiments show that our method outperforms existing image and event-based approaches across synthetic and real scenes and prove that our method can effectively optimize dynamic 3DGS with the help of event data.

[81] GTR-Bench: Evaluating Geo-Temporal Reasoning in Vision-Language Models

Qinghongbing Xie,Zhaoyuan Xia,Feng Zhu,Lijun Gong,Ziyue Li,Rui Zhao,Long Zeng

Main category: cs.CV

TL;DR: GTR-Bench是一个新的评估基准，用于测试视觉语言模型（VLMs）在地理时空推理上的能力，揭示了当前模型的多项不足。

Details

Motivation: 现有基准主要关注基于图像/视频或地图的单视角推理，而缺乏对地理时空多视角联合推理的评估，这对交通管理和应急响应等领域至关重要。

Contribution: 提出了GTR-Bench，一个专注于大规模摄像头网络中移动目标地理时空推理的新基准，并揭示了当前VLMs的三大缺陷。

Method: 基于多视角视频和非重叠视野的地图数据，设计了需要联合推理和时空预测的任务，评估了10多种主流VLMs的性能。

Result: 即使是表现最佳的专有模型Gemini-2.5-Pro（34.9%），也显著落后于人类表现（78.61%），并发现了模型在时空推理上的三大不足。

Insight: VLMs在地理时空推理中存在上下文利用不平衡、时间预测能力弱以及地图与多视角视频对齐能力不足等问题。

Abstract: Recently spatial-temporal intelligence of Visual-Language Models (VLMs) has attracted much attention due to its importance for Autonomous Driving, Embodied AI and General Artificial Intelligence. Existing spatial-temporal benchmarks mainly focus on egocentric perspective reasoning with images/video context, or geographic perspective reasoning with graphics context (eg. a map), thus fail to assess VLMs’ geographic spatial-temporal intelligence with both images/video and graphics context, which is important for areas like traffic management and emergency response. To address the gaps, we introduce Geo-Temporal Reasoning benchmark (GTR-Bench), a novel challenge for geographic temporal reasoning of moving targets in a large-scale camera network. GTR-Bench is more challenging as it requires multiple perspective switches between maps and videos, joint reasoning across multiple videos with non-overlapping fields of view, and inference over spatial-temporal regions that are unobserved by any video context. Evaluations of more than 10 popular VLMs on GTR-Bench demonstrate that even the best proprietary model, Gemini-2.5-Pro (34.9%), significantly lags behind human performance (78.61%) on geo-temporal reasoning. Moreover, our comprehensive analysis on GTR-Bench reveals three primary deficiencies of current models for geo-temporal reasoning. (1) VLMs’ reasoning is impaired by an imbalanced utilization of spatial-temporal context. (2) VLMs are weak in temporal forecasting, which leads to worse performance on temporal-emphasized tasks than on spatial-emphasized tasks. (3) VLMs lack the proficiency to comprehend or align the map data with multi-view video inputs. We believe GTR-Bench offers valuable insights and opens up new opportunities for research and applications in spatial-temporal intelligence. Benchmark and code will be released at https://github.com/X-Luffy/GTR-Bench.

[82] An End-to-End Room Geometry Constrained Depth Estimation Framework for Indoor Panorama Images

Kanglin Ning,Ruzhao Chen,Penghong Wang,Xingtao Wang,Ruiqin Xiong,Xiaopeng Fan

Main category: cs.CV

TL;DR: 这篇论文提出了一种端到端的室内全景图像深度估计框架，通过结合房间几何约束（如布局预测和背景分割）解决了现有方法在处理房间角落和环境噪声时的不足。

Details

Motivation: 现有的单目360度全景深度估计方法过于关注像素级精度，导致房间角落过平滑且对环境噪声敏感。论文旨在通过引入房间几何约束改进这一问题。

Contribution: 1）提出了一种端到端的深度估计框架，结合布局预测和背景分割；2）设计了两种策略：基于房间几何的背景深度解析策略和背景分割引导的融合机制。

Method: 框架包括一个共享的特征编码器和任务特定的解码器（布局、深度和背景分割）。通过两种策略将几何约束（房间布局）和背景分割整合到深度估计中。

Result: 在Stanford2D3D、Matterport3D和Structured3D数据集上，实验结果表明该方法显著优于现有开源方法。

Insight: 房间几何信息（如布局）可以显著提升深度估计的准确性，尤其是在处理结构化环境时。背景分割的引入能够有效区分前景与背景，减少噪声干扰。

Abstract: Predicting spherical pixel depth from monocular $360^{\circ}$ indoor panoramas is critical for many vision applications. However, existing methods focus on pixel-level accuracy, causing oversmoothed room corners and noise sensitivity. In this paper, we propose a depth estimation framework based on room geometry constraints, which extracts room geometry information through layout prediction and integrates those information into the depth estimation process through background segmentation mechanism. At the model level, our framework comprises a shared feature encoder followed by task-specific decoders for layout estimation, depth estimation, and background segmentation. The shared encoder extracts multi-scale features, which are subsequently processed by individual decoders to generate initial predictions: a depth map, a room layout map, and a background segmentation map. Furthermore, our framework incorporates two strategies: a room geometry-based background depth resolving strategy and a background-segmentation-guided fusion mechanism. The proposed room-geometry-based background depth resolving strategy leverages the room layout and the depth decoder’s output to generate the corresponding background depth map. Then, a background-segmentation-guided fusion strategy derives fusion weights for the background and coarse depth maps from the segmentation decoder’s predictions. Extensive experimental results on the Stanford2D3D, Matterport3D and Structured3D datasets show that our proposed methods can achieve significantly superior performance than current open-source methods. Our code is available at https://github.com/emiyaning/RGCNet.

[83] Enhancing Visual Prompting through Expanded Transformation Space and Overfitting Mitigation

Shohei Enomoto

Main category: cs.CV

TL;DR: 论文提出了ACAVP方法，通过增强视觉提示（VP）的表达能力和减轻过拟合问题，显著提升了视觉提示的性能。ACAVP引入了仿射变换和颜色变换以提高表达性，并使用TrivialAugment数据增强缓解过拟合。实验表明，该方法在12个数据集上达到了VP方法的SOTA性能，甚至在平均准确率上超过了线性探测。

Details

Motivation: 传统的视觉提示（VP）方法虽然计算开销小且适用于黑盒模型，但其表达能力受限且容易过拟合，导致准确率较低。论文旨在通过增强VP的表达能力和解决过拟合问题来提升其性能。

Contribution: 1. 提出了ACAVP方法，通过引入仿射变换和颜色变换增强VP的表达能力。2. 发现并解决了VP训练中的过拟合问题，提出使用TrivialAugment数据增强方法。3. 在12个数据集上验证了ACAVP的优越性，其性能超过了现有VP方法甚至线性探测。

Method: ACAVP结合了仿射变换（用于创建任务特定的提示区域）和颜色变换（用于强调任务相关的视觉特征），以提升VP的表达能力。此外，使用TrivialAugment数据增强方法缓解过拟合问题。

Result: ACAVP在12个数据集上表现优异，达到了VP方法的SOTA性能，平均准确率超过线性探测，并在分布偏移下表现出更强的鲁棒性。

Insight: 研究揭示了数据增强对VP训练的重要性，表明TrivialAugment不仅对ACAVP有效，还能显著提升其他VP方法的性能。

Abstract: Visual prompting (VP) has emerged as a promising parameter-efficient fine-tuning approach for adapting pre-trained vision models to downstream tasks without modifying model parameters. Despite offering advantages like negligible computational overhead and compatibility with black-box models, conventional VP methods typically achieve lower accuracy than other adaptation approaches. Our analysis reveals two critical limitations: the restricted expressivity of simple additive transformation and a tendency toward overfitting when the parameter count increases. To address these challenges, we propose ACAVP (Affine, Color, and Additive Visual Prompting), which enhances VP’s expressive power by introducing complementary transformation operations: affine transformation for creating task-specific prompt regions while preserving original image information, and color transformation for emphasizing task-relevant visual features. Additionally, we identify that overfitting is a critical issue in VP training and introduce TrivialAugment as an effective data augmentation, which not only benefits our approach but also significantly improves existing VP methods, with performance gains of up to 12 percentage points on certain datasets. This demonstrates that appropriate data augmentation is universally beneficial for VP training. Extensive experiments across twelve diverse image classification datasets with two different model architectures demonstrate that ACAVP achieves state-of-the-art accuracy among VP methods, surpasses linear probing in average accuracy, and exhibits superior robustness to distribution shifts, all while maintaining minimal computational overhead during inference.

[84] PrismGS: Physically-Grounded Anti-Aliasing for High-Fidelity Large-Scale 3D Gaussian Splatting

Houqiang Zhong,Zhenglong Wu,Sihua Fu,Zihan Zheng,Xin Jin,Xiaoyun Zhang,Li Song,Qiang Hu

Main category: cs.CV

TL;DR: PrismGS提出了一个基于物理的3D高斯溅射正则化框架，通过多尺度监督和显式大小正则化，解决了大规模3D场景中的抗锯齿和优化不稳定问题。

Details

Motivation: 3D高斯溅射（3DGS）在紧凑场景中实现了实时逼真渲染，但在大规模城市环境中容易出现锯齿伪影和优化不稳定问题，尤其是在高分辨率渲染下。现有分治方法未能解决这一保真度差距。

Contribution: 1. 提出PrismGS框架，通过两种正则化方法（多尺度监督和显式大小正则化）改进3D高斯溅射的内在渲染行为。2. 展示了与现有管道的兼容性和高性能表现。

Method: 1. 金字塔多尺度监督：通过预过滤图像金字塔监督渲染一致性，学习抗锯齿表示。2. 显式大小正则化：基于物理的约束防止退化高斯原语的形成。

Result: 在MatrixCity、Mill-19和UrbanScene3D上的实验表明，PrismGS在4K渲染下显著提升了PSNR（约1.5 dB），同时保持了高质量和鲁棒性。

Insight: PrismGS通过物理基础和抗锯齿设计，成功解决了3D高斯溅射在大规模场景中的常见问题，为高保真渲染提供了实用方法。

Abstract: 3D Gaussian Splatting (3DGS) has recently enabled real-time photorealistic rendering in compact scenes, but scaling to large urban environments introduces severe aliasing artifacts and optimization instability, especially under high-resolution (e.g., 4K) rendering. These artifacts, manifesting as flickering textures and jagged edges, arise from the mismatch between Gaussian primitives and the multi-scale nature of urban geometry. While existing ``divide-and-conquer’’ pipelines address scalability, they fail to resolve this fidelity gap. In this paper, we propose PrismGS, a physically-grounded regularization framework that improves the intrinsic rendering behavior of 3D Gaussians. PrismGS integrates two synergistic regularizers. The first is pyramidal multi-scale supervision, which enforces consistency by supervising the rendering against a pre-filtered image pyramid. This compels the model to learn an inherently anti-aliased representation that remains coherent across different viewing scales, directly mitigating flickering textures. This is complemented by an explicit size regularization that imposes a physically-grounded lower bound on the dimensions of the 3D Gaussians. This prevents the formation of degenerate, view-dependent primitives, leading to more stable and plausible geometric surfaces and reducing jagged edges. Our method is plug-and-play and compatible with existing pipelines. Extensive experiments on MatrixCity, Mill-19, and UrbanScene3D demonstrate that PrismGS achieves state-of-the-art performance, yielding significant PSNR gains around 1.5 dB against CityGaussian, while maintaining its superior quality and robustness under demanding 4K rendering.

[85] IsoSignVid2Aud: Sign Language Video to Audio Conversion without Text Intermediaries

Harsh Kavediya,Vighnesh Nayak,Bheeshm Sharma,Balamurugan Palaniappan

Main category: cs.CV

TL;DR: 该论文提出了IsoSignVid2Aud框架，直接将孤立手语视频转换为语音，无需文本中间表示，避免了多阶段翻译系统的延迟和错误累积。

Details

Motivation: 解决手语到语音的直接翻译问题，避免传统多阶段系统的延迟和错误，尤其适用于教育应用和提示界面场景。

Contribution: 提出首个端到端框架，直接从手语视频生成语音，无需文本中介，并引入新颖的NMS算法处理非语法连续手语序列。

Method: 结合I3D特征提取模块、专用特征转换网络和音频生成流水线，利用NMS算法完成时间检测。

Result: 在ASL-Citizen-1500和WLASL-100数据集上取得Top-1准确率72.01%和78.67%，语音质量指标（PESQ:2.67, STOI:0.73）显示输出可理解。

Insight: 直接翻译框架在减少延迟和错误方面具有潜力，尤其适合非语法连续手语场景，但语音质量仍有改进空间。

Abstract: Sign language to spoken language audio translation is important to connect the hearing- and speech-challenged humans with others. We consider sign language videos with isolated sign sequences rather than continuous grammatical signing. Such videos are useful in educational applications and sign prompt interfaces. Towards this, we propose IsoSignVid2Aud, a novel end-to-end framework that translates sign language videos with a sequence of possibly non-grammatic continuous signs to speech without requiring intermediate text representation, providing immediate communication benefits while avoiding the latency and cascading errors inherent in multi-stage translation systems. Our approach combines an I3D-based feature extraction module with a specialized feature transformation network and an audio generation pipeline, utilizing a novel Non-Maximal Suppression (NMS) algorithm for the temporal detection of signs in non-grammatic continuous sequences. Experimental results demonstrate competitive performance on ASL-Citizen-1500 and WLASL-100 datasets with Top-1 accuracies of 72.01% and 78.67%, respectively, and audio quality metrics (PESQ: 2.67, STOI: 0.73) indicating intelligible speech output. Code is available at: https://github.com/BheeshmSharma/IsoSignVid2Aud_AIMLsystems-2025.

[86] AlignGS: Aligning Geometry and Semantics for Robust Indoor Reconstruction from Sparse Views

Yijie Gao,Houqiang Zhong,Tianchi Zhu,Zhengxue Cheng,Qiang Hu,Li Song

Main category: cs.CV

TL;DR: AlignGS提出了一种新的框架，通过端到端优化几何与语义来解决稀疏视图室内重建的挑战，利用2D基础模型的语义先验直接优化3D表示，显著提升了重建质量和几何一致性。

Details

Motivation: 稀疏视图下的室内3D重建存在几何模糊性问题，现有方法通常将语义视为被动特征，而AlignGS认为语义应为主动引导力量，从而提升重建的鲁棒性。

Contribution: 提出了AlignGS框架，首次实现了几何与语义的协同端到端优化，通过深度一致性和多面法线正则化等机制将语义先验直接作用于3D表示。

Method: 从2D基础模型中提取语义先验，并通过深度一致性、多面法线正则化等新颖机制，将语义指导直接嵌入几何优化过程。

Result: 在标准测试集上，AlignGS在新视角合成和几何准确性方面达到SOTA，验证了语义先验作为几何正则器的有效性。

Insight: 语义信息可以作为几何优化的主动引导力量，显著提升稀疏视图下重建的完整性和一致性。

Abstract: The demand for semantically rich 3D models of indoor scenes is rapidly growing, driven by applications in augmented reality, virtual reality, and robotics. However, creating them from sparse views remains a challenge due to geometric ambiguity. Existing methods often treat semantics as a passive feature painted on an already-formed, and potentially flawed, geometry. We posit that for robust sparse-view reconstruction, semantic understanding instead be an active, guiding force. This paper introduces AlignGS, a novel framework that actualizes this vision by pioneering a synergistic, end-to-end optimization of geometry and semantics. Our method distills rich priors from 2D foundation models and uses them to directly regularize the 3D representation through a set of novel semantic-to-geometry guidance mechanisms, including depth consistency and multi-faceted normal regularization. Extensive evaluations on standard benchmarks demonstrate that our approach achieves state-of-the-art results in novel view synthesis and produces reconstructions with superior geometric accuracy. The results validate that leveraging semantic priors as a geometric regularizer leads to more coherent and complete 3D models from limited input views. Our code is avaliable at https://github.com/MediaX-SJTU/AlignGS .

[87] Self-Supervised Learning Strategies for a Platform to Test the Toxicity of New Chemicals and Materials

Thomas Lautenschlager,Nils Friederich,Angelo Jovin Yamachui Sitcheu,Katja Nau,Gaëlle Hayot,Thomas Dickmeis,Ralf Mikut

Main category: cs.CV

TL;DR: 论文探讨了如何通过自监督学习策略为测试新化学品和材料毒性的平台提供高效解决方案，利用EmbryoNet数据集展示了自监督学习表示在区分不同化合物作用模式上的有效性。

Details

Motivation: 高通量毒性测试需要高效的自动化评估方法，而机器学习模型在此领域的应用面临关键挑战，尤其是如何有效识别毒物引发的变化。

Contribution: 论文的主要贡献是提出并证明自监督学习表示在高通量毒性测试中的有效性，能够区分不同化合物的作用模式。

Method: 研究采用自监督学习策略，利用EmbryoNet数据集（包含由不同化学物质引发的斑马鱼胚胎表型）学习表示。

Result: 分析表明，自监督学习学到的表示可以有效区分不同化合物的作用模式。

Insight: 自监督学习为毒性测试提供了高效的特征表示方法，可集成到实际毒性测试设备中，具有实际应用潜力。

Abstract: High-throughput toxicity testing offers a fast and cost-effective way to test large amounts of compounds. A key component for such systems is the automated evaluation via machine learning models. In this paper, we address critical challenges in this domain and demonstrate how representations learned via self-supervised learning can effectively identify toxicant-induced changes. We provide a proof-of-concept that utilizes the publicly available EmbryoNet dataset, which contains ten zebrafish embryo phenotypes elicited by various chemical compounds targeting different processes in early embryonic development. Our analysis shows that the learned representations using self-supervised learning are suitable for effectively distinguishing between the modes-of-action of different compounds. Finally, we discuss the integration of machine learning models in a physical toxicity testing device in the context of the TOXBOX project.

[88] MARC: Memory-Augmented RL Token Compression for Efficient Video Understanding

Peiran Wu,Zhuorui Yu,Yunze Liu,Chi-Hao Wu,Enmin Zhou,Junxiao Shen

Main category: cs.CV

TL;DR: MARC提出了一种基于记忆增强强化学习的令牌压缩方法，用于高效视频理解，显著降低计算成本和延迟。

Details

Motivation: 视觉语言模型在处理视频时因高帧率和长持续时间面临高计算成本，现有令牌压缩方法容易导致信息丢失和性能下降。

Contribution: 提出了MARC框架，结合结构化检索和强化学习蒸馏，实现了高效视频理解，显著减少视觉令牌和计算资源。

Method: 采用’retrieve-then-compress’策略，使用VMR选择关键片段，并通过C-GRPO框架从教师模型中蒸馏推理能力。

Result: 在六个视频基准测试中，MARC仅使用一帧的令牌便接近基线准确率，视觉令牌减少95%，GPU内存降低72%，延迟减少23.9%。

Insight: MARC展示了在资源受限环境中实时视频理解的潜力，适用于视频问答、监控和自动驾驶等场景。

Abstract: The rapid progress of large language models (LLMs) has laid the foundation for multimodal models. However, visual language models (VLMs) still face heavy computational costs when extended from images to videos due to high frame rates and long durations. Token compression is a promising solution, yet most existing training-free methods cause information loss and performance degradation. To overcome this, we propose \textbf{Memory-Augmented Reinforcement Learning-based Token Compression (MARC)}, which integrates structured retrieval and RL-based distillation. MARC adopts a \textit{retrieve-then-compress} strategy using a \textbf{Visual Memory Retriever (VMR)} to select key clips and a \textbf{Compression Group Relative Policy Optimization (C-GRPO)} framework to distil reasoning ability from a teacher to a student model. Experiments on six video benchmarks show that MARC achieves near-baseline accuracy using only one frame’s tokens – reducing visual tokens by \textbf{95%}, GPU memory by \textbf{72%}, and latency by \textbf{23.9%}. This demonstrates its potential for efficient, real-time video understanding in resource-constrained settings such as video QA, surveillance, and autonomous driving.

[89] ASBench: Image Anomalies Synthesis Benchmark for Anomaly Detection

Qunyi Zhang,Songan Zhang,Jinbao Wang,Xiaoning Lei,Guoyang Xie,Guannan Jiang,Zhichao Lu

Main category: cs.CV

TL;DR: ASBench是第一套专注于评估异常合成方法的综合基准框架，提出了四个关键评估维度，揭示了现有方法的局限性并提供未来研究方向。

Details

Motivation: 异常检测在制造业质量控制中至关重要，但异常样本有限且标注成本高。现有研究将异常合成作为异常检测的辅助部分，缺乏系统性评估。

Contribution: 提出了ASBench框架，首次全面评估异常合成方法，包括跨数据集和管道的泛化性能、合成与真实数据比例等四个维度。

Method: 引入四维评估标准，并进行大量实验验证，分析合成数据的内在指标与异常检测性能的关系。

Result: 实验揭示了当前异常合成方法的局限性，并提供了未来研究的可行方向。

Insight: 异常合成方法的系统性评估对实际应用至关重要，当前方法在多场景适配性上仍有改进空间。

Abstract: Anomaly detection plays a pivotal role in manufacturing quality control, yet its application is constrained by limited abnormal samples and high manual annotation costs. While anomaly synthesis offers a promising solution, existing studies predominantly treat anomaly synthesis as an auxiliary component within anomaly detection frameworks, lacking systematic evaluation of anomaly synthesis algorithms. Current research also overlook crucial factors specific to anomaly synthesis, such as decoupling its impact from detection, quantitative analysis of synthetic data and adaptability across different scenarios. To address these limitations, we propose ASBench, the first comprehensive benchmarking framework dedicated to evaluating anomaly synthesis methods. Our framework introduces four critical evaluation dimensions: (i) the generalization performance across different datasets and pipelines (ii) the ratio of synthetic to real data (iii) the correlation between intrinsic metrics of synthesis images and anomaly detection performance metrics , and (iv) strategies for hybrid anomaly synthesis methods. Through extensive experiments, ASBench not only reveals limitations in current anomaly synthesis methods but also provides actionable insights for future research directions in anomaly synthesis

[90] TTOM: Test-Time Optimization and Memorization for Compositional Video Generation

Leigang Qu,Ziyang Wang,Na Zheng,Wenjie Wang,Liqiang Nie,Tat-Seng Chua

Main category: cs.CV

TL;DR: TTOM提出了一种免训练的框架，通过测试时优化和记忆机制，提升视频基础模型在组合场景（如运动、数量和空间关系）中的生成性能，实现了更好的文本-图像对齐。

Details

Motivation: 现有视频基础模型在组合场景中表现不佳，传统的单样本干预方法（如潜在空间或注意力机制调整）难以满足需求，亟需一种通用的优化方法来提升跨模态对齐能力。

Contribution: 1. 提出TTOM框架，通过测试时优化和记忆机制动态调整生成内容；2. 引入流式生成设置和参数化记忆机制，支持灵活的历史上下文操作；3. 展示了TTOM的组合知识解耦能力和强泛化性。

Method: 1. 通过布局注意力目标优化新参数；2. 采用流式生成框架和记忆机制管理历史优化上下文；3. 支持插入、读取、更新和删除等操作。

Result: 在T2V-CompBench和Vbench基准测试中，TTOM表现优异，证明了其高效、实用和可扩展性。

Insight: TTOM不仅解决了组合视频生成的挑战，还展示了其在跨模态对齐中的潜力，为未来动态优化框架的设计提供了新思路。

Abstract: Video Foundation Models (VFMs) exhibit remarkable visual generation performance, but struggle in compositional scenarios (e.g., motion, numeracy, and spatial relation). In this work, we introduce Test-Time Optimization and Memorization (TTOM), a training-free framework that aligns VFM outputs with spatiotemporal layouts during inference for better text-image alignment. Rather than direct intervention to latents or attention per-sample in existing work, we integrate and optimize new parameters guided by a general layout-attention objective. Furthermore, we formulate video generation within a streaming setting, and maintain historical optimization contexts with a parametric memory mechanism that supports flexible operations, such as insert, read, update, and delete. Notably, we found that TTOM disentangles compositional world knowledge, showing powerful transferability and generalization. Experimental results on the T2V-CompBench and Vbench benchmarks establish TTOM as an effective, practical, scalable, and efficient framework to achieve cross-modal alignment for compositional video generation on the fly.

[91] CVD-STORM: Cross-View Video Diffusion with Spatial-Temporal Reconstruction Model for Autonomous Driving

Tianrui Zhang,Yichen Liu,Zilin Guo,Yuxin Guo,Jingcheng Ni,Chenjing Ding,Dan Xu,Lewei Lu,Zehuan Wu

Main category: cs.CV

TL;DR: 该论文提出了一种名为CVD-STORM的跨视角视频扩散模型，通过结合空间-时间重构VAE，能够生成高质量的多视角视频并提供4D重构能力。实验显示其在FID和FVD指标上表现优异。

Details

Motivation: 随着自动驾驶技术的发展，对高保真视频生成和多样化信息（如深度估计）的需求日益增长。现有方法在长期多视角视频生成和场景理解能力上仍有不足。

Contribution: 1. 提出CVD-STORM模型，实现跨视角视频生成与4D重构；2. 通过联合训练的空间-时间重构VAE提升生成质量；3. 提出高斯散射解码器，有效重构动态场景。

Method: 1. 使用辅助4D重构任务微调VAE，增强3D结构与时间动态编码能力；2. 将VAE整合到视频扩散过程中以提升生成效果；3. 联合训练高斯散射解码器以实现动态场景重构。

Result: 在FID和FVD指标上显著提升，高斯散射解码器成功重构动态场景，提供丰富的几何信息。

Insight: 通过结合扩散模型与VAE的4D重构能力，能够同时满足高保真视频生成和场景理解的需求，为自动驾驶环境模拟提供新思路。

Abstract: Generative models have been widely applied to world modeling for environment simulation and future state prediction. With advancements in autonomous driving, there is a growing demand not only for high-fidelity video generation under various controls, but also for producing diverse and meaningful information such as depth estimation. To address this, we propose CVD-STORM, a cross-view video diffusion model utilizing a spatial-temporal reconstruction Variational Autoencoder (VAE) that generates long-term, multi-view videos with 4D reconstruction capabilities under various control inputs. Our approach first fine-tunes the VAE with an auxiliary 4D reconstruction task, enhancing its ability to encode 3D structures and temporal dynamics. Subsequently, we integrate this VAE into the video diffusion process to significantly improve generation quality. Experimental results demonstrate that our model achieves substantial improvements in both FID and FVD metrics. Additionally, the jointly-trained Gaussian Splatting Decoder effectively reconstructs dynamic scenes, providing valuable geometric information for comprehensive scene understanding.

[92] A Large-scale Dataset for Robust Complex Anime Scene Text Detection

Ziyi Dong,Yurui Zhang,Changmao Li,Naomi Rue Golding,Qing Long

Main category: cs.CV

TL;DR: 论文提出了一个名为AnimeText的大规模数据集，专注于动漫场景中的复杂文本检测，填补了现有数据集在动漫文本多样性方面的空白。

Details

Motivation: 现有文本检测数据集主要针对自然或文档场景，其文本具有规则字体、单调颜色和有序布局的特点，而动漫场景中的文本风格多样、排列不规则且易与复杂视觉元素混淆。

Contribution: 论文的主要贡献是提出了AnimeText数据集，包含735K图像和4.2M标注文本块，具有分层标注和针对动漫场景设计的困难负样本。

Method: 通过引入大规模动漫场景数据集，并采用分层标注和困难负样本设计，提升了模型对动漫复杂文本的检测能力。

Result: 实验表明，使用AnimeText训练的模型在动漫文本检测任务中优于基于现有数据集训练的模型。

Insight: 动漫场景中的文本检测需要针对其多样性进行专门设计和标注，通用数据集的训练效果有限。

Abstract: Current text detection datasets primarily target natural or document scenes, where text typically appear in regular font and shapes, monotonous colors, and orderly layouts. The text usually arranged along straight or curved lines. However, these characteristics differ significantly from anime scenes, where text is often diverse in style, irregularly arranged, and easily confused with complex visual elements such as symbols and decorative patterns. Text in anime scene also includes a large number of handwritten and stylized fonts. Motivated by this gap, we introduce AnimeText, a large-scale dataset containing 735K images and 4.2M annotated text blocks. It features hierarchical annotations and hard negative samples tailored for anime scenarios. %Cross-dataset evaluations using state-of-the-art methods demonstrate that models trained on AnimeText achieve superior performance in anime text detection tasks compared to existing datasets. To evaluate the robustness of AnimeText in complex anime scenes, we conducted cross-dataset benchmarking using state-of-the-art text detection methods. Experimental results demonstrate that models trained on AnimeText outperform those trained on existing datasets in anime scene text detection tasks. AnimeText on HuggingFace: https://huggingface.co/datasets/deepghs/AnimeText

[93] The impact of abstract and object tags on image privacy classification

Darya Baranouskaya,Andrea Cavallaro

Main category: cs.CV

TL;DR: 本文探讨了对象标签和抽象标签在图像隐私分类中的效果，发现预算有限时抽象标签更有效，而标签数量较多时对象标签同样有用。

Details

Motivation: 图像隐私分类是一个依赖上下文且主观的任务，通常使用对象标签，但抽象标签可能更适合捕捉高层次信息。本文旨在比较两种标签的效果。

Contribution: 主要贡献在于证明了抽象标签在标签预算有限时优于对象标签，而标签数量充足时两者效果相当。

Method: 通过实验比较对象标签和抽象标签在图像隐私分类中的表现，分析了不同标签数量下的效果差异。

Result: 结果显示抽象标签在有限标签预算下更有效，而标签数量较多时两种标签的表现相当。

Insight: 未来的图像隐私分类器应考虑标签类型和数量的影响，尤其是在资源受限场景下优先使用抽象标签。

Abstract: Object tags denote concrete entities and are central to many computer vision tasks, whereas abstract tags capture higher-level information, which is relevant for tasks that require a contextual, potentially subjective scene understanding. Object and abstract tags extracted from images also facilitate interpretability. In this paper, we explore which type of tags is more suitable for the context-dependent and inherently subjective task of image privacy. While object tags are generally used for privacy classification, we show that abstract tags are more effective when the tag budget is limited. Conversely, when a larger number of tags per image is available, object-related information is as useful. We believe that these findings will guide future research in developing more accurate image privacy classifiers, informed by the role of tag types and quantity.

[94] Is Architectural Complexity Always the Answer? A Case Study on SwinIR vs. an Efficient CNN

Chandresh Sutariya,Nitin Singh

Main category: cs.CV

TL;DR: 本文通过比较SwinIR和轻量级CNN在低光图像恢复任务中的表现，发现尽管SwinIR性能更高（PSNR 39.03 dB），但CNN以显著更低的计算成本（训练时间短、模型小）实现了接近的性能（PSNR 37.4 dB），为资源受限的场景提供了实用选择。

Details

Motivation: 低光图像恢复任务是计算机视觉中的持久挑战，现有大型Transformer模型（如SwinIR）虽性能优越，但计算成本高昂。本文探讨性能和效率的权衡，为实际应用中资源受限的场景寻找更优解。

Contribution: 主要贡献是揭示了轻量级CNN在低光图像恢复任务中能以更低计算成本实现接近SwinIR的性能，为资源受限场景提供了可行方案。

Method: 通过实验比较SwinIR和标准轻量级CNN的性能与效率，重点关注PSNR指标、训练收敛速度和模型大小。

Result: SwinIR的PSNR为39.03 dB，但CNN以37.4 dB接近其性能，且CNN仅需10轮训练（SwinIR需132轮），模型大小仅为SwinIR的1/55。

Insight: 在某些任务中，轻量级CNN能以显著更低的计算成本提供接近最先进性能的效果，提醒研究者不应忽视简单模型的潜力。

Abstract: The simultaneous restoration of high-frequency details and suppression of severe noise in low-light imagery presents a significant and persistent challenge in computer vision. While large-scale Transformer models like SwinIR have set the state of the art in performance, their high computational cost can be a barrier for practical applications. This paper investigates the critical trade-off between performance and efficiency by comparing the state-of-the-art SwinIR model against a standard, lightweight Convolutional Neural Network (CNN) on this challenging task. Our experimental results reveal a nuanced but important finding. While the Transformer-based SwinIR model achieves a higher peak performance, with a Peak Signal-to-Noise Ratio (PSNR) of 39.03 dB, the lightweight CNN delivers a surprisingly competitive PSNR of 37.4 dB. Crucially, the CNN reached this performance after converging in only 10 epochs of training, whereas the more complex SwinIR model required 132 epochs. This efficiency is further underscored by the model’s size; the CNN is over 55 times smaller than SwinIR. This work demonstrates that a standard CNN can provide a near state-of-the-art result with significantly lower computational overhead, presenting a compelling case for its use in real-world scenarios where resource constraints are a primary concern.

[95] GraphEnet: Event-driven Human Pose Estimation with a Graph Neural Network

Gaurvi Goyal,Pham Cong Thuong,Arren Glover,Masayoshi Mizuno,Chiara Bartolozzi

Main category: cs.CV

TL;DR: GraphEnet是一种基于图神经网络（GNN）的事件驱动人体姿态估计方法，首次将GNN应用于事件相机数据，提出了一种新颖的偏移向量学习范式与基于置信度的池化方法。

Details

Motivation: 传统RGB相机的人体姿态估计方法在低延迟和低能耗场景（如便携设备和移动机器人）中存在局限，而事件相机因其稀疏输出的特性更适合此类应用。

Contribution: 1. 首次将GNN应用于事件相机数据的人体姿态估计；2. 提出了一种新颖的偏移向量学习范式与基于置信度的池化方法。

Method: GraphEnet利用事件相机输出的稀疏性，通过基于线的中间事件表示，结合GNN和偏移向量学习范式，实现高频率的2D人体姿态估计。

Result: 该方法在事件数据上实现了高效的人体姿态估计，代码已开源。

Insight: 通过利用事件数据的稀疏性和GNN的灵活性，GraphEnet展示了在低延迟、低能耗场景下人体姿态估计的新方向。

Abstract: Human Pose Estimation is a crucial module in human-machine interaction applications and, especially since the rise in deep learning technology, robust methods are available to consumers using RGB cameras and commercial GPUs. On the other hand, event-based cameras have gained popularity in the vision research community for their low latency and low energy advantages that make them ideal for applications where those resources are constrained like portable electronics and mobile robots. In this work we propose a Graph Neural Network, GraphEnet, that leverages the sparse nature of event camera output, with an intermediate line based event representation, to estimate 2D Human Pose of a single person at a high frequency. The architecture incorporates a novel offset vector learning paradigm with confidence based pooling to estimate the human pose. This is the first work that applies Graph Neural Networks to event data for Human Pose Estimation. The code is open-source at https://github.com/event-driven-robotics/GraphEnet-NeVi-ICCV2025.

[96] CIR-CoT: Towards Interpretable Composed Image Retrieval via End-to-End Chain-of-Thought Reasoning

Weihuang Lin,Yiwei Ma,Jiayi Ji,Xiaoshuai Sun,Rongrong Ji

Main category: cs.CV

TL;DR: 这篇论文提出了CIR-CoT，一种基于端到端思维链推理的可解释组合图像检索方法，通过生成显式推理链提升跨模态交互能力，同时实现透明的决策过程。

Details

Motivation: 现有组合图像检索方法（如基于CLIP或Qwen-VL）多为黑箱模型，缺乏解释性和处理复杂指令的能力，限制了其可信度和实用性。

Contribution: 1. 提出首个端到端检索导向的多模态大语言模型CIR-CoT，集成显式思维链推理；2. 为现有数据集（如FashionIQ、CIRR）构建结构化思维链标注；3. 模型在域内外数据集上均表现出色。

Method: 1. 通过三阶段标注（描述、推理、结论）生成结构化思维链数据；2. 模型微调以生成结构化输出，并将检索意图编码为专用嵌入向量。

Result: CIR-CoT在FashionIQ、CIRR等域内数据集上表现优异，并在域外数据集CIRCO上展现出良好的泛化性。

Insight: 显式推理链不仅能提高检索精度，还能增强模型的可解释性，为未来可信赖的多模态检索系统提供了新思路。

Abstract: Composed Image Retrieval (CIR), which aims to find a target image from a reference image and a modification text, presents the core challenge of performing unified reasoning across visual and semantic modalities. While current approaches based on Vision-Language Models (VLMs, e.g., CLIP) and more recent Multimodal Large Language Models (MLLMs, e.g., Qwen-VL) have shown progress, they predominantly function as ``black boxes.” This inherent opacity not only prevents users from understanding the retrieval rationale but also restricts the models’ ability to follow complex, fine-grained instructions. To overcome these limitations, we introduce CIR-CoT, the first end-to-end retrieval-oriented MLLM designed to integrate explicit Chain-of-Thought (CoT) reasoning. By compelling the model to first generate an interpretable reasoning chain, CIR-CoT enhances its ability to capture crucial cross-modal interactions, leading to more accurate retrieval while making its decision process transparent. Since existing datasets like FashionIQ and CIRR lack the necessary reasoning data, a key contribution of our work is the creation of structured CoT annotations using a three-stage process involving a caption, reasoning, and conclusion. Our model is then fine-tuned to produce this structured output before encoding its final retrieval intent into a dedicated embedding. Comprehensive experiments show that CIR-CoT achieves highly competitive performance on in-domain datasets (FashionIQ, CIRR) and demonstrates remarkable generalization on the out-of-domain CIRCO dataset, establishing a new path toward more effective and trustworthy retrieval systems.

[97] Towards Real-World Deepfake Detection: A Diverse In-the-wild Dataset of Forgery Faces

Junyu Shi,Minghui Li,Junguo Zuo,Zhifei Yu,Yipeng Lin,Shengshan Hu,Ziqi Zhou,Yechao Zhang,Wei Wan,Yinzhe Xu,Leo Yu Zhang

Main category: cs.CV

TL;DR: 论文提出了一个名为RedFace的真实世界深度伪造检测数据集，解决了现有数据集在多样性和技术覆盖上的不足，并通过实验验证了其对现实场景的适用性。

Details

Motivation: 现有的深度伪造检测数据集通常缺乏多样性和现实世界中的最新技术覆盖，导致检测方法在实际应用中效果不佳。RedFace旨在填补这一空白。

Contribution: 1. 提出了RedFace数据集，包含6万张伪造图像和1000段视频，覆盖多样化的深度伪造技术；2. 使用9个商业平台生成数据，模拟现实黑盒场景；3. 分析了现有检测方法在RedFace上的局限性。

Method: 1. 收集多样化的真实世界深度伪造数据；2. 使用商业平台生成数据；3. 实验评估包括跨域、域内和社交网络传播仿真。

Result: 实验表明现有检测方法在RedFace数据集上表现不佳，验证了其对现实场景的适用性和挑战性。

Insight: 1. 现实世界的深度伪造技术多样化且快速演变；2. 现有检测方法需适应更复杂和动态的场景；3. RedFace为未来研究提供了更贴近实际的基准。

Abstract: Deepfakes, leveraging advanced AIGC (Artificial Intelligence-Generated Content) techniques, create hyper-realistic synthetic images and videos of human faces, posing a significant threat to the authenticity of social media. While this real-world threat is increasingly prevalent, existing academic evaluations and benchmarks for detecting deepfake forgery often fall short to achieve effective application for their lack of specificity, limited deepfake diversity, restricted manipulation techniques.To address these limitations, we introduce RedFace (Real-world-oriented Deepfake Face), a specialized facial deepfake dataset, comprising over 60,000 forged images and 1,000 manipulated videos derived from authentic facial features, to bridge the gap between academic evaluations and real-world necessity. Unlike prior benchmarks, which typically rely on academic methods to generate deepfakes, RedFace utilizes 9 commercial online platforms to integrate the latest deepfake technologies found “in the wild”, effectively simulating real-world black-box scenarios.Moreover, RedFace’s deepfakes are synthesized using bespoke algorithms, allowing it to capture diverse and evolving methods used by real-world deepfake creators. Extensive experimental results on RedFace (including cross-domain, intra-domain, and real-world social network dissemination simulations) verify the limited practicality of existing deepfake detection schemes against real-world applications. We further perform a detailed analysis of the RedFace dataset, elucidating the reason of its impact on detection performance compared to conventional datasets. Our dataset is available at: https://github.com/kikyou-220/RedFace.

[98] Physics-Driven Spatiotemporal Modeling for AI-Generated Video Detection

Shuhai Zhang,ZiHao Lian,Jiahao Yang,Daiyuan Li,Guoxuan Pang,Feng Liu,Bo Han,Shutao Li,Mingkui Tan

Main category: cs.CV

TL;DR: 该论文提出了一种基于物理驱动的AI生成视频检测方法（NSG-VD），通过量化时空梯度比例来捕捉违反物理规律的异常，显著提升了检测性能。

Details

Motivation: AI生成的视频（如Sora）已达到近乎完美的视觉真实感，急需可靠的检测机制。现有方法难以建模高维时空动态并识别违反物理规律的细微异常。

Contribution: 1. 提出了一种称为归一化时空梯度（NSG）的统计量，用于量化空间概率梯度与时间密度变化的比率；2. 基于NSG提出了一种视频检测方法（NSG-VD），性能优于现有基线方法。

Method: 1. 利用预训练扩散模型开发NSG估计器，通过空间梯度近似和运动感知的时序建模；2. 将NSG特征的最大均值差异（MMD）作为检测指标。

Result: 实验表明，NSG-VD在召回率和F1分数上分别比现有基线方法提高了16.00%和10.75%。

Insight: 1. 物理驱动的建模能有效捕捉AI生成视频的异常；2. NSG特征的差异为检测提供了理论基础。

Abstract: AI-generated videos have achieved near-perfect visual realism (e.g., Sora), urgently necessitating reliable detection mechanisms. However, detecting such videos faces significant challenges in modeling high-dimensional spatiotemporal dynamics and identifying subtle anomalies that violate physical laws. In this paper, we propose a physics-driven AI-generated video detection paradigm based on probability flow conservation principles. Specifically, we propose a statistic called Normalized Spatiotemporal Gradient (NSG), which quantifies the ratio of spatial probability gradients to temporal density changes, explicitly capturing deviations from natural video dynamics. Leveraging pre-trained diffusion models, we develop an NSG estimator through spatial gradients approximation and motion-aware temporal modeling without complex motion decomposition while preserving physical constraints. Building on this, we propose an NSG-based video detection method (NSG-VD) that computes the Maximum Mean Discrepancy (MMD) between NSG features of the test and real videos as a detection metric. Last, we derive an upper bound of NSG feature distances between real and generated videos, proving that generated videos exhibit amplified discrepancies due to distributional shifts. Extensive experiments confirm that NSG-VD outperforms state-of-the-art baselines by 16.00% in Recall and 10.75% in F1-Score, validating the superior performance of NSG-VD. The source code is available at https://github.com/ZSHsh98/NSG-VD.

[99] Real-Time Motion-Controllable Autoregressive Video Diffusion

Kesen Zhao,Jiaxin Shi,Beier Zhu,Junbao Zhou,Xiaolong Shen,Yuan Zhou,Qianru Sun,Hanwang Zhang

Main category: cs.CV

TL;DR: 论文提出了一种名为AR-Drag的实时运动可控自回归视频扩散模型，通过强化学习和轨迹奖励模型优化了图像到视频生成的运动控制能力和生成质量。

Details

Motivation: 现有自回归视频扩散模型在实时运动中存在延迟问题，且控制信号简单或生成质量下降。需要一种能够在低延迟下实现高质量和多样化运动控制的解决方案。

Contribution: 提出了首个基于强化学习的few-step自回归视频扩散模型AR-Drag，支持实时图像到视频生成及多样化运动控制，同时显著降低了延迟。

Method: 1. 微调基础I2V模型以支持基础运动控制；2. 通过强化学习和轨迹奖励模型进一步优化；3. 设计了Self-Rollout机制保持马尔可夫性，并在去噪步骤中引入选择性随机性加速训练。

Result: AR-Drag在视觉保真度和运动对齐上表现优异，相比现有方法显著降低延迟，仅使用1.3B参数。

Insight: 通过强化学习和选择性随机性设计，可以在保持生成质量的同时显著提升模型的实时性和控制能力。

Abstract: Real-time motion-controllable video generation remains challenging due to the inherent latency of bidirectional diffusion models and the lack of effective autoregressive (AR) approaches. Existing AR video diffusion models are limited to simple control signals or text-to-video generation, and often suffer from quality degradation and motion artifacts in few-step generation. To address these challenges, we propose AR-Drag, the first RL-enhanced few-step AR video diffusion model for real-time image-to-video generation with diverse motion control. We first fine-tune a base I2V model to support basic motion control, then further improve it via reinforcement learning with a trajectory-based reward model. Our design preserves the Markov property through a Self-Rollout mechanism and accelerates training by selectively introducing stochasticity in denoising steps. Extensive experiments demonstrate that AR-Drag achieves high visual fidelity and precise motion alignment, significantly reducing latency compared with state-of-the-art motion-controllable VDMs, while using only 1.3B parameters. Additional visualizations can be found on our project page: https://kesenzhao.github.io/AR-Drag.github.io/.

[100] Improving Temporal Understanding Logic Consistency in Video-Language Models via Attention Enhancement

Chengzhi Li,Heyan Huang,Ping Jian,Zhen Yang,Yaning Tian

Main category: cs.CV

TL;DR: 该论文针对视频-语言模型（Video-LLMs）中逻辑不一致的问题，提出了一种基于注意力增强的方法TCAS，通过提升模型的时间分辨能力来改善其时间理解逻辑一致性。

Details

Motivation: 大语言模型（LLMs）生成的输出常存在自相矛盾，这一问题在视频-语言模型中尤为突出，影响了模型的可靠性和实际应用，但其根本原因尚未深入研究。

Contribution: 论文的主要贡献包括：（1）通过可解释性分析揭示了交叉注意力头无法区分不同时间戳的视频令牌是逻辑不一致的主因；（2）提出了TCAS方法，通过注意力增强提升模型的时间分辨能力；（3）实验表明该方法显著提升了模型的时间逻辑一致性，并在通用视频时间定位任务上取得性能提升。

Method: 提出了Temporally Conditioned Attention Sharpening（TCAS）方法，通过构建基于注意力区分度的增强目标，优化交叉注意力头的时间分辨能力，从而提升模型的时间逻辑一致性。

Result: 实验结果表明，TCAS显著提升了视频-语言模型的时间逻辑一致性，并在通用任务上表现更优，验证了时间逻辑一致性是时间理解的瓶颈。

Insight: 论文的洞察包括：（1）逻辑不一致的根源在于交叉注意力头的时间分辨能力不足；（2）提升时间逻辑一致性可以推动视频时间理解的进展。

Abstract: Large language models (LLMs) often generate self-contradictory outputs, which severely impacts their reliability and hinders their adoption in practical applications. In video-language models (Video-LLMs), this phenomenon recently draws the attention of researchers. Specifically, these models fail to provide logically consistent responses to rephrased questions based on their grounding outputs. However, the underlying causes of this phenomenon remain underexplored. In this work, we adopt an interpretability-driven approach to analyze, statistically summarize, and intervention the potential factors of the phenomenon. We find that one of the primary reasons for the inconsistency in responses lies in the inability of cross-modal attention heads to effectively distinguish video tokens across different timestamps. To address this, we propose an attention enhancement method called Temporally Conditioned Attention Sharpening (TCAS), which constructs an enhancement objective based on attention distinctions to enhance the model’s temporal resolution capability, thereby improving its temporal understanding logic consistency. Experimental results demonstrate that our method significantly enhances the temporal logic consistency of Video-LLMs. Further interpretability analyses reveal that our method indeed improves the temporal discriminability of attention heads, validating our conclusions. Additionally, our method achieves performance improvements in general video temporal grounding tasks, highlighting that temporal logic consistency is a bottleneck in temporal understanding. By enhancing consistency, our method drives significant progress in video temporal understanding.

Shian Du,Menghan Xia,Chang Liu,Quande Liu,Xintao Wang,Pengfei Wan,Xiangyang Ji

Main category: cs.CV

TL;DR: UniMMVSR 提出了一种统一的多模态框架，用于级联视频超分辨率任务，支持文本、图像和视频等多种生成条件，显著提升了视频生成细节和对多模态条件的符合度。

Details

Motivation: 现有研究主要局限在文本到视频任务中，忽视了其他生成条件（如图像和视频）的重要性，而这在多模态视频生成中对保真度至关重要。UniMMVSR 旨在填补这一空白。

Contribution: 提出了首个统一的多模态视频超分辨率框架 UniMMVSR，能够整合文本、图像和视频等多种条件，并通过实验验证其在生成高质量视频方面的优越性。

Method: 通过潜在视频扩散模型探索了条件注入策略、训练方案和数据混合技术，设计了不同的数据构建和条件利用方法，以确保模型能精确使用所有条件类型。

Result: UniMMVSR 在实验中显著优于现有方法，生成的视频细节更丰富且更符合多模态条件。此外，结合基础模型可实现 4K 视频的多模态引导生成。

Insight: 多模态条件是提升视频生成保真度的关键，而统一的框架设计可以充分利用这些条件，显著提升生成效果与应用潜力。

Abstract: Cascaded video super-resolution has emerged as a promising technique for decoupling the computational burden associated with generating high-resolution videos using large foundation models. Existing studies, however, are largely confined to text-to-video tasks and fail to leverage additional generative conditions beyond text, which are crucial for ensuring fidelity in multi-modal video generation. We address this limitation by presenting UniMMVSR, the first unified generative video super-resolution framework to incorporate hybrid-modal conditions, including text, images, and videos. We conduct a comprehensive exploration of condition injection strategies, training schemes, and data mixture techniques within a latent video diffusion model. A key challenge was designing distinct data construction and condition utilization methods to enable the model to precisely utilize all condition types, given their varied correlations with the target video. Our experiments demonstrate that UniMMVSR significantly outperforms existing methods, producing videos with superior detail and a higher degree of conformity to multi-modal conditions. We also validate the feasibility of combining UniMMVSR with a base model to achieve multi-modal guided generation of 4K video, a feat previously unattainable with existing techniques.

[102] Beyond Textual CoT: Interleaved Text-Image Chains with Deep Confidence Reasoning for Image Editing

Zhentao Zou,Zhengrong Yue,Kunpeng Du,Binlei Bao,Hanting Li,Haizhen Xie,Guozheng Xu,Yue Zhou,Yali Wang,Jie Hu,Xue Jiang,Xinghao Chen

Main category: cs.CV

TL;DR: 论文提出了多模态推理编辑框架MURE，通过交替的文本-图像推理链优化图像编辑任务，结合深度置信推理以减少误差，并在实验中显著提升了编辑效果。

Details

Motivation: 现有基于文本的Chain-of-Thought（CoT）方法在复杂视觉布局和细微空间关系上表现不足，缺乏明确的推理过程和视觉引导。为此，作者提出了一种多模态推理方法来解决这些问题。

Contribution: 1. 提出了MURE框架，实现交替的文本-图像推理链；2. 设计了多模态深度置信（MMDC）推理范式，减少LLM的幻觉现象；3. 发布了CoT-Edit-14K数据集，支持多模态推理研究。

Method: 1. 通过交替的文本-视觉推理链逐步分解编辑任务；2. 基于深度置信评分对推理路径进行剪枝，确保高质量编辑轨迹；3. 使用奖励模型评估推理路径的置信度。

Result: 在三个图像编辑基准测试中取得了显著改进，展示了MURE在高保真图像编辑任务中的有效性。

Insight: 多模态推理链的引入能够更好地捕捉视觉细节，而深度置信机制则提高了模型的可靠性，为复杂编辑任务提供了新思路。

Abstract: Image editing with natural language has gained significant popularity, yet existing methods struggle with intricate object intersections and fine-grained spatial relationships due to the lack of an explicit reasoning process. While Chain-of-Thought (CoT) has been explored to enhance reasoning, purely textual CoT or CoT augmented with coordinate information is fundamentally limited in its ability to represent intricate visual layouts and lacks the necessary visual cues to guide the generation of fine-grained, pixel-level details. To address these challenges, we propose Multimodal Reasoning Edit (MURE), a novel framework that shifts the visual editing process from purely text-based reasoning to a series of interleaved textual and visual rationales. Our framework performs image editing using a natively multimodal, interleaved text-image CoT. This approach generates a step-by-step chain of reasoning where a textual description is followed by a corresponding visual cue, such as a positional mask that defined intended edited regions or a representation of new content. Furthermore, to mitigate the hallucination phenomenon of large language models, we introduce Multimodal Deep Confidence (MMDC) reasoning paradigm. This paradigm explores a tree of visual reasoning paths at each step. By pruning low-quality branches using a deep confidence score from a reward model, it ensures the model consistently follows a high-quality trajectory towards the final edited result. The proposed method decomposes complex editing tasks into interdependent sub-tasks, achieving greater precision at each stage and yielding high-fidelity edited results. We define the formulation for interleaved text-image chains and release the first CoT-Edit-14K dataset, comprising 14K high-quality editing examples. Extensive experiments show that our method yields significant improvements across three image editing benchmarks.

[103] Robust Canonicalization through Bootstrapped Data Re-Alignment

Johann Schmidt,Sebastian Stober

Main category: cs.CV

TL;DR: 论文提出了一种基于自举（bootstrapping）的数据重新对齐方法，逐步减少方差并恢复对齐假设，从而在细粒度视觉分类任务中优于现有的等变性和规范化基线方法。

Details

Motivation: 细粒度视觉分类（FGVC）任务需要模型对细微视觉线索敏感，同时对空间变换（如不同方向和尺度）鲁棒。现有方法依赖数据增强或等变架构，但这些方法要么需求大模型，要么限制表达能力。规范化提供了一种替代方案，但通常假设训练数据对齐，而实际数据集无法满足这一假设。

Contribution: 提出了一种迭代的自举算法，通过逐步减少方差和恢复对齐假设，解决训练数据对齐不足的问题，并在四个FGVC基准测试中表现优于基线方法。

Method: 采用自举算法逐步重新对齐训练样本，减少方差并恢复对齐假设，同时为任意紧凑群建立了收敛保证。

Result: 在四个FGVC基准测试中，该方法表现优于等变性和规范化基线，与数据增强方法相当。

Insight: 通过迭代的自举方法可以减少训练数据对齐不足带来的偏差，提高规范化方法的鲁棒性，为细粒度视觉分类任务提供了一种高效解决方案。

Abstract: Fine-grained visual classification (FGVC) tasks, such as insect and bird identification, demand sensitivity to subtle visual cues while remaining robust to spatial transformations. A key challenge is handling geometric biases and noise, such as different orientations and scales of objects. Existing remedies rely on heavy data augmentation, which demands powerful models, or on equivariant architectures, which constrain expressivity and add cost. Canonicalization offers an alternative by shielding such biases from the downstream model. In practice, such functions are often obtained using canonicalization priors, which assume aligned training data. Unfortunately, real-world datasets never fulfill this assumption, causing the obtained canonicalizer to be brittle. We propose a bootstrapping algorithm that iteratively re-aligns training samples by progressively reducing variance and recovering the alignment assumption. We establish convergence guarantees under mild conditions for arbitrary compact groups, and show on four FGVC benchmarks that our method consistently outperforms equivariant, and canonicalization baselines while performing on par with augmentation.

[104] Adaptive Gradient Calibration for Single-Positive Multi-Label Learning in Remote Sensing Image Scene Classification

Chenying Liu,Gianmarco Perantoni,Lorenzo Bruzzone,Xiao Xiang Zhu

Main category: cs.CV

TL;DR: 提出了Adaptive Gradient Calibration (AdaGC)，一种针对遥感图像单正多标签学习的自适应梯度校准框架，结合Mixup和双EMA模块，显著提升了性能和鲁棒性。

Details

Motivation: 遥感图像的多标签分类（MLC）语义理解更全面，但完整标注成本高。单正多标签学习（SPML）作为一种实用替代方案，引入了监督模糊性问题，需要专门方法解决。

Contribution: 1. 提出AdaGC框架，结合梯度校准、Mixup和双EMA模块；2. 设计了自适应触发梯度校准的指标；3. 在两种噪声类型下验证了SOTA性能。

Method: 采用梯度校准机制、Mixup增强和双EMA模块生成鲁棒伪标签，并通过动态训练指标自适应触发梯度校准。

Result: 在两个遥感基准数据集上，AdaGC在多种噪声设置下均取得最优性能和强鲁棒性。

Insight: 梯度校准和自适应触发机制能有效缓解噪声过拟合，Mixup和双EMA模块进一步提升了伪标签的可靠性。

Abstract: Multi-label classification (MLC) offers a more comprehensive semantic understanding of Remote Sensing (RS) imagery compared to traditional single-label classification (SLC). However, obtaining complete annotations for MLC is particularly challenging due to the complexity and high cost of the labeling process. As a practical alternative, single-positive multi-label learning (SPML) has emerged, where each image is annotated with only one relevant label, and the model is expected to recover the full set of labels. While scalable, SPML introduces significant supervision ambiguity, demanding specialized solutions for model training. Although various SPML methods have been proposed in the computer vision domain, research in the RS context remains limited. To bridge this gap, we propose Adaptive Gradient Calibration (AdaGC), a novel and generalizable SPML framework tailored to RS imagery. AdaGC adopts a gradient calibration (GC) mechanism combined with Mixup and a dual exponential moving average (EMA) module for robust pseudo-label generation. To maximize AdaGC’s effectiveness, we introduce a simple yet theoretically grounded indicator to adaptively trigger GC after an initial warm-up stage based on training dynamics, thereby guaranteeing the effectiveness of GC in mitigating overfitting to label noise. Extensive experiments on two benchmark RS datasets under two distinct label noise types demonstrate that AdaGC achieves state-of-the-art (SOTA) performance while maintaining strong robustness across diverse settings.

[105] One Stone with Two Birds: A Null-Text-Null Frequency-Aware Diffusion Models for Text-Guided Image Inpainting

Haipeng Liu,Yang Wang,Meng Wang

Main category: cs.CV

TL;DR: NTN-Diff是一种基于空文本和空频率感知的扩散模型，通过分解不同频率带的语义一致性来解决文本引导图像修复中掩码和非掩码区域的语义一致性和非掩码区域保护问题。

Details

Motivation: 文本引导图像修复需要同时解决掩码和非掩码区域的语义一致性以及非掩码区域的保护问题，而现有方法无法同时满足这两点。作者观察到这一问题源于混合频率带的纠缠，尤其是中低频带在去噪过程中对文本提示的鲁棒性不同。

Contribution: 提出了NTN-Diff模型，通过将语义一致性分解为不同频率带的一致性，并在去噪过程中解耦中低频带，解决了上述两个挑战。模型在早期和晚期去噪阶段分别处理高频和低频噪声，实现了掩码和非掩码区域的语义一致性。

Method: 将去噪过程分为早期（高频噪声）和晚期（低频噪声）阶段，在中低频带解耦的基础上进行空文本和文本引导的去噪。稳定的中频带在文本引导下去噪并作为低频带去噪的指导，最终实现一致性。

Result: 实验表明NTN-Diff在文本引导图像修复任务中优于现有扩散模型。

Insight: 通过频率带解耦和分阶段去噪策略，可以同时解决语义一致性和区域保护问题，为文本引导图像修复提供了新思路。

Abstract: Text-guided image inpainting aims at reconstructing the masked regions as per text prompts, where the longstanding challenges lie in the preservation for unmasked regions, while achieving the semantics consistency between unmasked and inpainted masked regions. Previous arts failed to address both of them, always with either of them to be remedied. Such facts, as we observed, stem from the entanglement of the hybrid (e.g., mid-and-low) frequency bands that encode varied image properties, which exhibit different robustness to text prompts during the denoising process. In this paper, we propose a null-text-null frequency-aware diffusion models, dubbed \textbf{NTN-Diff}, for text-guided image inpainting, by decomposing the semantics consistency across masked and unmasked regions into the consistencies as per each frequency band, while preserving the unmasked regions, to circumvent two challenges in a row. Based on the diffusion process, we further divide the denoising process into early (high-level noise) and late (low-level noise) stages, where the mid-and-low frequency bands are disentangled during the denoising process. As observed, the stable mid-frequency band is progressively denoised to be semantically aligned during text-guided denoising process, which, meanwhile, serves as the guidance to the null-text denoising process to denoise low-frequency band for the masked regions, followed by a subsequent text-guided denoising process at late stage, to achieve the semantics consistency for mid-and-low frequency bands across masked and unmasked regions, while preserve the unmasked regions. Extensive experiments validate the superiority of NTN-Diff over the state-of-the-art diffusion models to text-guided diffusion models. Our code can be accessed from https://github.com/htyjers/NTN-Diff.

[106] A Multimodal Depth-Aware Method For Embodied Reference Understanding

Fevziye Irem Eyiokur,Dogucan Yaman,Hazım Kemal Ekenel,Alexander Waibel

Main category: cs.CV

TL;DR: 这篇论文提出了一种多模态深度感知方法（ERU框架），用于解决具身参考理解任务中的目标对象识别问题。通过结合LLM数据增强、深度图模态和深度感知决策模块，该方法在复杂场景中显著提升了目标检测的准确性。

Details

Motivation: 现有的开放词汇对象检测方法在多候选对象的模糊场景中表现不佳，而具身参考理解任务需要结合语言指令和指向线索来识别目标对象。因此，作者提出一种新方法以解决这一问题。

Contribution: 1) 提出了一个结合LLM数据增强、深度图模态和深度感知决策模块的多模态ERU框架；2) 在复杂或混乱环境中实现了语言和具身线索的鲁棒融合。

Method: 该方法通过LLM生成数据增强样本，引入深度图模态提供场景的空间信息，并结合深度感知决策模块进行目标对象的准确识别。

Result: 在两个数据集上的实验表明，该方法显著优于现有基准模型，实现了更准确和可靠的参考检测。

Insight: 深度信息的引入可以有效解决多候选对象场景中的歧义问题，而多模态融合（语言+视觉+深度）是提升具身参考理解性能的关键。

Abstract: Embodied Reference Understanding requires identifying a target object in a visual scene based on both language instructions and pointing cues. While prior works have shown progress in open-vocabulary object detection, they often fail in ambiguous scenarios where multiple candidate objects exist in the scene. To address these challenges, we propose a novel ERU framework that jointly leverages LLM-based data augmentation, depth-map modality, and a depth-aware decision module. This design enables robust integration of linguistic and embodied cues, improving disambiguation in complex or cluttered environments. Experimental results on two datasets demonstrate that our approach significantly outperforms existing baselines, achieving more accurate and reliable referent detection.

[107] Learning Neural Exposure Fields for View Synthesis

Michael Niemeyer,Fabian Manhardt,Marie-Julie Rakotosaona,Michael Oechsle,Christina Tsalicoglou,Keisuke Tateno,Jonathan T. Barron,Federico Tombari

Main category: cs.CV

TL;DR: 该论文提出了神经曝光场（NExF），用于在动态范围高的场景中实现高质量的3D重建和视图合成，通过学习每3D点的最优曝光值，避免了后处理或多曝光捕捉的需求。

Details

Motivation: 现有神经场景表征在包含强曝光变化的真实场景中表现不佳，亟需一种鲁棒的方法来解决这一问题。

Contribution: 提出了神经曝光场（NExF），能够预测3D点的最优曝光值；设计了一个联合优化场景表征和曝光场的系统；在真实场景数据集上实现了显著的性能提升。

Method: 通过神经场预测每3D点的曝光值，结合神经调节机制联合优化场景表征和曝光场。

Result: 在多个基准上优于现有方法，性能提升超过55%，且训练速度更快。

Insight: 3D空间中的曝光优化比传统的逐图像或逐像素方法更适用于高动态范围场景。

Abstract: Recent advances in neural scene representations have led to unprecedented quality in 3D reconstruction and view synthesis. Despite achieving high-quality results for common benchmarks with curated data, outputs often degrade for data that contain per image variations such as strong exposure changes, present, e.g., in most scenes with indoor and outdoor areas or rooms with windows. In this paper, we introduce Neural Exposure Fields (NExF), a novel technique for robustly reconstructing 3D scenes with high quality and 3D-consistent appearance from challenging real-world captures. In the core, we propose to learn a neural field predicting an optimal exposure value per 3D point, enabling us to optimize exposure along with the neural scene representation. While capture devices such as cameras select optimal exposure per image/pixel, we generalize this concept and perform optimization in 3D instead. This enables accurate view synthesis in high dynamic range scenarios, bypassing the need of post-processing steps or multi-exposure captures. Our contributions include a novel neural representation for exposure prediction, a system for joint optimization of the scene representation and the exposure field via a novel neural conditioning mechanism, and demonstrated superior performance on challenging real-world data. We find that our approach trains faster than prior works and produces state-of-the-art results on several benchmarks improving by over 55% over best-performing baselines.

[108] LTCA: Long-range Temporal Context Attention for Referring Video Object Segmentation

Cilin Yan,Jingyun Wang,Guoliang Kang

Main category: cs.CV

TL;DR: 论文提出了LTCA（长时序上下文注意力）机制，用于视频指代分割任务，通过稀疏局部注意力堆叠和全局查询设计，平衡局部和全局信息，显著提升了性能。

Details

Motivation: 现有的视频指代分割方法在处理长时序上下文时，要么采用全局注意力（计算复杂度高），要么使用密集局部注意力（难以平衡局部和全局信息），因此需要一种更高效的机制。

Contribution: 提出了LTCA机制，通过稀疏局部注意力堆叠和随机全局键选择平衡局部与全局信息；引入了全局查询直接编码全局上下文信息。

Method: 1. 堆叠稀疏局部注意力，设计跨帧的扩张窗口注意力；2. 每个查询从全局池中随机选择少量键增强全局性；3. 设计全局查询与所有其他查询交互。

Result: 在四个视频指代分割基准测试中达到了新的SOTA，MeViS数据集上分别提升11.3%和8.3%。

Insight: 稀疏局部注意力堆叠和随机全局键选择能有效平衡局部与全局信息，同时避免计算复杂度过高；全局查询的设计直接编码长时序上下文。

Abstract: Referring Video Segmentation (RVOS) aims to segment objects in videos given linguistic expressions. The key to solving RVOS is to extract long-range temporal context information from the interactions of expressions and videos to depict the dynamic attributes of each object. Previous works either adopt attention across all the frames or stack dense local attention to achieve a global view of temporal context. However, they fail to strike a good balance between locality and globality, and the computation complexity significantly increases with the increase of video length. In this paper, we propose an effective long-range temporal context attention (LTCA) mechanism to aggregate global context information into object features. Specifically, we aggregate the global context information from two aspects. Firstly, we stack sparse local attentions to balance the locality and globality. We design a dilated window attention across frames to aggregate local context information and perform such attention in a stack of layers to enable a global view. Further, we enable each query to attend to a small group of keys randomly selected from a global pool to enhance the globality. Secondly, we design a global query to interact with all the other queries to directly encode the global context information. Experiments show our method achieves new state-of-the-art on four referring video segmentation benchmarks. Notably, our method shows an improvement of 11.3% and 8.3% on the MeViS valu and val datasets respectively.

[109] Unlocking 3D Affordance Segmentation with 2D Semantic Knowledge

Yu Huang,Zelin Peng,Changsong Wen,Xiaokang Yang,Wei Shen

Main category: cs.CV

TL;DR: 论文提出了一种基于2D视觉基础模型的语义知识迁移方法，通过跨模态亲和力传递（CMAT）和跨模态功能分割Transformer（CAST）提升3D功能分割的性能。

Details

Motivation: 3D功能分割面临数据稀疏、噪声和几何模糊等挑战，现有方法依赖点云编码器作为通用特征提取器，导致功能边界语义不一致。

Contribution: 1. 提出CMAT预训练策略，将2D语义知识迁移到3D域；2. 设计CAST模型，集成多模态提示生成精准分割图。

Method: 1. CMAT通过重建、亲和力和多样性联合优化对齐3D编码器与2D语义；2. CAST结合多模态提示和CMAT预训练特征。

Result: 在标准基准测试中取得最先进的结果。

Insight: 2D视觉基础模型的语义知识可以显著提升3D功能分割的性能。

Abstract: Affordance segmentation aims to parse 3D objects into functionally distinct parts, bridging recognition and interaction for applications in robotic manipulation, embodied AI, and AR. While recent studies leverage visual or textual prompts to guide this process, they often rely on point cloud encoders as generic feature extractors, overlooking the intrinsic challenges of 3D data such as sparsity, noise, and geometric ambiguity. As a result, 3D features learned in isolation frequently lack clear and semantically consistent functional boundaries. To address this bottleneck, we propose a semantic-grounded learning paradigm that transfers rich semantic knowledge from large-scale 2D Vision Foundation Models (VFMs) into the 3D domain. Specifically, We introduce Cross-Modal Affinity Transfer (CMAT), a pre-training strategy that aligns a 3D encoder with lifted 2D semantics and jointly optimizes reconstruction, affinity, and diversity to yield semantically organized representations. Building on this backbone, we further design the Cross-modal Affordance Segmentation Transformer (CAST), which integrates multi-modal prompts with CMAT-pretrained features to generate precise, prompt-aware segmentation maps. Extensive experiments on standard benchmarks demonstrate that our framework establishes new state-of-the-art results for 3D affordance segmentation.

[110] LinVideo: A Post-Training Framework towards O(n) Attention in Efficient Video Generation

Yushi Huang,Xingtong Ge,Ruihao Gong,Chengtao Lv,Jun Zhang

Main category: cs.CV

TL;DR: LinVideo是一个高效的视频生成后训练框架，将自注意力模块替换为线性注意力，显著降低了计算成本，同时保持生成质量。

Details

Motivation: 视频扩散模型的自注意力机制计算复杂度为二次方，导致视频序列生成时成本高昂。线性注意力虽降低复杂度，但完全替换需要昂贵的预训练。LinVideo旨在无需数据的情况下高效完成替换，减少计算成本。

Contribution: 1. 提出LinVideo框架，通过选择性转移和分布式匹配目标，高效替换自注意力模块；2. 引入自动化层选择方法，避免手动调整；3. 实现了1.25-2.00倍加速，且生成质量无损。

Method: 1. 框架通过选择性转移自动选择替换的注意力层；2. 提出ADM目标，对齐采样轨迹中各时间步的样本分布，提升替换效果；3. 实验验证框架的高效性。

Result: 实验表明，LinVideo在保持生成质量的同时，实现1.25-2.00倍加速；蒸馏模型进一步降低15.92倍延迟。

Insight: 1. 注意力层的可替换性差异显著，自动化选择是关键；2. 分布式匹配目标能有效提升替换后的性能；3. 框架适用于高效视频生成任务。

Abstract: Video diffusion models (DMs) have enabled high-quality video synthesis. However, their computation costs scale quadratically with sequence length because self-attention has quadratic complexity. While linear attention lowers the cost, fully replacing quadratic attention requires expensive pretraining due to the limited expressiveness of linear attention and the complexity of spatiotemporal modeling in video generation. In this paper, we present LinVideo, an efficient data-free post-training framework that replaces a target number of self-attention modules with linear attention while preserving the original model’s performance. First, we observe a significant disparity in the replaceability of different layers. Instead of manual or heuristic choices, we frame layer selection as a binary classification problem and propose selective transfer, which automatically and progressively converts layers to linear attention with minimal performance impact. Additionally, to overcome the ineffectiveness and inefficiency of existing objectives for this transfer process, we introduce an anytime distribution matching (ADM) objective that aligns the distributions of samples across any timestep along the sampling trajectory. This objective is efficient and recovers model performance. Extensive experiments show that our method achieves a 1.25-2.00x speedup while preserving generation quality, and our 4-step distilled model further delivers a 15.92x latency reduction with minimal visual quality drop.

[111] Evaluating Small Vision-Language Models on Distance-Dependent Traffic Perception

Nikos Theodoridis,Tim Brophy,Reenu Mohandas,Ganesh Sistu,Fiachra Collins,Anthony Scanlan,Ciaran Eising

Main category: cs.CV

TL;DR: 该论文提出了一个专注于交通场景感知的视觉问答基准DTPQA，评估了小规模视觉-语言模型（VLMs）在远近距离感知任务中的表现，发现其性能显著低于人类水平。

Details

Motivation: 自动驾驶系统需要可靠的感知能力，尤其是对远距离物体的准确感知。现有VLMs在交通场景中的表现尚未充分验证，尤其是在小规模模型受限的计算资源下。

Contribution: 1. 提出了首个专注于交通场景感知的VQA基准DTPQA，附带距离标注；2. 评估了小规模VLMs的感知能力，揭示其性能瓶颈。

Method: 1. 构建DTPQA基准，过滤掉需要推理的问题，仅保留感知任务；2. 测试多款SOTA小规模VLMs在DTPQA上的表现，并与人类基准对比。

Result: 小规模VLMs的平均准确率约为60%，远低于人类的85%。尤其是左右区分等任务表现较差。

Insight: 小规模VLMs在交通场景的感知任务中仍有较大改进空间，尤其是在远距离感知和方向判断方面。

Abstract: Vision-Language Models (VLMs) are becoming increasingly powerful, demonstrating strong performance on a variety of tasks that require both visual and textual understanding. Their strong generalisation abilities make them a promising component for automated driving systems, which must handle unexpected corner cases. However, to be trusted in such safety-critical applications, a model must first possess a reliable perception system. Moreover, since critical objects and agents in traffic scenes are often at a distance, we require systems that are not “shortsighted”, i.e., systems with strong perception capabilities at both close (up to 20 meters) and long (30+ meters) range. With this in mind, we introduce Distance-Annotated Traffic Perception Question Answering (DTPQA), the first Visual Question Answering (VQA) benchmark focused solely on perception-based questions in traffic scenes, enriched with distance annotations. By excluding questions that require reasoning, we ensure that model performance reflects perception capabilities alone. Since automated driving hardware has limited processing power and cannot support large VLMs, our study centers on smaller VLMs. More specifically, we evaluate several state-of-the-art (SOTA) small VLMs on DTPQA and show that, despite the simplicity of the questions, these models significantly underperform compared to humans (~60% average accuracy for the best-performing small VLM versus ~85% human performance). However, it is important to note that the human sample size was relatively small, which imposes statistical limitations. We also identify specific perception tasks, such as distinguishing left from right, that remain particularly challenging for these models.

[112] UniVideo: Unified Understanding, Generation, and Editing for Videos

Cong Wei,Quande Liu,Zixuan Ye,Qiulin Wang,Xintao Wang,Pengfei Wan,Kun Gai,Wenhu Chen

Main category: cs.CV

TL;DR: UniVideo是一个统一的视频理解、生成和编辑框架，通过双流设计（MLLM用于指令理解，MMDiT用于视频生成）实现了多任务统一建模，并在多种视频任务中达到或超越任务专用模型的表现。

Details

Motivation: 现有的统一多模态模型主要集中在图像领域，视频领域的统一建模仍有限。UniVideo旨在扩展统一建模到视频领域，支持复杂的多模态指令理解与生成任务。

Contribution: 1. 提出双流设计的UniVideo框架，结合MLLM和MMDiT，实现视频任务的统一建模；2. 支持任务组合（如编辑+风格迁移）和未见过的自由形式编辑任务迁移；3. 在文本/图像到视频生成、上下文视频生成和编辑任务中达到SOTA。

Method: 采用双流架构：1. MLLM解析多模态指令；2. MMDiT负责视频生成与编辑，通过联合训练实现多任务统一。还支持视觉提示引导的视频生成。

Result: UniVideo在文本/图像到视频生成、上下文视频生成和编辑任务中表现优异，支持任务组合和未见任务的泛化能力。

Insight: 双流设计和多任务联合训练是实现视频统一建模的有效方法，且图像编辑数据的知识可迁移到视频编辑任务。

Abstract: Unified multimodal models have shown promising results in multimodal content generation and editing but remain largely limited to the image domain. In this work, we present UniVideo, a versatile framework that extends unified modeling to the video domain. UniVideo adopts a dual-stream design, combining a Multimodal Large Language Model (MLLM) for instruction understanding with a Multimodal DiT (MMDiT) for video generation. This design enables accurate interpretation of complex multimodal instructions while preserving visual consistency. Built on this architecture, UniVideo unifies diverse video generation and editing tasks under a single multimodal instruction paradigm and is jointly trained across them. Extensive experiments demonstrate that UniVideo matches or surpasses state-of-the-art task-specific baselines in text/image-to-video generation, in-context video generation and in-context video editing. Notably, the unified design of UniVideo enables two forms of generalization. First, UniVideo supports task composition, such as combining editing with style transfer, by integrating multiple capabilities within a single instruction. Second, even without explicit training on free-form video editing, UniVideo transfers its editing capability from large-scale image editing data to this setting, handling unseen instructions such as green-screening characters or changing materials within a video. Beyond these core capabilities, UniVideo also supports visual-prompt-based video generation, where the MLLM interprets visual prompts and guides the MMDiT during synthesis. To foster future research, we will release our model and code.

[113] Detecting Legend Items on Historical Maps Using GPT-4o with In-Context Learning

Sofia Kirsanova,Yao-Yi Chiang,Weiwei Duan

Main category: cs.CV

TL;DR: 论文提出了一种结合LayoutLMv3和GPT-4o的方法，通过上下文学习检测历史地图图例项及其描述，性能优于基线，F-1分数达88%，IoU达85%。

Details

Motivation: 历史地图图例的布局和格式不一致，导致自动提取困难。现有方法主要专注于分割或OCR，缺乏有效结构化匹配图例符号与描述的方法。

Contribution: 结合LayoutLMv3检测布局与GPT-4o的上下文学习，实现图例项和描述的检测与链接，提升结构化解析能力。

Method: 使用LayoutLMv3进行布局检测，GPT-4o通过上下文学习和结构化JSON提示完成图例项与描述的匹配。

Result: 实验表明方法优于基线，F-1分数88%，IoU达85%，并验证了提示设计、示例数量和布局对齐对性能的影响。

Insight: 结构化提示和上下文学习能显著提升图例解析性能，支持大规模历史地图的可搜索性和索引。

Abstract: Historical map legends are critical for interpreting cartographic symbols. However, their inconsistent layouts and unstructured formats make automatic extraction challenging. Prior work focuses primarily on segmentation or general optical character recognition (OCR), with few methods effectively matching legend symbols to their corresponding descriptions in a structured manner. We present a method that combines LayoutLMv3 for layout detection with GPT-4o using in-context learning to detect and link legend items and their descriptions via bounding box predictions. Our experiments show that GPT-4 with structured JSON prompts outperforms the baseline, achieving 88% F-1 and 85% IoU, and reveal how prompt design, example counts, and layout alignment affect performance. This approach supports scalable, layout-aware legend parsing and improves the indexing and searchability of historical maps across various visual styles.

[114] VideoVerse: How Far is Your T2V Generator from a World Model?

Zeqing Wang,Xinyu Wei,Bairui Li,Zhen Guo,Jinrui Zhang,Hongyang Wei,Keze Wang,Lei Zhang

Main category: cs.CV

TL;DR: VideoVerse是一个新的评测基准，专注于评估T2V模型在时间因果关系和世界知识理解方面的能力，填补了现有评测的不足，并对当前T2V模型的性能进行了系统分析。

Details

Motivation: 现有评测基准在评估T2V模型的能力（如时间因果性和世界知识）方面存在不足，无法区分最新模型的性能，因此需要一个新的全面评测工具。

Contribution: 提出了VideoVerse，一个专注于时间因果关系和世界知识评测的基准，包含多样化的视频和事件描述，设计了十个评测维度的二进制问题，并对当前T2V模型进行系统评估。

Method: 收集多样化的视频并提取事件级描述，重写为T2V提示，设计二进制评测问题；利用现代视觉语言模型开发基于QA的人类偏好对齐评测流程。

Result: VideoVerse包含300个提示、815个事件和793个评测问题，系统评估了开源和闭源T2V模型，揭示了其在构建世界模型方面的差距。

Insight: 时间因果关系和世界知识是T2V模型的关键能力，但目前模型在这方面仍有显著不足，需进一步改进。

Abstract: The recent rapid advancement of Text-to-Video (T2V) generation technologies, which are critical to build ``world models’’, makes the existing benchmarks increasingly insufficient to evaluate state-of-the-art T2V models. First, current evaluation dimensions, such as per-frame aesthetic quality and temporal consistency, are no longer able to differentiate state-of-the-art T2V models. Second, event-level temporal causality, which not only distinguishes video from other modalities but also constitutes a crucial component of world models, is severely underexplored in existing benchmarks. Third, existing benchmarks lack a systematic assessment of world knowledge, which are essential capabilities for building world models. To address these issues, we introduce VideoVerse, a comprehensive benchmark that focuses on evaluating whether a T2V model could understand complex temporal causality and world knowledge in the real world. We collect representative videos across diverse domains (e.g., natural landscapes, sports, indoor scenes, science fiction, chemical and physical experiments) and extract their event-level descriptions with inherent temporal causality, which are then rewritten into text-to-video prompts by independent annotators. For each prompt, we design a suite of binary evaluation questions from the perspective of dynamic and static properties, with a total of ten carefully defined evaluation dimensions. In total, our VideoVerse comprises 300 carefully curated prompts, involving 815 events and 793 binary evaluation questions. Consequently, a human preference aligned QA-based evaluation pipeline is developed by using modern vision-language models. Finally, we perform a systematic evaluation of state-of-the-art open-source and closed-source T2V models on VideoVerse, providing in-depth analysis on how far the current T2V generators are from world models.

[115] Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency

Kaiwen Zheng,Yuji Wang,Qianli Ma,Huayu Chen,Jintao Zhang,Yogesh Balaji,Jianfei Chen,Ming-Yu Liu,Jun Zhu,Qinsheng Zhang

Main category: cs.CV

TL;DR: 该论文首次将连续时间一致性蒸馏扩展到大规模文本到图像和视频任务中，提出了一种新的得分正则化连续时间一致性模型（rCM），显著提升了生成质量和多样性，同时大幅加速了扩散采样。

Details

Motivation: 连续时间一致性模型（sCM）虽然在学术规模扩散加速中表现出色，但在大规模文本到图像和视频任务中的应用仍面临基础设施挑战和质量限制。作者旨在解决这些问题，并提出一种改进方法。

Contribution: 1) 开发了并行兼容的FlashAttention-2 JVP内核，支持10B参数以上模型的训练；2) 揭示了sCM在细节生成中的质量限制，并提出得分正则化rCM以改进；3) 在大规模模型和视频任务中验证了rCM的性能。

Method: 作者提出了得分正则化连续时间一致性模型（rCM），通过引入得分蒸馏作为长跳跃正则化器，补充sCM的“模式覆盖”目标，结合“模式寻求”反向散度提升质量。

Result: rCM在质量指标上匹配或超越现有最佳蒸馏方法DMD2，同时提供更高的多样性，仅需1~~4步即可生成高质量样本，加速扩散采样15~~50倍。

Insight: 得分蒸馏作为一种正则化器，可以有效弥补sCM在细节生成中的不足，为大模型蒸馏提供了一种实用且理论扎实的框架。

Abstract: This work represents the first effort to scale up continuous-time consistency distillation to general application-level image and video diffusion models. Although continuous-time consistency model (sCM) is theoretically principled and empirically powerful for accelerating academic-scale diffusion, its applicability to large-scale text-to-image and video tasks remains unclear due to infrastructure challenges in Jacobian-vector product (JVP) computation and the limitations of standard evaluation benchmarks. We first develop a parallelism-compatible FlashAttention-2 JVP kernel, enabling sCM training on models with over 10 billion parameters and high-dimensional video tasks. Our investigation reveals fundamental quality limitations of sCM in fine-detail generation, which we attribute to error accumulation and the “mode-covering” nature of its forward-divergence objective. To remedy this, we propose the score-regularized continuous-time consistency model (rCM), which incorporates score distillation as a long-skip regularizer. This integration complements sCM with the “mode-seeking” reverse divergence, effectively improving visual quality while maintaining high generation diversity. Validated on large-scale models (Cosmos-Predict2, Wan2.1) up to 14B parameters and 5-second videos, rCM matches or surpasses the state-of-the-art distillation method DMD2 on quality metrics while offering notable advantages in diversity, all without GAN tuning or extensive hyperparameter searches. The distilled models generate high-fidelity samples in only $1\sim4$ steps, accelerating diffusion sampling by $15\times\sim50\times$. These results position rCM as a practical and theoretically grounded framework for advancing large-scale diffusion distillation.

[116] Gaze on the Prize: Shaping Visual Attention with Return-Guided Contrastive Learning

Andrew Lee,Ian Chuang,Dechen Gao,Kai Fukazawa,Iman Soltani

Main category: cs.CV

TL;DR: 这篇论文提出了一种名为‘Gaze on the Prize’的框架，通过引入可学习的视觉注视机制（Gaze）和基于回报的自监督信号（Prize），解决视觉强化学习（RL）中样本效率低和稳定性差的问题。

Details

Motivation: 视觉RL代理需要处理高维图像数据，但其中仅一小部分像素与任务相关，导致代理浪费大量资源在无关特征上，学习效率低下且不稳定。论文受人类视觉注视机制启发，试图通过注意力机制提升任务相关特征的聚焦能力。

Contribution: 1. 提出了一种可学习的视觉注视机制（Gaze），通过自监督信号（Prize）引导注意力聚焦任务相关特征；
2. 设计了一种基于回报差异的对比学习方法，利用正负样本和三元组训练注意力机制；
3. 在不修改基线算法或超参数的情况下，显著提高了样本效率并解决了基线无法学习的任务。

Method: 1. 使用基于回报差异的自监督信号，将相似的视觉表征分为正负样本；
2. 构建对比学习三元组，训练注意力机制生成可区分的表征；
3. 在ManiSkill3基准测试中验证了方法的有效性。

Result: 在ManiSkill3基准测试中，该方法实现了高达2.4倍的样本效率提升，并能解决基线方法无法学习的任务。

Insight: 回报差异可以揭示任务相关性：如果两个相似的表征导致不同的结果，那么它们的区分特征很可能是任务相关的。这一洞察为视觉RL中的注意力机制提供了新的训练信号。

Abstract: Visual Reinforcement Learning (RL) agents must learn to act based on high-dimensional image data where only a small fraction of the pixels is task-relevant. This forces agents to waste exploration and computational resources on irrelevant features, leading to sample-inefficient and unstable learning. To address this, inspired by human visual foveation, we introduce Gaze on the Prize. This framework augments visual RL with a learnable foveal attention mechanism (Gaze), guided by a self-supervised signal derived from the agent’s experience pursuing higher returns (the Prize). Our key insight is that return differences reveal what matters most: If two similar representations produce different outcomes, their distinguishing features are likely task-relevant, and the gaze should focus on them accordingly. This is realized through return-guided contrastive learning that trains the attention to distinguish between the features relevant to success and failure. We group similar visual representations into positives and negatives based on their return differences and use the resulting labels to construct contrastive triplets. These triplets provide the training signal that teaches the attention mechanism to produce distinguishable representations for states associated with different outcomes. Our method achieves up to 2.4x improvement in sample efficiency and can solve tasks that the baseline fails to learn, demonstrated across a suite of manipulation tasks from the ManiSkill3 benchmark, all without modifying the underlying algorithm or hyperparameters.

[117] Hierarchical Spatial Algorithms for High-Resolution Image Quantization and Feature Extraction

Noor Islam S. Mohammad

Main category: cs.CV

TL;DR: 该论文提出了一种模块化的空间图像处理框架，整合了灰度量化、颜色和亮度增强、图像锐化、双向变换管道以及几何特征提取技术，展示了其在实时图像分析和计算机视觉中的潜力。

Details

Motivation: 随着高分辨率图像的广泛应用，传统的图像处理方法在效率和准确性上存在不足。论文旨在通过模块化框架解决这一问题，提升图像处理的灵活性和性能。

Contribution: 提出了一个模块化的图像处理框架，整合了多种技术（如灰度量化、颜色增强、几何特征提取），并通过双向变换管道实现了高精度的图像处理流程。

Method: 采用逐步强度变换量化灰度图像，结合RGB和YCrCb空间的直方图均衡化增强颜色，通过HSV值通道调整亮度，使用3x3卷积核锐化图像，并整合Canny边缘检测、Hough变换和Harris角点检测提取几何特征。

Result: 双向变换管道的准确率达到76.10%（正向）和74.80%（反向），几何特征提取的相似度为81.87%，展示了方法的鲁棒性和实时性。

Insight: 论文表明，模块化的空间图像处理方法能显著提升图像分析的效率和准确性，尤其在实时应用中具有重要价值。

Abstract: This study introduces a modular framework for spatial image processing, integrating grayscale quantization, color and brightness enhancement, image sharpening, bidirectional transformation pipelines, and geometric feature extraction. A stepwise intensity transformation quantizes grayscale images into eight discrete levels, producing a posterization effect that simplifies representation while preserving structural detail. Color enhancement is achieved via histogram equalization in both RGB and YCrCb color spaces, with the latter improving contrast while maintaining chrominance fidelity. Brightness adjustment is implemented through HSV value-channel manipulation, and image sharpening is performed using a 3 * 3 convolution kernel to enhance high-frequency details. A bidirectional transformation pipeline that integrates unsharp masking, gamma correction, and noise amplification achieved accuracy levels of 76.10% and 74.80% for the forward and reverse processes, respectively. Geometric feature extraction employed Canny edge detection, Hough-based line estimation (e.g., 51.50{\deg} for billiard cue alignment), Harris corner detection, and morphological window localization. Cue isolation further yielded 81.87% similarity against ground truth images. Experimental evaluation across diverse datasets demonstrates robust and deterministic performance, highlighting its potential for real-time image analysis and computer vision.

[118] Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Tools

Zhenlong Yuan,Xiangyan Qu,Chengxuan Qian,Rui Chen,Jing Tang,Lei Sun,Xiangxiang Chu,Dapeng Zhang,Yiwei Wang,Yujun Cai,Shuo Li

Main category: cs.CV

TL;DR: 论文提出了Video-STAR框架，通过结合上下文子运动分解和工具增强的强化学习，解决了开放词汇动作识别中语义相似动作的区分问题。

Details

Motivation: 现有的多模态大语言模型（MLLMs）依赖文本先验，难以在开放词汇场景中区分语义相似的动作。

Contribution: 1. 提出Video-STAR框架，将动作分解为判别性子运动以进行细粒度匹配；2. 动态调用领域特定工具实现跨模态交错推理；3. 设计分层奖励机制平衡工具使用效率和推理结构一致性。

Method: 采用上下文子运动分解和工具增强的强化学习，通过分层奖励机制动态优化子运动匹配和工具调用。

Result: 在HMDB-51、UCF-101、SSv2等多个数据集上达到SOTA性能，显著优于现有方法。

Insight: 结合子运动分解和工具增强的方法能够有效减少跨模态幻觉，提升开放词汇动作识别的鲁棒性和泛化能力。

Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable potential in bridging visual and textual reasoning, yet their reliance on text-centric priors often limits their ability to disentangle semantically similar actions in open-vocabulary scenarios. To address this, we propose Video-STAR, a framework that harmonizes contextual sub-motion decomposition with tool-augmented reinforcement learning for open-vocabulary action recognition (OVAR). Unlike prior methods that treat actions as monolithic entities, our approach innovatively decomposes actions into discriminative sub-motions for fine-grained matching while dynamically invoking domain-specific tools for cross-modal interleaving, thereby enabling category-specific reasoning capacity and reducing cross-modal hallucination. Moreover, by designing a hierarchical reward that balances tool-usage efficiency, sub-motion relevance, and structural coherence in reasoning, our method autonomously leverages external tools to prioritize sub-motion patterns without explicit supervision, transmitting from text-centric reasoning to visually grounded inference. Extensive evaluations on HMDB-51, UCF-101, SSv2, Kinetics-400, and Kinetics-600 datasets demonstrate our state-of-the-art performance, outperforming existing methods in distinguishing fine-grained actions and handling cross-modal hallucination, validating our excellent robustness and generalization.

[119] The Visual Iconicity Challenge: Evaluating Vision-Language Models on Sign Language Form-Meaning Mapping

Onur Keleş,Aslı Özyürek,Gerardo Ortega,Kadir Gökgö,Esam Ghaleb

Main category: cs.CV

TL;DR: 该论文提出了《视觉象似性挑战》，通过三个任务评估视觉语言模型在手语中的象似性表现，发现模型的性能仍远低于人类基线，但部分模型在音系形式和象似性评分任务中表现出相关性。

Details

Motivation: 手语中的象似性（形式与意义的相似性）为评估视觉语言模型的视觉基础能力提供了天然测试环境。论文旨在研究模型是否能从动态的人类动作中恢复这些映射关系。

Contribution: 1. 设计了《视觉象似性挑战》基准，包含音系形式预测、透明度和象似性评分三个任务；2. 评估了13种先进视觉语言模型，并与人类基线对比；3. 发现模型性能低于人类，但在某些任务中表现出相关性。

Method: 论文选择了13种视觉语言模型，在零样本和少样本设置下进行评估。任务包括：（i）音系符号形式预测（如手势、位置），（ii）透明度（从视觉形式推断意义），（iii）象似性分级评分。

Result: 1. 音系形式预测中，模型表现部分接近人类，但仍低于基线；2. 透明度任务中模型表现较差；3. 少数顶级模型与人类的象似性评分相关性适中。发现模型对音系形式的预测能力与象似性评分相关性较强。

Insight: 论文表明视觉语言模型在视觉基础任务中仍有改进空间，尤其是透明度和象似性评分任务。验证了音系形式预测与象似性评分的相关性，提示可以通过人类中心信号和具身学习方法提升模型的视觉基础能力。

Abstract: Iconicity, the resemblance between linguistic form and meaning, is pervasive in signed languages, offering a natural testbed for visual grounding. For vision-language models (VLMs), the challenge is to recover such essential mappings from dynamic human motion rather than static context. We introduce the \textit{Visual Iconicity Challenge}, a novel video-based benchmark that adapts psycholinguistic measures to evaluate VLMs on three tasks: (i) phonological sign-form prediction (e.g., handshape, location), (ii) transparency (inferring meaning from visual form), and (iii) graded iconicity ratings. We assess $13$ state-of-the-art VLMs in zero- and few-shot settings on Sign Language of the Netherlands and compare them to human baselines. On \textit{phonological form prediction}, VLMs recover some handshape and location detail but remain below human performance; on \textit{transparency}, they are far from human baselines; and only top models correlate moderately with human \textit{iconicity ratings}. Interestingly, \textit{models with stronger phonological form prediction correlate better with human iconicity judgment}, indicating shared sensitivity to visually grounded structure. Our findings validate these diagnostic tasks and motivate human-centric signals and embodied learning methods for modelling iconicity and improving visual grounding in multimodal models.

[120] InstructX: Towards Unified Visual Editing with MLLM Guidance

Chong Mou,Qichao Sun,Yanze Wu,Pengze Zhang,Xinghui Li,Fulong Ye,Songtao Zhao,Qian He

Main category: cs.CV

TL;DR: InstructX是一个统一的图像和视频编辑框架，利用多模态大型语言模型（MLLM）指导扩散模型，通过图像数据训练实现视频编辑能力，并在单一模型中统一处理图像和视频编辑任务。

Details

Motivation: 随着MLLM在视觉理解和推理方面的进步，研究者希望利用MLLM提升扩散模型的编辑性能。然而，MLLM的设计选择和与扩散模型的集成仍然是一个挑战，尤其是在视频编辑等复杂任务中。

Contribution: 1. 分析了MLLM与扩散模型的集成设计选择。2. 展示了在图像数据上的训练可以无需显式监督地实现视频编辑能力，缓解了视频数据稀缺的问题。3. 提出了一种统一框架，在单一模型中处理图像和视频编辑任务。

Method: InstructX通过深入研究MLLM与扩散模型的集成，分析了图像和视频在统一建模中的合作与区别，结合模态特定的MLLM特征，实现了图像和视频编辑任务的统一处理。

Result: 实验表明，该方法能够广泛处理图像和视频编辑任务，并达到最先进的性能。

Insight: 图像数据训练可以隐式地提升视频编辑能力，且模态特定的MLLM特征是统一建模的关键。

Abstract: With recent advances in Multimodal Large Language Models (MLLMs) showing strong visual understanding and reasoning, interest is growing in using them to improve the editing performance of diffusion models. Despite rapid progress, most studies lack an in-depth analysis of MLLM design choices. Moreover, the integration of MLLMs and diffusion models remains an open challenge in some difficult tasks, such as video editing. In this paper, we present InstructX, a unified framework for image and video editing. Specifically, we conduct a comprehensive study on integrating MLLMs and diffusion models for instruction-driven editing across diverse tasks. Building on this study, we analyze the cooperation and distinction between images and videos in unified modeling. (1) We show that training on image data can lead to emergent video editing capabilities without explicit supervision, thereby alleviating the constraints imposed by scarce video training data. (2) By incorporating modality-specific MLLM features, our approach effectively unifies image and video editing tasks within a single model. Extensive experiments demonstrate that our method can handle a broad range of image and video editing tasks and achieves state-of-the-art performance.

[121] MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restoration

Lu Liu,Chunlei Cai,Shaocheng Shen,Jianfeng Liang,Weimin Ouyang,Tianxiao Ye,Jian Mao,Huiyu Duan,Jiangchao Yao,Xiaoyun Zhang,Qiang Hu,Guangtao Zhai

Main category: cs.CV

TL;DR: MoA-VR提出了一种基于多智能体系统的视频修复方法，通过三个协同工作的智能体（退化识别、路由与修复、修复质量评估）模拟专业人工修复过程，实现对复杂退化问题的全面处理。

Details

Motivation: 实际视频常因采集和传输条件导致多种退化问题（如噪声、压缩伪影、低光失真）。现有方法需专业手动选择模型或依赖单一架构，难以泛化处理多样退化问题。受专家经验启发，MoA-VR旨在通过多智能体协作实现通用视频修复。

Contribution: 1. 首个基于多智能体（Mixture-of-Agents）的视频修复系统；2. 构建大规模高分辨率视频退化识别基准和VLM驱动的退化识别器；3. 引入LLM驱动的自适应路由器和VLM修复质量评估模型。

Method: 方法分为三部分：1. VLM驱动的退化识别器；2. LLM驱动的路由器自主学习修复策略；3. VLM设计的修复质量评估模型（基于Res-VQ数据集）。

Result: 实验表明，MoA-VR在多样化和复合退化问题上表现优异，客观指标和感知质量均超越现有基线方法。

Insight: MoA-VR展示了多模态智能和模块化推理在通用视频修复系统中的潜力，为未来研究提供了新方向。

Abstract: Real-world videos often suffer from complex degradations, such as noise, compression artifacts, and low-light distortions, due to diverse acquisition and transmission conditions. Existing restoration methods typically require professional manual selection of specialized models or rely on monolithic architectures that fail to generalize across varying degradations. Inspired by expert experience, we propose MoA-VR, the first \underline{M}ixture-\underline{o}f-\underline{A}gents \underline{V}ideo \underline{R}estoration system that mimics the reasoning and processing procedures of human professionals through three coordinated agents: Degradation Identification, Routing and Restoration, and Restoration Quality Assessment. Specifically, we construct a large-scale and high-resolution video degradation recognition benchmark and build a vision-language model (VLM) driven degradation identifier. We further introduce a self-adaptive router powered by large language models (LLMs), which autonomously learns effective restoration strategies by observing tool usage patterns. To assess intermediate and final processed video quality, we construct the \underline{Res}tored \underline{V}ideo \underline{Q}uality (Res-VQ) dataset and design a dedicated VLM-based video quality assessment (VQA) model tailored for restoration tasks. Extensive experiments demonstrate that MoA-VR effectively handles diverse and compound degradations, consistently outperforming existing baselines in terms of both objective metrics and perceptual quality. These results highlight the potential of integrating multimodal intelligence and modular reasoning in general-purpose video restoration systems.

[122] To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Models

Jiayun Luo,Wan-Cyuan Fan,Lyuyang Wang,Xiangteng He,Tanzila Rahman,Purang Abolmaesumi,Leonid Sigal

Main category: cs.CV

TL;DR: 这篇论文研究了大型视觉语言模型（LVLM）中视觉Transformer（ViT）生成的注意力汇聚点（attention sinks），发现这些高范数值的视觉token蕴含了图像的语义信息，但对LVLM的性能影响却被忽视。论文提出了训练无关和基于训练的方法，验证了有效利用这些token可以显著提升视觉推理任务的表现。

Details

Motivation: 现有研究主要关注LLM中的注意力汇聚点，但忽略了ViT中生成的视觉token对LVLM理解和推理的关键作用。这些高范数值的视觉token（ViT attention sinks）是否以及如何影响模型性能尚不清楚。

Contribution: 1. 首次系统地研究了ViT生成的注意力汇聚点及其语义重要性；2. 提出了定性定量分析方法；3. 设计了训练无关和基于训练的方法，显著提升了LVLM在视觉推理任务中的表现。

Method: 1. 识别ViT中的高范数值视觉token（ViT attention sinks）；2. 通过定性（可视化）和定量（任务性能）分析其语义信息；3. 提出两种方法：训练无关的直接利用和基于训练的优化策略。

Result: 实验表明，充分利用ViT attention sinks可以显著提升LVLM在多类视觉推理任务中的性能，证明了这些token在提升模型推理能力中的重要性。

Insight: ViT生成的视觉token并非均匀重要，高范数值的token蕴含更多语义信息，但现有模型架构未能充分利用。通过显式关注这些token，可以进一步挖掘LVLM的潜力。

Abstract: Large Vision Language Models (LVLMs) have recently emerged as powerful architectures capable of understanding and reasoning over both visual and textual information. These models typically rely on two key components: a Vision Transformer (ViT) and a Large Language Model (LLM). ViT encodes visual content into a sequence of image tokens and serves as the perceptual front-end – the eyes of the model. In contrast, the LLM interprets these tokens to perform high-level reasoning, generates responses, and functions as the cognitive core – the brain of the model. However, it remains unclear which visual tokens contribute most significantly to understanding and reasoning, and how effectively these signals are propagated from ViT to the LLM. While most existing works have focused on identifying attention sinks, low-semantic tokens receiving disproportionately high attention, within the LLM, we shift the focus to the vision encoder by identifying a class of high-norm visual tokens from ViT, referred to as ViT attention sinks – a problem that has been rarely studied but is indeed very important for LVLMs. Our findings show that these ViT sinks encapsulate high-level semantic concepts from images, allowing the LLM to perform more effective understanding and reasoning. Despite their importance, these sink tokens are often overlooked in existing LVLM architectures. To explore their contribution, we present both qualitative and quantitative analyses of the information embedded in these sink tokens. We also propose both training-free and training-based approaches to better leverage how this information is interpreted by the LLM, and to what extent. By explicitly utilizing these tokens, we demonstrate substantial improvements across a range of LVLMs and visual reasoning tasks, highlighting the untapped potential of ViT attention sinks in enhancing visual reasoning.

[123] SliceFine: The Universal Winning-Slice Hypothesis for Pretrained Networks

Md Kowsher,Ali O. Polat,Ehsan Mohammady Ardehaly,Mehrdad Salehi,Zia Ghiasi,Prasanth Murali,Chen Chen

Main category: cs.CV

TL;DR: 论文提出了一个理论框架，解释了为什么在预训练模型中微调随机选择的子网络（切片）足以适应下游任务。作者通过证明预训练网络的‘通用获胜切片’特性，提出了SliceFine方法，这是一种无需引入新参数的高效微调技术。

Details

Motivation: 当前参数高效微调（PEFT）方法通常需要引入额外参数（如适配器），导致效率下降。本文旨在从理论上解释预训练模型的冗余性，并提出一种更高效的微调方法。

Contribution: 1. 提出了‘通用获胜切片假设’，解释了预训练模型的切片特性；2. 设计了SliceFine方法，通过仅更新权重切片实现高效微调；3. 理论和实验验证了方法的有效性。

Method: 1. 理论分析预训练网络的频谱平衡和任务能量特性；2. 提出SliceFine方法，仅更新权重切片，无需引入新参数；3. 在语言和视觉任务上进行实验验证。

Result: SliceFine在性能上与现有PEFT方法相当，同时在训练速度、内存效率和模型紧凑性上显著提升。

Insight: 预训练网络存在天然的冗余性，仅更新部分切片即可高效适应下游任务，无需复杂调整或额外参数。

Abstract: This paper presents a theoretical framework explaining why fine tuning small, randomly selected subnetworks (slices) within pre trained models can be sufficient for downstream adaptation. We prove that pretrained networks exhibit a universal winning slice property arising from two phenomena: (1) spectral balance the eigenspectra of different weight matrix slices are remarkably similar; and (2) high task energy their backbone representations retain rich, task relevant features. This leads to the Universal Winning Slice Hypothesis, which provides a theoretical foundation for parameter efficient fine tuning (PEFT) in large scale models. Inspired by this, we propose SliceFine, a PEFT method that exploits this inherent redundancy by updating only selected slices of the original weights introducing zero new parameters, unlike adapter-based approaches. Empirically, SliceFine matches the performance of state of the art PEFT methods across language and vision tasks, while significantly improving training speed, memory efficiency, and model compactness. Our work bridges theory and practice, offering a theoretically grounded alternative to existing PEFT techniques.

[124] FlexTraj: Image-to-Video Generation with Flexible Point Trajectory Control

Zhiyuan Zhang,Can Wang,Dongdong Chen,Jing Liao

Main category: cs.CV

TL;DR: FlexTraj 是一个支持灵活点轨迹控制的图像到视频生成框架，通过统一的点基运动表示和高效的序列拼接方案，实现了多粒度和非对齐条件下的可控视频生成。

Details

Motivation: 现有视频生成方法在轨迹控制上通常依赖对齐条件或密集监督，FlexTraj 旨在通过统一的点基运动表示和高效的训练策略，实现更灵活、更鲁棒的轨迹控制。

Contribution: 1. 提出了一种统一的点基运动表示方法；2. 设计了高效的序列拼接方案，替代传统的条件注入方法；3. 采用了退火训练策略，减少对齐条件的依赖。

Method: 1. 用分割ID、轨迹ID和颜色通道编码点轨迹；2. 通过序列拼接注入轨迹条件；3. 采用退火训练策略，逐步减少对齐条件的监督。

Result: FlexTraj 支持多粒度的轨迹控制，并在多项应用中表现优异，如运动克隆、拖拽式视频生成、相机重定向等。

Insight: 点基运动表示和序列拼接的结合可以显著提升视频生成的可控性和效率，同时退火训练策略有效缓解了对对齐条件的依赖性。

Abstract: We present FlexTraj, a framework for image-to-video generation with flexible point trajectory control. FlexTraj introduces a unified point-based motion representation that encodes each point with a segmentation ID, a temporally consistent trajectory ID, and an optional color channel for appearance cues, enabling both dense and sparse trajectory control. Instead of injecting trajectory conditions into the video generator through token concatenation or ControlNet, FlexTraj employs an efficient sequence-concatenation scheme that achieves faster convergence, stronger controllability, and more efficient inference, while maintaining robustness under unaligned conditions. To train such a unified point trajectory-controlled video generator, FlexTraj adopts an annealing training strategy that gradually reduces reliance on complete supervision and aligned condition. Experimental results demonstrate that FlexTraj enables multi-granularity, alignment-agnostic trajectory control for video generation, supporting various applications such as motion cloning, drag-based image-to-video, motion interpolation, camera redirection, flexible action control and mesh animations.

[125] SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models

Hongxing Li,Dingming Li,Zixuan Wang,Yuchen Yan,Hang Wu,Wenqi Zhang,Yongliang Shen,Weiming Lu,Jun Xiao,Yueting Zhuang

Main category: cs.CV

TL;DR: 论文提出了一种渐进式训练方法SpatialLadder，用于提升视觉语言模型的空间推理能力，通过三阶段训练框架（空间感知、空间理解和复杂推理）显著提升了性能。

Details

Motivation: 当前视觉语言模型在空间推理任务中表现不佳，主要原因是缺乏层次化的感知和理解基础。

Contribution: 1. 提出了多模态数据集SpatialLadder-26k；2. 设计了渐进式三阶段训练框架；3. 实现了SOTA性能（23.4%提升）。

Method: 1. 通过目标定位建立空间感知；2. 通过多维度任务培养空间理解；3. 利用强化学习加强复杂推理。

Result: SpatialLadder模型在基准测试中性能超越GPT-4o和Gemini-2.0-Flash，泛化能力提升7.2%。

Insight: 渐进式训练从感知到推理对构建鲁棒的空间智能至关重要。

Abstract: Spatial reasoning remains a fundamental challenge for Vision-Language Models (VLMs), with current approaches struggling to achieve robust performance despite recent advances. We identify that this limitation stems from a critical gap: existing methods attempt to learn spatial reasoning directly without establishing the hierarchical foundations of perception and understanding. To address this challenge, we present a comprehensive methodology for building spatial intelligence progressively. We introduce SpatialLadder-26k, a multimodal dataset containing 26,610 samples spanning object localization, single image, multi-view, and video spatial reasoning tasks, constructed through a standardized pipeline that ensures systematic coverage across modalities. Building on this dataset, we design a three-stage progressive training framework that (1) establishes spatial perception through object localization, (2) develops spatial understanding through multi-dimensional spatial tasks, and (3) strengthens complex reasoning via reinforcement learning with verifiable rewards. This approach yields SpatialLadder, a 3B-parameter model that achieves state-of-the-art performance on spatial reasoning benchmarks, with 23.4% average improvement over the base model, surpassing GPT-4o by 20.8% and Gemini-2.0-Flash by 10.1%. Notably, SpatialLadder maintains strong generalization with 7.2% improvement on out-of-domain benchmarks, demonstrating that progressive training from perception to reasoning is essential for robust spatial intelligence.

[126] Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editing

Rishubh Parihar,Or Patashnik,Daniil Ostashev,R. Venkatesh Babu,Daniel Cohen-Or,Kuan-Chieh Wang

Main category: cs.CV

TL;DR: Kontinuous Kontext通过引入编辑强度的连续标量控制，扩展了基于指令的图像编辑模型的精细控制能力，实现了从细微到大幅度的统一编辑调整。

Details

Motivation: 现有的基于指令的图像编辑方法虽直观强大，但缺乏对编辑强度的精细控制，限制了用户体验，因此需要一种能够连续调整编辑强度的解决方案。

Contribution: 1）提出Kontinuous Kontext框架，扩展现有模型以支持编辑强度的连续控制；2）设计轻量级投影网络将标量输入映射到调制空间；3）合成高质量的数据集用于训练。

Method: 1）扩展现有多模态编辑模型，引入标量编辑强度输入；2）训练投影网络将标量和文本指令映射到调制系数；3）利用生成模型合成数据集并通过筛选确保质量。

Result: Kontinuous Kontext支持多种编辑操作（如风格化、属性、材质等）的连续强度调整，无需针对特定属性的训练。

Insight: 通过引入连续标量控制，实现了编辑强度的灵活调整，增强了用户对编辑过程的控制力，同时保持了模型的通用性。

Abstract: Instruction-based image editing offers a powerful and intuitive way to manipulate images through natural language. Yet, relying solely on text instructions limits fine-grained control over the extent of edits. We introduce Kontinuous Kontext, an instruction-driven editing model that provides a new dimension of control over edit strength, enabling users to adjust edits gradually from no change to a fully realized result in a smooth and continuous manner. Kontinuous Kontext extends a state-of-the-art image editing model to accept an additional input, a scalar edit strength which is then paired with the edit instruction, enabling explicit control over the extent of the edit. To inject this scalar information, we train a lightweight projector network that maps the input scalar and the edit instruction to coefficients in the model’s modulation space. For training our model, we synthesize a diverse dataset of image-edit-instruction-strength quadruplets using existing generative models, followed by a filtering stage to ensure quality and consistency. Kontinuous Kontext provides a unified approach for fine-grained control over edit strength for instruction driven editing from subtle to strong across diverse operations such as stylization, attribute, material, background, and shape changes, without requiring attribute-specific training.

[127] MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with Holistic Platform and Adaptive Hybrid Policy Optimization

Xiangyu Zhao,Junming Lin,Tianhao Liang,Yifan Zhou,Wenhao Chai,Yuzhe Gu,Weiyun Wang,Kai Chen,Gen Luo,Wenwei Zhang,Junchi Yan,Hua Yang,Haodong Duan,Xue Yang

Main category: cs.CV

TL;DR: MM-HELIX针对多模态大语言模型（MLLMs）的长链反思推理能力不足，提出了一个多维基准数据集和自适应混合策略优化方法，显著提升了模型性能。

Details

Motivation: 当前多模态大语言模型在数学和逻辑推理任务中表现优异，但在需要迭代思考和回溯的长链反思推理任务中表现不足。为此，作者研究了这一能力的现状，并提出解决方法。

Contribution: 1. 构建了MM-HELIX基准数据集（1,260个样本）；2. 生成了MM-HELIX-100K（10万高质量样本）用于指令微调；3. 提出自适应混合策略优化（AHPO），结合离线监督和在线优化。

Method: 通过数据合成引擎生成基准数据集，并提出AHPO训练策略，动态统一离线监督和在线优化。使用Step-Elicited Response Generation生成大规模训练数据。

Result: 在Qwen2.5-VL-7B模型上，AHPO方法在MM-HELIX基准上提升了18.6%的准确率，并在通用数学和逻辑任务中平均提升5.7%。

Insight: 长链反思推理能力可通过高质量数据生成和策略优化有效提升，为开发更强MLLMs提供了新思路。

Abstract: While current Multimodal Large Language Models (MLLMs) have demonstrated proficiency in reasoning tasks such as mathematics and logic, their capacity for long-chain reflective reasoning, a prerequisite for solving complex real-world problems, remains largely underexplored. In this work, we first conduct an extensive empirical investigation to evaluate this capability. Leveraging a carefully designed data synthesis engine, we construct MM-HELIX, a multimodal benchmark consisting 1,260 samples of 42 challenging synthetic tasks that require iterative thinking and backtracking. Empirical results on this benchmark reveal that existing MLLMs exhibit significant performance deficits in long-chain reflective reasoning. To address this limitation, we generate post-training data and further explore learning paradigms for exploiting such data. We first develop the Step-Elicited Response Generation pipeline to create MM-HELIX-100K, a large-scale dataset of 100k high-quality, reflective reasoning traces for instruction-tuning stage. Given that standard Reinforcement Learning fails on complex tasks due to sparse reward signals and catastrophic forgetting after Supervised Fine-Tuning, we propose Adaptive Hybrid Policy Optimization (AHPO), a novel training strategy that dynamically unifies offline supervision and online optimization into a single stage. This strategy enables the model to learn from expert data when rewards are sparse and conduct independent exploration once proficient. When applied to the Qwen2.5-VL-7B baseline, our method achieves a +18.6% accuracy improvement on MM-HELIX benchmark and demonstrates strong generalization with a +5.7% average performance gain on general mathematic and logic tasks. Our work demonstrate that reflective reasoning in MLLMs can be effectively learned and generalized, paving the way for developing more capable MLLMs.

[128] VideoNorms: Benchmarking Cultural Awareness of Video Language Models

Nikhil Reddy Varimalla,Yunfei Xu,Arkadiy Saakyan,Meng Fan Wang,Smaranda Muresan

Main category: cs.CV

TL;DR: VideoNorms是一个评估视频大语言模型文化意识的基准数据集，包含1000多个（视频片段，文化规范）对标注。研究发现，模型在文化规范违反、中国文化背景和非语言证据提供方面表现较差。

Details

Motivation: 部署全球的视频大语言模型需理解文化背景，但缺乏评估其文化意识的基准。

Contribution: 1) 引入VideoNorms基准数据集；2) 提出人机协作标注框架；3) 揭示模型在文化意识方面的常见问题。

Method: 基于言语行为理论和人机协作框架，标注视频片段的文化规范（遵守/违反）并提供语言与非语言证据。

Result: 模型在文化规范违反、中国文化背景和非语言证据识别上表现弱于美国文化和语言证据；正式非幽默场景下表现差。

Insight: 需加强模型的文化基础训练，VideoNorms为填补这一空白提供了工具。

Abstract: As Video Large Language Models (VideoLLMs) are deployed globally, they require understanding of and grounding in the relevant cultural background. To properly assess these models’ cultural awareness, adequate benchmarks are needed. We introduce VideoNorms, a benchmark of over 1000 (video clip, norm) pairs from US and Chinese cultures annotated with socio-cultural norms grounded in speech act theory, norm adherence and violations labels, and verbal and non-verbal evidence. To build VideoNorms, we use a human-AI collaboration framework, where a teacher model using theoretically-grounded prompting provides candidate annotations and a set of trained human experts validate and correct the annotations. We benchmark a variety of open-weight VideoLLMs on the new dataset which highlight several common trends: 1) models performs worse on norm violation than adherence; 2) models perform worse w.r.t Chinese culture compared to the US culture; 3) models have more difficulty in providing non-verbal evidence compared to verbal for the norm adhere/violation label and struggle to identify the exact norm corresponding to a speech-act; and 4) unlike humans, models perform worse in formal, non-humorous contexts. Our findings emphasize the need for culturally-grounded video language model training - a gap our benchmark and framework begin to address.

[129] ARTDECO: Towards Efficient and High-Fidelity On-the-Fly 3D Reconstruction with Structured Scene Representation

Guanghao Li,Kerui Ren,Linning Xu,Zhewen Zheng,Changjian Jiang,Xin Gao,Bo Dai,Jian Pu,Mulin Yu,Jiangmiao Pang

Main category: cs.CV

TL;DR: ARTDECO提出了一种结合前馈模型效率和SLAM可靠性的统一框架，用于实时3D重建，通过结构化高斯表示和层次化渲染策略实现了高效且高质量的建模。

Details

Motivation: 当前3D重建方法在效率和保真度之间存在权衡：逐场景优化计算成本高，而前馈模型实时性虽好但精度不足。

Contribution: 1. 提出统一框架ARTDECO，结合前馈模型和SLAM的优势；2. 设计结构化高斯表示和层次化渲染策略；3. 在多种数据集上验证了高效和高保真度的表现。

Method: 1. 使用3D基础模型进行姿态估计和点预测；2. 通过高斯解码器生成结构化3D高斯；3. 采用层次化表示和LoD感知渲染策略减少冗余。

Result: 在八项基准测试中，ARTDECO实现了接近SLAM的实时性、前馈系统的鲁棒性和逐场景优化的重建质量。

Insight: 结构化表示和层次化渲染是关键创新，为实时高保真3D重建提供了实用路径。

Abstract: On-the-fly 3D reconstruction from monocular image sequences is a long-standing challenge in computer vision, critical for applications such as real-to-sim, AR/VR, and robotics. Existing methods face a major tradeoff: per-scene optimization yields high fidelity but is computationally expensive, whereas feed-forward foundation models enable real-time inference but struggle with accuracy and robustness. In this work, we propose ARTDECO, a unified framework that combines the efficiency of feed-forward models with the reliability of SLAM-based pipelines. ARTDECO uses 3D foundation models for pose estimation and point prediction, coupled with a Gaussian decoder that transforms multi-scale features into structured 3D Gaussians. To sustain both fidelity and efficiency at scale, we design a hierarchical Gaussian representation with a LoD-aware rendering strategy, which improves rendering fidelity while reducing redundancy. Experiments on eight diverse indoor and outdoor benchmarks show that ARTDECO delivers interactive performance comparable to SLAM, robustness similar to feed-forward systems, and reconstruction quality close to per-scene optimization, providing a practical path toward on-the-fly digitization of real-world environments with both accurate geometry and high visual fidelity. Explore more demos on our project page: https://city-super.github.io/artdeco/.

Yunzhe Xu,Yiyuan Pan,Zhe Liu

Main category: cs.CV

TL;DR: Memoir提出了一种基于想象力的检索机制，用于记忆持久性视觉语言导航（VLN），通过选择性检索环境观测和行为历史，显著提升了导航性能。

Details

Motivation: 现有记忆持久性VLN方法存在局限性：缺乏高效的记忆访问机制，且主要存储环境观测而忽略导航行为模式。Memoir旨在通过想象力检索机制解决这些问题。

Contribution: 1）提出语言条件的世界模型生成检索查询；2）设计混合视角级内存存储观测和行为模式；3）提出经验增强导航模型整合检索知识。

Method: 1）世界模型想象未来状态作为查询；2）混合内存锚定观测和行为模式；3）导航模型通过专用编码器整合检索知识。

Result: 在多样化VLN基准测试中，Memoir显著优于基线方法（IR2R上提升5.4% SPL），训练速度提升8.3倍，推理内存减少74%。

Insight: 预测性检索环境和行为记忆是高效导航的关键，想象力引导的范式仍有巨大潜力（73.3% vs 93.4%上限）。

Abstract: Vision-and-Language Navigation (VLN) requires agents to follow natural language instructions through environments, with memory-persistent variants demanding progressive improvement through accumulated experience. Existing approaches for memory-persistent VLN face critical limitations: they lack effective memory access mechanisms, instead relying on entire memory incorporation or fixed-horizon lookup, and predominantly store only environmental observations while neglecting navigation behavioral patterns that encode valuable decision-making strategies. We present Memoir, which employs imagination as a retrieval mechanism grounded by explicit memory: a world model imagines future navigation states as queries to selectively retrieve relevant environmental observations and behavioral histories. The approach comprises: 1) a language-conditioned world model that imagines future states serving dual purposes: encoding experiences for storage and generating retrieval queries; 2) Hybrid Viewpoint-Level Memory that anchors both observations and behavioral patterns to viewpoints, enabling hybrid retrieval; and 3) an experience-augmented navigation model that integrates retrieved knowledge through specialized encoders. Extensive evaluation across diverse memory-persistent VLN benchmarks with 10 distinctive testing scenarios demonstrates Memoir’s effectiveness: significant improvements across all scenarios, with 5.4% SPL gains on IR2R over the best memory-persistent baseline, accompanied by 8.3x training speedup and 74% inference memory reduction. The results validate that predictive retrieval of both environmental and behavioral memories enables more effective navigation, with analysis indicating substantial headroom (73.3% vs 93.4% upper bound) for this imagination-guided paradigm. Code at https://github.com/xyz9911/Memoir.

[131] VideoCanvas: Unified Video Completion from Arbitrary Spatiotemporal Patches via In-Context Conditioning

Minghong Cai,Qiulin Wang,Zongli Ye,Wenze Liu,Quande Liu,Weicai Ye,Xintao Wang,Pengfei Wan,Kun Gai,Xiangyu Yue

Main category: cs.CV

TL;DR: VideoCanvas提出了一种统一的视频补全方法，支持用户通过任意时空补丁生成视频，解决了现有潜在视频扩散模型中时间模糊性问题，并通过零参数新增实现了精细控制。

Details

Motivation: 现有视频生成任务（如第一帧图像转视频、修复、扩展和插值）缺乏统一的灵活框架，且潜在视频扩散模型因因果VAE的时间模糊性难以实现帧级精确控制。

Contribution: 1) 提出VideoCanvas框架，统一了多种可控视频生成任务；2) 通过In-Context Conditioning（ICC）和Temporal RoPE Interpolation解决时间模糊性问题；3) 开发了首个评测基准VideoCanvasBench。

Method: 采用混合条件策略，空间控制利用零填充，时间控制通过Temporal RoPE Interpolation分配连续分数位置，从而实现像素帧级控制而不修改骨干网络。

Result: 实验表明，VideoCanvas在灵活性和统一性上优于现有方法，达到新SOTA。

Insight: 通过解耦时空控制，无需额外参数即可实现精细控制，为视频生成提供了新思路。

Abstract: We introduce the task of arbitrary spatio-temporal video completion, where a video is generated from arbitrary, user-specified patches placed at any spatial location and timestamp, akin to painting on a video canvas. This flexible formulation naturally unifies many existing controllable video generation tasks–including first-frame image-to-video, inpainting, extension, and interpolation–under a single, cohesive paradigm. Realizing this vision, however, faces a fundamental obstacle in modern latent video diffusion models: the temporal ambiguity introduced by causal VAEs, where multiple pixel frames are compressed into a single latent representation, making precise frame-level conditioning structurally difficult. We address this challenge with VideoCanvas, a novel framework that adapts the In-Context Conditioning (ICC) paradigm to this fine-grained control task with zero new parameters. We propose a hybrid conditioning strategy that decouples spatial and temporal control: spatial placement is handled via zero-padding, while temporal alignment is achieved through Temporal RoPE Interpolation, which assigns each condition a continuous fractional position within the latent sequence. This resolves the VAE’s temporal ambiguity and enables pixel-frame-aware control on a frozen backbone. To evaluate this new capability, we develop VideoCanvasBench, the first benchmark for arbitrary spatio-temporal video completion, covering both intra-scene fidelity and inter-scene creativity. Experiments demonstrate that VideoCanvas significantly outperforms existing conditioning paradigms, establishing a new state of the art in flexible and unified video generation.

[132] SciVideoBench: Benchmarking Scientific Video Reasoning in Large Multimodal Models

Andong Deng,Taojiannan Yang,Shoubin Yu,Lincoln Spencer,Mohit Bansal,Chen Chen,Serena Yeung-Levy,Xiaohan Wang

Main category: cs.CV

TL;DR: SciVideoBench是一个专门用于评估科学视频推理能力的基准测试，包含1000个来自尖端科学实验视频的多选题，覆盖25个学科领域，旨在挑战大型多模态模型的高级认知能力。

Details

Motivation: 当前视频基准主要针对一般场景，依赖感知/识别任务，而科学领域的复杂视频推理能力评估不足。SciVideoBench填补了这一空白。

Contribution: 提出了SciVideoBench，一个专注于科学视频推理的基准测试，挑战模型的领域知识、时空感知和逻辑推理能力。

Method: 通过半自动系统从25个学科的科学实验视频中提取1000个多选题，每个问题需要高级认知能力。

Result: 评估发现现有大型多模态模型（如Gemini 2.5 Pro和Qwen2.5-VL）表现不佳，表明视频推理能力仍有提升空间。

Insight: SciVideoBench揭示了模型在复杂推理任务中的不足，为未来多模态AI的发展提供了明确方向。

Abstract: Large Multimodal Models (LMMs) have achieved remarkable progress across various capabilities; however, complex video reasoning in the scientific domain remains a significant and challenging frontier. Current video benchmarks predominantly target general scenarios where perception/recognition is heavily relied on, while with relatively simple reasoning tasks, leading to saturation and thus failing to effectively evaluate advanced multimodal cognitive skills. To address this critical gap, we introduce SciVideoBench, a rigorous benchmark specifically designed to assess advanced video reasoning in scientific contexts. SciVideoBench consists of 1,000 carefully crafted multiple-choice questions derived from cutting-edge scientific experimental videos spanning over 25 specialized academic subjects and verified by a semi-automatic system. Each question demands sophisticated domain-specific knowledge, precise spatiotemporal perception, and intricate logical reasoning, effectively challenging models’ higher-order cognitive abilities. Our evaluation highlights significant performance deficits in state-of-the-art proprietary and open-source LMMs, including Gemini 2.5 Pro and Qwen2.5-VL, indicating substantial room for advancement in video reasoning capabilities. Detailed analyses of critical factors such as reasoning complexity and visual grounding provide valuable insights and clear direction for future developments in LMMs, driving the evolution of truly capable multimodal AI co-scientists. We hope SciVideoBench could fit the interests of the community and help to push the boundary of cutting-edge AI for border science.

Maham Tanveer,Yang Zhou,Simon Niklaus,Ali Mahdavi Amiri,Hao Zhang,Krishna Kumar Singh,Nanxuan Zhao

Main category: cs.CV

TL;DR: 论文提出MultiCOIN,一个多模态可控的视频插帧框架,支持深度过渡、运动轨迹、文本提示等多模态控制,解决了现有方法难以生成复杂运动和缺乏精细控制的问题。

Details

Motivation: 现有视频插帧方法无法生成复杂运动,且缺乏对中间帧细节的精细控制,难以满足多样化的用户意图。

Contribution: 提出首个支持多模态控制的视频插帧框架MultiCOIN,通过将控制信号统一为点表示,并设计双分支生成器分别处理内容和运动,实现了灵活且精细的视频插帧。

Method: 采用Diffusion Transformer作为视频生成模型,将多模态控制统一为稀疏点表示,设计内容和运动双分支生成器,并提出分阶段训练策略。

Result: 实验表明,多模态控制能生成动态、可定制且上下文准确的视频插帧。

Insight: 将多模态控制统一为点表示为视频插帧提供了灵活性和精细控制,双分支设计有效分离了内容和运动特征的生成。

Abstract: Video inbetweening creates smooth and natural transitions between two image frames, making it an indispensable tool for video editing and long-form video synthesis. Existing works in this domain are unable to generate large, complex, or intricate motions. In particular, they cannot accommodate the versatility of user intents and generally lack fine control over the details of intermediate frames, leading to misalignment with the creative mind. To fill these gaps, we introduce \modelname{}, a video inbetweening framework that allows multi-modal controls, including depth transition and layering, motion trajectories, text prompts, and target regions for movement localization, while achieving a balance between flexibility, ease of use, and precision for fine-grained video interpolation. To achieve this, we adopt the Diffusion Transformer (DiT) architecture as our video generative model, due to its proven capability to generate high-quality long videos. To ensure compatibility between DiT and our multi-modal controls, we map all motion controls into a common sparse and user-friendly point-based representation as the video/noise input. Further, to respect the variety of controls which operate at varying levels of granularity and influence, we separate content controls and motion controls into two branches to encode the required features before guiding the denoising process, resulting in two generators, one for motion and the other for content. Finally, we propose a stage-wise training strategy to ensure that our model learns the multi-modal controls smoothly. Extensive qualitative and quantitative experiments demonstrate that multi-modal controls enable a more dynamic, customizable, and contextually accurate visual narrative.

[134] NaViL: Rethinking Scaling Properties of Native Multimodal Large Language Models under Data Constraints

Changyao Tian,Hao Li,Gen Luo,Xizhou Zhu,Weijie Su,Hanming Deng,Jinguo Zhu,Jie Shao,Ziran Zhu,Yunpeng Liu,Lewei Lu,Wenhai Wang,Hongsheng Li,Jifeng Dai

Main category: cs.CV

TL;DR: 本文研究了在数据受限条件下原生多模态大语言模型（MLLMs）的扩展性，提出了NaViL模型，并通过实验验证了其性能和设计空间的优化。

Details

Motivation: 现有的MLLMs通常采用组合式训练范式，即通过连续多模态预训练将预训练视觉编码器与大语言模型（LLMs）连接。然而，这种分离训练方式使得多模态扩展性难以探索。本文旨在通过端到端的原生训练方式，探索MLLMs的设计空间和扩展性。

Contribution: 1. 提出了NaViL，一种原生训练的MLLM模型；2. 系统地研究了在数据受限条件下MLLMs的设计空间和扩展性；3. 发现了视觉编码器与LLMs之间的正相关扩展关系；4. 提出了简单且成本高效的训练方法。

Method: 1. 采用端到端的原生训练方式；2. 研究不同设计选择，确定最优元架构；3. 探索视觉编码器与LLMs的扩展关系；4. 提出NaViL模型及训练方法。

Result: 在14个多模态基准测试中，NaViL表现出与现有MLLMs竞争的性能，验证了其设计和扩展性的有效性。

Insight: 原生MLLMs的训练方式具有潜力，视觉编码器与LLMs的扩展关系为正相关，未来研究可进一步优化设计空间和训练效率。

Abstract: Compositional training has been the de-facto paradigm in existing Multimodal Large Language Models (MLLMs), where pre-trained vision encoders are connected with pre-trained LLMs through continuous multimodal pre-training. However, the multimodal scaling property of this paradigm remains difficult to explore due to the separated training. In this paper, we focus on the native training of MLLMs in an end-to-end manner and systematically study its design space and scaling property under a practical setting, i.e., data constraint. Through careful study of various choices in MLLM, we obtain the optimal meta-architecture that best balances performance and training cost. After that, we further explore the scaling properties of the native MLLM and indicate the positively correlated scaling relationship between visual encoders and LLMs. Based on these findings, we propose a native MLLM called NaViL, combined with a simple and cost-effective recipe. Experimental results on 14 multimodal benchmarks confirm the competitive performance of NaViL against existing MLLMs. Besides that, our findings and results provide in-depth insights for the future study of native MLLMs.

[135] D$^2$GS: Depth-and-Density Guided Gaussian Splatting for Stable and Accurate Sparse-View Reconstruction

Meixi Song,Xin Lin,Dizhe Zhang,Haodong Li,Xiangtai Li,Bo Du,Lu Qi

Main category: cs.CV

TL;DR: D$^2$GS提出了一种统一的框架，通过深度和密度引导的高斯剔除策略以及距离感知的保真度增强模块，解决了稀疏视图条件下3D高斯泼溅（3DGS）的过拟合和欠拟合问题，显著提升了重建质量和稳定性。

Details

Motivation: 稀疏视图条件下，3DGS技术在近相机区域容易因高斯密度过高而过拟合，而在远距离区域则因高斯覆盖不足而欠拟合。这些问题导致重建性能下降和不稳定。

Contribution: 1. 提出深度和密度引导的高斯剔除策略（Depth-and-Density Guided Dropout），减少过拟合；2. 提出距离感知的保真度增强模块（Distance-Aware Fidelity Enhancement），改善欠拟合；3. 引入新的评估指标，量化高斯分布的稳定性。

Method: 1. 使用高斯密度和深度信息自适应剔除冗余高斯；2. 针对远距离区域设计目标监督机制，增强重建保真度；3. 通过新指标评估高斯分布的稳定性。

Result: 在多种数据集上的实验表明，D$^2$GS显著提升了稀疏视图条件下的视觉质量和鲁棒性。

Insight: 高斯分布的密度和深度信息是解决稀疏视图问题的关键；自适应调整和有针对性的监督可以提升3DGS的稳定性和准确性。

Abstract: Recent advances in 3D Gaussian Splatting (3DGS) enable real-time, high-fidelity novel view synthesis (NVS) with explicit 3D representations. However, performance degradation and instability remain significant under sparse-view conditions. In this work, we identify two key failure modes under sparse-view conditions: overfitting in regions with excessive Gaussian density near the camera, and underfitting in distant areas with insufficient Gaussian coverage. To address these challenges, we propose a unified framework D$^2$GS, comprising two key components: a Depth-and-Density Guided Dropout strategy that suppresses overfitting by adaptively masking redundant Gaussians based on density and depth, and a Distance-Aware Fidelity Enhancement module that improves reconstruction quality in under-fitted far-field areas through targeted supervision. Moreover, we introduce a new evaluation metric to quantify the stability of learned Gaussian distributions, providing insights into the robustness of the sparse-view 3DGS. Extensive experiments on multiple datasets demonstrate that our method significantly improves both visual quality and robustness under sparse view conditions. The project page can be found at: https://insta360-research-team.github.io/DDGS-website/.

[136] MATRIX: Multimodal Agent Tuning for Robust Tool-Use Reasoning

Tajamul Ashraf,Umair Nawaz,Abdelrahman M. Shaker,Rao Anwer,Philip Torr,Fahad Shahbaz Khan,Salman Khan

Main category: cs.CV

TL;DR: MATRIX提出了一种自动合成多模态轨迹的框架，通过构建M-TRACE数据集和Pref-X偏好对，优化VLM控制器的工具使用推理能力，显著提升了多模态工具使用的性能。

Details

Motivation: 现有视觉语言模型作为控制器时，由于高质量多模态轨迹稀缺和人工标注成本高，其工具使用和推理能力受限。

Contribution: 1) 构建M-TRACE数据集（28.5K多模态任务，177K已验证轨迹）；2) 提出Pref-X偏好对（11K自动生成）；3) 开发MATRIX Agent，优化工具推理能力。

Method: 1) 自动合成多模态轨迹；2) 通过模仿学习和偏好学习优化VLM控制器；3) 结合M-TRACE和Pref-X进行训练。

Result: 在Agent-X、GTA和GAIA三个基准测试中，MATRIX显著超越开源和闭源VLM，证明了其多模态工具使用的有效性。

Insight: 自动合成数据和偏好学习是提升多模态工具使用能力的关键；大规模验证数据集（M-TRACE）为模型提供了丰富的训练资源。

Abstract: Vision language models (VLMs) are increasingly deployed as controllers with access to external tools for complex reasoning and decision-making, yet their effectiveness remains limited by the scarcity of high-quality multimodal trajectories and the cost of manual annotation. We address this challenge with a vision-centric agent tuning framework that automatically synthesizes multimodal trajectories, generates step-wise preference pairs, and trains a VLM controller for robust tool-use reasoning. Our pipeline first constructs M-TRACE, a large-scale dataset of 28.5K multimodal tasks with 177K verified trajectories, enabling imitation-based trajectory tuning. Building on this, we develop MATRIX Agent, a controller finetuned on M-TRACE for step-wise tool reasoning. To achieve finer alignment, we further introduce Pref-X, a set of 11K automatically generated preference pairs, and optimize MATRIX on it via step-wise preference learning. Across three benchmarks, Agent-X, GTA, and GAIA, MATRIX consistently surpasses both open- and closed-source VLMs, demonstrating scalable and effective multimodal tool use. Our data and code is avaliable at https://github.com/mbzuai-oryx/MATRIX.

cs.SI [Back]

[137] From Keywords to Clusters: AI-Driven Analysis of YouTube Comments to Reveal Election Issue Salience in 2024

Raisa M. Simoes,Timoteo Kelly,Eduardo J. Simoes,Praveen Rao

Main category: cs.SI

TL;DR: 论文探讨了两种数据科学方法，通过AI技术分析2024年总统选举期间YouTube评论，揭示选民最关注的问题。结果表明移民和民主是最常提及的问题，而通胀重要性被高估。

Details

Motivation: 研究旨在通过AI驱动的方法分析YouTube评论，揭示选举中最关键的问题，提供与传统调查不同的视角。

Contribution: 首次结合自然语言处理和聚类分析，量化了YouTube评论中选民关注的问题，验证了在线数据在选举分析中的潜力。

Method: 使用自然语言处理和聚类分析技术，从右翼（华尔街日报）和左翼（纽约时报）YouTube视频的8000多条评论中提取高频议题。

Result: 移民和民主是最常提及的议题，身份政治次之，通胀重要性较低。结果与传统调查部分吻合，但颠覆了对通胀的预期。

Insight: 在线评论分析的动态数据可能比传统调查更准确反映选民关注点，为选举预测提供了新工具。

Abstract: This paper aims to explore two competing data science methodologies to attempt answering the question, “Which issues contributed most to voters’ choice in the 2024 presidential election?” The methodologies involve novel empirical evidence driven by artificial intelligence (AI) techniques. By using two distinct methods based on natural language processing and clustering analysis to mine over eight thousand user comments on election-related YouTube videos from one right leaning journal, Wall Street Journal, and one left leaning journal, New York Times, during pre-election week, we quantify the frequency of selected issue areas among user comments to infer which issues were most salient to potential voters in the seven days preceding the November 5th election. Empirically, we primarily demonstrate that immigration and democracy were the most frequently and consistently invoked issues in user comments on the analyzed YouTube videos, followed by the issue of identity politics, while inflation was significantly less frequently referenced. These results corroborate certain findings of post-election surveys but also refute the supposed importance of inflation as an election issue. This indicates that variations on opinion mining, with their analysis of raw user data online, can be more revealing than polling and surveys for analyzing election outcomes.

cs.GR [Back]

[138] SViM3D: Stable Video Material Diffusion for Single Image 3D Generation

Andreas Engelhardt,Mark Boss,Vikram Voletti,Chun-Han Yao,Hendrik P. A. Lensch,Varun Jampani

Main category: cs.GR

TL;DR: SViM3D是一个从单张图像预测多视角一致的PBR材质框架，通过扩展视频扩散模型，联合输出空间变化的PBR参数和表面法线，支持高质量的重新光照和新视角合成。

Details

Motivation: 当前视频扩散模型虽能高效从单张图像重建3D物体，但反射率仍依赖简单材质模型或需额外步骤估计，限制了重新光照和外观编辑的能力。

Contribution: 1. 扩展视频扩散模型，联合预测PBR参数和表面法线；2. 引入新机制提升在不适定问题中的质量；3. 在多个数据集上展示重新光照和新视角合成的先进性能。

Method: 基于潜在视频扩散模型，显式控制相机参数，生成多视角一致的PBR参数和法线图，并引入多种提升质量的机制。

Result: 在AR/VR、影视游戏等领域展示了高质量的可重光照3D资产生成能力，并在数据集上优于现有方法。

Insight: 结合视频扩散模型与显式相机控制，可实现高效且高质量的3D材质生成，拓展到多领域应用。

Abstract: We present Stable Video Materials 3D (SViM3D), a framework to predict multi-view consistent physically based rendering (PBR) materials, given a single image. Recently, video diffusion models have been successfully used to reconstruct 3D objects from a single image efficiently. However, reflectance is still represented by simple material models or needs to be estimated in additional steps to enable relighting and controlled appearance edits. We extend a latent video diffusion model to output spatially varying PBR parameters and surface normals jointly with each generated view based on explicit camera control. This unique setup allows for relighting and generating a 3D asset using our model as neural prior. We introduce various mechanisms to this pipeline that improve quality in this ill-posed setting. We show state-of-the-art relighting and novel view synthesis performance on multiple object-centric datasets. Our method generalizes to diverse inputs, enabling the generation of relightable 3D assets useful in AR/VR, movies, games and other visual media.

[139] X2Video: Adapting Diffusion Models for Multimodal Controllable Neural Video Rendering

Zhitong Huang,Mohan Zhang,Renhan Wang,Rui Tang,Hao Zhu,Jing Liao

Main category: cs.GR

TL;DR: X2Video 是一种扩散模型，支持通过内部分量（如反照率、法线、粗糙度等）和多模态控制（图像和文本提示）生成逼真视频，通过混合自注意力保证时间一致性，并提出了递归采样方法用于长视频生成。

Details

Motivation: 现有视频生成方法缺乏对内在通道（如材质、光照等）的精确控制，且难以在多模态输入（图像和文本）下保持时间一致性。X2Video 旨在填补这一空白。

Contribution: 1. 首次将扩散模型扩展到支持内在通道和多模态控制的视频生成。2. 提出混合自注意力和掩码交叉注意力机制，增强时间一致性和多模态交互。3. 提出递归采样方法，用于生成长视频。4. 发布了InteriorVideo数据集。

Method: 1. 扩展XRGB模型为视频生成，采用混合自注意力保证时间一致性。2. 使用掩码交叉注意力分离全局和局部文本提示。3. 提出递归采样方法，结合关键帧预测和插值生成长视频。

Result: X2Video 能够生成长且时间一致的逼真视频，支持多模态控制和参数化编辑（如颜色、材质等）。定性和定量实验验证了其优越性。

Insight: 1. 内在通道和多模态控制的结合是实现高可控视频生成的有效途径。2. 时间一致性的关键是高效的注意力机制。

Abstract: We present X2Video, the first diffusion model for rendering photorealistic videos guided by intrinsic channels including albedo, normal, roughness, metallicity, and irradiance, while supporting intuitive multi-modal controls with reference images and text prompts for both global and local regions. The intrinsic guidance allows accurate manipulation of color, material, geometry, and lighting, while reference images and text prompts provide intuitive adjustments in the absence of intrinsic information. To enable these functionalities, we extend the intrinsic-guided image generation model XRGB to video generation by employing a novel and efficient Hybrid Self-Attention, which ensures temporal consistency across video frames and also enhances fidelity to reference images. We further develop a Masked Cross-Attention to disentangle global and local text prompts, applying them effectively onto respective local and global regions. For generating long videos, our novel Recursive Sampling method incorporates progressive frame sampling, combining keyframe prediction and frame interpolation to maintain long-range temporal consistency while preventing error accumulation. To support the training of X2Video, we assembled a video dataset named InteriorVideo, featuring 1,154 rooms from 295 interior scenes, complete with reliable ground-truth intrinsic channel sequences and smooth camera trajectories. Both qualitative and quantitative evaluations demonstrate that X2Video can produce long, temporally consistent, and photorealistic videos guided by intrinsic conditions. Additionally, X2Video effectively accommodates multi-modal controls with reference images, global and local text prompts, and simultaneously supports editing on color, material, geometry, and lighting through parametric tuning. Project page: https://luckyhzt.github.io/x2video

cs.AI [Back]

[140] Evaluation of LLMs for Process Model Analysis and Optimization

Akhil Kumar,Jianliang Leon Zhao,Om Dobariya

Main category: cs.AI

TL;DR: 本文探讨了多种LLM（如ChatGPT）在零样本设置下理解和分析BPMN流程模型的能力，发现LLM可作为业务流程设计的助手。

Details

Motivation: 研究LLM是否能够通过自然语言界面理解流程模型、发现错误并进行深度推理，以支持业务流程设计和优化。

Contribution: 1. 验证了未经训练的LLM在零样本设置下对BPMN流程模型的理解能力；2. 比较了不同LLM的表现差异；3. 展示了LLM在业务流程设计中的辅助作用。

Method: 使用交互式对话方式评估LLM对BPMN流程模型的语法、逻辑和语义层面的理解能力及其推理表现。

Result: 研究表明，LLM（如ChatGPT）能有效理解流程模型并回答相关查询，且在深度推理中表现出拟人化特性。

Insight: LLM可作为业务流程设计的高效助手，但其表现因模型而异，仍需进一步优化和验证。

Abstract: In this paper, we report our experience with several LLMs for their ability to understand a process model in an interactive, conversational style, find syntactical and logical errors in it, and reason with it in depth through a natural language (NL) interface. Our findings show that a vanilla, untrained LLM like ChatGPT (model o3) in a zero-shot setting is effective in understanding BPMN process models from images and answering queries about them intelligently at syntactic, logic, and semantic levels of depth. Further, different LLMs vary in performance in terms of their accuracy and effectiveness. Nevertheless, our empirical analysis shows that LLMs can play a valuable role as assistants for business process designers and users. We also study the LLM’s “thought process” and ability to perform deeper reasoning in the context of process analysis and optimization. We find that the LLMs seem to exhibit anthropomorphic properties.

[141] CompassLLM: A Multi-Agent Approach toward Geo-Spatial Reasoning for Popular Path Query

Md. Nazmul Islam Ananto,Shamit Fatin,Mohammed Eunus Ali,Md Rizwan Parvez

Main category: cs.AI

TL;DR: CompassLLM是一个多智能体框架，利用大语言模型（LLMs）的空间和图推理能力解决热门路径查询问题，通过SEARCH和GENERATE两阶段实现高精度和竞争性性能。

Details

Motivation: 传统方法和机器学习在热门路径查询中需要模型训练和参数调整，而LLMs在空间和图推理中的能力为这一领域提供了新思路。

Contribution: 提出CompassLLM多智能体框架，结合LLMs的能力解决热门路径查询问题，无需模型训练，支持动态数据更新。

Method: 采用两阶段管道：SEARCH阶段识别热门路径，GENERATE阶段合成新路径；利用LLMs进行推理。

Result: 在真实和合成数据集上，CompassLLM在SEARCH阶段表现出高精度，在GENERATE阶段具有竞争性性能且成本效益显著。

Insight: LLMs在空间推理中的应用潜力巨大，多智能体框架为动态数据场景提供了灵活且高效的解决方案。

Abstract: The popular path query - identifying the most frequented routes between locations from historical trajectory data - has important applications in urban planning, navigation optimization, and travel recommendations. While traditional algorithms and machine learning approaches have achieved success in this domain, they typically require model training, parameter tuning, and retraining when accommodating data updates. As Large Language Models (LLMs) demonstrate increasing capabilities in spatial and graph-based reasoning, there is growing interest in exploring how these models can be applied to geo-spatial problems. We introduce CompassLLM, a novel multi-agent framework that intelligently leverages the reasoning capabilities of LLMs into the geo-spatial domain to solve the popular path query. CompassLLM employs its agents in a two-stage pipeline: the SEARCH stage that identifies popular paths, and a GENERATE stage that synthesizes novel paths in the absence of an existing one in the historical trajectory data. Experiments on real and synthetic datasets show that CompassLLM demonstrates superior accuracy in SEARCH and competitive performance in GENERATE while being cost-effective.

[142] Test-Time Matching: Unlocking Compositional Reasoning in Multimodal Models

Yinglun Zhu,Jiancheng Zhang,Fuzhi Tang

Main category: cs.AI

TL;DR: 论文提出了一种称为Test-Time Matching (TTM)的算法，通过利用组匹配分数和测试时优化，显著提升了多模态模型的组合推理能力，超越了GPT-4和人类表现。

Details

Motivation: 前沿AI模型在组合推理任务中表现不佳，现有评估指标可能低估了模型的真实能力。作者希望通过改进评估方法和优化策略，释放模型的隐藏潜力。

Contribution: 1) 提出了一种新的组匹配分数，更准确地评估模型能力；2) 提出TTM算法，通过测试时优化进一步提升性能；3) 在多个基准测试中实现了显著的性能提升，甚至超越人类水平。

Method: 1) 引入组匹配分数以更好地利用组结构；2) 在测试时通过过拟合组匹配提升性能；3) 设计TTM算法，通过迭代自优化进一步改进模型表现。

Result: SigLIP-B16和GPT-4.1在Winoground等任务中表现显著提升，TTM算法在无监督情况下实现了85.7%的相对增益，并在16个数据集中表现一致优秀。

Insight: 现有评估方法可能低估模型能力，测试时优化和多模态模型的组合推理能力仍有巨大潜力。

Abstract: Frontier AI models have achieved remarkable progress, yet recent studies suggest they struggle with compositional reasoning, often performing at or below random chance on established benchmarks. We revisit this problem and show that widely used evaluation metrics systematically underestimate model capability. To address this, we introduce a group matching score that better exploits group structure and reveals substantial hidden capability in both contrastive vision-language models (VLMs) and multimodal large language models (MLLMs). Moreover, simply overfitting to the induced group matchings at test time transfers this hidden capability into higher scores under standard evaluation metrics, closing much of the reported gap. This adjustment enables SigLIP-B16 to surpass all previous results and GPT-4.1 to yield the first result surpassing estimated human performance on Winoground. Building on this insight, we propose Test-Time Matching (TTM), an iterative, self-improving algorithm that further bootstraps model performance without any external supervision. TTM delivers additional, non-trivial improvements: for example, TTM enables SigLIP-B16 to surpass GPT-4.1 on MMVP-VLM, establishing a new state of the art. Importantly, TTM remains broadly effective even on benchmarks without metric-induced effects or group structures, achieving relative gains up to 85.7% on challenging datasets such as WhatsUp. Across 16 dataset variants spanning diverse setups, our experiments demonstrate that TTM consistently improves model performance and advances the frontier of compositional reasoning.

Alhim Vera,Karen Sanchez,Carlos Hinojosa,Haidar Bin Hamid,Donghoon Kim,Bernard Ghanem

Main category: cs.AI

TL;DR: 论文提出了一个可复现的仿真框架，用于评估生成式智能体在多模态环境中的安全性、一致性和社会动态表现，揭示了当前架构在多模态安全推理上的局限性。

Details

Motivation: 尽管大语言模型和视觉语言模型的发展使智能体能够在丰富环境中自主行动，但其在多模态安全、一致性和信任方面的推理能力仍然有限。论文旨在填补这一研究空白。

Contribution: 1. 提出一个可复现的多模态仿真框架；2. 引入SocialMetrics评估智能体的行为和社会动态；3. 揭示了当前智能体在多模态安全推理上的局限性。

Method: 通过分层记忆、动态规划和多模态感知能力装备智能体，并使用SocialMetrics量化行为指标（如计划修订、不安全到安全的转化率）。实验在三种模型（Claude、GPT-4o mini、Qwen-VL）上进行。

Result: 智能体在多模态安全推理上的成功率仅为55%，且在误导性视觉信息的影响下，45%的不安全行为被接受。不同模型的转化率分别为75%（Claude）、55%（GPT-4o mini）、58%（Qwen-VL）。

Insight: 当前智能体在多模态环境中存在明显的局限性，尤其是对视觉信息的过度信任可能导致安全隐患。论文为未来研究提供了标准化的评估平台。

Abstract: Can generative agents be trusted in multimodal environments? Despite advances in large language and vision-language models that enable agents to act autonomously and pursue goals in rich settings, their ability to reason about safety, coherence, and trust across modalities remains limited. We introduce a reproducible simulation framework for evaluating agents along three dimensions: (1) safety improvement over time, including iterative plan revisions in text-visual scenarios; (2) detection of unsafe activities across multiple categories of social situations; and (3) social dynamics, measured as interaction counts and acceptance ratios of social exchanges. Agents are equipped with layered memory, dynamic planning, multimodal perception, and are instrumented with SocialMetrics, a suite of behavioral and structural metrics that quantifies plan revisions, unsafe-to-safe conversions, and information diffusion across networks. Experiments show that while agents can detect direct multimodal contradictions, they often fail to align local revisions with global safety, reaching only a 55 percent success rate in correcting unsafe plans. Across eight simulation runs with three models - Claude, GPT-4o mini, and Qwen-VL - five agents achieved average unsafe-to-safe conversion rates of 75, 55, and 58 percent, respectively. Overall performance ranged from 20 percent in multi-risk scenarios with GPT-4o mini to 98 percent in localized contexts such as fire/heat with Claude. Notably, 45 percent of unsafe actions were accepted when paired with misleading visuals, showing a strong tendency to overtrust images. These findings expose critical limitations in current architectures and provide a reproducible platform for studying multimodal safety, coherence, and social dynamics.

[144] oMeBench: Towards Robust Benchmarking of LLMs in Organic Mechanism Elucidation and Reasoning

Ruiling Xu,Yifan Zhang,Qingyun Wang,Carl Edwards,Heng Ji

Main category: cs.AI

TL;DR: 论文介绍了oMeBench，一个用于评估大型语言模型（LLMs）在有机反应机理推理中的首个大规模专家标注基准，并提出了动态评估框架oMeS，结果显示当前模型在多步推理中存在不足，但通过提示策略和微调可显著提升性能。

Details

Motivation: 有机反应机理是理解化学反应性和设计新分子的基础，但目前尚不清楚LLMs在此任务中的表现是否反映真实的化学推理能力。因此，需要建立基准和评估框架以量化模型的化学推理能力。

Contribution: 1. 提出了首个大规模专家标注的有机机理推理基准oMeBench；2. 开发了动态评估框架oMeS；3. 分析了当前LLMs的性能并展示改进方法（如提示和微调）。

Method: 1. 构建包含10,000+标注机理步骤的基准数据集；2. 设计oMeS框架，结合步级逻辑和化学相似性进行动态评估；3. 测试多种LLMs并提出改进策略（提示和微调）。

Result: 结果显示当前LLMs在多步推理中表现不足，但通过提示和微调可将性能提升50%。

Insight: LLMs在化学任务中虽具潜力，但其推理能力仍需提升；oMeBench为未来AI系统的化学推理研究提供了重要工具。

Abstract: Organic reaction mechanisms are the stepwise elementary reactions by which reactants form intermediates and products, and are fundamental to understanding chemical reactivity and designing new molecules and reactions. Although large language models (LLMs) have shown promise in understanding chemical tasks such as synthesis design, it is unclear to what extent this reflects genuine chemical reasoning capabilities, i.e., the ability to generate valid intermediates, maintain chemical consistency, and follow logically coherent multi-step pathways. We address this by introducing oMeBench, the first large-scale, expert-curated benchmark for organic mechanism reasoning in organic chemistry. It comprises over 10,000 annotated mechanistic steps with intermediates, type labels, and difficulty ratings. Furthermore, to evaluate LLM capability more precisely and enable fine-grained scoring, we propose oMeS, a dynamic evaluation framework that combines step-level logic and chemical similarity. We analyze the performance of state-of-the-art LLMs, and our results show that although current models display promising chemical intuition, they struggle with correct and consistent multi-step reasoning. Notably, we find that using prompting strategy and fine-tuning a specialist model on our proposed dataset increases performance by 50% over the leading closed-source model. We hope that oMeBench will serve as a rigorous foundation for advancing AI systems toward genuine chemical reasoning.

[145] VoiceAgentBench: Are Voice Assistants ready for agentic tasks?

Dhruv Jain,Harshit Shukla,Gautam Rajeev,Ashish Kulkarni,Chandra Khatri,Shubham Agarwal

Main category: cs.AI

TL;DR: VoiceAgentBench是一个全面的语音助手机器人能力评测基准，旨在评估语音语言模型在真实场景中的表现，涵盖多语言、文化背景和对抗鲁棒性。

Details

Motivation: 现有语音评测基准主要关注孤立能力（如转录或问答），缺乏对包含多语言和文化理解的系统性评估，以及对对抗鲁棒性的测试。

Contribution: 引入VoiceAgentBench，包含5500多个合成的语音查询，支持英语、印地语和5种印度语言，模拟真实语言和文化多样性，并通过新颖的采样算法最大化声学和说话人多样性。

Method: 设计了涵盖单工具调用、多工具工作流、多轮交互和安全评估的评测任务，并通过基于说话人嵌入的采样算法模拟说话人变异性。

Result: 实验揭示了当前语音语言模型在上下文工具协调任务、印度语言泛化和对抗鲁棒性方面的显著局限性。

Insight: 当前的语音语言模型在多语言和文化多样性处理、复杂的任务协调以及对抗鲁棒性方面仍有明显挑战，未来需进一步改进。

Abstract: Large-scale Speech Language Models (SpeechLMs) have enabled voice assistants capable of understanding natural spoken queries and performing complex tasks. However, existing speech benchmarks primarily focus on isolated capabilities such as transcription, or question-answering, and do not systematically evaluate agentic scenarios encompassing multilingual and cultural understanding, as well as adversarial robustness. To address this, we introduce VoiceAgentBench, a comprehensive benchmark designed to evaluate SpeechLMs in realistic spoken agentic settings. It comprises over 5,500 synthetic spoken queries, including dialogues grounded in Indian context, covering single-tool invocations, multi-tool workflows, multi-turn interactions, and safety evaluations. The benchmark supports English, Hindi, and 5 other Indian languages, reflecting real-world linguistic and cultural diversity. We simulate speaker variability using a novel sampling algorithm that selects audios for TTS voice conversion based on its speaker embeddings, maximizing acoustic and speaker diversity. Our evaluation measures tool selection accuracy, structural consistency, and the correctness of tool invocations, including adversarial robustness. Our experiments reveal significant gaps in contextual tool orchestration tasks, Indic generalization, and adversarial robustness, exposing critical limitations of current SpeechLMs.

[146] AutoQual: An LLM Agent for Automated Discovery of Interpretable Features for Review Quality Assessment

Xiaochong Lan,Jie Feng,Yinxing Liu,Xinlei Shi,Yong Li

Main category: cs.AI

TL;DR: AutoQual是一个基于LLM的智能体框架，用于自动化发现可解释的特征，解决了传统方法和深度学习在黑盒模型和可扩展性上的局限性，并在大规模A/B测试中验证了其有效性。

Details

Motivation: 在线评论质量评估对电商平台至关重要，但传统方法依赖手工特征且难以跨领域扩展，深度学习方法则缺乏可解释性。

Contribution: 提出了AutoQual框架，自动将数据中的隐含知识转化为明确、可计算的特征，支持迭代特征生成和工具自主实现。

Method: 模拟人类研究过程，通过反思生成特征假设，利用工具实现特征操作化，并在持久化记忆中积累经验。

Result: 在大规模A/B测试中，AutoQual显著提升了用户平均浏览评论数和转化率。

Insight: AutoQual展示了LLM在特征发现中的潜力，为缺乏大规模标注数据的领域提供了解决方案。

Abstract: Ranking online reviews by their intrinsic quality is a critical task for e-commerce platforms and information services, impacting user experience and business outcomes. However, quality is a domain-dependent and dynamic concept, making its assessment a formidable challenge. Traditional methods relying on hand-crafted features are unscalable across domains and fail to adapt to evolving content patterns, while modern deep learning approaches often produce black-box models that lack interpretability and may prioritize semantics over quality. To address these challenges, we propose AutoQual, an LLM-based agent framework that automates the discovery of interpretable features. While demonstrated on review quality assessment, AutoQual is designed as a general framework for transforming tacit knowledge embedded in data into explicit, computable features. It mimics a human research process, iteratively generating feature hypotheses through reflection, operationalizing them via autonomous tool implementation, and accumulating experience in a persistent memory. We deploy our method on a large-scale online platform with a billion-level user base. Large-scale A/B testing confirms its effectiveness, increasing average reviews viewed per user by 0.79% and the conversion rate of review readers by 0.27%.

[147] R-Horizon: How Far Can Your Large Reasoning Model Really Go in Breadth and Depth?

Yi Lu,Jianing Wang,Linsen Guo,Wei He,Hongyin Tang,Tao Gui,Xuanjing Huang,Xuezhi Cao,Wei Wang,Xunliang Cai

Main category: cs.AI

TL;DR: R-HORIZON 是一种评估和增强大型推理模型（LRMs）长时程推理能力的方法，通过构建多步推理基准和强化学习验证奖励（RLVR），解决了现有基准对复杂、长时程任务评估不足的问题。

Details

Motivation: 现有基准主要关注单时程任务，无法全面评估 LRMs 在处理复杂、长时程推理任务时的能力，这在模型实际应用中是一个重要缺陷。

Contribution: 提出了 R-HORIZON，一种通过查询组合激发长时程推理行为的评估方法，并构建了复杂的多步推理基准；利用 R-HORIZON 数据结合 RLVR 显著提升了 LRMs 的性能。

Method: 通过查询组合设计长时程推理任务，构建多步推理基准；进一步利用这些数据结合强化学习验证奖励（RLVR）进行模型训练。

Result: 实验表明，先进 LRMs 在长时程任务上表现显著下降；RLVR 结合 R-HORIZON 数据不仅提升了多时程任务性能，还在标准任务上提高了 7.5 分（AIME2024）。

Insight: LRMs 的有效推理长度有限，且在多个问题间分配思考资源的能力不足；R-HORIZON 提供了一种低成本、可控的范式，用于增强和评估长时程推理能力。

Abstract: Recent trends in test-time scaling for reasoning models (e.g., OpenAI o1, DeepSeek-R1) have led to remarkable improvements through long Chain-of-Thought (CoT). However, existing benchmarks mainly focus on immediate, single-horizon tasks, failing to adequately evaluate models’ ability to understand and respond to complex, long-horizon scenarios. To address this incomplete evaluation of Large Reasoning Models (LRMs), we propose R-HORIZON, a method designed to stimulate long-horizon reasoning behaviors in LRMs through query composition. Based on R-HORIZON, we construct a long-horizon reasoning benchmark, comprising complex multi-step reasoning tasks with interdependent problems that span long reasoning horizons. Through comprehensive evaluation of LRMs using the R-HORIZON benchmark, we find that even the most advanced LRMs suffer significant performance degradation. Our analysis reveals that LRMs exhibit limited effective reasoning length and struggle to allocate thinking budget across multiple problems appropriately. Recognizing these limitations, we use R-HORIZON to construct long-horizon reasoning data for reinforcement learning with verified rewards (RLVR). Compared to training with single-horizon data, RLVR with R-HORIZON not only substantially improves performance on the multi-horizon reasoning tasks, but also promotes accuracy on standard reasoning tasks, with an increase of 7.5 on AIME2024. These results position R-HORIZON as a scalable, controllable, and low-cost paradigm for enhancing and evaluating the long-horizon reasoning capabilities of LRMs.

[148] Beyond Pass@k: Breadth-Depth Metrics for Reasoning Boundaries

Marius Dragoi,Ioana Pintilie,Florin Gogianu,Florin Brad

Main category: cs.AI

TL;DR: 本文提出了Cover@tau指标，用于更准确地评估模型的推理边界，避免了Pass@k在大k值时可能误导的问题。

Details

Motivation: Pass@k在大k值时可能因随机猜测而误导推理能力的评估，尤其是在离散答案空间中，因此需要一种更准确的指标。

Contribution: 提出了Cover@tau指标，能明确捕捉模型在可靠性阈值下的推理能力，而非依赖大量采样。

Method: 通过Cover@tau衡量模型在tau比例完成度下的问题解决能力，避免随机猜测的干扰。

Result: 实验表明，Cover@tau能更准确地反映模型的推理边界，并改变了RLVR模型的排名。

Insight: 离散任务中的评估需关注可靠性阈值，而非单纯的成功率，Cover@tau为此提供了新视角。

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a powerful paradigm to improve Large Language Models on reasoning tasks such as coding, math or logic. To assess the reasoning boundary (the fraction of problems a model can solve) researchers often report Pass@k at large sampling budgets. Recent results reveal a crossover phenomenon: while RLVR models outperform the base model at small k values, the base model usually outperforms them when sampling a very large number of completions. This has been interpreted as evidence that base models have a larger reasoning boundary. We argue that on tasks with discrete answer spaces, such as math with numeric outputs, Pass@k at large k reflects the increasingly higher chance of success in the limit of the number of trials rather than genuine reasoning, and can therefore be misleading. We propose Cover@tau, which measures the fraction of problems that a model can solve for which at least a tau proportion of completions are correct. Unlike Pass@k, Cover@tau captures reasoning under an explicit reliability threshold: models that rely on random guessing degrade rapidly as tau increases. We evaluate several RLVR models using Cover@tau-based metrics and illustrate how the relative rankings of popular algorithms change compared to Pass@1, offering a different perspective on reasoning boundaries.

[149] Looking to Learn: Token-wise Dynamic Gating for Low-Resource Vision-Language Modelling

Bianca-Mihaela Ganescu,Suchir Salhan,Andrew Caines,Paula Buttery

Main category: cs.AI

TL;DR: 论文提出了一种轻量级解码器架构，通过动态门控实现对视觉和语言信息的自适应融合，在低资源视觉语言建模任务中实现了竞争性或更优的性能。

Details

Motivation: 在BabyLM Challenge 2025的视觉赛道限制下，需要重新思考模型如何高效整合多模态信息，以适应有限的数据资源。

Contribution: 主要贡献包括：(1) 基于动态门控的自适应视觉语言信息融合，(2) 特征调制和通道注意力机制以最大化有限视觉信息的利用，(3) 辅助对比目标以增强视觉基础。

Method: 采用轻量级解码器架构，引入token-wise动态门控、特征调制和通道注意力机制，并通过辅助对比目标优化模型训练。

Result: 在BLiMP、BLiMP Supplement、EWoK、Winoground和VQA五个基准测试中表现优于或接近多模态基线。动态门控还发现了无需显式监督的可解释模式。

Insight: 动态门控是一种高效的视觉语言学习方法，即使在严格约束下也能提供可解释性和性能。全局图像嵌入和方法的数据分割可能导致信息瓶颈和训练不稳定。

Abstract: Training vision-language models on cognitively-plausible amounts of data requires rethinking how models integrate multimodal information. Within the constraints of the Vision track for the BabyLM Challenge 2025, we propose a lightweight decoder-based architecture with (1) token-wise dynamic gating for adaptive fusion of linguistic and visual cues, (2) feature modulation and channel attention to maximise the utility of limited visual information and (3) auxiliary contrastive objectives for visual grounding. Evaluation on five benchmarks (BLiMP, BLiMP Supplement, EWoK, Winoground and VQA) shows competitive or superior performance to multimodal baselines. More notably, our dynamic gate discovers interpretable patterns without explicit supervision, favouring visual cues for content words and linguistic cues for function words. While we identify limitations in the Challenge constraints, such as the information bottleneck created by global image embeddings and training instability from the dataset split, our findings establish dynamic gating as a powerful tool for efficient multimodal learning, offering both interpretability and performance even under severe constraints.

[150] How to Teach Large Multimodal Models New Skills

Zhen Zhu,Yiming Gong,Yao Xiao,Yaoyao Liu,Derek Hoiem

Main category: cs.AI

TL;DR: 该论文研究了如何在不对大型多模态模型（LMMs）的先前能力造成损害的前提下教授新技能，提出两种简单的调优方法以减少遗忘。

Details

Motivation: 大型多模态模型在顺序微调新技能时可能会遗忘先前习得的能力，因此需要研究如何在不损害模型通用性能的情况下教授新技能。

Contribution: 论文的主要贡献是提出了两种简单的调优方法（仅更新自注意力投影层或MLP Gate&Up层），在教授新技能的同时最小化对通用性能的影响。

Method: 通过在五个目标技能上进行顺序微调，并使用八个基准测试监控通用能力，论文分析了输出标记分布的变化，并提出两种参数更新策略。

Result: 提出的两种调优方法在多个模型和任务中实现了目标技能的有效学习，同时显著保留了模型的通用性能。

Insight: 论文发现输出标记分布的简单计数偏差可以作为遗忘的指标，这表明模型能力的部分恢复可能与参数更新的局部性有关。

Abstract: How can we teach large multimodal models (LMMs) new skills without erasing prior abilities? We study sequential fine-tuning on five target skills while monitoring general ability on eight held-out benchmarks across three model families. We observe that apparent “forgetting” on held-out tasks after narrow fine-tuning can partly recover at later stages. We trace this behavior to a measurable shift in the output token distribution, manifested through a simple counting-bias probe that co-varies with forgetting. Guided by this picture, we identify two simple, robust tuning recipes that learn strongly while limiting drift: (i) updating only the self-attention projection layers, and (ii) updating only the MLP Gate&Up while freezing the Down projection. Across models and tasks, these choices deliver strong target gains while largely preserving held-out performance. Code is available at https://github.com/jessemelpolio/LMM_CL

[151] CaRT: Teaching LLM Agents to Know When They Know Enough

Grace Liu,Yuxiao Qu,Jeff Schneider,Aarti Singh,Aviral Kumar

Main category: cs.AI

TL;DR: 这篇论文提出了CaRT方法，通过反事实轨迹和推理训练LLM代理，使其能够在信息收集中知道何时停止，从而提高任务效率和成功率。

Details

Motivation: 许多任务需要模型在多轮交互中策略性地收集相关信息，然后再执行任务。模型不仅需要知道如何有效获取信息，还需要知道何时停止收集并做出决策，以避免过度思考或偏离任务。

Contribution: 论文的贡献包括：1)形式化了信息收集和终止决策的问题；2)提出了CaRT方法，通过反事实轨迹和推理训练LLM学习何时终止；3)在医疗诊断和数学问题解决两个领域中验证了CaRT的有效性。

Method: CaRT通过微调LLM使用反事实轨迹对，一种是终止合理的情况，另一种是终止不合理的情况。模型通过语言推理解释终止决策的理由，并将这种能力通过微调注入基础LLM中。

Result: 实验表明，CaRT在交互式医疗诊断和数学问题解决领域中提高了信息收集的效率和任务成功率。

Insight: 反事实轨迹和语言推理的结合是训练LLM代理在复杂任务中做出高效决策的有效方法。这种方法不仅适用于特定领域，还可能泛化到其他需要多步决策的场景。

Abstract: Many tasks require learned models to strategically gather relevant information over multiple rounds of interaction before actually acting on a task. Strategic information gathering requires models to know not only how to effectively acquire information, but also when to stop gathering information and make a decision, in order to avoid overthinking or getting derailed when acting. In this paper, we formalize this problem and introduce Counterfactuals and Reasoning for Termination (CaRT), an approach for teaching LLMs when to stop seeking information. To appropriately learn when to terminate, CaRT fine-tunes LLMs using counterfactual pairs of trajectories, one where termination is appropriate and a minimally modified version of the same trajectory where it is not. It trains the LLM to explain the rationale for the termination decision in either case via verbal reasoning, and imbues this capability into the base LLM via fine-tuning. We instantiate CaRT in two domains: interactive medical diagnosis and math problem solving. In both domains, we find that CaRT improves the efficiency of information gathering and task success rate compared to other fine-tuning methods.

[152] Agent Learning via Early Experience

Kai Zhang,Xiangchao Chen,Bo Liu,Tianci Xue,Zeyi Liao,Zhihan Liu,Xiyao Wang,Yuting Ning,Zhaorun Chen,Xiaohan Fu,Jian Xie,Yuxuan Sun,Boyu Gou,Qi Qi,Zihang Meng,Jianwei Yang,Ning Zhang,Xian Li,Ashish Shah,Dat Huynh,Hengduo Li,Zi Yang,Sara Cao,Lawrence Jang,Shuyan Zhou,Jiacheng Zhu,Huan Sun,Jason Weston,Yu Su,Yifan Wu

Main category: cs.AI

TL;DR: 论文提出了一种名为’早期经验’的学习范式，通过智能体自身的交互数据来改进性能，避免了依赖专家数据或强化学习的限制，并在多样环境中验证了其有效性。

Details

Motivation: 现有语言智能体主要依赖专家数据进行监督微调，难以扩展到复杂任务且泛化能力差。此外，强化学习在许多环境中由于缺乏可验证奖励或长序列问题难以应用。因此，需要一种介于两者之间的学习方式。

Contribution: 1. 提出’早期经验’范式，利用智能体自身交互数据学习，无需奖励信号；2. 研究了两种策略：隐式世界建模和自我反思；3. 在八个多样环境中验证方法的有效性。

Method: 1. 隐式世界建模：利用收集的状态数据学习环境动态；2. 自我反思：通过从次优行动中学习改进推理与决策。两种策略均基于智能体的早期经验数据。

Result: 方法在多样环境中显著提升了智能体的性能和跨域泛化能力，并为进一步的强化学习提供了良好基础。

Insight: 早期经验为智能体学习提供了实用桥梁，弥补了模仿学习与完全基于经验学习的鸿沟，尤其在缺乏奖励信号的环境中表现出显著优势。

Abstract: A long-term goal of language agents is to learn and improve through their own experience, ultimately outperforming humans in complex, real-world tasks. However, training agents from experience data with reinforcement learning remains difficult in many environments, which either lack verifiable rewards (e.g., websites) or require inefficient long-horizon rollouts (e.g., multi-turn tool use). As a result, most current agents rely on supervised fine-tuning on expert data, which is challenging to scale and generalizes poorly. This limitation stems from the nature of expert demonstrations: they capture only a narrow range of scenarios and expose the agent to limited environment diversity. We address this limitation with a middle-ground paradigm we call early experience: interaction data generated by the agent’s own actions, where the resulting future states serve as supervision without reward signals. Within this paradigm we study two strategies of using such data: (1) Implicit world modeling, which uses collected states to ground the policy in environment dynamics; and (2) Self-reflection, where the agent learns from its suboptimal actions to improve reasoning and decision-making. We evaluate across eight diverse environments and multiple model families. Our approaches consistently improve effectiveness and out-of-domain generalization, highlighting the value of early experience. Moreover, in environments with verifiable rewards, our results provide promising signals that early experience offers a strong foundation for subsequent reinforcement learning, positioning it as a practical bridge between imitation learning and fully experience-driven agents.

cs.IR [Back]

[153] Who Stole Your Data? A Method for Detecting Unauthorized RAG Theft

Peiyang Liu,Ziqiang Cui,Di Liang,Wei Ye

Main category: cs.IR

TL;DR: 该论文提出了一种检测RAG系统中未经授权数据窃取的方法，通过引入一个新数据集RPD和双层级水印技术，结合统计假设检验，实现了高效的数据保护。

Details

Motivation: RAG技术虽提升了LLM的性能，但也加剧了大规模数据盗用的风险，因此需要一种有效的方法来检测和保护知识产权。

Contribution: 1. 提出了专为RAG抄袭检测设计的数据集RPD；2. 开发了一种双层级水印技术，结合语义和词汇层面的保护，并通过统计假设检验框架增强检测能力。

Method: 1. 构建多样化数据集RPD；2. 设计双层级水印系统（语义+词汇）；3. 使用统计假设检验框架进行证据积累和检测。

Result: 实验表明，该方法在不同查询量、防御提示和检索参数下均有效，且能抵抗对抗性规避技术。

Insight: 论文为RAG系统中的知识产权保护提供了基础性框架，同时展示了水印技术与统计方法结合的潜力。

Abstract: Retrieval-augmented generation (RAG) enhances Large Language Models (LLMs) by mitigating hallucinations and outdated information issues, yet simultaneously facilitates unauthorized data appropriation at scale. This paper addresses this challenge through two key contributions. First, we introduce RPD, a novel dataset specifically designed for RAG plagiarism detection that encompasses diverse professional domains and writing styles, overcoming limitations in existing resources. Second, we develop a dual-layered watermarking system that embeds protection at both semantic and lexical levels, complemented by an interrogator-detective framework that employs statistical hypothesis testing on accumulated evidence. Extensive experimentation demonstrates our approach’s effectiveness across varying query volumes, defense prompts, and retrieval parameters, while maintaining resilience against adversarial evasion techniques. This work establishes a foundational framework for intellectual property protection in retrieval-augmented AI systems.

[154] TaoSR-AGRL: Adaptive Guided Reinforcement Learning Framework for E-commerce Search Relevance

Jianhui Yang,Yiming Jin,Pengkun Jiao,Chenhe Dong,Zerui Huang,Shaowei Yao,Xiaojiang Zhou,Dan Ou,Haihong Tang

Main category: cs.IR

TL;DR: 论文提出了一种面向电商搜索相关性的自适应引导强化学习框架TaoSR-AGRL，通过规则感知的奖励塑造和自适应引导回放，解决了现有方法在复杂业务规则和长尾查询下的推理能力不足问题。

Details

Motivation: 电商搜索中的查询-商品相关性预测对用户体验和业务转化至关重要。现有基于大型语言模型（LLMs）的方法在处理复杂规则和长尾查询时推理能力不足，且强化学习方法因稀疏终端奖励而收敛缓慢。

Contribution: 提出了TaoSR-AGRL框架，包含两项创新：(1) 规则感知的奖励塑造，将最终相关性判断分解为密集的结构化奖励；(2) 自适应引导回放，通过注入目标真实数据引导策略避开违规推理模式。

Method: 通过规则感知的奖励塑造和自适应引导回放两项技术优化LLM推理能力。前者对齐领域相关标准提供密集奖励，后者在训练中识别低精度轨迹并注入真实数据引导。

Result: 在淘宝搜索的大规模真实数据集和在线评估中，TaoSR-AGRL显著优于DPO和GRPO基线，提升了相关性准确性、规则遵守性和训练稳定性，并已成功部署服务于数亿用户。

Insight: 密集的结构化奖励和有针对性的真实数据引导能够有效提升LLM在复杂业务场景中的推理能力和训练效率，为电商搜索优化提供了一种新思路。

Abstract: Query-product relevance prediction is fundamental to e-commerce search and has become even more critical in the era of AI-powered shopping, where semantic understanding and complex reasoning directly shape the user experience and business conversion. Large Language Models (LLMs) enable generative, reasoning-based approaches, typically aligned via supervised fine-tuning (SFT) or preference optimization methods like Direct Preference Optimization (DPO). However, the increasing complexity of business rules and user queries exposes the inability of existing methods to endow models with robust reasoning capacity for long-tail and challenging cases. Efforts to address this via reinforcement learning strategies like Group Relative Policy Optimization (GRPO) often suffer from sparse terminal rewards, offering insufficient guidance for multi-step reasoning and slowing convergence. To address these challenges, we propose TaoSR-AGRL, an Adaptive Guided Reinforcement Learning framework for LLM-based relevance prediction in Taobao Search Relevance. TaoSR-AGRL introduces two key innovations: (1) Rule-aware Reward Shaping, which decomposes the final relevance judgment into dense, structured rewards aligned with domain-specific relevance criteria; and (2) Adaptive Guided Replay, which identifies low-accuracy rollouts during training and injects targeted ground-truth guidance to steer the policy away from stagnant, rule-violating reasoning patterns toward compliant trajectories. TaoSR-AGRL was evaluated on large-scale real-world datasets and through online side-by-side human evaluations on Taobao Search. It consistently outperforms DPO and standard GRPO baselines in offline experiments, improving relevance accuracy, rule adherence, and training stability. The model trained with TaoSR-AGRL has been successfully deployed in the main search scenario on Taobao, serving hundreds of millions of users.

[155] VersionRAG: Version-Aware Retrieval-Augmented Generation for Evolving Documents

Daniel Huwiler,Kurt Stockinger,Jonathan Fürst

Main category: cs.IR

TL;DR: VersionRAG是一个版本感知的RAG框架，通过层级图结构显式建模文档演化，解决了版本化文档问答中的时间有效性检查问题，显著提升了性能。

Details

Motivation: 现有RAG系统在处理版本化文档时表现不佳，主要是由于缺乏对文档演化的显式建模和时间有效性检查，导致回答版本敏感问题的准确性较低。

Contribution: 1. 提出VersionRAG，显式建模文档演化；2. 设计基于意图分类的查询路由机制；3. 构建VersionQA基准测试集，推动未来研究。

Method: VersionRAG通过层次图结构捕捉文档版本序列、内容边界和状态变化，检索时基于意图分类动态路由查询，实现版本感知过滤和变更追踪。

Result: 在VersionQA基准测试中，VersionRAG达到90%的准确率，远超基线方法（58%-64%），且在处理隐式变更检测时表现出色（60%）。

Insight: 显式建模文档演化和动态查询路由是提升版本化文档问答性能的关键，同时VersionRAG的高效性使其适用于大规模部署。

Abstract: Retrieval-Augmented Generation (RAG) systems fail when documents evolve through versioning-a ubiquitous characteristic of technical documentation. Existing approaches achieve only 58-64% accuracy on version-sensitive questions, retrieving semantically similar content without temporal validity checks. We present VersionRAG, a version-aware RAG framework that explicitly models document evolution through a hierarchical graph structure capturing version sequences, content boundaries, and changes between document states. During retrieval, VersionRAG routes queries through specialized paths based on intent classification, enabling precise version-aware filtering and change tracking. On our VersionQA benchmark-100 manually curated questions across 34 versioned technical documents-VersionRAG achieves 90% accuracy, outperforming naive RAG (58%) and GraphRAG (64%). VersionRAG reaches 60% accuracy on implicit change detection where baselines fail (0-10%), demonstrating its ability to track undocumented modifications. Additionally, VersionRAG requires 97% fewer tokens during indexing than GraphRAG, making it practical for large-scale deployment. Our work establishes versioned document QA as a distinct task and provides both a solution and benchmark for future research.

[156] ReasonEmbed: Enhanced Text Embeddings for Reasoning-Intensive Document Retrieval

Jianlyu Chen,Junwei Lan,Chaofan Li,Defu Lian,Zheng Liu

Main category: cs.IR

TL;DR: 该论文提出了ReasonEmbed，一种专为推理密集型文档检索设计的文本嵌入模型，包含三项关键技术贡献：ReMixer数据合成方法、Redapter自适应学习算法以及在不同规模骨干网络上实现的ReasonEmbed模型。

Details

Motivation: 当前文本嵌入模型在处理推理密集型文档检索任务时表现不佳，主要原因是数据集质量不高和模型对推理强度的动态适应能力不足。

Contribution: 1. 提出ReMixer数据合成方法，解决了以往合成数据集中的平凡性问题；2. 设计Redapter自适应学习算法，动态调整样本权重；3. 实现了在不同规模骨干网络上的ReasonEmbed模型，性能显著提升。

Method: 1. ReMixer通过合成82K高质量训练样本解决数据集问题；2. Redapter根据样本推理强度动态调整权重；3. ReasonEmbed在多种骨干网络上实现。

Result: ReasonEmbed-Qwen3-8B在BRIGHT基准测试中取得了38.1的nDCG@10高分，显著优于现有模型。

Insight: 高质量合成数据和动态权重调整对提升推理密集型任务的性能至关重要。

Abstract: In this paper, we introduce ReasonEmbed, a novel text embedding model developed for reasoning-intensive document retrieval. Our work includes three key technical contributions. First, we propose ReMixer, a new data synthesis method that overcomes the triviality problem prevalent in previous synthetic datasets, enabling large-scale production of 82K high-quality training samples. Second, we design Redapter, a self-adaptive learning algorithm that dynamically adjusts training each sample’s weight based on its reasoning intensity. This allows the model to effectively capture the complex semantic relationships between queries and documents. Third, we implement ReasonEmbed across multiple backbones of varying sizes, all of which achieve superior performance on reasoning-intensive retrieval tasks. Notably, our ReasonEmbed-Qwen3-8B model offers a record-high nDCG@10 score of 38.1 on the BRIGHT benchmark, which significantly outperforms existing text embedding models. We will fully open-source our created resources in ReasonEmbed to push forward the research advancement in this field.

cs.LG [Back]

[157] ConCuR: Conciseness Makes State-of-the-Art Kernel Generation

Lingcheng Kong,Jiateng Wei,Hanzhang Shen,Huan Wang

Main category: cs.LG

TL;DR: 论文提出了一个名为ConCuR的数据集和KernelCoder模型，通过简洁而信息丰富的推理痕迹生成高质量CUDA内核，解决了内核生成任务中高质量数据稀缺的问题，显著提升了性能。

Details

Motivation: 内核生成任务面临高质量数据稀缺的挑战，大多数高质量内核是专有的且未开源。为此，论文提出了一种生成和筛选高质量CUDA内核及其推理痕迹的流程。

Contribution: 1. 提出ConCuR数据集和KernelCoder模型；2. 发现简洁且信息丰富的推理痕迹能提升内核生成效果；3. 提出平均推理长度作为任务难度指标。

Method: 通过生成和筛选高质量CUDA内核及其推理痕迹构建数据集，并训练KernelCoder模型。推理痕迹的简洁性和信息量是关键。

Result: 在KernelBench测试中，KernelCoder显著优于现有最佳模型QwQ-32B和所有开源模型，甚至超越前沿模型如DeepSeek-V3.1-Think和Claude-4-sonnet。

Insight: 推理痕迹的简洁性和信息量对内核生成任务至关重要，未来可用类似方法改进数据质量。

Abstract: GPU kernel generation by LLMs has recently experienced rapid development, leveraging test-time scaling and reinforcement learning techniques. However, a key challenge for kernel generation is the scarcity of high-quality data, as most high-quality kernels are proprietary and not open-source. This challenge prevents us from leveraging supervised fine-tuning to align LLMs to the kernel generation task. To address this challenge, we develop a pipeline that generates and curates high-quality CUDA kernels with reasoning traces, motivated by a critical observation that concise yet informative reasoning traces result in robust generation of high-performance kernels. Using this pipeline, we construct our dataset ConCuR and introduce our model KernelCoder, which is the first model trained on a curated dataset consisting of PyTorch, reasoning, and CUDA kernel pairs, to our knowledge. In the KernelBench setup, our model achieves significant improvements over the existing top-performing model, QwQ-32B, and outperforms all open-source models fine-tuned for kernel generation, as well as frontier models such as DeepSeek-V3.1-Think and Claude-4-sonnet. Finally, we show that the average reasoning length can serve as a metric to assess the difficulty of kernel generation tasks. The observations, metrics, and our data collection and curation pipeline can help obtain better data in the kernel generation task in the future.

[158] Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts

Yeskendir Koishekenov,Aldo Lipani,Nicola Cancedda

Main category: cs.LG

TL;DR: 该论文提出了Encode-Think-Decode (ETD)方法，通过在推理时迭代少量关键层来增强大语言模型的推理能力，无需增加参数或训练数据规模。

Details

Motivation: 现有方法提升语言模型推理能力通常依赖于增加模型参数或训练数据规模，但这增加了计算成本。研究发现推理任务的关键计算集中在某些层，因此提出ETD方法以低成本增强推理能力。

Contribution: 引入ETD方法，通过训练模型在中间阶段迭代关键层，显著增强推理能力；提出自适应深度策略，动态调整每标记的计算量。

Method: ETD方法选择推理相关的少量层进行迭代训练，保留原始架构和参数规模；自适应深度策略根据输入动态调整计算。

Result: 在17个推理基准测试中表现出色，如GSM8K上相对准确率提升28.4%，MATH上提升36%（基于OLMo-2 1B模型）。

Insight: 递归隐式推理是一种简单有效的方式，可以在不改变模型架构的情况下显著提升推理能力。

Abstract: Most efforts to improve the reasoning capabilities of large language models (LLMs) involve either scaling the number of parameters and the size of training data, or scaling inference computation by letting models generate complex chains of thought. Motivated by interpretability studies showing that the crucial computation required for reasoning tasks is concentrated in a limited range of layers, we introduce Encode-Think-Decode (ETD), a method that enhances the reasoning capabilities of a base model by training it to iterate over a small subset of reasoning-relevant layers during the mid-training stage. ETD amplifies latent reasoning while preserving the original architecture, parameter count, hyperparameters, and training data composition. When iterating on the selected layers at inference time, ETD models yield substantial gains on 17 reasoning benchmarks, including +28.4% relative accuracy improvement on GSM8K and +36% on MATH with the OLMo-2 1B Base model. We also explore an adaptive depth strategy that adjusts the computation per input token. Our results show that recursive latent reasoning offers a simple and effective path to stronger LLM reasoning.

[159] LiveThinking: Enabling Real-Time Efficient Reasoning for AI-Powered Livestreaming via Reinforcement Learning

Yuhan Sun,Zhiwei Huang,Wanqing Cui,Shaopan Xiong,Yazhi Guo,Meiguang Jin,Junfeng Ma

Main category: cs.LG

TL;DR: 为了解决电商直播中数字形象实时响应的问题，本文提出了LiveThinking框架，通过两阶段优化（模型蒸馏和强化学习）显著降低了计算成本和延迟，同时提升了互动效果和商业表现。

Details

Motivation: AI驱动的电商直播需要数字形象能够实时响应用户以提升互动效果，但常用的大型推理模型（LRMs）计算成本高、延迟大，不适合实时场景。

Contribution: 1. 通过蒸馏670B参数的LRM为轻量级30B MoE模型（仅3B活跃参数），显著降低了计算成本；2. 利用强化学习（GRPO）压缩推理路径，平衡正确性、实用性和简洁性，实现亚秒级延迟。

Method: 1. 第一阶段：使用拒绝采样微调（RFT）将大模型蒸馏为轻量级MoE模型；2. 第二阶段：引入GRPO强化学习，通过多目标奖励函数优化推理路径。

Result: 在淘宝直播中，LiveThinking将计算成本降低了30倍，响应正确性提升3.3%，实用性提升21.8%，并显著提升了GMV。

Insight: 结合蒸馏技术和强化学习可以高效优化大型模型在实时场景中的表现，同时平衡多个性能指标。

Abstract: In AI-powered e-commerce livestreaming, digital avatars require real-time responses to drive engagement, a task for which high-latency Large Reasoning Models (LRMs) are ill-suited. We introduce LiveThinking, a practical two-stage optimization framework to bridge this gap. First, we address computational cost by distilling a 670B teacher LRM into a lightweight 30B Mixture-of-Experts (MoE) model (3B active) using Rejection Sampling Fine-Tuning (RFT). This reduces deployment overhead but preserves the teacher’s verbose reasoning, causing latency. To solve this, our second stage employs reinforcement learning with Group Relative Policy Optimization (GRPO) to compress the model’s reasoning path, guided by a multi-objective reward function balancing correctness, helpfulness, and brevity. LiveThinking achieves a 30-fold reduction in computational cost, enabling sub-second latency. In real-world application on Taobao Live, it improved response correctness by 3.3% and helpfulness by 21.8%. Tested by hundreds of thousands of viewers, our system led to a statistically significant increase in Gross Merchandise Volume (GMV), demonstrating its effectiveness in enhancing user experience and commercial performance in live, interactive settings.

[160] MultiFair: Multimodal Balanced Fairness-Aware Medical Classification with Dual-Level Gradient Modulation

Md Zubair,Hao Zheng,Nussdorf Jonathan,Grayson W. Armstrong,Lucy Q. Shen,Gabriela Wilson,Yu Tian,Xingquan Zhu,Min Shi

Main category: cs.LG

TL;DR: MultiFair提出了一种新型多模态医学分类方法，通过双重梯度调制解决模态不平衡和群体不公平问题，优于现有方法。

Details

Motivation: 现有医学决策系统依赖多源数据确保可靠性，但当前多模态学习模型常忽略模态学习和群体公平性问题，导致结果偏差。

Contribution: 提出MultiFair方法，通过双重梯度调制（模态和群体层面）动态平衡多模态学习和公平性。

Method: 采用双重梯度调制技术，分别在模态和群体层面调整优化的方向和幅度，确保平衡学习和公平性。

Result: 在多个多模态医学数据集上验证，MultiFair优于现有多模态学习和公平性学习方法。

Insight: 模态不平衡和群体不公平问题相互关联，动态梯度调制是解决此类多任务优化问题的有效手段。

Abstract: Medical decision systems increasingly rely on data from multiple sources to ensure reliable and unbiased diagnosis. However, existing multimodal learning models fail to achieve this goal because they often ignore two critical challenges. First, various data modalities may learn unevenly, thereby converging to a model biased towards certain modalities. Second, the model may emphasize learning on certain demographic groups causing unfair performances. The two aspects can influence each other, as different data modalities may favor respective groups during optimization, leading to both imbalanced and unfair multimodal learning. This paper proposes a novel approach called MultiFair for multimodal medical classification, which addresses these challenges with a dual-level gradient modulation process. MultiFair dynamically modulates training gradients regarding the optimization direction and magnitude at both data modality and group levels. We conduct extensive experiments on two multimodal medical datasets with different demographic groups. The results show that MultiFair outperforms state-of-the-art multimodal learning and fairness learning methods.

[161] MLLM4TS: Leveraging Vision and Multimodal Language Models for General Time-Series Analysis

Qinghua Liu,Sam Heshmati,Zheda Mai,Zubin Abraham,John Paparrizos,Liu Ren

Main category: cs.LG

TL;DR: MLLM4TS提出了一种新颖的框架，通过结合多模态大型语言模型（MLLM）和视觉表示，实现通用的时间序列分析。该方法将时间序列数据转换为视觉表示，并通过视觉补丁对齐策略捕捉时空依赖关系。

Details

Motivation: 时间序列数据的复杂时序依赖和跨通道交互带来了分析上的挑战。受人类通过视觉检查时间序列的启发，探讨视觉表示是否能提升自动化分析的性能。

Contribution: 1. 提出了MLLM4TS框架，首次将MLLM与视觉表示结合用于时间序列分析；2. 设计了特定视觉分支和时间感知视觉补丁对齐策略；3. 在预测和生成任务中验证了方法的有效性。

Method: 1. 将时间序列数据转换为水平堆叠的彩色编码线图；2. 使用时间感知视觉补丁对齐策略对齐视觉补丁和时间片段；3. 融合数值数据的细粒度时序细节和视觉表示的全局上下文信息。

Result: 在标准基准测试中，MLLM4TS在分类、异常检测和预测等任务中均表现出色，证明了视觉模态与预训练语言模型结合的有效性。

Insight: 视觉表示能够补充和增强时间序列分析的性能，多模态融合为通用时间序列分析提供了新思路。

Abstract: Effective analysis of time series data presents significant challenges due to the complex temporal dependencies and cross-channel interactions in multivariate data. Inspired by the way human analysts visually inspect time series to uncover hidden patterns, we ask: can incorporating visual representations enhance automated time-series analysis? Recent advances in multimodal large language models have demonstrated impressive generalization and visual understanding capability, yet their application to time series remains constrained by the modality gap between continuous numerical data and discrete natural language. To bridge this gap, we introduce MLLM4TS, a novel framework that leverages multimodal large language models for general time-series analysis by integrating a dedicated vision branch. Each time-series channel is rendered as a horizontally stacked color-coded line plot in one composite image to capture spatial dependencies across channels, and a temporal-aware visual patch alignment strategy then aligns visual patches with their corresponding time segments. MLLM4TS fuses fine-grained temporal details from the numerical data with global contextual information derived from the visual representation, providing a unified foundation for multimodal time-series analysis. Extensive experiments on standard benchmarks demonstrate the effectiveness of MLLM4TS across both predictive tasks (e.g., classification) and generative tasks (e.g., anomaly detection and forecasting). These results underscore the potential of integrating visual modalities with pretrained language models to achieve robust and generalizable time-series analysis.

[162] Self-Improving LLM Agents at Test-Time

Emre Can Acikgoz,Cheng Qian,Heng Ji,Dilek Hakkani-Tür,Gokhan Tur

Main category: cs.LG

TL;DR: 论文提出了一种新的测试时自改进方法（TT-SI），通过识别模型不确定的样本（自感知）、生成类似样本（自数据增强）并在测试时进行微调（自改进），显著提升了语言模型代理的性能，同时减少了训练数据需求。

Details

Motivation: 传统语言模型微调依赖大量训练数据，但数据收集和训练成本高，且无法保证模型能处理复杂场景或具备更好的泛化能力。现有方法未区分样本是否提供新信息，导致资源浪费。

Contribution: 提出了测试时自改进方法（TT-SI），通过自感知、自数据增强和自改进三步，显著提升模型性能（+5.48%准确率），同时减少68倍训练样本量。

Method: 1. 识别模型不确定的样本（自感知）；2. 生成类似样本（自数据增强）；3. 在测试时微调模型（自改进）。对比了TT-SI和TT-D（测试时蒸馏）两种方法。

Result: TT-SI在所有基准测试中平均提升5.48%准确率，且训练样本需求仅为传统方法的1/68。

Insight: 测试时自我改进是一种高效且低成本的新范式，展示了模型在运行过程中自我演化的潜力。

Abstract: One paradigm of language model (LM) fine-tuning relies on creating large training datasets, under the assumption that high quantity and diversity will enable models to generalize to novel tasks after post-training. In practice, gathering large sets of data is inefficient, and training on them is prohibitively expensive; worse, there is no guarantee that the resulting model will handle complex scenarios or generalize better. Moreover, existing techniques rarely assess whether a training sample provides novel information or is redundant with the knowledge already acquired by the model, resulting in unnecessary costs. In this work, we explore a new test-time self-improvement method to create more effective and generalizable agentic LMs on-the-fly. The proposed algorithm can be summarized in three steps: (i) first it identifies the samples that model struggles with (self-awareness), (ii) then generates similar examples from detected uncertain samples (self-data augmentation), and (iii) uses these newly generated samples at test-time fine-tuning (self-improvement). We study two variants of this approach: Test-Time Self-Improvement (TT-SI), where the same model generates additional training examples from its own uncertain cases and then learns from them, and contrast this approach with Test-Time Distillation (TT-D), where a stronger model generates similar examples for uncertain cases, enabling student to adapt using distilled supervision. Empirical evaluations across different agent benchmarks demonstrate that TT-SI improves the performance with +5.48% absolute accuracy gain on average across all benchmarks and surpasses other standard learning methods, yet using 68x less training samples. Our findings highlight the promise of TT-SI, demonstrating the potential of self-improvement algorithms at test-time as a new paradigm for building more capable agents toward self-evolution.

[163] MMM: Quantum-Chemical Molecular Representation Learning for Combinatorial Drug Recommendation

Chongmyung Kwon,Yujin Kim,Seoeun Park,Yunji Lee,Charmgil Hong

Main category: cs.LG

TL;DR: 该论文提出了一种名为MMM的新框架，通过结合量子化学的分子电子局域函数（ELF）三维信息改进药物表示学习，从而优化组合药物推荐中的药物-药物相互作用（DDI）预测。

Details

Motivation: 现有药物推荐系统通常使用图神经网络（GNN）表示药物结构，但其简化的离散形式无法充分捕捉分子结合亲和力和反应性，限制了DDI预测的准确性。

Contribution: 提出了MMM框架，首次将三维量子化学信息（ELF）融入药物表示学习，设计了一种全局电子特性与局部子结构交互的双向图编码器结合的模型。

Method: 通过ELF生成三维电子密度图，结合全局电子特性特征和双向图编码器建模局部子结构交互，学习药物的互补特性。

Result: 在MIMIC-III数据集上，MMM相比GNN-based SafeDrug模型显著提升了F1-score (p=0.0387)、Jaccard (p=0.0112)和DDI率(p=0.0386)。

Insight: 量子化学的三维表示能够提升药物相互作用的预测精度，为临床实践中更安全的组合药物推荐提供了新思路。

Abstract: Drug recommendation is an essential task in machine learning-based clinical decision support systems. However, the risk of drug-drug interactions (DDI) between co-prescribed medications remains a significant challenge. Previous studies have used graph neural networks (GNNs) to represent drug structures. Regardless, their simplified discrete forms cannot fully capture the molecular binding affinity and reactivity. Therefore, we propose Multimodal DDI Prediction with Molecular Electron Localization Function (ELF) Maps (MMM), a novel framework that integrates three-dimensional (3D) quantum-chemical information into drug representation learning. It generates 3D electron density maps using the ELF. To capture both therapeutic relevance and interaction risks, MMM combines ELF-derived features that encode global electronic properties with a bipartite graph encoder that models local substructure interactions. This design enables learning complementary characteristics of drug molecules. We evaluate MMM in the MIMIC-III dataset (250 drugs, 442 substructures), comparing it with several baseline models. In particular, a comparison with the GNN-based SafeDrug model demonstrates statistically significant improvements in the F1-score (p = 0.0387), Jaccard (p = 0.0112), and the DDI rate (p = 0.0386). These results demonstrate the potential of ELF-based 3D representations to enhance prediction accuracy and support safer combinatorial drug prescribing in clinical practice.

[164] Opponent Shaping in LLM Agents

Marta Emili Garcia Segura,Stephen Hailes,Mirco Musolesi

Main category: cs.LG

TL;DR: 本文首次研究了基于大语言模型（LLM）的自主代理人中的对手塑造（Opponent Shaping, OS）能力，提出了一种适用于Transformer架构的模型无关OS方法ShapeLLM，并在多种博弈环境中验证了LLM代理人能够引导对手的学习动态进而影响其行为。

Details

Motivation: 随着LLM作为自主代理人在现实世界中的广泛应用，多代理人交互变得不可避免。理解其战略行为及是否能够仅通过交互塑造对手的学习动态成为关键问题。

Contribution: 提出了ShapeLLM，这是首个针对Transformer架构的模型无关OS方法，证明了LLM代理人能够在竞争性和协作性博弈中成功影响对手的学习动态。

Method: ShapeLLM是对现有模型无关OS方法的改进，适用于Transformer架构，避免了高阶导数需求和扩展性限制。实验涵盖多种博弈环境，如囚徒困境和猎鹿博弈。

Result: 实验表明，LLM代理人能够在竞争性游戏中引导对手达到可被利用的均衡，在协作性游戏中促进协调并提升集体福利。

Insight: LLM代理人不仅能够塑造对手行为，还能被塑造，凸显了OS在LLM多代理人研究中的重要性。

Abstract: Large Language Models (LLMs) are increasingly being deployed as autonomous agents in real-world environments. As these deployments scale, multi-agent interactions become inevitable, making it essential to understand strategic behavior in such systems. A central open question is whether LLM agents, like reinforcement learning agents, can shape the learning dynamics and influence the behavior of others through interaction alone. In this paper, we present the first investigation of opponent shaping (OS) with LLM-based agents. Existing OS algorithms cannot be directly applied to LLMs, as they require higher-order derivatives, face scalability constraints, or depend on architectural components that are absent in transformers. To address this gap, we introduce ShapeLLM, an adaptation of model-free OS methods tailored for transformer-based agents. Using ShapeLLM, we examine whether LLM agents can influence co-players’ learning dynamics across diverse game-theoretic environments. We demonstrate that LLM agents can successfully guide opponents toward exploitable equilibria in competitive games (Iterated Prisoner’s Dilemma, Matching Pennies, and Chicken) and promote coordination and improve collective welfare in cooperative games (Iterated Stag Hunt and a cooperative version of the Prisoner’s Dilemma). Our findings show that LLM agents can both shape and be shaped through interaction, establishing opponent shaping as a key dimension of multi-agent LLM research.

[165] Reinforcing Diffusion Models by Direct Group Preference Optimization

Yihong Luo,Tianyang Hu,Jing Tang

Main category: cs.LG

TL;DR: 该论文提出了直接组偏好优化（DGPO），一种新的在线强化学习算法，用于解决扩散模型中强化学习方法适配的挑战。DGPO直接利用组级偏好信息，避免了低效的随机策略，显著提升了训练速度和性能。

Details

Motivation: 现有的强化学习方法（如GRPO）在扩散模型中的应用受限，因为它们需要随机策略，而高效的扩散采样器通常是确定性的。DGPO旨在解决这一冲突，避免依赖低效的高斯噪声。

Contribution: DGPO是一种创新的在线强化学习算法，无需策略梯度框架，直接利用组级偏好信息，实现了高效训练和优异性能。

Method: DGPO通过直接优化组级偏好，避免了随机策略的需求，采用了高效的确定性ODE采样器，显著提高训练速度。

Result: 实验表明，DGPO的训练速度比现有方法快约20倍，并在域内和域外奖励指标上表现更优。

Insight: DGPO的成功表明，组级偏好信息可以替代传统的策略梯度方法，为扩散模型的强化学习提供了新的方向。

Abstract: While reinforcement learning methods such as Group Relative Preference Optimization (GRPO) have significantly enhanced Large Language Models, adapting them to diffusion models remains challenging. In particular, GRPO demands a stochastic policy, yet the most cost-effective diffusion samplers are based on deterministic ODEs. Recent work addresses this issue by using inefficient SDE-based samplers to induce stochasticity, but this reliance on model-agnostic Gaussian noise leads to slow convergence. To resolve this conflict, we propose Direct Group Preference Optimization (DGPO), a new online RL algorithm that dispenses with the policy-gradient framework entirely. DGPO learns directly from group-level preferences, which utilize relative information of samples within groups. This design eliminates the need for inefficient stochastic policies, unlocking the use of efficient deterministic ODE samplers and faster training. Extensive results show that DGPO trains around 20 times faster than existing state-of-the-art methods and achieves superior performance on both in-domain and out-of-domain reward metrics. Code is available at https://github.com/Luo-Yihong/DGPO.

[166] Mix- and MoE-DPO: A Variational Inference Approach to Direct Preference Optimization

Jason Bohne,Pawel Polak,David Rosenberg,Brian Bloniarz,Gary Kazantsev

Main category: cs.LG

TL;DR: 这篇论文提出了Mix-和MoE-DPO框架，通过引入软混合模型和混合专家（MoE）架构，扩展了直接偏好优化（DPO），并采用变分推理方法优化专家分配策略，以实现多任务和多样化偏好下的高效对齐。

Details

Motivation: 现有的DPO方法依赖单一模型，限制了其在多任务和多样化偏好分布中的表达能力和适应性。因此，作者希望通过引入混合模型和MoE架构来增强模型的灵活性和性能。

Contribution: 1. 提出了Mix-和MoE-DPO框架，结合软混合模型和MoE架构；2. 采用变分推理方法优化专家分配策略；3. 提供通用函数逼近、专家策略特化和上下文对齐三大优势。

Method: 基于变分推断的潜在变量模型，优化变分证据下界（ELBO），学习专家分配策略。支持共享基础架构与专家特定策略头的组合，以及完全独立的专家模型。

Result: 实验验证表明，Mix-和MoE-DPO在多偏好数据集和多种模型规模下均表现优异，提供了一种强大的、可扩展的LLM对齐方法。

Insight: 通过混合模型和MoE架构，能够更灵活地适应多样化偏好和多任务场景，提升模型的表达能力和泛化性能。

Abstract: Direct Preference Optimization (DPO) has recently emerged as a simple and effective alternative to reinforcement learning from human feedback (RLHF) for aligning large language models (LLMs) with user preferences. However, existing DPO formulations rely on a single monolithic model, which limits their expressivity in multi-task settings and their adaptability to heterogeneous or diverse preference distributions. In this work, we propose Mix- and MoE-DPO, a framework that extends DPO with both soft mixture models and mixture-of-experts (MoE) architectures, using a stochastic variational inference approach. Our method introduces a latent-variable model over expert assignments and optimizes a variational evidence lower bound (ELBO), enabling stable and efficient learning of specialized expert policies from preference data. Mix- and MoE-DPO provides three key advantages over standard DPO: (i) generalization via universal function approximation through mixtures; (ii) reward and policy specialization through expert components tailored to distinct preference modes; and (iii) contextual alignment through input-dependent soft gating that enables user-specific mixture policies. Our framework supports both shared base architectures with expert-specific policy heads and fully independent expert models, allowing flexible trade-offs between parameter efficiency and specialization. We validate our approach on a variety of model sizes and multi-preference datasets, demonstrating that Mix- and MoE-DPO offers a powerful and scalable method for preference-based LLM alignment.

[167] FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Experts

Heming Zou,Yunliang Zang,Wutong Xu,Yao Zhu,Xiangyang Ji

Main category: cs.LG

TL;DR: FlyLoRA是一种基于隐式MoE的LoRA变体，通过引入秩级专家激活和隐式路由器，解决了参数干扰问题，同时提升了多任务模型合并的性能。

Details

Motivation: LoRA在基础模型的参数高效微调中广泛应用，但存在参数干扰问题，影响性能。现有的MoE-based LoRA变体虽能缓解单任务内的相关性，但在多任务模型合并中无效，且引入了额外路由器参数。受果蝇嗅觉回路启发，FlyLoRA旨在解决这些问题。

Contribution: 1) 提出秩级专家激活机制；2) 设计隐式路由器，统一专家路由和下投影；3) 利用随机矩阵的正交性缓解任务间干扰，无需显式路由器。

Method: FlyLoRA通过冻结稀疏随机投影矩阵替换传统密集可训练矩阵，实现隐式MoE设计，消除了显式路由器的需求，同时保持计算效率。

Result: 在四个领域的实验中（通用知识理解、科学问答、数学推理和代码生成），FlyLoRA一致优于现有方法。

Insight: FlyLoRA展示了生物结构如何启发AI技术创新，特别是在参数高效性和任务解耦方面。

Abstract: Low-Rank Adaptation (LoRA) is a widely used parameter-efficient fine-tuning method for foundation models, but it suffers from parameter interference, resulting in suboptimal performance. Although Mixture-of-Experts (MoE)-based LoRA variants show promise in mitigating intra-task correlations in single-task instruction tuning, they introduce additional router parameters and remain ineffective in multi-task model merging where inter-task interference arises. Inspired by the fly olfactory circuit, we propose FlyLoRA, an implicit MoE-based LoRA variant that introduces: (1) rank-wise expert activation in the up-projection matrix, and (2) an implicit router that unifies expert routing and down-projection, where a frozen sparse random projection matrix replaces the traditional dense trainable version. This design resolves the trade-off between intra-task decorrelation and computational efficiency by eliminating the need for an explicit router, while inherently mitigating inter-task interference due to the orthogonality property of random matrices. Extensive experiments across four domains – general knowledge understanding, scientific question answering, mathematical reasoning, and code generation – demonstrate consistent performance improvements over existing methods. Beyond empirical gains, FlyLoRA highlights how biological structures can inspire innovations in AI technologies. Code is available at https://github.com/gfyddha/FlyLoRA.

[168] Better Together: Leveraging Unpaired Multimodal Data for Stronger Unimodal Models

Sharut Gupta,Shobhita Sundaram,Chenyu Wang,Stefanie Jegelka,Phillip Isola

Main category: cs.LG

TL;DR: UML是一种模态无关的训练范式，通过交替处理不同模态的输入并共享参数，利用未配对的辅助多模态数据提升单模态任务的表示学习效果。

Details

Motivation: 传统的多模态学习依赖配对数据，研究探讨了是否可以利用未配对的多模态数据直接增强单模态表示学习的效果。

Contribution: 提出了UML（Unpaired Multimodal Learner）框架，利用未配对的多模态数据提升单模态任务的性能。

Method: UML通过参数共享和交替处理不同模态的输入，利用多模态数据的结构信息，无需显式的配对数据。

Result: 理论上证明未配对数据可以提供比单模态训练更丰富的表示；实验表明在多模态辅助数据（如文本、音频、图像）下，单模态任务性能显著提升。

Insight: 未配对的多模态数据可以作为单模态任务的强大辅助资源，揭示了模态间的共享结构对表示学习的潜在价值。

Abstract: Traditional multimodal learners find unified representations for tasks like visual question answering, but rely heavily on paired datasets. However, an overlooked yet potentially powerful question is: can one leverage auxiliary unpaired multimodal data to directly enhance representation learning in a target modality? We introduce UML: Unpaired Multimodal Learner, a modality-agnostic training paradigm in which a single model alternately processes inputs from different modalities while sharing parameters across them. This design exploits the assumption that different modalities are projections of a shared underlying reality, allowing the model to benefit from cross-modal structure without requiring explicit pairs. Theoretically, under linear data-generating assumptions, we show that unpaired auxiliary data can yield representations strictly more informative about the data-generating process than unimodal training. Empirically, we show that using unpaired data from auxiliary modalities – such as text, audio, or images – consistently improves downstream performance across diverse unimodal targets such as image and audio. Our project page: https://unpaired-multimodal.github.io/

[169] xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learning

Cheng Qian,Zuxin Liu,Shirley Kokane,Akshara Prabhakar,Jielin Qiu,Haolin Chen,Zhiwei Liu,Heng Ji,Weiran Yao,Shelby Heinecke,Silvio Savarese,Caiming Xiong,Huan Wang

Main category: cs.LG

TL;DR: xRouter is a reinforcement learning-based LLM orchestration系统 that dynamically routes tasks between expensive premium and lightweight models, optimizing for cost-performance trade-offs without manual rule设计。

Details

Motivation: Modern LLM部署面临着成本和性能之间的权衡：高价模型性能强但昂贵，轻量模型经济但处理复杂任务能力差。现有的静态规则和启发式方法无法充分利用这种频谱。

Contribution: 提出了xRouter，一个基于强化学习的路由系统，能够动态选择直接回答或调用外部模型，通过显式的成本感知奖励实现端到端训练，无需人工规则。

Method: xRouter利用强化学习训练路由决策，奖励函数显式编码成本-性能权衡，涵盖完整的RL框架（奖励计算、成本统计）及部署评估流程。

Result: xRouter在多样化基准测试中实现了显著的性价比优化（如相似任务完成率下成本大幅降低），并对学习路由的可靠性提供了实证见解。

Insight: 研究发现，小型开放模型难以学习复杂的编排行为，而强化学习是实现动态、成本感知路由的有效方法。

Abstract: Modern LLM deployments confront a widening cost-performance spectrum: premium models deliver strong reasoning but are expensive, while lightweight models are economical yet brittle on complex tasks. Static escalation rules and keyword heuristics under-utilize this spectrum and fail to adapt across task types. We present xRouter, a tool-calling-based routing system in which a learned router can either answer directly or invoke one or more external models. The router is trained end-to-end with reinforcement learning using an explicit, cost-aware reward that encodes cost-performance trade-offs, eliminating the need for hand-engineered routing rules. Our implementation encompasses the full reinforcement learning framework, including reward and cost accounting, as well as the deployment and evaluation pipelines. Across diverse benchmarks, xRouter achieves strong cost-performance trade-offs (e.g., substantial cost reductions at comparable task completion rates), and provides empirical insights into what reliably helps learned routing and what does not, ranging from model trainability to the difficulty of eliciting sophisticated orchestration behaviors in small open models. We hope these findings and our open implementation will serve as a practical substrate for advancing learned, cost-aware LLM orchestration.

cs.HC [Back]

[170] Sentiment Matters: An Analysis of 200 Human-SAV Interactions

Lirui Guo,Michael G. Burke,Wynita M. Griggs

Main category: cs.HC

TL;DR: 这篇论文通过分析200个人类与共享自动驾驶汽车(SAV)的互动数据，研究了情感对SAV接受度和服务质量的影响，并比较了基于LLM的情感分析工具与传统方法的性能。

Details

Motivation: 随着共享自动驾驶汽车(SAV)在交通系统中的普及，人类与SAV的有效互动成为重要研究领域。论文旨在通过提供一个开放的数据集，推动这一领域的研究。

Contribution: 1. 提供了一个包含200个互动案例的开源数据集，涵盖文本交互和心理因素的调查数据。2. 使用随机森林模型和弦图识别了SAV接受度和服务质量的关键预测因素，发现情感极性是重要影响因素。3. 比较了LLM与传统情感分析工具的性能，发现LLM在零样本情况下表现更优。

Method: 1. 收集了200个互动案例的文本数据和心理调查数据。2. 使用随机森林和弦图分析关键影响因素。3. 比较了基于LLM和传统TextBlob方法的情感分析效果。

Result: 1. 情感极性是SAV接受度和服务质量的关键预测因素。2. LLM在零样本情况下优于传统情感分析工具，但仍存在局限性。

Insight: 情感在人类与SAV互动中至关重要，设计SAV对话界面时应注重情感表达；LLM在情感分析中潜力巨大，但需进一步优化。

Abstract: Shared Autonomous Vehicles (SAVs) are likely to become an important part of the transportation system, making effective human-SAV interactions an important area of research. This paper introduces a dataset of 200 human-SAV interactions to further this area of study. We present an open-source human-SAV conversational dataset, comprising both textual data (e.g., 2,136 human-SAV exchanges) and empirical data (e.g., post-interaction survey results on a range of psychological factors). The dataset’s utility is demonstrated through two benchmark case studies: First, using random forest modeling and chord diagrams, we identify key predictors of SAV acceptance and perceived service quality, highlighting the critical influence of response sentiment polarity (i.e., perceived positivity). Second, we benchmark the performance of an LLM-based sentiment analysis tool against the traditional lexicon-based TextBlob method. Results indicate that even simple zero-shot LLM prompts more closely align with user-reported sentiment, though limitations remain. This study provides novel insights for designing conversational SAV interfaces and establishes a foundation for further exploration into advanced sentiment modeling, adaptive user interactions, and multimodal conversational systems.

cs.RO [Back]

[171] IntentionVLA: Generalizable and Efficient Embodied Intention Reasoning for Human-Robot Interaction

Yandu Chen,Kefan Gu,Yuqing Wen,Yucheng Zhao,Tiancai Wang,Liqiang Nie

Main category: cs.RO

TL;DR: IntentionVLA提出了一种结合感知与推理的Vision-Language-Action框架，通过课程训练和高效推理机制，显著提升了机器人在复杂人机交互中的隐性意图推理能力。

Details

Motivation: 现有VLA模型主要基于多模态任务预训练，缺乏对具体嵌入场景的意图推理能力，无法满足复杂人机交互的需求。

Contribution: 提出IntentionVLA框架，结合课程训练和高效推理机制，实现了对隐性意图的推理和感知能力的有效结合。

Method: 利用精心设计的推理数据（意图推断、空间定位和紧凑推理）进行预训练，并在微调阶段利用推理输出指导动作生成。

Result: 在直接指令任务上比基线模型π0高18%。在隐性意图任务上比ECoT高28%。在零样本任务上实现了40%的成功率。

Insight: 通过推理密集型预训练和推理引导的动作生成，IntentionVLA展示了下一代人机交互系统的潜力。

Abstract: Vision-Language-Action (VLA) models leverage pretrained vision-language models (VLMs) to couple perception with robotic control, offering a promising path toward general-purpose embodied intelligence. However, current SOTA VLAs are primarily pretrained on multimodal tasks with limited relevance to embodied scenarios, and then finetuned to map explicit instructions to actions. Consequently, due to the lack of reasoning-intensive pretraining and reasoning-guided manipulation, these models are unable to perform implicit human intention reasoning required for complex, real-world interactions. To overcome these limitations, we propose \textbf{IntentionVLA}, a VLA framework with a curriculum training paradigm and an efficient inference mechanism. Our proposed method first leverages carefully designed reasoning data that combine intention inference, spatial grounding, and compact embodied reasoning, endowing the model with both reasoning and perception capabilities. In the following finetuning stage, IntentionVLA employs the compact reasoning outputs as contextual guidance for action generation, enabling fast inference under indirect instructions. Experimental results show that IntentionVLA substantially outperforms $\pi_0$, achieving 18% higher success rates with direct instructions and 28% higher than ECoT under intention instructions. On out-of-distribution intention tasks, IntentionVLA achieves over twice the success rate of all baselines, and further enables zero-shot human-robot interaction with 40% success rate. These results highlight IntentionVLA as a promising paradigm for next-generation human-robot interaction (HRI) systems.

Erjia Xiao,Lingfeng Zhang,Yingbo Tang,Hao Cheng,Renjing Xu,Wenbo Ding,Lei Zhou,Long Chen,Hangjun Ye,Xiaoshuai Hao

Main category: cs.RO

TL;DR: 本文介绍了团队在IROS 2025 RoboSense挑战赛社会导航赛道中的参赛技术，提出了一种基于主动风险感知的改进方法，提升了自动驾驶代理在动态人群环境中的导航能力。

Details

Motivation: 为了解决自动驾驶代理在动态人类环境中导航时的社会规范遵循问题（如安全距离和避碰），团队提出了一种改进方法，旨在增强代理的空间感知能力和主动避碰行为。

Contribution: 主要贡献是引入了一个主动风险感知模块（Proactive Risk Perception Module），通过学习预测周围人类的基于距离的碰撞风险分数，提升了代理的社会导航性能。

Method: 基于Falcon模型进行改进，增加了碰撞风险理解能力。该方法利用RGB-D观测数据学习风险分数，使代理能够更早地识别潜在碰撞风险并采取规避行为。

Result: 在Social-HM3D基准测试中，该方法显著提升了代理的动态避碰能力和社会规范遵循表现，最终在16支参赛队伍中排名第二。

Insight: 主动风险感知可以通过预测碰撞风险分数增强代理的空间意识，尤其是在拥挤的动态环境中。这表明未来社会导航系统可以进一步从风险预测中受益。

Abstract: In this report, we describe the technical details of our submission to the IROS 2025 RoboSense Challenge Social Navigation Track. This track focuses on developing RGBD-based perception and navigation systems that enable autonomous agents to navigate safely, efficiently, and socially compliantly in dynamic human-populated indoor environments. The challenge requires agents to operate from an egocentric perspective using only onboard sensors including RGB-D observations and odometry, without access to global maps or privileged information, while maintaining social norm compliance such as safe distances and collision avoidance. Building upon the Falcon model, we introduce a Proactive Risk Perception Module to enhance social navigation performance. Our approach augments Falcon with collision risk understanding that learns to predict distance-based collision risk scores for surrounding humans, which enables the agent to develop more robust spatial awareness and proactive collision avoidance behaviors. The evaluation on the Social-HM3D benchmark demonstrates that our method improves the agent’s ability to maintain personal space compliance while navigating toward goals in crowded indoor scenes with dynamic human agents, achieving 2nd place among 16 participating teams in the challenge.

Haolin Yang,Yuxing Long,Zhuoyuan Yu,Zihan Yang,Minghan Wang,Jiapeng Xu,Yihan Wang,Ziyan Yu,Wenzhe Cai,Lei Kang,Hao Dong

Main category: cs.RO

TL;DR: NavSpace是一个专注于评估导航代理空间感知与推理能力的基准测试，包含六类任务和1,228对轨迹-指令。通过评估22种导航代理（包括SOTA模型和多模态大语言模型），揭示了其在空间智能方面的表现，并提出新模型SNav，优于现有方法。

Details

Motivation: 现有导航基准主要关注语义理解，忽视了空间感知与推理能力的系统性评估。NavSpace的提出填补了这一空白，旨在更全面地评估导航代理的空间智能。

Contribution: 提出NavSpace基准，包含六类任务和1,228对轨迹-指令；评估22种导航代理，揭示空间智能表现；提出新模型SNav，在NavSpace和真实机器人测试中表现优异。

Method: NavSpace设计了六类任务以评估空间智能；提出SNav模型，结合空间感知与推理能力。

Result: SNav在NavSpace和真实机器人测试中表现优于现有导航代理。

Insight: 空间智能是导航代理的重要能力，NavSpace为未来研究提供了系统性评估工具。

Abstract: Instruction-following navigation is a key step toward embodied intelligence. Prior benchmarks mainly focus on semantic understanding but overlook systematically evaluating navigation agents’ spatial perception and reasoning capabilities. In this work, we introduce the NavSpace benchmark, which contains six task categories and 1,228 trajectory-instruction pairs designed to probe the spatial intelligence of navigation agents. On this benchmark, we comprehensively evaluate 22 navigation agents, including state-of-the-art navigation models and multimodal large language models. The evaluation results lift the veil on spatial intelligence in embodied navigation. Furthermore, we propose SNav, a new spatially intelligent navigation model. SNav outperforms existing navigation agents on NavSpace and real robot tests, establishing a strong baseline for future work.

[174] DexMan: Learning Bimanual Dexterous Manipulation from Human and Generated Videos

Jhen Hsieh,Kuan-Hsun Tu,Kuo-Han Hung,Tsung-Wei Ke

Main category: cs.RO

TL;DR: DexMan是一个自动化框架，将人类视觉演示转化为双手机器人操作技能，无需额外硬件或精确标注，通过接触奖励和强化学习实现高性能操作。

Details

Motivation: 现有方法通常依赖简化模型（如浮动手）或昂贵的数据采集设备（如深度传感器或运动捕捉），限制了泛化能力和数据规模。DexMan旨在直接从第三人称视频中学习操作技能，降低成本并提升通用性。

Contribution: 1. 提出无需精确标注或额外硬件的框架DexMan；2. 通过接触奖励提升强化学习策略；3. 兼容真实与合成视频生成技能；4. 在TACO和OakInk-v2基准上实现SOTA性能。

Method: 1. 从视频中估计手-物体姿态；2. 通过接触奖励优化策略；3. 结合强化学习训练人形机器人；4. 支持合成数据生成大规模数据集。

Result: 1. TACO基准上ADD-S和VSD指标分别提升0.08和0.12；2. OakInk-v2任务成功率提升19%。

Insight: 接触奖励和合成数据生成显著提升了双手机器人操作的泛化能力，为大规模数据集和通用技能学习提供了新思路。

Abstract: We present DexMan, an automated framework that converts human visual demonstrations into bimanual dexterous manipulation skills for humanoid robots in simulation. Operating directly on third-person videos of humans manipulating rigid objects, DexMan eliminates the need for camera calibration, depth sensors, scanned 3D object assets, or ground-truth hand and object motion annotations. Unlike prior approaches that consider only simplified floating hands, it directly controls a humanoid robot and leverages novel contact-based rewards to improve policy learning from noisy hand-object poses estimated from in-the-wild videos. DexMan achieves state-of-the-art performance in object pose estimation on the TACO benchmark, with absolute gains of 0.08 and 0.12 in ADD-S and VSD. Meanwhile, its reinforcement learning policy surpasses previous methods by 19% in success rate on OakInk-v2. Furthermore, DexMan can generate skills from both real and synthetic videos, without the need for manual data collection and costly motion capture, and enabling the creation of large-scale, diverse datasets for training generalist dexterous manipulation.

[175] DexNDM: Closing the Reality Gap for Dexterous In-Hand Rotation via Joint-Wise Neural Dynamics Model

Xueyi Liu,He Wang,Li Yi

Main category: cs.RO

TL;DR: DexNDM提出了一种基于联合神经动力学模型的框架，通过在仿真中训练的策略实现现实中广泛物体的灵巧旋转，解决了从仿真到现实的‘现实鸿沟’问题。

Details

Motivation: 灵巧手操控物体的旋转在机器人学中是一个挑战，主要是因为仿真策略难以迁移到现实世界。复杂的接触动力学导致了‘现实鸿沟’，限制了现有方法的应用范围。

Contribution: 1. 提出了联合神经动力学模型，通过适应仿真动作弥合现实鸿沟；2. 开发了高效的数据收集策略，减少人为干预；3. 实现了对形状复杂、高长宽比和小尺寸物体的广泛泛化。

Method: 1. 基于联合的动态模型分解动力学，学习每个关节的动态特性；2. 压缩系统级影响为低维变量；3. 结合自主数据收集策略获取多样化现实数据。

Result: 单一策略成功旋转了形状复杂、高长宽比和小尺寸的物体，适应多种手腕方向和旋转轴。

Insight: 1. 联合动力学分解和低维压缩是实现高效泛化的关键；2. 自主数据收集策略显著提升了模型的适应性。

Abstract: Achieving generalized in-hand object rotation remains a significant challenge in robotics, largely due to the difficulty of transferring policies from simulation to the real world. The complex, contact-rich dynamics of dexterous manipulation create a “reality gap” that has limited prior work to constrained scenarios involving simple geometries, limited object sizes and aspect ratios, constrained wrist poses, or customized hands. We address this sim-to-real challenge with a novel framework that enables a single policy, trained in simulation, to generalize to a wide variety of objects and conditions in the real world. The core of our method is a joint-wise dynamics model that learns to bridge the reality gap by effectively fitting limited amount of real-world collected data and then adapting the sim policy’s actions accordingly. The model is highly data-efficient and generalizable across different whole-hand interaction distributions by factorizing dynamics across joints, compressing system-wide influences into low-dimensional variables, and learning each joint’s evolution from its own dynamic profile, implicitly capturing these net effects. We pair this with a fully autonomous data collection strategy that gathers diverse, real-world interaction data with minimal human intervention. Our complete pipeline demonstrates unprecedented generality: a single policy successfully rotates challenging objects with complex shapes (e.g., animals), high aspect ratios (up to 5.33), and small sizes, all while handling diverse wrist orientations and rotation axes. Comprehensive real-world evaluations and a teleoperation application for complex tasks validate the effectiveness and robustness of our approach. Website: https://meowuu7.github.io/DexNDM/

[176] NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videos

Hongyu Li,Lingfeng Sun,Yafei Hu,Duy Ta,Jennifer Barry,George Konidaris,Jiahui Fu

Main category: cs.RO

TL;DR: NovaFlow是一个零样本机器人操作框架，通过从生成视频中提取可操作的对象流，实现跨平台的未知任务执行，无需任务演示或特定硬件训练。

Details

Motivation: 现有机器人操作方法通常需要任务分布内数据或依赖于特定硬件的微调，限制了跨平台的迁移能力。NovaFlow旨在解决这一问题，实现零样本的任务执行。

Contribution: 提出了一个无需演示的机器人操作框架NovaFlow，通过生成视频和提取3D对象流，实现跨硬件的零样本任务执行。

Method: 1. 使用视频生成模型根据任务描述合成视频；2. 利用现成感知模块提取3D可操作对象流；3. 对刚性物体计算相对位姿并转化为机器人动作；4. 对可变形对象使用基于粒子的动力学模型进行规划。

Result: 在刚性、关节和可变形物体的操作任务中，NovaFlow在Franka机械臂和Spot四足机器人上实现了有效的零样本执行。

Insight: 将高层次任务理解与低层次控制解耦，使框架能够自然跨硬件迁移，适用于多种对象类型。

Abstract: Enabling robots to execute novel manipulation tasks zero-shot is a central goal in robotics. Most existing methods assume in-distribution tasks or rely on fine-tuning with embodiment-matched data, limiting transfer across platforms. We present NovaFlow, an autonomous manipulation framework that converts a task description into an actionable plan for a target robot without any demonstrations. Given a task description, NovaFlow synthesizes a video using a video generation model and distills it into 3D actionable object flow using off-the-shelf perception modules. From the object flow, it computes relative poses for rigid objects and realizes them as robot actions via grasp proposals and trajectory optimization. For deformable objects, this flow serves as a tracking objective for model-based planning with a particle-based dynamics model. By decoupling task understanding from low-level control, NovaFlow naturally transfers across embodiments. We validate on rigid, articulated, and deformable object manipulation tasks using a table-top Franka arm and a Spot quadrupedal mobile robot, and achieve effective zero-shot execution without demonstrations or embodiment-specific training. Project website: https://novaflow.lhy.xyz/.

Table of Contents

cs.CL [Back]

[1] Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation

[2] LASER: An LLM-based ASR Scoring and Evaluation Rubric

[3] Meaningful Pose-Based Sign Language Evaluation

[4] Populism Meets AI: Advancing Populism Research with LLMs

[5] MAPRO: Recasting Multi-Agent Prompt Optimization as Maximum a Posteriori Inference

[6] AsyncSpade: Efficient Test-Time Scaling with Asynchronous Sparse Decoding

[7] Can Lessons From Human Teams Be Applied to Multi-Agent Systems? The Role of Structure, Diversity, and Interaction Dynamics

[8] Can Speech LLMs Think while Listening?

[9] When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMs

[10] ParsTranslit: Truly Versatile Tajik-Farsi Transliteration

[11] Deploying Tiny LVLM Judges for Real-World Evaluation of Chart Models: Lessons Learned and Best Practices

[12] Multi-Task Pre-Finetuning of Lightweight Transformer Encoders for Text Classification and NER

[13] Linguistic Patterns in Pandemic-Related Content: A Comparative Analysis of COVID-19, Constraint, and Monkeypox Datasets

[14] Role-Conditioned Refusals: Evaluating Access Control Reasoning in Large Language Models

[15] Causality Guided Representation Learning for Cross-Style Hate Speech Detection

[16] SUBQRAG: sub-question driven dynamic graph rag

[17] Multilingual Knowledge Graph Completion via Efficient Multilingual Knowledge Sharing

[18] ToolExpander: Extending the Frontiers of Tool-Using Reinforcement Learning to Weak LLMs

[19] OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment

[20] Parallel Test-Time Scaling for Latent Reasoning Models

[21] Test-Time Reasoners Are Strategic Multiple-Choice Test-Takers

[22] ToolLibGen: Scalable Automatic Tool Creation and Aggregation for LLM Reasoning

[23] Curing Miracle Steps in LLM Mathematical Reasoning with Rubric Rewards

[24] The Unintended Trade-off of AI Alignment:Balancing Hallucination Mitigation and Safety in LLMs

[25] LLM4Cell: A Survey of Large Language and Agentic Models for Single-Cell Biology

[26] HiPRAG: Hierarchical Process Rewards for Efficient Agentic Retrieval Augmented Generation

[27] AdaSwitch: Adaptive Switching Generation for Knowledge Distillation

[28] Do LLMs Really Need 10+ Thoughts for “Find the Time 1000 Days Later”? Towards Structural Understanding of LLM Overthinking

[29] CS3-Bench: Evaluating and Enhancing Speech-to-Speech LLMs for Mandarin-English Code-Switching

[30] Contrastive Weak-to-strong Generalization

[31] Metric Calculating Benchmark: Code-Verifiable Complicate Instruction Following Benchmark for Large Language Models

[32] ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recall

[33] Towards Human-Like Grading: A Unified LLM-Enhanced Framework for Subjective Question Evaluation

[34] STEPER: Step-wise Knowledge Distillation for Enhancing Reasoning Ability in Multi-Step Retrieval-Augmented Language Models

[35] Comprehensiveness Metrics for Automatic Evaluation of Factual Recall in Text Generation

[36] Vision-Enabled LLMs in Historical Lexicography: Digitising and Enriching Estonian-German Dictionaries from the 17th and 18th Centuries

[37] A$^2$Search: Ambiguity-Aware Question Answering with Reinforcement Learning

[38] LightReasoner: Can Small Language Models Teach Large Language Models Reasoning?

[39] Active Confusion Expression in Large Language Models: Leveraging World Models toward Better Social Reasoning

[40] Leveraging Author-Specific Context for Scientific Figure Caption Generation: 3rd SciCap Challenge

[41] Learning on the Job: An Experience-Driven Self-Evolving Agent for Long-Horizon Tasks

[42] ChatGPT as a Translation Engine: A Case Study on Japanese-English

[43] A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Models

[44] The Price of Thought: A Multilingual Analysis of Reasoning, Performance, and Cost of Negotiation in Large Language Models

[45] Lossless Vocabulary Reduction for Auto-Regressive Language Models

[46] Evaluating LLM-Generated Legal Explanations for Regulatory Compliance in Social Media Influencer Marketing

[47] ARM2: Adaptive Reasoning Model with Vision Understanding and Executable Code

[48] MetricalARGS: A Taxonomy for Studying Metrical Poetry with LLMs

[49] Training-Free Group Relative Policy Optimization

[50] Memory Retrieval and Consolidation in Large Language Models through Function Tokens

[51] LLMs Learn to Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactions

[52] Investigating Counterclaims in Causality Extraction from Text

[53] The Alignment Waltz: Jointly Training Agents to Collaborate for Safety

[54] Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling

[55] Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window

[56] If Probable, Then Acceptable? Understanding Conditional Acceptability Judgments in Large Language Models

[57] ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shaping

[58] LeWiDi-2025 at NLPerspectives: The Third Edition of the Learning with Disagreements Shared Task

[59] DeepPrune: Parallel Scaling without Inter-trace Redundancy

[60] Which Heads Matter for Reasoning? RL-Guided KV Cache Compression

[61] CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards

[62] ArenaBencher: Automatic Benchmark Evolution via Multi-Model Competitive Evaluation

cs.CV [Back]

[63] DynamicEval: Rethinking Evaluation for Dynamic Text-to-Video Synthesis

[64] PickStyle: Video-to-Video Style Transfer with Context-Style Adapters

[65] TRAVL: A Recipe for Making Video-Language Models Better Judges of Physics Implausibility

[66] Cross-Modal Attention Guided Unlearning in Vision-Language Models

[67] MaizeStandCounting (MaSC): Automated and Accurate Maize Stand Counting from UAV Imagery Using Image Processing and Deep Learning

[68] PIT-QMM: A Large Multimodal Model For No-Reference Point Cloud Quality Assessment

[69] Dual-Stream Alignment for Action Segmentation

[70] Once Is Enough: Lightweight DiT-Based Video Virtual Try-On via One-Time Garment Appearance Injection

[71] MONKEY: Masking ON KEY-Value Activation Adapter for Personalization

[72] Automatic Text Box Placement for Supporting Typographic Design

[73] TCIP: Threshold-Controlled Iterative Pyramid Network for Deformable Medical Image Registration

[74] Controllable Video Synthesis via Variational Inference

[75] Hybrid CNN-BYOL Approach for Fault Detection in Induction Motors Using Thermal Images

[76] Mutual Learning for Hashing: Unlocking Strong Hash Functions from Weak Supervision

[77] RePainter: Empowering E-commerce Object Removal via Spatial-matting Reinforcement Learning