cs.CL [Total: 35]
cs.CV [Total: 68]
cs.GR [Total: 1]
cs.RO [Total: 1]
cs.LG [Total: 4]
cs.NI [Total: 1]
cs.CY [Total: 1]
cs.AI [Total: 4]

cs.CL [Back]

[1] AriadneMem: Threading the Maze of Lifelong Memory for LLM Agents cs.CL | cs.AI | cs.IR | cs.LGPDF

Wenhui Zhu, Xiwen Chen, Zhipeng Wang, Jingjing Wang, Xuanzhao Dong

TL;DR: 本文提出了AriadneMem，一个为长视野LLM智能体设计的结构化记忆系统，旨在解决长期对话中证据分散和状态更新的挑战。它采用解耦的两阶段流程：离线构建阶段通过熵感知门控过滤噪声并应用冲突感知粗化来合并静态副本；在线推理阶段则通过算法桥接发现和单次拓扑感知合成来重建逻辑路径。

Details

Motivation: 现有长视野LLM智能体的记忆系统在固定上下文预算下难以处理长期对话中的两个持久挑战：一是多跳答案所需的证据分散在不同时间点（证据不连贯），二是信息演变（如日程变更）导致与旧静态记录冲突（状态更新）。

Result: 在基于GPT-4o的LoCoMo实验中，AriadneMem相比强基线在Multi-Hop F1上提升了15.2%，Average F1提升了9.0%。通过将推理卸载到图层，系统仅使用497个上下文令牌，总运行时间减少了77.8%。

Insight: 创新点在于将记忆构建与推理解耦为两阶段，并引入图结构来显式建模时间边以处理状态演变。其核心是离线阶段的熵感知门控与冲突感知粗化，以及在线阶段的算法桥接发现，这避免了昂贵的迭代规划，实现了高效、准确的长期记忆管理。

Abstract: Long-horizon LLM agents require memory systems that remain accurate under fixed context budgets. However, existing systems struggle with two persistent challenges in long-term dialogue: (i) \textbf{disconnected evidence}, where multi-hop answers require linking facts distributed across time, and (ii) \textbf{state updates}, where evolving information (e.g., schedule changes) creates conflicts with older static logs. We propose AriadneMem, a structured memory system that addresses these failure modes via a decoupled two-phase pipeline. In the \textbf{offline construction phase}, AriadneMem employs \emph{entropy-aware gating} to filter noise and low-information message before LLM extraction and applies \emph{conflict-aware coarsening} to merge static duplicates while preserving state transitions as temporal edges. In the \textbf{online reasoning phase}, rather than relying on expensive iterative planning, AriadneMem executes \emph{algorithmic bridge discovery} to reconstruct missing logical paths between retrieved facts, followed by \emph{single-call topology-aware synthesis}. On LoCoMo experiments with GPT-4o, AriadneMem improves \textbf{Multi-Hop F1 by 15.2%} and \textbf{Average F1 by 9.0%} over strong baselines. Crucially, by offloading reasoning to the graph layer, AriadneMem reduces \textbf{total runtime by 77.8%} using only \textbf{497} context tokens. The code is available at https://github.com/LLM-VLM-GSL/AriadneMem.

[2] From Conflict to Consensus: Boosting Medical Reasoning via Multi-Round Agentic RAG cs.CL | cs.AI | cs.IRPDF

Wenhao Wu, Zhentao Tang, Yafu Li, Shixiong Kai, Mingxuan Yuan

TL;DR: 本文提出了一种名为MA-RAG（多轮代理RAG）的框架，用于提升大型语言模型在复杂医学问答中的推理能力。该框架通过一个代理式的多轮精炼循环，迭代地利用候选答案之间的语义冲突来驱动外部证据检索，并优化内部推理历史，最终将冲突转化为共识，以提高答案的准确性和可靠性。

Details

Motivation: 大型语言模型在医学问答中表现出强大的推理能力，但存在产生幻觉和知识过时的问题，这在医疗领域具有关键风险。现有的检索增强生成方法依赖于嘈杂的token级信号，且缺乏复杂推理所需的多轮精炼能力。

Result: 在7个医学问答基准测试上的广泛评估表明，MA-RAG持续超越了竞争性的推理时扩展和RAG基线方法，在骨干模型的基础上平均准确率大幅提升了+6.8个百分点。

Insight: 创新点在于将自洽性原理扩展为一种主动信号，利用候选答案间的不一致性来驱动多轮代理推理和检索，并模仿了一种提升机制，迭代地最小化残差误差以达成稳定、高保真的医学共识。这为复杂领域推理提供了一种新的、迭代式的冲突到共识的优化范式。

Abstract: Large Language Models (LLMs) exhibit high reasoning capacity in medical question-answering, but their tendency to produce hallucinations and outdated knowledge poses critical risks in healthcare fields. While Retrieval-Augmented Generation (RAG) mitigates these issues, existing methods rely on noisy token-level signals and lack the multi-round refinement required for complex reasoning. In the paper, we propose MA-RAG (Multi-Round Agentic RAG), a framework that facilitates test-time scaling for complex medical reasoning by iteratively evolving both external evidence and internal reasoning history within an agentic refinement loop. At each round, the agent transforms semantic conflict among candidate responses into actionable queries to retrieve external evidence, while optimizing history reasoning traces to mitigate long-context degradation. MA-RAG extends the self-consistency principle by leveraging the lack of consistency as a proactive signal for multi-round agentic reasoning and retrieval, and mirrors a boosting mechanism that iteratively minimizes the residual error toward a stable, high-fidelity medical consensus. Extensive evaluations across 7 medical Q&A benchmarks show that MA-RAG consistently surpasses competitive inference-time scaling and RAG baselines, delivering substantial +6.8 points on average accuracy over the backbone model. Our code is available at this url.

[3] SE-Search: Self-Evolving Search Agent via Memory and Dense Reward cs.CLPDF

Jian Li, Yizhang Jin, Dongqi Liu, Hang Ding, Jiafu Wu

TL;DR: 本文提出SE-Search，一种通过记忆净化、原子查询训练和密集奖励实现自我进化的搜索智能体，旨在解决现有检索增强生成（RAG）方法在自主多轮信息检索中积累无关文档和依赖稀疏强化学习信号的问题。该方法采用Think-Search-Memorize策略，在单跳和多跳问答基准测试中显著优于基线模型。

Details

Motivation: 现有检索增强生成（RAG）方法在作为自主多轮信息检索过程时，容易积累无关或噪声文档，并且依赖稀疏的强化学习信号，导致搜索效率低下。

Result: 在单跳和多跳问答基准测试中，SE-Search-3B模型显著优于强基线模型，相比Search-R1实现了10.8个百分点的绝对提升和33.8%的相对增益。

Insight: 创新点在于整合了记忆净化机制以保留关键证据并过滤无关内容，引入原子查询训练以生成更简短多样的查询来改善证据获取，以及采用密集奖励提供细粒度反馈以加速训练过程，从而实现了搜索行为的在线自我进化。

Abstract: Retrieval augmented generation (RAG) reduces hallucinations and factual errors in large language models (LLMs) by conditioning generation on retrieved external knowledge. Recent search agents further cast RAG as an autonomous, multi-turn information-seeking process. However, existing methods often accumulate irrelevant or noisy documents and rely on sparse reinforcement learning signals. We propose \textbf{S}elf-\textbf{E}volving \textbf{Search}, a Self-Evolving Search agent that improves online search behavior through three components, memory purification, atomic query training, and dense rewards. SE-Search follows a \textit{Think-Search-Memorize} strategy that retains salient evidence while filtering irrelevant content. Atomic query training promotes shorter and more diverse queries, improving evidence acquisition. Dense rewards provide fine-grained feedback that speeds training. Experiments on single-hop and multi-hop question answering benchmarks show that \texttt{SE-Search-3B} outperforms strong baselines, yielding a $10.8$ point absolute improvement and a $33.8%$ relative gain over Search-R1.\footnote{We will make the code and model weights publicly available upon acceptance.}

[4] Language Model Goal Selection Differs from Humans’ in an Open-Ended Task cs.CL | cs.AI | cs.CYPDF

Gaia Molinaro, Dave August, Danielle Perszyk, Anne G. E. Collins

TL;DR: 本文通过一项开放式学习任务，评估了大型语言模型（LLM）在目标选择上与人类行为的相似性，发现包括GPT-5、Gemini 2.5 Pro、Claude Sonnet 4.5和Centaur在内的先进模型与人类行为存在显著差异，模型倾向于利用单一解决方案或表现不佳，而人类则表现出渐进探索和个体多样性。

Details

Motivation: 随着LLM越来越多地自主选择目标而非仅完成人类定义的任务，其目标选择是否反映人类偏好尚不明确，本文旨在直接测试LLM作为人类目标选择代理的有效性。

Result: 在受控的开放式学习任务中，四个先进模型（GPT-5、Gemini 2.5 Pro、Claude Sonnet 4.5和Centaur）均与人类行为有显著分歧，人类表现出渐进探索和个体多样性，而模型大多利用单一解决方案（奖励破解）或性能低下，且同一模型的实例间变异性低；即使专为模拟人类训练的Centaur模型也未能有效捕捉人类目标选择，思维链推理和角色引导仅带来有限改进。

Insight: 论文的创新点在于直接比较LLM与人类在开放式任务中的目标选择行为，揭示了当前模型在模拟人类探索性和多样性方面的局限性；客观分析表明，这警示了在个人助理、科学发现和政策研究等应用中用现有模型替代人类目标选择的风险，强调了人类决策的独特性。

Abstract: As large language models (LLMs) get integrated into human decision-making, they are increasingly choosing goals autonomously rather than only completing human-defined ones, assuming they will reflect human preferences. However, human-LLM similarity in goal selection remains largely untested. We directly assess the validity of LLMs as proxies for human goal selection in a controlled, open-ended learning task borrowed from cognitive science. Across four state-of-the-art models (GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5, and Centaur), we find substantial divergence from human behavior. While people gradually explore and learn to achieve goals with diversity across individuals, most models exploit a single identified solution (reward hacking) or show surprisingly low performance, with distinct patterns across models and little variability across instances of the same model. Even Centaur, explicitly trained to emulate humans in experimental settings, poorly captures people’s goal selection. Chain-of-thought reasoning and persona steering provide limited improvements. These findings highlight the uniqueness of human goal selection, cautioning against replacing it with current models in applications such as personal assistance, scientific discovery, and policy research.

[5] PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agents cs.CL | cs.AI | cs.IRPDF

Ke Yang, Zixi Chen, Xuan He, Jize Jiang, Michel Galley

TL;DR: 本文提出了PlugMem，一种任务无关的插件式记忆模块，可附加到任意大型语言模型（LLM）智能体上，旨在解决现有记忆设计要么任务特定且不可迁移、要么任务无关但效果不佳（因任务相关性低和原始记忆检索导致的上下文爆炸）的问题。该方法受认知科学启发，将情景记忆结构化为一个紧凑、可扩展的以知识为中心的记忆图，显式表示命题性和规范性知识，从而实现高效的任务相关知识检索与推理。

Details

Motivation: 解决LLM智能体在复杂环境中运行时，现有长期记忆设计存在的任务特定性（不可迁移）与任务无关性（效果差，因任务相关性低和原始记忆检索导致的上下文爆炸）之间的矛盾。

Result: 在三个异构基准测试（长程对话问答、多跳知识检索和网页智能体任务）上，PlugMem始终优于任务无关基线，并超过了任务特定的记忆设计，同时在统一的信息论分析下实现了最高的信息密度。

Insight: 核心创新在于将记忆的组织和访问单元从实体或文本块转变为抽象知识，构建了一个以知识为中心的记忆图，这借鉴了认知科学，并实现了任务无关的高效记忆检索与推理，区别于GraphRAG等方法。

Abstract: Long-term memory is essential for large language model (LLM) agents operating in complex environments, yet existing memory designs are either task-specific and non-transferable, or task-agnostic but less effective due to low task-relevance and context explosion from raw memory retrieval. We propose PlugMem, a task-agnostic plugin memory module that can be attached to arbitrary LLM agents without task-specific redesign. Motivated by the fact that decision-relevant information is concentrated as abstract knowledge rather than raw experience, we draw on cognitive science to structure episodic memories into a compact, extensible knowledge-centric memory graph that explicitly represents propositional and prescriptive knowledge. This representation enables efficient memory retrieval and reasoning over task-relevant knowledge, rather than verbose raw trajectories, and departs from other graph-based methods like GraphRAG by treating knowledge as the unit of memory access and organization instead of entities or text chunks. We evaluate PlugMem unchanged across three heterogeneous benchmarks (long-horizon conversational question answering, multi-hop knowledge retrieval, and web agent tasks). The results show that PlugMem consistently outperforms task-agnostic baselines and exceeds task-specific memory designs, while also achieving the highest information density under a unified information-theoretic analysis. Code and data are available at https://github.com/TIMAN-group/PlugMem.

[6] TTSR: Test-Time Self-Reflection for Continual Reasoning Improvement cs.CL | cs.AI | cs.LGPDF

Haoyang He, Zihua Rong, Liangjie Zhao, Yunjia Zhao, Lan Yang

TL;DR: 本文提出了一种名为TTSR的测试时自反思训练框架，旨在通过单一预训练语言模型在测试时交替扮演学生和教师的角色，以解决测试时训练中伪标签不可靠和无法针对性适应模型推理弱点的问题，从而持续提升大语言模型的推理能力。

Details

Motivation: 解决测试时训练面临的两大挑战：测试问题难度高导致自生成伪标签不可靠，以及现有方法缺乏有效机制来针对性适应模型特定的推理弱点，导致学习效率低下。

Result: 在多个具有挑战性的数学推理基准测试上的实验结果表明，TTSR能持续提升推理性能，并且在不同模型主干和通用领域推理任务上具有良好的泛化能力。

Insight: 核心创新点在于引入了教师-学生角色交替的自反思机制，教师分析学生失败的推理轨迹并总结其弱点，进而合成有针对性的变体问题，形成一个可学习的持续自我进化循环，为测试时稳定、持续的推理改进提供了有效途径。

Abstract: Test-time Training enables model adaptation using only test questions and offers a promising paradigm for improving the reasoning ability of large language models (LLMs). However, it faces two major challenges: test questions are often highly difficult, making self-generated pseudo-labels unreliable, and existing methods lack effective mechanisms to adapt to a model’s specific reasoning weaknesses, leading to inefficient learning. To address these issues, we propose \textbf{TTSR}, a self-reflective test-time self-evolving training framework. TTSR employs a single pretrained language model that alternates between the roles of a \textit{Student} and a \textit{Teacher} at test time. The Student focuses on solving problems and learning from synthesized variant questions, while the Teacher analyzes the Student’s failed reasoning trajectories, summarizes recurring reasoning weaknesses, and synthesizes targeted variant questions accordingly. This process guides the model to improve within a learnable regime through a continual self-evolving loop. Experimental results on multiple challenging mathematical reasoning benchmarks show that TTSR consistently improves reasoning performance and generalizes well across different model backbones and general-domain reasoning tasks. These findings suggest that teacher-mediated self-reflection provides an effective pathway for stable and continual reasoning improvement at test time.

[7] TATRA: Training-Free Instance-Adaptive Prompting Through Rephrasing and Aggregation cs.CL | cs.AIPDF

Bartosz Dziuba, Kacper Kuchta, Paweł Batorski, Przemysław Spurek, Paul Swoboda

TL;DR: TATRA是一种无需训练、实例自适应的提示方法，通过动态重述和聚合来为每个输入实例构建特定的少样本提示，无需任务特定训练数据或昂贵的迭代优化，在文本分类和数学推理基准测试中表现优异。

Details

Motivation: 解决大型语言模型（LLMs）对提示措辞高度敏感的问题，现有自动提示工程方法通常需要任务特定训练集、昂贵的迭代优化且无法灵活适应新任务。

Result: 在标准文本分类基准测试中，TATRA匹配或超越了依赖训练数据和广泛搜索的强提示优化基线；在数学推理基准测试（GSM8K和DeepMath）上达到了最先进的性能（SOTA）。

Insight: 创新点在于提出了一种无需训练数据、无需任务特定优化循环的实例自适应提示构建方法，通过动态合成上下文示例来提升性能，表明为每个实例构建有效的上下文示例比针对整个任务运行昂贵的优化循环更为重要。

Abstract: Large Language Models (LLMs) have improved substantially alignment, yet their behavior remains highly sensitive to prompt phrasing. This brittleness has motivated automated prompt engineering, but most existing methods (i) require a task-specific training set, (ii) rely on expensive iterative optimization to produce a single dataset-level prompt, and (iii) must be rerun from scratch for each new task. We introduce TATRA, a dataset-free prompting method that constructs instance-specific few-shot prompts by synthesizing on-the-fly examples to accompany a user-provided instruction. TATRA requires no labeled training data and avoids task-specific optimization loops, while retaining the benefits of demonstration-based prompting. Across standard text classification benchmarks, TATRA matches or improves over strong prompt-optimization baselines that depend on training data and extensive search. On mathematical reasoning benchmarks, TATRA achieves state-of-the-art performance on GSM8K and DeepMath, outperforming methods that explicitly optimize prompts on those tasks. Our results suggest that per-instance construction of effective in-context examples is more important than running long, expensive optimization loops to produce a single prompt per task. We will make all code publicly available upon acceptance of the paper. Code is available at https://github.com/BMD223/TATRA

[8] Benchmarking Legal RAG: The Promise and Limits of AI Statutory Surveys cs.CLPDF

Mohamed Afane, Emaan Hariri, Derek Ouyang, Daniel E. Ho

TL;DR: 本文评估了三种新兴工具（STARA、Westlaw AI和Lexis+ AI）在法律RAG基准LaborBench上的表现，发现STARA在布尔任务上准确率达83%，远优于商业平台和标准RAG，并通过错误分析揭示了DOL律师自身存在遗漏，修正后STARA准确率可达92%。

Details

Motivation: 针对法律AI领域RAG系统缺乏系统性基准的问题，基于LaborBench基准，评估新兴工具在法律条文检索与生成任务上的实际性能与局限性。

Result: 在LaborBench基准的布尔任务上，STARA准确率为83%（修正DOL遗漏后达92%），而Westlaw AI和Lexis+ AI分别仅为58%和64%，均低于标准RAG的70%。

Insight: 研究揭示了商业AI法律工具性能不足、人工标注数据存在遗漏的关键问题，并提出了提升多司法管辖区法律AI系统准确性的具体设计原则，为法律RAG的发展提供了实证基础和实用指导。

Abstract: Retrieval-augmented generation (RAG) offers significant potential for legal AI, yet systematic benchmarks are sparse. Prior work introduced LaborBench to benchmark RAG models based on ostensible ground truth from an exhaustive, multi-month, manual enumeration of all U.S. state unemployment insurance requirements by U.S. Department of Labor (DOL) attorneys. That prior work found poor performance of standard RAG (70% accuracy on Boolean tasks). Here, we assess three emerging tools not previously evaluated on LaborBench: the Statutory Research Assistant (STARA), a custom statutory research tool, and two commercial tools by Westlaw and LexisNexis marketing AI statutory survey capabilities. We make five main contributions. First, we show that STARA achieves substantial performance gains, boosting accuracy to 83%. Second, we show that commercial platforms fare poorly, with accuracy of 58% (Westlaw AI) and 64% (Lexis+ AI), even worse than standard RAG. Third, we conduct a comprehensive error analysis, comparing our outputs to those compiled by DOL attorneys, and document both reasoning errors, such as confusion between related legal concepts and misinterpretation of statutory exceptions, and retrieval failures, where relevant statutory provisions are not captured. Fourth, we discover that many apparent errors are actually significant omissions by DOL attorneys themselves, such that STARA’s actual accuracy is 92%. Fifth, we chart the path forward for legal RAG through concrete design principles, offering actionable guidance for building AI systems capable of accurate multi-jurisdictional legal research.

[9] Developing an AI Assistant for Knowledge Management and Workforce Training in State DOTs cs.CL | cs.AI | cs.IRPDF

Divija Amaram, Lu Gao, Gowtham Reddy Gudla, Tejaswini Sanjay Katale

TL;DR: 本文提出了一种基于检索增强生成（RAG）的多智能体框架，旨在解决州交通部门（DOTs）中知识管理和员工培训的挑战。该系统通过整合结构化文档检索、视觉语言模型对技术图表进行语义编码，并利用多个专门智能体进行迭代式查询优化与质量评估，从而为工程师提供实时、基于上下文的精准信息支持。

Details

Motivation: 解决州交通机构因传统知识管理方法（如静态文档、课堂培训）导致的知识碎片化、信息检索效率低下以及资深工程师退休带来的专业知识流失问题，以支持现场问题解决和培训任务。

Result: 论文未在摘要中提及具体的定量实验结果或基准测试，但提出了一个集成了开放权重视觉语言模型和大型语言模型的系统框架，旨在提升信息检索的准确性和响应生成的质量。

Insight: 创新点在于将多智能体架构引入RAG框架，通过分工协作（检索、生成、评估、查询优化）实现迭代改进与质量控制；同时，利用视觉语言模型将技术图表转换为可检索的语义文本，实现了多模态知识的统一索引与利用，增强了系统对复杂技术文档的理解和响应能力。

Abstract: Effective knowledge management is critical for preserving institutional expertise and improving the efficiency of workforce training in state transportation agencies. Traditional approaches, such as static documentation, classroom-based instruction, and informal mentorship, often lead to fragmented knowledge transfer, inefficiencies, and the gradual loss of expertise as senior engineers retire. Moreover, given the enormous volume of technical manuals, guidelines, and research reports maintained by these agencies, it is increasingly challenging for engineers to locate relevant information quickly and accurately when solving field problems or preparing for training tasks. These limitations hinder timely decision-making and create steep learning curves for new personnel in maintenance and construction operations. To address these challenges, this paper proposes a Retrieval-Augmented Generation (RAG) framework with a multi-agent architecture to support knowledge management and decision making. The system integrates structured document retrieval with real-time, context-aware response generation powered by a large language model (LLM). Unlike conventional single-pass RAG systems, the proposed framework employs multiple specialized agents for retrieval, answer generation, evaluation, and query refinement, which enables iterative improvement and quality control. In addition, the system incorporates an open-weight vision-language model to convert technical figures into semantic textual representations, which allows figure-based knowledge to be indexed and retrieved alongside text. Retrieved text and figure-based context are then provided to an open-weight large language model, which generates the final responses grounded in the retrieved evidence.

[10] HumanLM: Simulating Users with State Alignment Beats Response Imitation cs.CL | cs.AIPDF

Shirley Wu, Evelyn Choi, Arpandeep Khatua, Zhanghan Wang, Joy He-Yueya

TL;DR: 本文提出了HumanLM，一种通过状态对齐而非简单响应模仿来模拟真实用户的训练框架。该方法的核心是让模型生成与真实用户潜在心理状态（如信念和情绪）对齐的自然语言隐状态，并基于此合成用户响应。

Details

Motivation: 现有的大语言模型用户模拟器大多只模仿表面语言模式，无法反映真实用户的底层状态（如信念和情绪），限制了用户中心应用的发展。

Result: 在包含6个大规模数据集、总计26k用户和216k响应的Humanual基准测试中，HumanLM显著优于其他方法，在LLM评判器的对齐分数上平均相对提升16.3%。在111名参与者的实时模拟研究中，HumanLM获得了与真实用户响应最高的相似度和有竞争力的人类相似度分数。

Insight: 创新点在于将用户模拟从响应模仿提升到状态对齐，通过强化学习使模型生成与真实用户心理状态维度对齐的隐状态，从而更准确地反映用户内在驱动因素。这为构建更真实、可解释的用户模拟器提供了新思路。

Abstract: Large Language Models (LLMs) are increasingly used to simulate how specific users respond to a given context, enabling more user-centric applications that rely on user feedback. However, existing user simulators mostly imitate surface-level patterns and language styles, which fail to reflect the underlying states of real users (e.g., beliefs and emotions). To address these limitations, we propose a novel training framework, HumanLM, which builds user simulators that accurately reflect real users. Our key insight is that, in addition to generating responses, the model should generate natural-language latent states that align with ground-truth responses through reinforcement learning. These latent states correspond to a set of psychologically grounded state dimensions that drive how real users respond. HumanLM further synthesizes these aligned latent states into responses that accurately represent real users. For extensive evaluation, we develop Humanual, a comprehensive benchmark for simulating real users based on public data. Humanual consists of six large-scale datasets with 26k users and 216k responses in total, spanning diverse tasks such as generating user responses to daily life issues, political blogs, and chat sessions with LLM assistants. Across datasets, HumanLM significantly outperforms alternative approaches, achieving an average relative improvement of 16.3% in alignment scores from an LLM judge. In a real-time simulation study with 111 participants, HumanLM achieves the highest similarity to real user responses and competitive human-likeness scores.

[11] Draft-Conditioned Constrained Decoding for Structured Generation in LLMs cs.CL | cs.AI | cs.LGPDF

Avinash Reddy, Thayne T. Walker, James S. Ide, Amrit Singh Bedi

TL;DR: 本文提出了一种名为Draft-Conditioned Constrained Decoding (DCCD) 的训练无关推理方法，用于提升大语言模型在生成结构化输出（如JSON、API调用）时的准确性和有效性。该方法将生成过程解耦为两个步骤：先生成一个无约束的草稿进行语义规划，再基于此草稿进行约束解码以保证语法正确性。

Details

Motivation: 解决现有约束解码方法在模型对有效延续分配低概率时，会扭曲生成过程，导致输出虽然语法正确但语义错误的问题，确保LLM生成的结构化输出既有效又准确。

Result: 在结构化推理基准测试（如GSM8K）上，DCCD相比标准约束解码方法，将严格结构化准确率最高提升了24个百分点（例如，在1B模型上从15.2%提升至39.0%），并且能让更小的模型达到或超过更大模型的性能，显著提升了参数效率。

Insight: 核心创新在于将语义规划（草稿生成）与结构强制执行（约束解码）解耦，通过先验草稿来增加有效概率质量并减少硬约束带来的累积“投影税”。这是一种无需训练、通过改进推理过程来提升性能的简单有效方法。

Abstract: Large language models (LLMs) are increasingly used to generate executable outputs, JSON objects, and API calls, where a single syntax error can make the output unusable. Constrained decoding enforces validity token-by-token via masking and renormalization, but it can distort generation when the model assigns low probability mass to valid continuations, pushing decoding toward locally valid yet semantically incorrect trajectories. We propose \emph{Draft-Conditioned Constrained Decoding (DCCD)}, a simple two-step, training-free inference procedure that decouples semantic planning from structural enforcement: an unconstrained draft is generated first, and constrained decoding is then applied, conditioned on this draft, to guarantee validity. We analyze DCCD through a KL-projection view, showing that draft conditioning increases feasible mass and reduces the cumulative “projection tax” induced by hard constraints, with an optional best-of-$K$ draft selection. Across structured reasoning benchmarks, DCCD improves strict structured accuracy by up to +24 percentage points over standard constrained decoding (e.g., 15.2% to 39.0% on GSM8K with a 1B model), and enables smaller model pairs to match or exceed much larger constrained baselines, yielding substantial gains in parameter efficiency.

[12] M-QUEST – Meme Question-Understanding Evaluation on Semantics and Toxicity cs.CL | cs.AI | cs.LGPDF

Stefano De Giorgis, Ting-Chih Chen, Filip Ilievski

TL;DR: 本文提出了M-QUEST基准，用于评估大型语言模型在理解网络表情包（meme）语义和毒性方面的常识推理能力。该工作首先建立了一个包含文本、视觉、场景、背景知识等十个维度的语义框架来解析表情包含义，并基于此框架半自动生成了一个包含609个问答对的基准数据集。作者评估了八个开源大语言模型，发现具有指令微调和推理能力的模型表现更优，但实用推理问题仍具挑战性。

Details

Motivation: 现有研究缺乏一个能够系统识别表情包含义构成要素的整体架构，特别是在毒性检测方面，表情包依赖常识知识，使得检测变得困难。本文旨在填补这一空白，为自动从表情包中提取知识提供一个语义框架和评估基准。

Result: 在自建的M-QUEST基准（包含307个表情包及其609个问答对）上评估了八个开源大语言模型。结果表明，当前模型对有毒表情包解释的常识推理能力因维度和模型架构而异，具有指令微调和推理能力的模型（如某些经过专门训练的模型）显著优于其他模型，但在涉及实用推理的问题上仍存在困难。

Insight: 创新点在于提出了一个系统性的、包含十个关键维度的语义框架来解构表情包的含义，并基于此创建了一个专注于毒性评估和原因推理的问答式基准。这为多模态内容安全和常识推理的交叉研究提供了新的评估工具和方向。从客观角度看，将复杂的多模态理解任务分解为可评估的、基于常识的问答对，是一种有效且可解释的评估方法。

Abstract: Internet memes are a powerful form of online communication, yet their nature and reliance on commonsense knowledge make toxicity detection challenging. Identifying key features for meme interpretation and understanding, is a crucial task. Previous work has been focused on some elements contributing to the meaning, such as the Textual dimension via OCR, the Visual dimension via object recognition, upper layers of meaning like the Emotional dimension, Toxicity detection via proxy variables, such as hate speech detection, and sentiment analysis. Nevertheless, there is still a lack of an overall architecture able to formally identify elements contributing to the meaning of a meme, and be used in the sense-making process. In this work, we present a semantic framework and a corresponding benchmark for automatic knowledge extraction from memes. First, we identify the necessary dimensions to understand and interpret a meme: Textual material, Visual material, Scene, Background Knowledge, Emotion, Semiotic Projection, Analogical Mapping, Overall Intent, Target Community, and Toxicity Assessment. Second, the framework guides a semi-automatic process of generating a benchmark with commonsense question-answer pairs about meme toxicity assessment and its underlying reason. The resulting benchmark M-QUEST consists of 609 question-answer pairs for 307 memes. Thirdly, we evaluate eight open-source large language models on their ability to correctly solve M-QUEST. Our results show that current models’ commonsense reasoning capabilities for toxic meme interpretation vary depending on the dimension and architecture. Models with instruction tuning and reasoning capabilities significantly outperform the others, though pragmatic inference questions remain challenging. We release code, benchmark, and prompts to support future research intersecting multimodal content safety and commonsense reasoning.

[13] The Influence of Iconicity in Transfer Learning for Sign Language Recognition cs.CL | cs.AI | cs.CVPDF

Keren Artiaga, Conor Lynch, Haithem Afli, Mohammed Hasanuzzaman

TL;DR: 该研究探讨了手语识别中图标性（iconicity）对迁移学习效果的影响，通过比较中文-阿拉伯语和希腊语-弗拉芒语两对手语的图标性符号迁移性能，发现从源语言到目标语言的图标性迁移能提升识别准确率。

Details

Motivation: 现有手语识别研究多依赖ImageNet等视觉数据集的迁移学习，但忽略了手语本身的图标性特征对跨语言知识迁移的影响，本文旨在探究图标性相似性是否对迁移学习效果具有必要性。

Result: 实验结果显示，从中文到阿拉伯语的图标性迁移使阿拉伯语手语识别准确率提升7.02%，从希腊语到弗拉芒语的迁移则提升1.07%，表明图标性迁移能带来性能增益。

Insight: 创新点在于首次系统评估手语图标性对迁移学习的影响，揭示了跨语言图标性相似性可作为提升手语识别迁移效果的关键因素，为手语识别研究提供了新的数据选择视角。

Abstract: Most sign language recognition research relies on Transfer Learning (TL) from vision-based datasets such as ImageNet. Some extend this to alternatively available language datasets, often focusing on signs with cross-linguistic similarities. This body of work examines the necessity of these likenesses on effective knowledge transfer by comparing TL performance between iconic signs of two different sign language pairs: Chinese to Arabic and Greek to Flemish. Google Mediapipe was utilised as an input feature extractor, enabling spatial information of these signs to be processed with a Multilayer Perceptron architecture and the temporal information with a Gated Recurrent Unit. Experimental results showed a 7.02% improvement for Arabic and 1.07% for Flemish when conducting iconic TL from Chinese and Greek respectively.

[14] From We to Me: Theory Informed Narrative Shift with Abductive Reasoning cs.CL | cs.AIPDF

Jaikrishna Manojkumar Patil, Divyagna Bavikadi, Kaustuv Mukherji, Ashby Steward-Nolan, Peggy-Jean Allin

TL;DR: 本文提出了一种基于社会科学理论和溯因推理的神经符号方法，用于实现叙事转换，即改变文本的叙事框架（如从集体主义转向个人主义）同时保留其核心信息。该方法通过自动提取规则来推导所需的故事元素，从而指导大语言模型进行一致且有针对性的叙事转换。实验表明，该方法在多个LLM上均能有效实现叙事转换，同时保持与原始故事的高语义相似度。

Details

Motivation: 当前的大语言模型在保持核心信息不变的前提下，将文本从一个叙事框架转换到另一个（例如从“我们”到“我”）的任务上存在显著困难。本文旨在解决这一叙事转换的挑战。

Result: 在多个LLM（GPT-4o, Llama-4, Grok-4, Deepseek-R1）上进行了评估。以GPT-4o为例，在从集体主义到个人主义的叙事转换任务上，该方法比零样本LLM基线性能提升了55.88%，同时在保持与原始故事的语义相似度上也有显著提升（KL散度改善了40.4%）。在相反方向的转换上也取得了相当的改进。

Insight: 主要创新点在于将社会科学理论（叙事框架）与溯因推理相结合，形成一种神经符号方法来自动推导转换规则，从而系统性地指导LLM完成复杂的叙事转换任务。这为LLM在需要理论指导的细粒度文本生成与编辑任务上提供了新思路。

Abstract: Effective communication often relies on aligning a message with an audience’s narrative and worldview. Narrative shift involves transforming text to reflect a different narrative framework while preserving its original core message–a task we demonstrate is significantly challenging for current Large Language Models (LLMs). To address this, we propose a neurosymbolic approach grounded in social science theory and abductive reasoning. Our method automatically extracts rules to abduce the specific story elements needed to guide an LLM through a consistent and targeted narrative transformation. Across multiple LLMs, abduction-guided transformed stories shifted the narrative while maintaining the fidelity with the original story. For example, with GPT-4o we outperform the zero-shot LLM baseline by 55.88% for collectivistic to individualistic narrative shift while maintaining superior semantic similarity with the original stories (40.4% improvement in KL divergence). For individualistic to collectivistic transformation, we achieve comparable improvements. We show similar performance across both directions for Llama-4, and Grok-4 and competitive performance for Deepseek-R1.

[15] IntPro: A Proxy Agent for Context-Aware Intent Understanding via Retrieval-conditioned Inference cs.CL | cs.AI | cs.LGPDF

Guanming Liu, Meng Wu, Peng Zhang, Yu Zhang, Yubo Shu

TL;DR: 本文提出IntPro，一种基于检索条件推理的代理智能体，用于上下文感知的意图理解。该方法通过设计意图解释来抽象上下文信号与表达意图之间的联系，并将其存储在个人意图历史库中供检索。通过监督微调和多轮组相对策略优化训练，使智能体学会何时利用历史意图模式、何时直接推理。

Details

Motivation: 现有方法通常将意图理解视为静态识别任务，忽略了用户积累的意图模式，这些模式可为更准确和可泛化的理解提供有价值的参考。上下文感知意图理解需要同时推理即时上下文和驱动行为的潜在动机，具有内在挑战性。

Result: 在三个不同场景（Highlight-Intent、MIntRec2.0和Weibo Post-Sync）上的实验表明，IntPro在不同场景和模型类型中均实现了强大的意图理解性能，并具备有效的上下文感知推理能力。

Insight: 创新点在于将意图理解从静态识别转变为基于检索条件推理的动态过程，通过构建个人意图历史库和设计工具感知奖励函数的多轮优化训练，使智能体能够自适应地利用历史模式进行上下文感知推理。

Abstract: Large language models (LLMs) have become integral to modern Human-AI collaboration workflows, where accurately understanding user intent serves as a crucial step for generating satisfactory responses. Context-aware intent understanding, which involves inferring user intentions from situational environments, is inherently challenging because it requires reasoning over both the immediate context and the user’s underlying motivations that drive their behavior. Moreover, existing approaches often treat intent understanding as a static recognition task, overlooking users’ accumulated intent patterns that could provide valuable references for more accurate and generalizable understanding. To address this gap, we propose IntPro, a proxy agent that learns to adapt to individual users via retrieval-conditioned intent inference. We design intent explanations that abstract how contextual signals connect to expressed intents, and store them in an individual intent history library for retrieval. We train IntPro through supervised fine-tuning on retrieval-conditioned trajectories and multi-turn Group Relative Policy Optimization (GRPO) with tool-aware reward functions, enabling the agent to learn when to leverage historical intent patterns and when to infer directly. Experiments across three diverse scenarios (Highlight-Intent, MIntRec2.0, and Weibo Post-Sync) demonstrate that IntPro achieves strong intent understanding performance with effective context-aware reasoning capabilities across different scenarios and model types.

[16] Certainty robustness: Evaluating LLM stability under self-challenging prompts cs.CL | cs.AIPDF

Mohammadreza Saadat, Steve Nemzer

TL;DR: 本文提出了一个名为Certainty Robustness Benchmark的两轮评估框架，用于评估大型语言模型在受到自我挑战提示（如不确定性询问和明确反驳）时的稳定性与适应性。该研究基于LiveBench的200个推理和数学问题，对四个最先进的LLM进行了测试，区分了合理的自我修正与不合理的答案变化。

Details

Motivation: 现有基准主要评估单轮准确性、真实性或置信度校准，但未能捕捉模型在交互环境中受到挑战时的行为。本文旨在填补这一空白，研究LLM在面临自我挑战提示时的可靠性。

Result: 研究揭示了模型在交互可靠性上的显著差异，这些差异无法仅通过基线准确性解释：一些模型在对话压力下放弃了正确答案，而另一些模型则表现出对挑战的强抵抗力和置信度与正确性之间更好的对齐。

Insight: 本文的创新点在于将确定性鲁棒性确立为一个独特且关键的LLM评估维度，强调了在交互设置中评估模型稳定性的重要性，这对模型对齐、可信度和实际部署具有重要影响。

Abstract: Large language models (LLMs) often present answers with high apparent confidence despite lacking an explicit mechanism for reasoning about certainty or truth. While existing benchmarks primarily evaluate single-turn accuracy, truthfulness or confidence calibration, they do not capture how models behave when their responses are challenged in interactive settings. We introduce the Certainty Robustness Benchmark, a two-turn evaluation framework that measures how LLMs balance stability and adaptability under self-challenging prompts such as uncertainty (“Are you sure?”) and explicit contradiction (“You are wrong!”), alongside numeric confidence elicitation. Using 200 reasoning and mathematics questions from LiveBench, we evaluate four state-of-the-art LLMs and distinguish between justified self-corrections and unjustified answer changes. Our results reveal substantial differences in interactive reliability that are not explained by baseline accuracy alone: some models abandon correct answers under conversational pressure, while others demonstrate strong resistance to challenge and better alignment between confidence and correctness. These findings identify certainty robustness as a distinct and critical dimension of LLM evaluation, with important implications for alignment, trustworthiness and real-world deployment.

[17] PulseLM: A Foundation Dataset and Benchmark for PPG-Text Learning cs.CL | cs.AIPDF

Hung Manh Pham, Jinyang Wu, Xiao Ma, Yiming Zhang, Yixin Xu

TL;DR: 本文介绍了PulseLM，一个大规模的光电容积脉搏波（PPG）-文本数据集，旨在通过统一的封闭式问答（QA）框架连接原始PPG波形与自然语言。该数据集整合了来自15个公开来源的PPG记录，将异构注释统一为12个常见的生理学QA任务，包含131万个标准化的10秒PPG片段和315万个问答对。

Details

Motivation: 现有PPG数据集通常以数值测量或任务特定标签的形式提供监督，限制了其在基于语言的生理推理和多模态基础模型中的适用性，因此需要构建一个能够桥接原始PPG波形与自然语言的大规模数据集。

Result: 论文建立了使用多模态PPG感知大语言模型的基线基准，为研究多模态生理推理、跨数据集泛化和基于PPG的语言模型的可扩展基准测试提供了标准化基础。

Insight: 创新点在于通过统一的封闭式问答形式将异构PPG数据标准化，创建了首个大规模PPG-文本配对数据集，为PPG与语言模型的结合提供了新的基准和评估协议。

Abstract: Photoplethysmography (PPG) is a widely used non-invasive sensing modality for continuous cardiovascular and physiological monitoring across clinical, laboratory, and wearable settings. While existing PPG datasets support a broad range of downstream tasks, they typically provide supervision in the form of numerical measurements or task-specific labels, limiting their suitability for language-based physiological reasoning and multimodal foundation models. In this work, we introduce PulseLM, a large-scale PPG-text dataset designed to bridge raw PPG waveforms and natural language through a unified, closed-ended question answering (QA) formulation. PulseLM aggregates PPG recordings from fifteen publicly available sources and harmonizes heterogeneous annotations into twelve common physiologically QA tasks. The dataset comprises 1.31 million standardized 10-second PPG segments, associated with 3.15 million question-answer pairs. We further define reproducible preprocessing, supervision, and evaluation protocols and establish baseline benchmarks using multimodal PPG-aware large language models. PulseLM provides a standardized foundation for studying multimodal physiological reasoning, cross-dataset generalization, and scalable benchmarking of PPG-based language models. The data and code can be found publicly available at: https://github.com/manhph2211/PulseLM.

[18] Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations cs.CL | cs.AI | cs.LGPDF

Ashwath Vaithinathan Aravindan, Mayank Kejriwal

TL;DR: 本文通过系统评估13个不同规模（3B至1.5T参数）的大语言模型对五种链式思维扰动类型的鲁棒性，揭示了模型在数学推理任务中面对中间步骤干扰时的脆弱性模式与规模缩放效应。

Details

Motivation: 链式思维提示已成为激发大语言模型推理能力的基础技术，但其对中间推理步骤扰动的鲁棒性尚未得到充分理解，本文旨在填补这一空白。

Result: 在数学推理任务上，不同扰动类型导致不同程度的准确率下降：MathError对小模型影响最严重（损失50-60%），但随规模扩大有明显改善；UnitConversion对所有规模模型都具挑战性（即使最大模型也有20-30%损失）；ExtraSteps影响最小（0-6%损失）；Sycophancy影响中等（小模型损失7%）；SkippedSteps造成中等损害（15%损失）。模型规模对部分扰动具有防护作用，但对维度推理任务防护有限。

Insight: 创新点在于首次系统构建了链式思维扰动的分类学（五类扰动），并通过大规模实验揭示了模型鲁棒性与规模之间的异质性关系（符合幂律模式），强调特定任务鲁棒性评估的必要性，为多阶段推理管道部署提供了实证依据。

Abstract: Chain-of-Thought (CoT) prompting has emerged as a foundational technique for eliciting reasoning from Large Language Models (LLMs), yet the robustness of this approach to corruptions in intermediate reasoning steps remains poorly understood. This paper presents a comprehensive empirical evaluation of LLM robustness to a structured taxonomy of 5 CoT perturbation types: \textit{MathError, UnitConversion, Sycophancy, SkippedSteps,} and \textit{ExtraSteps}. We evaluate 13 models spanning three orders of magnitude in parameter count (3B to 1.5T\footnote{Assumed parameter count of closed models}), testing their ability to complete mathematical reasoning tasks despite perturbations injected at different points in the reasoning chain. Our key findings reveal heterogeneous vulnerability patterns: MathError perturbations produce the most severe degradation in small models (50-60% accuracy loss) but show strong scaling benefits; UnitConversion remains challenging across all scales (20-30% loss even for largest models); ExtraSteps incur minimal accuracy degradation (0-6%) regardless of scale; Sycophancy produces modest effects (7% loss for small models); and SkippedSteps cause intermediate damage (15% loss). Scaling relationships follow power-law patterns, with model size serving as a protective factor against some perturbations but offering limited defense against dimensional reasoning tasks. These findings have direct implications for deploying LLMs in multi-stage reasoning pipelines and underscore the necessity of task-specific robustness assessments and mitigation strategies. The code and results are available \href{https://github.com/Mystic-Slice/CoTPerturbation}{here}.

[19] The CompMath-MCQ Dataset: Are LLMs Ready for Higher-Level Math? cs.CLPDF

Bianca Raimondi, Francesco Pivi, Davide Evangelista, Maurizio Gabbrielli

TL;DR: 该论文介绍了CompMath-MCQ数据集，这是一个用于评估大语言模型在研究生水平和计算数学领域高级推理能力的多项选择题基准。数据集包含1500道由研究生课程教授原创的题目，涵盖线性代数、数值优化、向量微积分、概率和基于Python的科学计算等主题。

Details

Motivation: 现有的大语言模型数学推理评估主要集中于初等问题、竞赛风格题目或形式化定理证明，而研究生水平和计算数学领域相对未被充分探索，因此需要一个新的基准来填补这一空白。

Result: 在CompMath-MCQ数据集上对最先进的大语言模型进行的基线测试结果表明，高级计算数学推理仍然是一个重大挑战。

Insight: 论文的创新点在于创建了一个原创的、无数据泄露风险的研究生级计算数学多项选择题基准，并通过跨LLM分歧和专家评审确保题目有效性，为客观、可重复的评估提供了新工具。

Abstract: The evaluation of Large Language Models (LLMs) on mathematical reasoning has largely focused on elementary problems, competition-style questions, or formal theorem proving, leaving graduate-level and computational mathematics relatively underexplored. We introduce CompMath-MCQ, a new benchmark dataset for assessing LLMs on advanced mathematical reasoning in a multiple-choice setting. The dataset consists of 1{,}500 originally authored questions by professors of graduate-level courses, covering topics including Linear Algebra, Numerical Optimization, Vector Calculus, Probability, and Python-based scientific computing. Three option choices are provided for each question, with exactly one of them being correct. To ensure the absence of data leakage, all questions are newly created and not sourced from existing materials. The validity of questions is verified through a procedure based on cross-LLM disagreement, followed by manual expert review. By adopting a multiple-choice format, our dataset enables objective, reproducible, and bias-free evaluation through lm_eval library. Baseline results with state-of-the-art LLMs indicate that advanced computational mathematical reasoning remains a significant challenge. We release CompMath-MCQ at the following link: https://github.com/biancaraimondi/CompMath-MCQ.git

[20] Compressed Sensing for Capability Localization in Large Language Models cs.CLPDF

Anna Bair, Yixuan Even Xu, Mingjie Sun, J. Zico Kolter

TL;DR: 该论文发现大型语言模型（LLMs）的许多特定能力（如数学推理、代码生成）高度集中在Transformer架构中少数特定的注意力头上。通过一种基于压缩感知的方法，论文能够识别出这些稀疏且功能特定的组件，并验证了这种能力局部化是Transformer语言模型的一个普遍组织原则。

Details

Motivation: 动机在于探索大型语言模型中特定能力（如数学推理）的实现机制，并验证这些能力是否由模型中稀疏、局部化的组件（如特定的注意力头）所负责，这有助于模型的可解释性、编辑和AI安全。

Result: 在Llama和Qwen模型（1B到8B参数）以及包括数学能力和代码生成在内的多种能力上进行验证。结果表明，仅归零（zero out）五个任务特定的注意力头，就能使相关基准测试的性能下降高达65%，同时基本不影响无关任务。

Insight: 创新点在于利用压缩感知思想，通过策略性“敲除”（knockouts）和少量模型评估来识别负责特定能力的稀疏注意力头。这揭示了Transformer语言模型具有模块化组织，特定能力由稀疏且功能独立的组件实现，为模型解释和编辑提供了新途径。

Abstract: Large language models (LLMs) exhibit a wide range of capabilities, including mathematical reasoning, code generation, and linguistic behaviors. We show that many capabilities are highly localized to small subsets of attention heads within Transformer architectures. Zeroing out as few as five task-specific heads can degrade performance by up to $65%$ on standard benchmarks measuring the capability of interest, while largely preserving performance on unrelated tasks. We introduce a compressed sensing based method that exploits the sparsity of these heads to identify them via strategic knockouts and a small number of model evaluations. We validate these findings across Llama and Qwen models ranging from 1B to 8B parameters and a diverse set of capabilities including mathematical abilities and code generation, revealing a modular organization in which specialized capabilities are implemented by sparse, functionally distinct components. Overall, our results suggest that capability localization is a general organizational principle of Transformer language models, with implications for interpretability, model editing, and AI safety. Code is released at https://github.com/locuslab/llm-components.

[21] Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs cs.CL | cs.AIPDF

Mingyu Jin, Yutong Yin, Jingcheng Niu, Qingcheng Zeng, Wujiang Xu

TL;DR: 本文研究了大型语言模型在处理不同难度（即分布外偏移程度）输入时，其内部表征的变化规律。研究发现了一个普遍且可量化的现象：随着任务难度增加（如推理问题变难、上下文变长或答案选项增多），LLMs最后一层隐藏状态会变得显著稀疏，即’偏移越远，表征越稀疏’。这种稀疏性与难度的关系在不同模型和领域中都存在，表明语言模型通过将计算集中在特定子空间来应对陌生或复杂输入。基于此，作者设计了一种利用表征稀疏性来安排少样本示例的策略，即稀疏性引导的课程上下文学习，从而显著提升了模型性能。

Details

Motivation: 动机是探究大型语言模型在面对分布外或难度递增的输入时，其内部表征如何自适应变化，以理解模型处理复杂或陌生任务的机制。

Result: 论文通过一系列受控分析和学习动态解释，证明了稀疏性是一种稳定分布外推理的自适应机制，而非偶然现象。在此基础上提出的SG-ICL策略，在少样本上下文学习场景中带来了显著的性能提升。

Insight: 论文宣称的创新点在于揭示了LLMs中表征稀疏性与任务难度（OOD偏移程度）之间的普遍量化关系，并利用这一机制设计了SG-ICL策略。从客观角度看，该研究为理解LLMs的内部工作机制提供了新的机制性见解，并将表征稀疏性这一属性转化为可操作的性能提升方法，具有借鉴意义。

Abstract: In this work, we investigate how Large Language Models (LLMs) adapt their internal representations when encountering inputs of increasing difficulty, quantified as the degree of out-of-distribution (OOD) shift. We reveal a consistent and quantifiable phenomenon: as task difficulty increases, whether through harder reasoning questions, longer contexts, or adding answer choices, the last hidden states of LLMs become substantially sparser. In short, \textbf{\textit{the farther the shift, the sparser the representations}}. This sparsity–difficulty relation is observable across diverse models and domains, suggesting that language models respond to unfamiliar or complex inputs by concentrating computation into specialized subspaces in the last hidden state. Through a series of controlled analyses with a learning dynamic explanation, we demonstrate that this sparsity is not incidental but an adaptive mechanism for stabilizing reasoning under OOD. Leveraging this insight, we design \textit{Sparsity-Guided Curriculum In-Context Learning (SG-ICL)}, a strategy that explicitly uses representation sparsity to schedule few-shot demonstrations, leading to considerable performance enhancements. Our study provides new mechanistic insights into how LLMs internalize OOD challenges. The source code is available at the URL: https://github.com/MingyuJ666/sparsityLLM.

[22] Tucano 2 Cool: Better Open Source LLMs for Portuguese cs.CL | cs.AIPDF

Nicholas Kluge Corrêa, Aniket Sen, Shiza Fatimah, Sophia Falk, Lennard Landgraf

TL;DR: 本文介绍了Tucano 2，一个完全开源的、参数规模在0.5-37亿之间的大型语言模型套件，旨在弥补葡萄牙语开源LLM的不足。通过扩展和提升数据集GigaVerbo-v2的质量与规模，并引入新的合成数据集GigaVerbo-v2 Synth以及两个后训练数据集，该套件支持在检索增强生成、编码、工具使用、思维链推理等多个领域进行训练。研究设计了预训练和持续预训练方案，在多个葡萄牙语基准测试中达到了最先进的性能，并提供了全面的评估套件和所有相关资源。

Details

Motivation: 解决葡萄牙语开源大型语言模型发展中的不足和空白，为葡萄牙语NLP社区提供高质量、可复现且可扩展的模型资源。

Result: 在多个葡萄牙语建模基准测试中达到了最先进的性能。

Insight: 通过构建大规模、高质量且多样化的数据集（包括合成数据），并设计专门的预训练、持续预训练及后训练方案，有效提升了葡萄牙语LLM的能力；同时，提供完整的训练方案、评估套件和开源资源，促进了研究的可复现性和社区发展。

Abstract: We present Tucano 2, a fully open suite of large language models (LLMs) with 0.5-3.7 billion parameters, designed to address certain gaps in open-source development for Portuguese LLMs. Following our previous works, we now extend our dataset, GigaVerbo-v2, to a new degree of quality and scale, while also introducing a new synthetic dataset, GigaVerbo-v2 Synth, aimed at filling missing gaps in GigaVerbo-v2, and two post-training datasets, GigaVerbo-v2 SFT and GigaVerbo-v2 Preferences, that allow Portuguese LLMs to be trained in domains like retrieval augmented generation, coding, tool use, chain-of-thought reasoning, and many other domains of interest. Through extensive ablation studies, we design both pretraining and continual pretraining recipes for the Tucano 2 suite (Base, Instruct, and Think), which achieve state-of-the-art performance on several Portuguese-language modeling benchmarks. We also extend and refine the evaluation harness introduced in our earlier work, yielding a comprehensive evaluation suite that provides strong signals across different pretraining, continual pretraining, and post-training regimes. All artifacts associated with Tucano 2 are openly released, including training recipes, logs, and source code, ensuring that our work is reproducible, accessible, and extendable by the broader Portuguese NLP community.

[23] ByteFlow: Language Modeling through Adaptive Byte Compression without a Tokenizer cs.CL | cs.LGPDF

Chunyuan Deng, Sanket Lokegaonkar, Colin Lockard, Besnik Fetahu, Nasser Zalmout

TL;DR: 本文提出了一种名为ByteFlow Net的新型分层架构，该架构完全摒弃了传统的预定义子词分词器，通过自适应字节压缩实现语言建模。该方法基于潜在表示的编码率进行压缩驱动的分割，利用Top-K选择保持静态计算图，从而让模型能够从原始字节流中学习语义上有意义的单元分割。实验表明，该方法在性能上优于基于BPE的Transformer和先前的字节级架构。

Details

Motivation: 现代语言模型仍依赖固定、预定义的子词分词器，一旦分词器训练完成，模型只能在该固定粒度上操作，这导致即使是在强大的推理模型中也会出现脆弱和反直觉的行为。因此，本文旨在解决分词器带来的限制，探索更自适应、基于信息基础的语言建模方法。

Result: 实验证明，基于压缩的分块策略带来了显著的性能提升，ByteFlow Net在多个基准测试中优于基于BPE的Transformer和先前的字节级架构，实现了SOTA水平。

Insight: 创新点在于完全移除分词器，通过自适应字节压缩实现端到端的语言建模，利用编码率驱动的分割和Top-K选择机制，使模型能够根据输入自适应调整内部表示粒度，从而更有效地捕捉语义信息。从客观角度看，这种方法为构建更自适应、信息基础的语言模型开辟了新路径，具有重要的借鉴意义。

Abstract: Modern language models still rely on fixed, pre-defined subword tokenizations. Once a tokenizer is trained, the LM can only operate at this fixed level of granularity, which often leads to brittle and counterintuitive behaviors even in otherwise strong reasoning models. We introduce \textbf{ByteFlow Net}, a new hierarchical architecture that removes tokenizers entirely and instead enables models to learn their own segmentation of raw byte streams into semantically meaningful units. ByteFlow Net performs compression-driven segmentation based on the coding rate of latent representations, yielding adaptive boundaries \emph{while preserving a static computation graph via Top-$K$ selection}. Unlike prior self-tokenizing methods that depend on brittle heuristics with human-designed inductive biases, ByteFlow Net adapts its internal representation granularity to the input itself. Experiments demonstrate that this compression-based chunking strategy yields substantial performance gains, with ByteFlow Net outperforming both BPE-based Transformers and previous byte-level architectures. These results suggest that end-to-end, tokenizer-free modeling is not only feasible but also more effective, opening a path toward more adaptive and information-grounded language models.

[24] MIND: Unified Inquiry and Diagnosis RL with Criteria Grounded Clinical Supports for Psychiatric Consultation cs.CL | cs.AIPDF

Guoyi Li, Shihao Xu, Jiatong Ma, Yunyun Han, Jianhua Chen

TL;DR: 本文提出MIND框架，一种用于精神科咨询的统一询问-诊断强化学习方法。该方法通过构建标准驱动的精神科推理库（PRB）来提供临床支持，并利用基于量规的过程奖励和轨迹修正机制，以优化多轮对话中的信息获取和诊断决策。

Details

Motivation: 现有方法在精神科咨询中面临两个挑战：缺乏标准驱动的临床支持，导致临床断言缺乏依据；在多轮交互中难以避免询问偏离主题或效率低下，无法优化提问策略。

Result: 在广泛实验中，MIND在诊断准确性、共情交互质量、可解释性和泛化性方面持续优于强基线方法。

Insight: 创新点包括构建标准驱动的精神科推理库（PRB）来提供可重用的临床支持，以及引入基于量规的过程奖励和值感知轨迹修正机制，以细粒度监督中间决策步骤并联合优化信息获取与诊断决策。

Abstract: Large language models (LLMs) have advanced medical dialogue systems, yet psychiatric consultation poses substantially higher demands due to subjective ambiguity and comorbidity complexity: an agent must continuously extract psychopathological cues from incomplete and inconsistent patient reports in multi-turn interactions and perform rigorous differential diagnostic reasoning. However, existing methods face two fundamental challenges. First, without criteria-grounded clinical supports, they are prone to unsupported clinical assertions when symptoms are atypical or underspecified. Second, in multi-turn interactions, they struggle to mitigate inquiry drift (off-topic or low-yield questioning) and optimize questioning strategies. To address these challenges, we propose MIND, a unified inquiry–diagnosis reinforcement learning framework for psychiatric consultation. Specifically, we build a Criteria-Grounded Psychiatric Reasoning Bank (PRB) that summarizes dialogue context into clinical retrieval states, retrieves semantically similar reference consultations, and distills reusable criteria-grounded clinical supports to guide criteria-aligned inquiry and reasoning. Building on this foundation, MIND enforces explicit clinical reasoning with rubric-based process rewards to provide fine-grained supervision over intermediate decision steps, and incorporates a value-aware trajectory rectification mechanism to jointly improve information acquisition and diagnostic decision-making across turns. Extensive experiments demonstrate that MIND consistently outperforms strong baselines in diagnostic accuracy, empathetic interaction quality, interpretability, and generalization.

[25] Confidence-Calibrated Small-Large Language Model Collaboration for Cost-Efficient Reasoning cs.CL | cs.AIPDF

Chuang Zhang, Zizhen Zhu, Yihao Wei, Bing Tian, Junyi Liu

TL;DR: 本文提出了COREA系统，通过级联小型语言模型（SLM）和大型语言模型（LLM）来平衡复杂推理任务中的准确性和成本。系统首先使用SLM尝试回答问题，并输出答案和置信度分数；若置信度低于预设阈值，则将问题转交给LLM处理。通过基于强化学习的训练算法校准SLM的置信度，实验表明该方法在多个数据集和模型上同时提升了SLM的推理能力和置信度校准。

Details

Motivation: 解决LLM推理能力强但成本高，而SLM成本低但能力不足的问题，旨在通过两者的协作实现成本效益高的推理。

Result: 在域外数学和非数学数据集上，相比单独使用LLM，COREA分别降低了21.5%和16.8%的成本，且绝对pass@1下降在2%以内，在多个数据集和模型骨架上验证了其有效性。

Insight: 创新点包括级联SLM和LLM的协作框架，以及基于强化学习的置信度校准奖励机制，以提升SLM的置信度校准和整体系统的成本效率。

Abstract: Large language models (LLMs) demonstrate superior reasoning capabilities compared to small language models (SLMs), but incur substantially higher costs. We propose COllaborative REAsoner (COREA), a system that cascades an SLM with an LLM to achieve a balance between accuracy and cost in complex reasoning tasks. COREA first attempts to answer questions using the SLM, which outputs both an answer and a verbalized confidence score. Questions with confidence below a predefined threshold are deferred to the LLM for more accurate resolution. We introduce a reinforcement learning-based training algorithm that aligns the SLM’s confidence through an additional confidence calibration reward. Extensive experiments demonstrate that our method jointly improves the SLM’s reasoning ability and confidence calibration across diverse datasets and model backbones. Compared to using the LLM alone, COREA reduces cost by 21.5% and 16.8% on out-of-domain math and non-math datasets, respectively, with only an absolute pass@1 drop within 2%.

[26] T2S-Bench & Structure-of-Thought: Benchmarking and Prompting Comprehensive Text-to-Structure Reasoning cs.CL | cs.AIPDF

Qinsi Wang, Hancheng Ye, Jinhee Kim, Jinghan Ke, Yifei Wang

TL;DR: 本文提出了Structure of Thought (SoT)提示技术和T2S-Bench基准。SoT通过引导大语言模型构建中间文本结构来提升文本处理性能。T2S-Bench是首个评估模型文本到结构能力的基准，包含跨6个科学领域的1.8K个样本和32种结构类型。实验表明，现有模型在复杂推理和结构提取任务上仍有很大提升空间，而SoT提示和基于T2S-Bench的微调能显著提升模型性能。

Details

Motivation: 受人类处理复杂阅读任务时标记关键点、推断关系并结构化信息以指导理解的启发，探索大语言模型是否能从显式的文本结构中受益，以提升其文本处理性能。

Result: 在45个主流模型上的评估显示，多跳推理任务的平均准确率仅为52.1%，最先进的模型在端到端提取任务中的节点准确率也仅为58.1%。在Qwen2.5-7B-Instruct模型上，SoT提示在八项文本处理任务上平均带来+5.7%的提升，而在T2S-Bench上微调后，提升进一步增加到+8.6%。

Insight: 论文的创新点在于提出了SoT提示技术，通过显式引导模型构建中间文本结构来提升性能，并构建了首个综合性文本到结构推理基准T2S-Bench。客观来看，将结构化思维过程显式化作为中间步骤，以及创建专门评估结构推理能力的基准，是提升模型复杂文本理解能力的有效且可借鉴的路径。

Abstract: Think about how human handles complex reading tasks: marking key points, inferring their relationships, and structuring information to guide understanding and responses. Likewise, can a large language model benefit from text structure to enhance text-processing performance? To explore it, in this work, we first introduce Structure of Thought (SoT), a prompting technique that explicitly guides models to construct intermediate text structures, consistently boosting performance across eight tasks and three model families. Building upon this insight, we present T2S-Bench, the first benchmark designed to evaluate and improve text-to-structure capabilities of models. T2S-Bench includes 1.8K samples across 6 scientific domains and 32 structural types, rigorously constructed to ensure accuracy, fairness, and quality. Evaluation on 45 mainstream models reveals substantial improvement potential: the average accuracy on the multi-hop reasoning task is only 52.1%, and even the most advanced model achieves 58.1% node accuracy in end-to-end extraction. Furthermore, on Qwen2.5-7B-Instruct, SoT alone yields an average +5.7% improvement across eight diverse text-processing tasks, and fine-tuning on T2S-Bench further increases this gain to +8.6%. These results highlight the value of explicit text structuring and the complementary contributions of SoT and T2S-Bench. Dataset and eval code have been released at https://t2s-bench.github.io/T2S-Bench-Page/.

[27] Monitoring Emergent Reward Hacking During Generation via Internal Activations cs.CL | cs.AIPDF

Patrick Wilhelm, Thorsten Wittkopp, Odej Kao

TL;DR: 该论文提出了一种基于内部激活的监控方法，用于在大型语言模型生成过程中检测由涌现错位导致的奖励黑客行为。该方法通过在残差流激活上训练稀疏自编码器，并应用轻量级线性分类器来产生令牌级别的奖励黑客活动估计。研究发现，内部激活模式能可靠地区分奖励黑客与良性行为，并可推广到未见过的混合策略适配器，且在思维链推理中表现出模型依赖的时间结构。

Details

Motivation: 微调后的大型语言模型可能出现由涌现错位导致的奖励黑客行为，仅从最终输出难以检测。先前研究主要关注完成响应层面的奖励黑客，而该论文旨在探索是否能在生成过程中识别此类行为。

Result: 在多个模型系列和微调混合实验中，内部激活模式能可靠地区分奖励黑客与良性行为，并泛化到未见过的混合策略适配器。奖励黑客信号通常出现较早，在整个推理过程中持续存在，且在弱指定奖励目标下通过思维链提示增加测试时计算会被放大。

Insight: 创新点在于提出了一种基于内部激活的监控方法，能够在生成过程中早期检测奖励黑客，这比基于输出的评估提供了更早的涌现错位信号。该方法使用稀疏自编码器和轻量级线性分类器，实现了高效的令牌级别监控，为微调语言模型的部署后安全监控提供了更鲁棒的补充手段。

Abstract: Fine-tuned large language models can exhibit reward-hacking behavior arising from emergent misalignment, which is difficult to detect from final outputs alone. While prior work has studied reward hacking at the level of completed responses, it remains unclear whether such behavior can be identified during generation. We propose an activation-based monitoring approach that detects reward-hacking signals from internal representations as a model generates its response. Our method trains sparse autoencoders on residual stream activations and applies lightweight linear classifiers to produce token-level estimates of reward-hacking activity. Across multiple model families and fine-tuning mixtures, we find that internal activation patterns reliably distinguish reward-hacking from benign behavior, generalize to unseen mixed-policy adapters, and exhibit model-dependent temporal structure during chain-of-thought reasoning. Notably, reward-hacking signals often emerge early, persist throughout reasoning, and can be amplified by increased test-time compute in the form of chain-of-thought prompting under weakly specified reward objectives. These results suggest that internal activation monitoring provides a complementary and earlier signal of emergent misalignment than output-based evaluation, supporting more robust post-deployment safety monitoring for fine-tuned language models.

Tom Kouwenhoven, Michiel van der Meer, Max van Duijn

TL;DR: 本文通过系统测试17个开源大语言模型在192个平衡版错误信念测试变体上的表现，采用贝叶斯逻辑回归分析模型规模和训练后处理对社会认知能力的影响，发现模型规模扩大虽有益但非绝对，且命题态度（如‘X认为’）的显式表达会根本改变响应模式；指令微调可部分缓解此效应，而推理导向的微调则会放大它；通过分析OLMo 2训练过程，揭示交叉效应在预训练阶段出现，表明模型习得了与心理状态词汇相关的刻板响应模式；最后，向量操控技术分离出‘思考’向量作为观察到的FBT行为的因果驱动因素。

Details

Motivation: 针对大语言模型在错误信念测试中评估社会认知能力时存在的可靠性问题，如数据污染、模型细节不足和控制不一致，本文旨在通过更严谨的实验设计来探究模型规模和后训练如何影响社会认知能力。

Result: 在Trott等人（2023）的192个FBT变体基准上，模型规模扩大通常提升性能，但存在交叉效应：显式命题态度（如‘X认为’）会改变响应模式；指令微调部分缓解此效应，推理导向微调则放大它；OLMo 2训练分析显示交叉效应在预训练阶段出现；向量操控实验分离出‘思考’向量作为FBT行为的因果驱动。

Insight: 创新点包括：使用贝叶斯逻辑回归量化模型规模和后训练对社会认知的影响，揭示命题态度表达的交叉效应及其与训练过程的关联，并通过向量操控技术因果性地识别心理状态词汇（如‘思考’）在模型行为中的关键作用，为理解LLM社会推理机制提供了新视角。

Abstract: The False Belief Test (FBT) has been the main method for assessing Theory of Mind (ToM) and related socio-cognitive competencies. For Large Language Models (LLMs), the reliability and explanatory potential of this test have remained limited due to issues like data contamination, insufficient model details, and inconsistent controls. We address these issues by testing 17 open-weight models on a balanced set of 192 FBT variants (Trott et al. 2023) using Bayesian Logistic regression to identify how model size and post-training affect socio-cognitive competence. We find that scaling model size benefits performance, but not strictly. A cross-over effect reveals that explicating propositional attitudes (X thinks) fundamentally alters response patterns. Instruction tuning partially mitigates this effect, but further reasoning-oriented finetuning amplifies it. In a case study analysing social reasoning ability throughout OLMo 2 training, we show that this cross-over effect emerges during pre-training, suggesting that models acquire stereotypical response patterns tied to mental-state vocabulary that can outweigh other scenario semantics. Finally, vector steering allows us to isolate a think vector as the causal driver of observed FBT behaviour.

[29] Bielik-Q2-Sharp: A Comparative Study of Extreme 2-bit Quantization Methods for a Polish 11B Language Model cs.CL | cs.AIPDF

Jakub Prejzner

TL;DR: 本文介绍了Bielik-Q2-Sharp，这是首个对波兰语大语言模型进行极端2位量化方法的系统性学术评估。研究以Bielik-11B-v2.3-Instruct为基础模型，在波兰语语料库上比较了六种先进的训练后量化方法，并公开了所有模型和评估结果。

Details

Motivation: 动机是填补对波兰语大语言模型进行极端2位量化（即每参数约2位）的系统性评估空白，旨在探索在保持模型性能的同时，显著减少模型存储空间的方法。

Result: 在22个波兰语基准测试上，最佳变体（QuIP# E8P12）达到71.92%的准确率，接近IQ2_XXS基线的72.07%，但模型大小略有增加（3.26 GB vs. ~2.6 GB）。在eq_bench推理基准上，该方法得分为47.14，优于基线的43.53（提升3.6个百分点）。QTIP方法在每比特效率上表现最佳，在约2.4位每权重（bpw）和3.27 GB大小下达到79.4%的MC acc_norm，与VPTQ质量相当但体积小35%。

Insight: 创新点包括首次对波兰语LLM进行系统的2位量化比较，揭示了基于旋转的量化方法在保持对数似然质量的同时，在自回归生成任务上可能出现灾难性失败的现象（MC-生成解离现象），并证明了在极低预算（285美元）下由独立研究者完成此类研究的可行性。QTIP在压缩效率上表现突出，为资源受限场景提供了参考。

Abstract: We present Bielik-Q2-Sharp, the first systematic academic evaluation of extreme 2-bit quantization applied to a Polish large language model. Using Bielik-11B-v2.3-Instruct (11B parameters, Mistral architecture) as our base model, we compare six state-of-the-art post-training quantization methods – QuIP#, SpinQuant+GPTQ, ButterflyQuant, QTIP, VPTQ, and AQLM – all calibrated on a Polish-language corpus (CulturaX-PL) with shared Hessian matrices. Our best variant (QuIP# E8P12) achieves 71.92% across 22 Polish benchmarks versus 72.07% for the IQ2_XXS baseline – within statistical noise, at a modest size premium (3.26 GB vs. ~2.6 GB). On eq_bench, our method scores 47.14 versus 43.53 (+3.6pp), suggesting superior preservation of higher-order reasoning. QTIP achieves the best per-bit efficiency (79.4% MC acc_norm at ~2.4 bpw, 3.27 GB), matching VPTQ’s quality at 35% smaller size. We additionally document a MC-generation dissociation phenomenon where rotation-based methods preserve log-likelihood quality but fail catastrophically at autoregressive generation. The entire project was conducted by a single independent researcher on cloud GPUs (vast.ai) within a $285 budget. All models, Hessians, and evaluation logs are publicly available.

[30] Retrieval or Representation? Reassessing Benchmark Gaps in Multilingual and Visually Rich RAG cs.CLPDF

Martin Asenov, Kenza Benkirane, Dan Goldwater, Aneiss Ghodsi

TL;DR: 本文通过系统实验证明，在多语言和视觉丰富文档的检索增强生成（RAG）基准测试中，性能提升主要源于更好的文档表示（如转录和预处理），而非检索机制本身。研究发现，即使使用传统的BM25检索方法，通过改进文档表示也能大幅缩小与先进端到端多模态检索器的性能差距。

Details

Motivation: 旨在澄清在多语言和视觉丰富文档的RAG基准测试中，观察到的性能提升究竟主要归因于检索机制的进步，还是文档表示（如转录、预处理）的改进，以正确评估技术进展并引导研究方向。

Result: 在保持检索机制（BM25）固定的情况下，通过改进文档的转录和预处理方法，能够在多语言和视觉丰富的基准测试（如相关RAG基准）上大幅缩小与先进端到端多模态检索器的性能差距，甚至达到相当水平。

Insight: 论文的核心创新点在于揭示了文档表示质量对RAG基准测试结果的巨大影响，并倡导将基准测试解耦为分别评估文档表示（转录）能力和检索能力的部分，这有助于更准确地归因性能提升并聚焦有效的研究方向。

Abstract: Retrieval-augmented generation (RAG) is a common way to ground language models in external documents and up-to-date information. Classical retrieval systems relied on lexical methods such as BM25, which rank documents by term overlap with corpus-level weighting. End-to-end multimodal retrievers trained on large query-document datasets claim substantial improvements over these approaches, especially for multilingual documents with complex visual layouts. We demonstrate that better document representation is the primary driver of benchmark improvements. By systematically varying transcription and preprocessing methods while holding the retrieval mechanism fixed, we demonstrate that BM25 can recover large gaps on multilingual and visual benchmarks. Our findings call for decomposed evaluation benchmarks that separately measure transcription and retrieval capabilities, enabling the field to correctly attribute progress and focus effort where it matters.

[31] Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory cs.CL | cs.LGPDF

Zhenting Wang, Huancheng Chen, Jiayun Wang, Wei Wei

TL;DR: 本文提出Memex，一种基于索引经验记忆的机制，用于解决大型语言模型（LLM）智能体在长视野任务中因有限上下文窗口导致的瓶颈问题。Memex通过维护一个包含简洁结构化摘要和稳定索引的紧凑工作上下文，并将完整的交互细节存储在外部经验数据库中，从而在不丢弃证据的情况下压缩上下文。通过强化学习框架MemexRL优化写入和读取行为，智能体学习如何总结、归档、索引和检索信息，相比仅依赖摘要的方法，显著减少了信息损失。

Details

Motivation: 现有LLM智能体在长视野任务中受限于有限的上下文窗口，随着轨迹增长，保留工具输出和中间推理变得不可行，现有解决方案（如截断或运行摘要）本质上是信息有损的，因为它们压缩或丢弃了过去的证据。

Result: 在具有挑战性的长视野任务上，经过MemexRL训练的Memex智能体在显著减小工作上下文的同时，提高了任务成功率。

Insight: 创新点在于引入了索引经验记忆机制，将上下文压缩与证据保留解耦，并通过强化学习优化索引的写入和检索策略，理论上分析了在有限解引用下保持决策质量并控制上下文计算复杂度的潜力，为长序列处理提供了更高效且信息保留更好的记忆管理方案。

Abstract: Large language model (LLM) agents are fundamentally bottlenecked by finite context windows on long-horizon tasks. As trajectories grow, retaining tool outputs and intermediate reasoning in-context quickly becomes infeasible: the working context becomes prohibitively long, eventually exceeds the context budget, and makes distant evidence harder to use even when it is still present. Existing solutions typically shorten context through truncation or running summaries, but these methods are fundamentally lossy because they compress or discard past evidence itself. We introduce Memex, an indexed experience memory mechanism that instead compresses context without discarding evidence. Memex maintains a compact working context consisting of concise structured summaries and stable indices, while storing full-fidelity underlying interactions in an external experience database under those indices. The agent can then decide when to dereference an index and recover the exact past evidence needed for the current subgoal. We optimize both write and read behaviors with our reinforcement learning framework MemexRL, using reward shaping tailored to indexed memory usage under a context budget, so the agent learns what to summarize, what to archive, how to index it, and when to retrieve it. This yields a substantially less lossy form of long-horizon memory than summary-only approaches. We further provide a theoretical analysis showing the potential of the Memex loop to preserve decision quality with bounded dereferencing while keeping effective in-context computation bounded as history grows. Empirically, on challenging long-horizon tasks, Memex agent trained with MemexRL improves task success while using a significantly smaller working context.

[32] Position: Vector Prompt Interfaces Should Be Exposed to Enable Customization of Large Language Models cs.CLPDF

Liangwei Yang, Shiyu Wang, Haolin Chen, Rithesh Murthy, Ming Zhu

TL;DR: 这篇立场论文主张，随着大语言模型从研究原型转向实际系统，定制化已成为核心瓶颈。作者认为仅基于文本的提示不足以实现可扩展、稳定且仅需推理的定制，因此建议模型提供商应将向量提示输入作为公开接口的一部分，以支持更有效的LLM定制。

Details

Motivation: 解决当前基于文本提示的LLM定制方法在可扩展性、稳定性和仅推理定制方面的局限性，推动更高效的模型控制接口。

Result: 诊断性证据表明，向量提示调优随着监督增加持续改进，而基于文本的提示优化早期即饱和；向量提示表现出密集的全局注意力模式，表明其具有独特的控制机制。

Insight: 创新点在于提出将向量提示作为公开接口，以实现更稳定和可扩展的LLM定制，同时论证了在标准黑盒威胁模型下，暴露向量提示不会显著增加模型泄漏风险，为社区重新思考提示接口设计提供了新视角。

Abstract: As large language models (LLMs) transition from research prototypes to real-world systems, customization has emerged as a central bottleneck. While text prompts can already customize LLM behavior, we argue that text-only prompting does not constitute a suitable control interface for scalable, stable, and inference-only customization. This position paper argues that model providers should expose \emph{vector prompt inputs} as part of the public interface for customizing LLMs. We support this position with diagnostic evidence showing that vector prompt tuning continues to improve with increasing supervision whereas text-based prompt optimization saturates early, and that vector prompts exhibit dense, global attention patterns indicative of a distinct control mechanism. We further discuss why inference-only customization is increasingly important under realistic deployment constraints, and why exposing vector prompts need not fundamentally increase model leakage risk under a standard black-box threat model. We conclude with a call to action for the community to rethink prompt interfaces as a core component of LLM customization.

[33] $V_1$: Unifying Generation and Self-Verification for Parallel Reasoners cs.CLPDF

Harman Singh, Xiuyu Li, Kusha Sareen, Monishwaran Maheswaran, Sijun Tan

TL;DR: 这篇论文提出了一个名为$V_1$的统一框架，用于并行推理任务中的生成和自我验证。该框架包含两个核心组件：$V_1$-Infer，一种基于锦标赛排名的、不确定性引导的推理算法，能动态分配计算资源；以及$V_1$-PairRL，一个强化学习框架，用于联合训练一个模型同时作为生成器和成对自我验证器。

Details

Motivation: 解决复杂推理任务中测试时扩展（test-time scaling）的一个关键瓶颈：验证。现有方法通常通过标量评分独立评估候选方案，但作者发现模型在成对自我验证（pairwise self-verification）方面能力更强。因此，需要一种能统一生成和验证、并更有效识别正确解决方案的方法。

Result: 在代码生成（LiveCodeBench, CodeContests, SWE-Bench）和数学推理（AIME, HMMT）基准测试上，$V_1$-Infer将Pass@1指标提升了高达10%，优于逐点验证和最近的测试时扩展方法，且效率显著更高。$V_1$-PairRL在测试时扩展上比标准RL和逐点联合训练提升了7-9%，在代码生成场景中，其基础Pass@1比标准RL提升了高达8.7%。

Insight: 主要创新点在于提出了成对自我验证优于独立标量验证的见解，并基于此构建了统一生成与验证的框架。具体创新包括：1）不确定性引导的锦标赛排名算法，能动态优化计算分配；2）联合训练单一模型同时执行生成和成对验证的强化学习框架，使验证器能适应生成器不断变化的输出分布。这为高效利用推理时计算提供了新思路。

Abstract: Test-time scaling for complex reasoning tasks shows that leveraging inference-time compute, by methods such as independently sampling and aggregating multiple solutions, results in significantly better task outcomes. However, a critical bottleneck is verification: sampling is only effective if correct solutions can be reliably identified among candidates. While existing approaches typically evaluate candidates independently via scalar scoring, we demonstrate that models are substantially stronger at pairwise self-verification. Leveraging this insight, we introduce $V_1$, a framework that unifies generation and verification through efficient pairwise ranking. $V_1$ comprises two components: $V_1$-Infer, an uncertainty-guided algorithm using a tournament-based ranking that dynamically allocates self-verification compute to candidate pairs whose relative correctness is most uncertain; and $V_1$-PairRL, an RL framework that jointly trains a single model as both generator and pairwise self-verifier, ensuring the verifier adapts to the generator’s evolving distribution. On code generation (LiveCodeBench, CodeContests, SWE-Bench) and math reasoning (AIME, HMMT) benchmarks, $V_1$-Infer improves Pass@1 by up to $10%$ over pointwise verification and outperforms recent test-time scaling methods while being significantly more efficient. Furthermore, $V_1$-PairRL achieves $7$–$9%$ test-time scaling gains over standard RL and pointwise joint training, and improves base Pass@1 by up to 8.7% over standard RL in a code-generation setting.

[34] AILS-NTUA at SemEval-2026 Task 12: Graph-Based Retrieval and Reflective Prompting for Abductive Event Reasoning cs.CLPDF

Nikolas Karafyllis, Maria Lymperaiou, Giorgos Filandrianos, Athanasios Voulodimos, Giorgos Stamou

TL;DR: 本文介绍了在SemEval-2026 Task 12（溯因事件推理任务）中获得第一名的三阶段系统，该系统结合了基于图的检索、通过反思性提示演化优化的LLM驱动溯因推理以及事后一致性强制机制，在评估阶段以0.95的准确率排名榜首。

Details

Motivation: 旨在解决多标签因果推理任务中的溯因事件推理问题，并探究不同模型在该任务中存在的系统性偏差。

Result: 在SemEval-2026 Task 12的评估阶段，系统以0.95的准确率排名第一。对14个模型（7个家族）的跨模型错误分析揭示了因果链不完整性、近因偏好和显著性偏差这三个共享的归纳偏差，其跨家族收敛性（原因计数减少51%）表明这些是多标签因果推理中系统性的而非模型特定的失败模式。

Insight: 创新点在于将基于图的检索、通过反思性提示演化优化的LLM推理与事后一致性强制相结合的三阶段系统架构。更深层的洞察是，通过跨模型分析识别出多标签因果推理中普遍存在的系统性归纳偏差，这为理解和改进此类模型的推理能力提供了重要方向。

Abstract: We present a winning three-stage system for SemEval 2026 Task12: Abductive Event Reasoning that combines graph-based retrieval, LLM-driven abductive reasoning with prompt design optimized through reflective prompt evolution, and post-hoc consistency enforcement; our system ranks first on the evaluation-phase leaderboard with an accuracy score of 0.95. Cross-model error analysis across 14 models (7families) reveals three shared inductive biases: causal chain incompleteness, proximate cause preference, and salience bias, whose cross-family convergence (51% cause-count reduction) indicates systematic rather than model-specific failure modes in multi-label causal reasoning.

[35] AgentIR: Reasoning-Aware Retrival for Deep Research Agents cs.CLPDF

Zijian Chen, Xueguang Ma, Shengyao Zhuang, Jimmy Lin, Akari Asai

TL;DR: 本文提出了一种面向深度研究智能体的推理感知检索方法AgentIR，通过联合嵌入智能体的推理轨迹与查询来提升检索性能，并设计了数据合成方法DR-Synth生成训练数据。实验表明，该方法在BrowseComp-Plus基准上显著优于传统检索模型。

Details

Motivation: 现有检索系统完全忽略了深度研究智能体在搜索前生成的显式自然语言推理过程，而这些推理包含了丰富的意图和上下文信息，因此需要开发能够利用这些信号的检索范式。

Result: 在BrowseComp-Plus基准测试中，AgentIR-4B与通义深度研究智能体结合实现了68%的准确率，显著优于规模是其两倍的传统嵌入模型（50%）和BM25（37%），达到了新的SOTA水平。

Insight: 创新点在于提出了推理感知检索范式，将智能体的推理轨迹作为检索的额外输入，并设计了从标准QA数据集合成训练数据的方法，有效利用了智能体特有的中间思维信号。

Abstract: Deep Research agents are rapidly emerging as primary consumers of modern retrieval systems. Unlike human users who issue and refine queries without documenting their intermediate thought processes, Deep Research agents generate explicit natural language reasoning before each search call, revealing rich intent and contextual information that existing retrievers entirely ignore. To exploit this overlooked signal, we introduce: (1) Reasoning-Aware Retrieval, a retrieval paradigm that jointly embeds the agent’s reasoning trace alongside its query; and (2) DR-Synth, a data synthesis method that generates Deep Research retriever training data from standard QA datasets. We demonstrate that both components are independently effective, and their combination yields a trained embedding model, AgentIR-4B, with substantial gains. On the challenging BrowseComp-Plus benchmark, AgentIR-4B achieves 68% accuracy with the open-weight agent Tongyi-DeepResearch, compared to 50% with conventional embedding models twice its size, and 37% with BM25. Code and data are available at: https://texttron.github.io/AgentIR/.

cs.CV [Back]

[36] Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning cs.CVPDF

Anas Zafar, Leema Krishna Murali, Ashish Vashist

TL;DR: 本文提出了一种超越准确率的评估框架，用于衡量多模态医学VQA任务中的视觉依赖性和视觉基础能力。研究发现，仅文本的强化学习（RLVR）在多个医学VQA基准（PathVQA、PMC-VQA、SLAKE、VQA-RAD）上能达到与图像-文本RLVR相当的准确率，但实际视觉依赖度很低甚至为负，表明模型可能利用文本捷径而非真正依赖视觉信息。

Details

Motivation: 当前多模态医学VQA的评估协议可能无法有效衡量模型对视觉信息的因果依赖，导致仅文本模型也能取得高准确率，掩盖了视觉基础能力的缺失。

Result: 在PathVQA上，仅文本RLVR的视觉依赖分数（VRS）为-0.09，表明使用不匹配图像时性能反而更好；图像-文本RLVR的整体图像敏感度（IS）降至39.8%。在VQA-RAD上，两种模型准确率均为63%，但仅文本RLVR在空白图像下仍保持81%性能，而图像-文本RLVR的图像敏感度仅为29%。模型在68-74%的响应中生成了视觉声明，但其中38-43%是未基于图像的幻觉推理（HVRR）。

Insight: 创新点在于引入了反事实评估框架（使用真实、空白和乱序图像）和三个新指标（VRS、IS、HVRR）来量化视觉依赖和幻觉。核心洞察是仅依赖准确率的奖励会导致模型利用文本捷径，未来需要显式强化视觉依赖的评估协议和训练目标。

Abstract: Recent work shows that text-only reinforcement learning with verifiable rewards (RLVR) can match or outperform image-text RLVR on multimodal medical VQA benchmarks, suggesting current evaluation protocols may fail to measure causal visual dependence. We introduce a counterfactual evaluation framework using real, blank, and shuffled images across four medical VQA benchmarks: PathVQA, PMC-VQA, SLAKE, and VQA-RAD. Beyond accuracy, we measure Visual Reliance Score (VRS), Image Sensitivity (IS), and introduce Hallucinated Visual Reasoning Rate (HVRR) to detect cases where models generate visual claims despite producing image-invariant answers. Our findings reveal that RLVR improves accuracy while degrading visual grounding: text-only RLVR achieves negative VRS on PathVQA (-0.09), performing better with mismatched images, while image-text RLVR reduces image sensitivity to 39.8% overall despite improving accuracy. On VQA-RAD, both variants achieve 63% accuracy through different mechanisms: text-only RLVR retains 81% performance with blank images, while image-text RLVR shows only 29% image sensitivity. Models generate visual claims in 68-74% of responses, yet 38-43% are ungrounded (HVRR). These findings demonstrate that accuracy-only rewards enable shortcut exploitation, and progress requires grounding-aware evaluation protocols and training objectives that explicitly enforce visual dependence.

[37] Proact-VL: A Proactive VideoLLM for Real-Time AI Companions cs.CVPDF

Weicai Yan, Yuhong Dai, Qi Ran, Haodong Li, Wang Lin

TL;DR: 本文提出了Proact-VL，一个用于实时AI伴侣的主动式视频语言模型框架，旨在解决在连续流输入下的低延迟推理、自主决定响应时机以及控制生成内容的质量与数量以满足实时约束等挑战。该工作通过游戏解说员和游戏引导员两个场景实例化AI伴侣，并引入了包含单人解说、双人解说和用户引导三种代表性场景的大规模Live Gaming Benchmark数据集。

Details

Motivation: 动机是构建具有类人交互体验的实时AI伴侣，需要解决在持续流媒体输入下的低延迟推理、自主决策响应时机以及平衡生成内容质量与数量以满足实时性要求这三个核心挑战。

Result: 大量实验表明，Proact-VL在保持强大视频理解能力的同时，实现了优越的响应延迟和质量，证明了其在实时交互应用中的实用性。

Insight: 创新点在于提出了一个将多模态语言模型塑造成主动、实时交互代理的通用框架，并创建了适用于自动评估的实时游戏基准测试集，以量化衡量AI伴侣在低延迟和内容质量方面的性能。

Abstract: Proactive and real-time interactive experiences are essential for human-like AI companions, yet face three key challenges: (1) achieving low-latency inference under continuous streaming inputs, (2) autonomously deciding when to respond, and (3) controlling both quality and quantity of generated content to meet real-time constraints. In this work, we instantiate AI companions through two gaming scenarios, commentator and guide, selected for their suitability for automatic evaluation. We introduce the Live Gaming Benchmark, a large-scale dataset with three representative scenarios: solo commentary, co-commentary, and user guidance, and present Proact-VL, a general framework that shapes multimodal language models into proactive, real-time interactive agents capable of human-like environment perception and interaction. Extensive experiments show Proact-VL achieves superior response latency and quality while maintaining strong video understanding capabilities, demonstrating its practicality for real-time interactive applications.

[38] Beyond Pixel Histories: World Models with Persistent 3D State cs.CV | cs.AI | cs.LGPDF

Samuel Garcin, Thomas Walker, Steven McDonagh, Tim Pearce, Hakan Bilen

TL;DR: 本文提出PERSIST，一种新型世界模型范式，通过模拟潜在3D场景（环境、相机和渲染器）的演化来生成交互式视频。该方法克服了现有模型缺乏显式3D表示和空间记忆受限的问题，实现了具有持久空间记忆和几何一致性的视频合成。

Details

Motivation: 现有交互式世界模型通常缺乏3D环境表示，导致3D一致性需从数据中隐式学习，且空间记忆受限于短期时间上下文窗口，这影响了用户体验并阻碍了下游任务（如智能体训练）。

Result: 定量指标和定性用户研究表明，PERSIST在空间记忆、3D一致性和长时程稳定性方面相比现有方法有显著提升，能够生成连贯演化的3D世界。

Insight: 创新点在于引入显式潜在3D场景演化模拟，实现了持久空间记忆和几何一致性；客观分析认为其支持从单图像合成多样3D环境，并允许在3D空间中进行细粒度、几何感知的环境编辑与控制，为生成式AI提供了更结构化、可控的3D世界建模框架。

Abstract: Interactive world models continually generate video by responding to a user’s actions, enabling open-ended generation capabilities. However, existing models typically lack a 3D representation of the environment, meaning 3D consistency must be implicitly learned from data, and spatial memory is restricted to limited temporal context windows. This results in an unrealistic user experience and presents significant obstacles to down-stream tasks such as training agents. To address this, we present PERSIST, a new paradigm of world model which simulates the evolution of a latent 3D scene: environment, camera, and renderer. This allows us to synthesize new frames with persistent spatial memory and consistent geometry. Both quantitative metrics and a qualitative user study show substantial improvements in spatial memory, 3D consistency, and long-horizon stability over existing methods, enabling coherent, evolving 3D worlds. We further demonstrate novel capabilities, including synthesising diverse 3D environments from a single image, as well as enabling fine-grained, geometry-aware control over generated experiences by supporting environment editing and specification directly in 3D space. Project page: https://francelico.github.io/persist.github.io

[39] Phys4D: Fine-Grained Physics-Consistent 4D Modeling from Video Diffusion cs.CV | cs.AI | cs.ROPDF

Haoran Lu, Shang Wu, Jianshu Zhang, Maojiang Su, Guo Ye

TL;DR: Phys4D是一个从视频扩散模型中学习物理一致的4D世界表示的管道。它采用三阶段训练范式，逐步将外观驱动的视频扩散模型提升为物理一致的4D表示，包括伪监督预训练、基于物理的监督微调和基于模拟的强化学习，并引入了4D世界一致性评估来超越外观指标。

Details

Motivation: 当前视频扩散模型作为大规模生成世界模型取得了显著进展，但往往在细粒度物理一致性上存在不足，表现出随时间推移物理上不合理的动态。

Result: 实验结果表明，与外观驱动的基线方法相比，Phys4D在细粒度时空和物理一致性方面有显著提升，同时保持了强大的生成性能。

Insight: 创新点在于提出了一个渐进式的三阶段训练范式来增强生成模型的物理一致性，并引入了专门的4D世界一致性评估指标来量化几何一致性、运动稳定性和长时程物理合理性，超越了传统的外观评估。

Abstract: Recent video diffusion models have achieved impressive capabilities as large-scale generative world models. However, these models often struggle with fine-grained physical consistency, exhibiting physically implausible dynamics over time. In this work, we present \textbf{Phys4D}, a pipeline for learning physics-consistent 4D world representations from video diffusion models. Phys4D adopts \textbf{a three-stage training paradigm} that progressively lifts appearance-driven video diffusion models into physics-consistent 4D world representations. We first bootstrap robust geometry and motion representations through large-scale pseudo-supervised pretraining, establishing a foundation for 4D scene modeling. We then perform physics-grounded supervised fine-tuning using simulation-generated data, enforcing temporally consistent 4D dynamics. Finally, we apply simulation-grounded reinforcement learning to correct residual physical violations that are difficult to capture through explicit supervision. To evaluate fine-grained physical consistency beyond appearance-based metrics, we introduce a set of \textbf{4D world consistency evaluation} that probe geometric coherence, motion stability, and long-horizon physical plausibility. Experimental results demonstrate that Phys4D substantially improves fine-grained spatiotemporal and physical consistency compared to appearance-driven baselines, while maintaining strong generative performance. Our project page is available at https://sensational-brioche-7657e7.netlify.app/

[40] PhyPrompt: RL-based Prompt Refinement for Physically Plausible Text-to-Video Generation cs.CV | cs.AIPDF

Shang Wu, Chenwei Xu, Zhuofan Xia, Weijian Li, Lie Lu

TL;DR: 本文提出了PhyPrompt，一个基于强化学习的两阶段提示词优化框架，旨在提升文本到视频生成模型的物理合理性。该框架首先使用专注于物理推理的思维链数据集微调大语言模型，以整合物理原理；然后应用带有动态奖励课程的组相对策略优化，逐步从语义保真度转向物理常识优化。该方法在VideoPhy2基准上显著提升了物理合理性和语义一致性，并展示了在多种T2V架构上的零样本迁移能力。

Details

Motivation: 现有最先进的文本到视频生成器尽管视觉质量高，但经常违反物理定律。作者认为这源于提示词中物理约束不足，而非模型本身的限制。手动添加物理细节虽有效但需要专业知识且难以扩展，因此需要自动化的提示词优化方法。

Result: 在VideoPhy2基准测试中，PhyPrompt-7B模型实现了40.8%的联合成功率（提升8.6个百分点），物理常识得分从55.8%提升至66.8%（提升11个百分点），语义一致性从43.4%提升至47.8%（提升4.4个百分点）。该方法超越了GPT-4o（提升3.8%）和参数大100倍的DeepSeek-V3（提升2.2%），并在多种T2V架构（如Lavie, VideoCrafter2, CogVideoX-5B）上实现了零样本迁移，最高提升达16.8%。

Insight: 论文的创新点在于：1）提出了一个两阶段的强化学习框架，将物理知识整合与动态奖励课程相结合；2）通过组相对策略优化和动态奖励课程（从语义保真度逐步转向物理常识），实现了超越传统多目标权衡的协同优化，证明了组合式提示词发现的潜力；3）展示了领域专用的强化学习与组合课程在提升物理感知生成方面，优于通用的模型规模扩展方法。

Abstract: State-of-the-art text-to-video (T2V) generators frequently violate physical laws despite high visual quality. We show this stems from insufficient physical constraints in prompts rather than model limitations: manually adding physics details reliably produces physically plausible videos, but requires expertise and does not scale. We present PhyPrompt, a two-stage reinforcement learning framework that automatically refines prompts for physically realistic generation. First, we fine-tune a large language model on a physics-focused Chain-of-Thought dataset to integrate principles like object motion and force interactions while preserving user intent. Second, we apply Group Relative Policy Optimization with a dynamic reward curriculum that initially prioritizes semantic fidelity, then progressively shifts toward physical commonsense. This curriculum achieves synergistic optimization: PhyPrompt-7B reaches 40.8% joint success on VideoPhy2 (8.6pp gain), improving physical commonsense by 11pp (55.8% to 66.8%) while simultaneously increasing semantic adherence by 4.4pp (43.4% to 47.8%). Remarkably, our curriculum exceeds single-objective training on both metrics, demonstrating compositional prompt discovery beyond conventional multi-objective trade-offs. PhyPrompt outperforms GPT-4o (+3.8% joint) and DeepSeek-V3 (+2.2%, 100$\times$ larger) using only 7B parameters. The approach transfers zero-shot across diverse T2V architectures (Lavie, VideoCrafter2, CogVideoX-5B) with up to 16.8% improvement, establishing that domain-specialized reinforcement learning with compositional curricula surpasses general-purpose scaling for physics-aware generation.

[41] PinCLIP: Large-scale Foundational Multimodal Representation at Pinterest cs.CVPDF

Josh Beal, Eric Kim, Jinfeng Rao, Rex Wu, Dmitry Kislyuk

TL;DR: PinCLIP是Pinterest开发的大规模视觉表示学习方法，旨在通过视觉语言模型学习图文对齐来增强其检索和排序模型。它采用了一种新颖的混合视觉Transformer架构，结合了VLM骨干网络和混合融合机制，以捕捉不同粒度的多模态内容表示。除了标准的图文对齐目标，还引入了邻居对齐目标来建模Pinterest Pin-Board图中的多模态表示交叉融合。

Details

Motivation: 尽管多模态视觉语言模型在许多领域取得了成功，但由于训练目标差异和服务效率瓶颈等问题，将其整合到推荐和检索系统中仍然具有挑战性。本文旨在解决这些问题，为Pinterest的检索和排序模型开发一个高效的多模态表示学习方案。

Result: 离线评估显示，PinCLIP在多模态检索任务中优于最先进的基线模型（如Qwen），性能提升20%。在线A/B测试证明了其显著的商业影响，包括在Pinterest所有主要界面上带来大量用户参与度提升。特别地，PinCLIP显著缓解了“冷启动”问题，使有机内容的Repin增加了15%，新广告的点击率提高了8.7%。

Insight: 论文的创新点在于提出了一种混合视觉Transformer架构和混合融合机制，以及一个新颖的邻居对齐目标，该目标利用Pinterest的Pin-Board图结构来建模多模态表示的交叉融合。从客观角度看，将图结构信息（邻居关系）整合到多模态表示学习中，以解决推荐系统中的冷启动问题，是一个值得借鉴的思路。

Abstract: While multi-modal Visual Language Models (VLMs) have demonstrated significant success across various domains, the integration of VLMs into recommendation and retrieval systems remains a challenge, due to issues like training objective discrepancies and serving efficiency bottlenecks. This paper introduces PinCLIP, a large-scale visual representation learning approach developed to enhance retrieval and ranking models at Pinterest by leveraging VLMs to learn image-text alignment. We propose a novel hybrid Vision Transformer architecture that utilizes a VLM backbone and a hybrid fusion mechanism to capture multi-modality content representation at varying granularities. Beyond standard image-to-text alignment objectives, we introduce a neighbor alignment objective to model the cross-fusion of multi-modal representations within the Pinterest Pin-Board graph. Offline evaluations show that PinCLIP outperforms state-of-the-art baselines, such as Qwen, by 20% in multi-modal retrieval tasks. Online A/B testing demonstrates significant business impact, including substantial engagement gains across all major surfaces in Pinterest. Notably, PinCLIP significantly addresses the “cold-start” problem, enhancing fresh content distribution with a 15% Repin increase in organic content and 8.7% higher click for new Ads.

[42] Modeling Cross-vision Synergy for Unified Large Vision Model cs.CVPDF

Shengqiong Wu, Lanhu Wu, Mingyang Bao, Wenhao Xu, Hanwang Zhang

TL;DR: 本文提出了PolyV，一个统一的大型视觉模型，旨在实现跨视觉模态（图像、视频、3D）的协同，而不仅仅是功能集成。它在架构上采用由动态模态路由器协调的稀疏混合专家模型，在训练上结合模态特定预训练和从粗到细的协同调优。

Details

Motivation: 现有统一大型视觉模型主要追求功能集成，但忽视了跨视觉模态协同这一更深层目标，即利用不同视觉模态间的互补先验进行推理的能力。

Result: 在涵盖图像、视频和3D理解的10个基准测试（包括需要空间或时间先验的协同数据集）上，PolyV始终优于现有模型，相比其骨干网络平均提升超过10%。

Insight: 创新点在于从架构和训练两个层面系统性地建模跨模态协同：架构上通过动态路由的稀疏MoE实现专家专业化与双向交互；训练上通过知识蒸馏和对象/关系级对齐实现从粗到细的协同调优，为真正的协同视觉推理提供了统一框架。

Abstract: Recent advances in large vision models (LVMs) have shifted from modality-specific designs toward unified architectures that jointly process images, videos, and 3D data. However, existing unified LVMs primarily pursue functional integration, while overlooking the deeper goal of cross-vision synergy: the ability to reason over complementary priors across visual modalities. To address this, we present PolyV, a unified LVM that achieves cross-vision synergy at both the architectural and training levels. Architecturally, PolyV adopts a sparse Mixture-of-Experts LVM coordinated by a dynamic modality router, allowing each expert to specialize in modality-specific priors while enabling bidirectional interaction and mutual refinement across modalities. Training-wise, a synergy-aware paradigm combines modality-specific pretraining with coarse-to-fine synergy tuning via knowledge distillation and object-/relation-level alignment. Extensive experiments on 10 benchmarks spanning image, video, and 3D understanding, including synergy-focused datasets requiring spatial or temporal priors, demonstrate that PolyV consistently outperforms existing models, achieving over 10% average improvement over its backbone. Overall, PolyV establishes a unified framework for synesthetic visual reasoning, advancing toward truly synergistic LVMs. Project page: https://sqwu.top/PolyV.

[43] Confidence-aware Monocular Depth Estimation for Minimally Invasive Surgery cs.CVPDF

Muhammad Asad, Emanuele Colleoni, Pritesh Mehta, Nicolas Toussaint, Ricardo Sanchez-Matilla

TL;DR: 本文提出了一种用于微创手术的置信度感知单目深度估计框架，旨在解决内窥镜视频中因烟雾、镜面反射、模糊和遮挡等因素导致的深度估计不准确问题，并通过引入置信度估计来提升模型的临床可靠性。

Details

Motivation: 微创手术中的单目深度估计受噪声和伪影影响，现有模型缺乏置信度输出，限制了其临床应用的可靠性。

Result: 在内部临床内窥镜数据集（StereoKP）上，该方法相比基线模型将密集深度估计精度提升了约8%，并在内部和公共数据集上验证了其能稳健量化预测置信度。

Insight: 创新点包括：使用微调立体匹配模型集成生成校准的置信度目标；提出置信度感知损失函数，使可靠像素主导训练；设计推理时置信度估计头，可输出逐像素置信度图以评估深度可靠性。

Abstract: Purpose: Monocular depth estimation (MDE) is vital for scene understanding in minimally invasive surgery (MIS). However, endoscopic video sequences are often contaminated by smoke, specular reflections, blur, and occlusions, limiting the accuracy of MDE models. In addition, current MDE models do not output depth confidence, which could be a valuable tool for improving their clinical reliability. Methods: We propose a novel confidence-aware MDE framework featuring three significant contributions: (i) Calibrated confidence targets: an ensemble of fine-tuned stereo matching models is used to capture disparity variance into pixel-wise confidence probabilities; (ii) Confidence-aware loss: Baseline MDE models are optimized with confidence-aware loss functions, utilizing pixel-wise confidence probabilities such that reliable pixels dominate training; and (iii) Inference-time confidence: a confidence estimation head is proposed with two convolution layers to predict per-pixel confidence at inference, enabling assessment of depth reliability. Results: Comprehensive experimental validation across internal and public datasets demonstrates that our framework improves depth estimation accuracy and can robustly quantify the prediction’s confidence. On the internal clinical endoscopic dataset (StereoKP), we improve dense depth estimation accuracy by ~8% as compared to the baseline model. Conclusion: Our confidence-aware framework enables improved accuracy of MDE models in MIS, addressing challenges posed by noise and artifacts in pre-clinical and clinical data, and allows MDE models to provide confidence maps that may be used to improve their reliability for clinical applications.

[44] An Effective Data Augmentation Method by Asking Questions about Scene Text Images cs.CVPDF

Xu Yao, Lei Kang

TL;DR: 本文提出了一种受视觉问答（VQA）启发的数据增强方法，用于增强场景文本识别（STR）和手写文本识别（HTR）模型的训练。该方法通过为图像-文本对生成关于字符级属性（如存在、位置、频率）的自然语言问题，并让模型回答这些问题，从而鼓励模型进行更细粒度的推理。

Details

Motivation: 解决传统OCR模型直接预测转录而缺乏对文本结构进行详细推理的问题，旨在通过结构化问答任务来增强OCR模型的训练。

Result: 在WordArt和Esposalles数据集上的实验表明，该方法相比基线模型取得了持续改进，显著降低了字符错误率（CER）和词错误率（WER）。

Insight: 创新点在于将VQA范式引入OCR训练，通过生成字符级属性的问答任务作为辅助学习目标，迫使模型对齐视觉特征与文本查询，进行联合推理，这是一种新颖的数据增强和模型正则化思路。

Abstract: Scene text recognition (STR) and handwritten text recognition (HTR) face significant challenges in accurately transcribing textual content from images into machine-readable formats. Conventional OCR models often predict transcriptions directly, which limits detailed reasoning about text structure. We propose a VQA-inspired data augmentation framework that strengthens OCR training through structured question-answering tasks. For each image-text pair, we generate natural-language questions probing character-level attributes such as presence, position, and frequency, with answers derived from ground-truth text. These auxiliary tasks encourage finer-grained reasoning, and the OCR model aligns visual features with textual queries to jointly reason over images and questions. Experiments on WordArt and Esposalles datasets show consistent improvements over baseline models, with significant reductions in both CER and WER. Our code is publicly available at https://github.com/xuyaooo/DataAugOCR.

[45] Hazard-Aware Traffic Scene Graph Generation cs.CVPDF

Yaoqi Huang, Julie Stephany Berrio, Mao Shan, Stewart Worrall

TL;DR: 本文提出了一种新的交通场景图生成任务，旨在通过构建以自车为中心的交通场景图来增强驾驶场景中的态势感知能力，重点关注突出危险与自车之间的交通特定关系。

Details

Motivation: 现有方法在检测特定语义类别和视觉显著区域方面表现良好，但缺乏评估安全相关性的能力，且通用空间谓词（仅针对前景对象或所有场景实体）不足以应对驾驶场景。

Result: 在Cityscapes数据集上创建了关系标注，并从5个角度评估了10项任务。对比实验和消融研究的结果证明了该方法在以自车为中心的危险感知交通场景理解方面的能力。

Insight: 创新点在于引入了交通场景图生成任务，并提出了一个利用交通事故数据和深度线索来补充视觉特征和语义信息进行推理的框架，输出通过颜色编码危险严重性并标注其影响机制和相对位置的直观场景图。

Abstract: Maintaining situational awareness in complex driving scenarios is challenging. It requires continuously prioritizing attention among extensive scene entities and understanding how prominent hazards might affect the ego vehicle. While existing studies excel at detecting specific semantic categories and visually salient regions, they lack the ability to assess safety-relevance. Meanwhile, the generic spatial predicates either for foreground objects only or for all scene entities modeled by existing scene graphs are inadequate for driving scenarios. To bridge this gap, we introduce a novel task, Traffic Scene Graph Generation, which captures traffic-specific relations between prominent hazards and the ego vehicle. We propose a novel framework that explicitly uses traffic accident data and depth cues to supplement visual features and semantic information for reasoning. The output traffic scene graphs provide intuitive guidelines that stress prominent hazards by color-coding their severity and notating their effect mechanism and relative location to the ego vehicle. We create relational annotations on Cityscapes dataset and evaluate our model on 10 tasks from 5 perspectives. The results in comparative experiments and ablation studies demonstrate our capacity in ego-centric reasoning for hazard-aware traffic scene understanding.

[46] Tracking Feral Horses in Aerial Video Using Oriented Bounding Boxes cs.CV | q-bio.QMPDF

Saeko Takizawa, Tamao Maeda, Shinya Yamamoto, Hiroaki Kawashima

TL;DR: 本文提出了一种基于定向边界框（OBB）的野马空中视频跟踪方法，通过头部朝向估计解决现有OBB检测器因180°角度限制导致的头部尾部混淆和跟踪跳变问题，提升了多动物跟踪的准确性。

Details

Motivation: 现有基于轴对齐边界框的跟踪方法在复杂背景、小目标、高密度和动物姿态变化的空中俯拍视频中性能下降，而现有OBB检测器存在180°角度限制，无法区分头尾并导致跟踪不稳定。

Result: 在299张测试图像上，所提方法达到了99.3%的准确率，优于单个检测模型，证明了其在基于OBB的鲁棒跟踪中的有效性。

Insight: 创新点在于提出了一种结合OBB裁剪、多检测器（头、尾、头尾）和基于IoU多数投票的头部朝向估计方法，有效解决了OBB角度歧义问题，提升了动物行为分析的跟踪连续性。

Abstract: The social structures of group-living animals such as feral horses are diverse and remain insufficiently understood, even within a single species. To investigate group dynamics, aerial videos are often utilized to track individuals and analyze their movement trajectories, which are essential for evaluating inter-individual interactions and comparing social behaviors. Accurate individual tracking is therefore crucial. In multi-animal tracking, axis-aligned bounding boxes (bboxes) are widely used; however, for aerial top-view footage of entire groups, their performance degrades due to complex backgrounds, small target sizes, high animal density, and varying body orientations. To address this issue, we employ oriented bounding boxes (OBBs), which include rotation angles and reduce unnecessary background. Nevertheless, current OBB detectors such as YOLO-OBB restrict angles within a 180$^{\circ}$ range, making it impossible to distinguish head from tail and often causing sudden 180$^{\circ}$ flips across frames, which severely disrupts continuous tracking. To overcome this limitation, we propose a head-orientation estimation method that crops OBB-centered patches, applies three detectors (head, tail, and head-tail), and determines the final label through IoU-based majority voting. Experiments using 299 test images show that our method achieves 99.3% accuracy, outperforming individual models, demonstrating its effectiveness for robust OBB-based tracking.

[47] RAGTrack: Language-aware RGBT Tracking with Retrieval-Augmented Generation cs.CVPDF

Hao Li, Yuhao Wang, Wenning Hao, Pingping Zhang, Dong Wang

TL;DR: 本文提出RAGTrack，一种基于检索增强生成（RAG）的语言感知RGB-Thermal（RGBT）跟踪框架。该方法通过引入多模态大语言模型（MLLMs）自动生成文本描述，构建了一个统一的视觉-语言建模框架，包含多模态Transformer编码器（MTE）、自适应令牌融合（ATF）和上下文感知推理模块（CRM），以解决现有RGBT跟踪器因缺乏语言引导而难以适应目标外观变化、以及存在冗余搜索区域和模态差异的问题。

Details

Motivation: 现有RGBT跟踪器仅依赖初始帧的视觉信息进行目标建模，缺乏语言引导，难以适应目标外观变化；同时，现有方法存在冗余搜索区域和异构模态差异，导致背景干扰。

Result: 在四个RGBT基准测试上的大量实验表明，该框架在各种挑战性场景下实现了最先进的（SOTA）性能。

Insight: 主要创新点包括：1) 首次将文本描述引入RGBT跟踪基准，通过MLLMs自动生成文本标注；2) 提出一个统一的检索增强生成（RAG）框架，通过自适应令牌融合（ATF）减少搜索冗余和模态差异，并通过上下文感知推理模块（CRM）进行时序语言推理以实现鲁棒的目标建模。从客观角度看，将语言模态与RGBT跟踪结合，并利用RAG机制进行动态知识维护和推理，是一个新颖且有潜力的方向。

Abstract: RGB-Thermal (RGBT) tracking aims to achieve robust object localization across diverse environmental conditions by fusing visible and thermal infrared modalities. However, existing RGBT trackers rely solely on initial-frame visual information for target modeling, failing to adapt to appearance variations due to the absence of language guidance. Furthermore, current methods suffer from redundant search regions and heterogeneous modality gaps, causing background distraction. To address these issues, we first introduce textual descriptions into RGBT tracking benchmarks. This is accomplished through a pipeline that leverages Multi-modal Large Language Models (MLLMs) to automatically produce texual annotations. Afterwards, we propose RAGTrack, a novel Retrieval-Augmented Generation framework for robust RGBT tracking. To this end, we introduce a Multi-modal Transformer Encoder (MTE) for unified visual-language modeling. Then, we design an Adaptive Token Fusion (ATF) to select target-relevant tokens and perform channel exchanges based on cross-modal correlations, mitigating search redundancies and modality gaps. Finally, we propose a Context-aware Reasoning Module (CRM) to maintain a dynamic knowledge base and employ a Retrieval-Augmented Generation (RAG) to enable temporal linguistic reasoning for robust target modeling. Extensive experiments on four RGBT benchmarks demonstrate that our framework achieves state-of-the-art performance across various challenging scenarios. The source code is available https://github.com/IdolLab/RAGTrack.

[48] CoRe-BT: A Multimodal Radiology-Pathology-Text Benchmark for Robust Brain Tumor Typing cs.CVPDF

Juampablo E. Heras Rivera, Daniel K. Low, Xavier Xiong, Jacob J. Ruzevick, Daniel D. Child

TL;DR: 本文提出了CoRe-BT，一个用于脑肿瘤分型的多模态（放射学-病理学-文本）基准数据集，旨在研究在模态缺失条件下的鲁棒多模态学习。该数据集包含310名患者的多序列脑部MRI、95例配对的病理切片图像和病理报告，并标注了肿瘤类型、分级及分割掩码。

Details

Motivation: 准确的脑肿瘤分型需要整合MRI、组织病理学和病理报告等异构临床证据，而这些信息在诊断时常常不完整。现有研究缺乏一个能系统评估模态缺失下多模型学习鲁棒性的基准。

Result: 基线实验证明了多模态融合的可行性，并突出了不同模态在临床相关分型任务中的互补贡献。通过比较仅使用MRI的模型与结合病理信息的多模态方法，评估了不同模态可用性下的肿瘤分型性能。

Insight: 创新点在于构建了一个包含多模态、有标注且模拟真实临床数据不完整性的基准测试平台（CoRe-BT），支持区域感知建模和辅助学习任务，为推进多模态胶质瘤分型和表征学习提供了接地气的测试环境。

Abstract: Accurate brain tumor typing requires integrating heterogeneous clinical evidence, including magnetic resonance imaging (MRI), histopathology, and pathology reports, which are often incomplete at the time of diagnosis. We introduce CoRe-BT, a cross-modal radiology-pathology-text benchmark for brain tumor typing, designed to study robust multimodal learning under missing modality conditions. The dataset comprises 310 patients with multi-sequence brain MRI (T1, T1c, T2, FLAIR), including 95 cases with paired H&E-stained whole-slide pathology images and pathology reports. All cases are annotated with tumor type and grade, and MRI volumes include expert-annotated tumor masks, enabling both region-aware modeling and auxiliary learning tasks. Tumors are categorized into six clinically relevant classes capturing the heterogeneity of common and rare glioma subtypes. We evaluate tumor typing under variable modality availability by comparing MRI-only models with multimodal approaches that incorporate pathology information when present. Baseline experiments demonstrate the feasibility of multimodal fusion and highlight complementary modality contributions across clinically relevant typing tasks. CoRe-BT provides a grounded testbed for advancing multimodal glioma typing and representation learning in realistic scenarios with incomplete clinical data.

[49] Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructions cs.CV | cs.AI | cs.CRPDF

Neha Nagaraja, Lan Zhang, Zhilong Wang, Bo Zhang, Pawan Patil

TL;DR: 本文研究了基于图像的提示注入攻击，这是一种针对多模态大语言模型的黑盒攻击方法，通过在自然图像中嵌入对抗性指令来覆盖模型行为。作者提出了一种端到端的IPI流程，结合了基于分割的区域选择、自适应字体缩放和背景感知渲染技术，以在保持模型可解释性的同时隐藏提示不被人类察觉。

Details

Motivation: 多模态大语言模型整合视觉和文本带来了新的安全漏洞，本文旨在探索图像中嵌入对抗性指令以劫持模型行为的攻击方法，以揭示这种集成引入的脆弱性。

Result: 在COCO数据集和GPT-4-turbo模型上评估了12种对抗性提示策略和多种嵌入配置，最有效的配置在隐蔽约束下实现了高达64%的攻击成功率，表明IPI能可靠地操纵模型输出。

Insight: 创新点在于提出了一种端到端的图像提示注入攻击流程，通过分割区域选择、自适应字体缩放和背景感知渲染来隐蔽嵌入对抗指令，这为多模态模型的安全防御提供了新的研究方向，强调了对抗性视觉提示注入的实际威胁。

Abstract: Multimodal Large Language Models (MLLMs) integrate vision and text to power applications, but this integration introduces new vulnerabilities. We study Image-based Prompt Injection (IPI), a black-box attack in which adversarial instructions are embedded into natural images to override model behavior. Our end-to-end IPI pipeline incorporates segmentation-based region selection, adaptive font scaling, and background-aware rendering to conceal prompts from human perception while preserving model interpretability. Using the COCO dataset and GPT-4-turbo, we evaluate 12 adversarial prompt strategies and multiple embedding configurations. The results show that IPI can reliably manipulate the output of the model, with the most effective configuration achieving up to 64% attack success under stealth constraints. These findings highlight IPI as a practical threat in black-box settings and underscore the need for defenses against multimodal prompt injection.

[50] InfinityStory: Unlimited Video Generation with World Consistency and Character-Aware Shot Transitions cs.CVPDF

Mohamed Elmoghany, Liangbing Zhao, Xiaoqian Shen, Subhojyoti Mukherjee, Yang Zhou

TL;DR: InfinityStory是一个用于生成长篇叙事视频的框架，通过背景一致性生成管道和过渡感知视频合成模块，解决了跨镜头背景一致性、多主体镜头间无缝过渡以及扩展到小时级叙事的挑战。

Details

Motivation: 解决视频合成中长篇幅叙事视频生成时背景一致性差、多主体镜头过渡不自然以及可扩展性有限的问题。

Result: 在VBench基准测试中，InfinityStory在背景一致性（88.94）、主体一致性（82.11）和整体平均排名（2.80）上均达到最高水平，表现出更好的稳定性、更平滑的过渡和更优的时间连贯性。

Insight: 创新点包括背景一致性生成管道以保持视觉连贯性，以及过渡感知视频合成模块处理多主体进出帧的复杂场景；客观分析认为其贡献还包括一个包含10,000个多主体过渡序列的合成数据集，覆盖了动态场景组合的不足领域。

Abstract: Generating long-form storytelling videos with consistent visual narratives remains a significant challenge in video synthesis. We present a novel framework, dataset, and a model that address three critical limitations: background consistency across shots, seamless multi-subject shot-to-shot transitions, and scalability to hour-long narratives. Our approach introduces a background-consistent generation pipeline that maintains visual coherence across scenes while preserving character identity and spatial relationships. We further propose a transition-aware video synthesis module that generates smooth shot transitions for complex scenarios involving multiple subjects entering or exiting frames, going beyond the single-subject limitations of prior work. To support this, we contribute with a synthetic dataset of 10,000 multi-subject transition sequences covering underrepresented dynamic scene compositions. On VBench, InfinityStory achieves the highest Background Consistency (88.94), highest Subject Consistency (82.11), and the best overall average rank (2.80), showing improved stability, smoother transitions, and better temporal coherence.

[51] Field imaging framework for morphological characterization of aggregates with computer vision: Algorithms and applications cs.CV | cs.AI | eess.IVPDF

Haohang Huang

TL;DR: 本文开发了一个用于骨料形态表征的现场成像框架，包括针对单个非重叠骨料的成像系统与分割体积估计算法、针对骨料堆的2D实例分割与形态分析方法，以及针对骨料堆3D点云分析的集成重建-分割-补全（RSC-3D）方法。

Details

Motivation: 解决现有骨料成像方法仅适用于受控条件下规则尺寸骨料的局限性，为多场景下的骨料形态表征提供现场解决方案。

Result: 在真实骨料堆上验证了集成RSC-3D方法，在捕捉和预测骨料不可见侧面方面表现出良好性能。

Insight: 创新性地提出了一个覆盖从2D到3D、从单个到堆叠场景的完整现场成像框架，特别是通过构建3D骨料粒子库和合成数据集来支持3D实例分割与形状补全网络的学习，实现了对野外复杂条件下骨料形态的高保真表征。

Abstract: Construction aggregates, including sand and gravel, crushed stone and riprap, are the core building blocks of the construction industry. State-of-the-practice characterization methods mainly relies on visual inspection and manual measurement. State-of-the-art aggregate imaging methods have limitations that are only applicable to regular-sized aggregates under well-controlled conditions. This dissertation addresses these major challenges by developing a field imaging framework for the morphological characterization of aggregates as a multi-scenario solution. For individual and non-overlapping aggregates, a field imaging system was designed and the associated segmentation and volume estimation algorithms were developed. For 2D image analyses of aggregates in stockpiles, an automated 2D instance segmentation and morphological analysis approach was established. For 3D point cloud analyses of aggregate stockpiles, an integrated 3D Reconstruction-Segmentation-Completion (RSC-3D) approach was established: 3D reconstruction procedures from multi-view images, 3D stockpile instance segmentation, and 3D shape completion to predict the unseen sides. First, a 3D reconstruction procedure was developed to obtain high-fidelity 3D models of collected aggregate samples, based on which a 3D aggregate particle library was constructed. Next, two datasets were derived from the 3D particle library for 3D learning: a synthetic dataset of aggregate stockpiles with ground-truth instance labels, and a dataset of partial-complete shape pairs, developed with varying-view raycasting schemes. A state-of-the-art 3D instance segmentation network and a 3D shape completion network were trained on the datasets, respectively. The application of the integrated approach was demonstrated on real stockpiles and validated with ground-truth, showing good performance in capturing and predicting the unseen sides of aggregates.

[52] InEdit-Bench: Benchmarking Intermediate Logical Pathways for Intelligent Image Editing Models cs.CV | cs.AIPDF

Zhiqiang Sheng, Xumeng Han, Zhiwei Zhang, Zenghui Xiong, Yifan Ding

TL;DR: 本文介绍了首个专注于评估图像编辑模型在中间逻辑路径推理能力的基准测试InEdit-Bench。该基准包含四个基本任务类别（状态转换、动态过程、时间序列和科学模拟）的精心标注测试用例，并提出了一套评估生成路径逻辑连贯性、视觉自然性及模型对路径约束遵循程度的细粒度标准。通过对14个代表性图像编辑模型的全面评估，揭示了该领域普遍存在的显著不足。

Details

Motivation: 当前多模态生成模型在图像编辑的静态任务上表现出色，但在需要动态推理的复杂场景中能力不足，无法建模从初始状态到最终状态的多步演化中的连贯中间逻辑路径。这种能力对于实现更深层次的程序和因果理解至关重要，因此需要系统性地衡量这一关键局限。

Result: 对14个代表性图像编辑模型在InEdit-Bench上的全面评估揭示了该领域存在显著且普遍的缺陷，表明现有模型在动态推理和中间路径生成方面能力不足。

Insight: 论文的创新点在于首次提出了专门针对图像编辑中间逻辑路径推理的标准化评估基准（InEdit-Bench），并设计了细粒度的评估标准。这为引导开发更具动态性、推理感知和智能的多模态生成模型提供了关键的评估工具和研究方向。从客观角度看，该工作填补了图像编辑模型动态推理能力评估的空白，其任务分类和评估标准的设计具有借鉴意义。

Abstract: Multimodal generative models have made significant strides in image editing, demonstrating impressive performance on a variety of static tasks. However, their proficiency typically does not extend to complex scenarios requiring dynamic reasoning, leaving them ill-equipped to model the coherent, intermediate logical pathways that constitute a multi-step evolution from an initial state to a final one. This capacity is crucial for unlocking a deeper level of procedural and causal understanding in visual manipulation. To systematically measure this critical limitation, we introduce InEdit-Bench, the first evaluation benchmark dedicated to reasoning over intermediate pathways in image editing. InEdit-Bench comprises meticulously annotated test cases covering four fundamental task categories: state transition, dynamic process, temporal sequence, and scientific simulation. Additionally, to enable fine-grained evaluation, we propose a set of assessment criteria to evaluate the logical coherence and visual naturalness of the generated pathways, as well as the model’s fidelity to specified path constraints. Our comprehensive evaluation of 14 representative image editing models on InEdit-Bench reveals significant and widespread shortcomings in this domain. By providing a standardized and challenging benchmark, we aim for InEdit-Bench to catalyze research and steer development towards more dynamic, reason-aware, and intelligent multimodal generative models.

[53] EvoPrune: Early-Stage Visual Token Pruning for Efficient MLLMs cs.CV | cs.AIPDF

Yuhao Chen, Bin Shan, Xin Ye, Cheng Chen

TL;DR: 本文提出了EvoPrune，一种用于多模态大语言模型（MLLMs）的早期视觉令牌剪枝方法，旨在通过在视觉编码阶段直接剪枝冗余令牌，解决高分辨率图像和视频场景中视觉令牌数量激增导致的推理效率瓶颈问题。

Details

Motivation: 现有视觉令牌剪枝方法主要在视觉编码之后进行，忽略了编码阶段本身产生的巨大计算开销，因此需要一种在编码过程中进行剪枝的方法来提升整体推理效率。

Result: 在图像和视频基准测试上的广泛实验验证了EvoPrune的有效性。特别是在VideoMME数据集上，EvoPrune实现了2倍的推理加速，且性能下降小于1%。

Insight: 创新点在于提出了在视觉编码的早期阶段进行层间剪枝的策略，并综合了令牌相似性、多样性和基于注意力的重要性作为剪枝指导，从而在编码过程中高效保留信息量最大的视觉令牌，为延迟敏感的MLLM部署提供了新思路。

Abstract: Multimodal Large Language Models (MLLMs) have shown strong performance in vision-language tasks, but their inference efficiency is severely limited by the exponential growth of visual tokens in complex scenarios such as high-resolution images and videos. Existing visual token pruning methods mainly operate after visual encoding, overlooking the substantial computational cost incurred during the encoding stage. To address this issue, we propose EvoPrune, an early-stage visual token pruning method for MLLMs that performs pruning directly during visual encoding. Specifically, EvoPrune employs a layer-wise pruning strategy guided by token similarity, diversity, and attention-based importance to retain the most informative visual tokens at selected encoding layers. Extensive experiments on image and video benchmarks validate the effectiveness of EvoPrune. In particular, on the VideoMME dataset, EvoPrune achieves 2$\times$ inference speedup with less than 1% performance degradation, demonstrating its potential for latency-sensitive MLLM deployment.

Seunghoi Kim, Chen Jin, Henry F. J. Tregidgo, Matteo Figini, Daniel C. Alexander

TL;DR: MPFlow是一种基于整流流的零样本多模态MRI重建框架，通过引入辅助MRI模态（如高质量结构扫描）来提升解剖保真度，无需重新训练生成先验。该方法采用自监督预训练策略PAMRI学习跨模态共享表示，并结合数据一致性和跨模态特征对齐引导采样，有效抑制内在和外在幻觉。在HCP和BraTS数据集上的实验表明，MPFlow仅用20%的采样步骤即可达到扩散基线的图像质量，同时将肿瘤幻觉降低超过15%（分割Dice分数）。

Details

Motivation: 解决零样本MRI重建中单模态无条件先验在严重病态问题下产生幻觉的局限性，利用临床中常规可用的互补MRI采集（如高质量结构扫描）来提升重建可靠性。

Result: 在HCP和BraTS基准测试中，MPFlow在图像质量上匹配扩散基线，仅需20%的采样步骤，并将肿瘤幻觉降低超过15%（以分割Dice分数衡量），实现了更高效可靠的零样本重建。

Insight: 创新点包括：多模态后验引导的流匹配框架，允许在推理时整合辅助模态；自监督预训练策略PAMRI学习跨模态共享表示；结合数据一致性和跨模态特征对齐的系统性采样引导，有效抑制幻觉。这为利用多模态信息提升生成先验的保真度和效率提供了新思路。

Abstract: Zero-shot MRI reconstruction relies on generative priors, but single-modality unconditional priors produce hallucinations under severe ill-posedness. In many clinical workflows, complementary MRI acquisitions (e.g. high-quality structural scans) are routinely available, yet existing reconstruction methods lack mechanisms to leverage this additional information. We propose MPFlow, a zero-shot multi-modal reconstruction framework built on rectified flow that incorporates auxiliary MRI modalities at inference time without retraining the generative prior to improve anatomical fidelity. Cross-modal guidance is enabled by our proposed self-supervised pretraining strategy, Patch-level Multi-modal MR Image Pretraining (PAMRI), which learns shared representations across modalities. Sampling is jointly guided by data consistency and cross-modal feature alignment using pre-trained PAMRI, systematically suppressing intrinsic and extrinsic hallucinations. Extensive experiments on HCP and BraTS show that MPFlow matches diffusion baselines on image quality using only 20% of sampling steps while reducing tumor hallucinations by more than 15% (segmentation dice score). This demonstrates that cross-modal guidance enables more reliable and efficient zero-shot MRI reconstruction.

[55] Glass Segmentation with Fusion of Learned and General Visual Features cs.CVPDF

Risto Ojala, Tristan Ellison, Mo Chen

TL;DR: 本文提出了一种名为LGNet的新型玻璃分割架构，该架构融合了通用视觉特征和任务特定学习特征。它采用双骨干网络，其中通用特征由冻结的DINOv3视觉基础模型提取，任务特定特征由监督训练的Swin模型生成。这些多尺度特征经过残差Squeeze-and-Excitation通道缩减后，输入Mask2Former解码器以生成最终分割掩码。

Details

Motivation: 解决从RGB图像中分割玻璃表面的挑战，因为玻璃作为透明材料明显缺乏视觉特征，而该任务对于场景理解和机器人技术至关重要。

Result: 在四个常用玻璃分割数据集上进行了评估，在多个准确率指标上达到了最先进（SOTA）水平。与之前的最先进方法相比，具有竞争力的推理速度，并且在使用更轻量级的DINOv3骨干变体时超越了它。

Insight: 主要创新点在于提出了一种融合通用基础模型特征（DINOv3）与任务特定模型特征（Swin）的双骨干架构，并通过残差SE通道缩减进行高效特征融合。这种设计利用了基础模型的强大通用表示能力和特定模型的针对性学习能力，以应对玻璃分割中特征稀缺的难题。

Abstract: Glass surface segmentation from RGB images is a challenging task, since glass as a transparent material distinctly lacks visual characteristics. However, glass segmentation is critical for scene understanding and robotics, as transparent glass surfaces must be identified as solid material. This paper presents a novel architecture for glass segmentation, deploying a dual-backbone producing general visual features as well as task-specific learned visual features. General visual features are produced by a frozen DINOv3 vision foundation model, and the task-specific features are generated with a Swin model trained in a supervised manner. Resulting multi-scale feature representations are downsampled with residual Squeeze-and-Excitation Channel Reduction, and fed into a Mask2Former Decoder, producing the final segmentation masks. The architecture was evaluated on four commonly used glass segmentation datasets, achieving state-of-the-art results on several accuracy metrics. The model also has a competitive inference speed compared to the previous state-of-the-art method, and surpasses it when using a lighter DINOv3 backbone variant. The implementation source code and model weights are available at: https://github.com/ojalar/lgnet

[56] QD-PCQA: Quality-Aware Domain Adaptation for Point Cloud Quality Assessment cs.CVPDF

Guohua Zhang, Jian Jin, Meiqin Liu, Chao Yao, Weisi Lin

TL;DR: 本文提出了一种名为QD-PCQA的质量感知域适应框架，用于解决无参考点云质量评估（NR-PCQA）中因标注数据稀缺导致的泛化能力不足问题。该框架通过从标注的图像数据中迁移质量先验知识到未标注的点云数据，包含两个核心策略：排名加权条件对齐（RCA）和质量引导特征增强（QFA），以增强对感知质量排名和特征对齐的关注。

Details

Motivation: NR-PCQA的泛化能力受限于标注点云数据集的稀缺性。由于人类视觉系统（HVS）的感知质量评估独立于媒体类型，从图像中学到的质量先验知识可以迁移到点云。这促使作者采用无监督域适应（UDA）方法，但现有UDA方法常忽略感知质量的关键特性（如对质量排名的敏感性和质量感知特征对齐），因此需要改进。

Result: 广泛的跨域实验表明，QD-PCQA在NR-PCQA任务中显著提高了泛化性能。

Insight: 创新点包括：1）排名加权条件对齐（RCA）策略，在一致质量水平下对齐特征并自适应强调错误排名的样本，以增强感知质量排名意识；2）质量引导特征增强（QFA）策略，通过质量引导风格混合、多层扩展和双域增强模块来增强感知特征对齐。从客观角度看，该方法将图像质量评估的先验知识有效迁移到点云领域，并针对感知质量特性设计了专门的域适应机制，提升了跨域泛化能力。

Abstract: No-Reference Point Cloud Quality Assessment (NR-PCQA) still struggles with generalization, primarily due to the scarcity of annotated point cloud datasets. Since the Human Visual System (HVS) drives perceptual quality assessment independently of media types, prior knowledge on quality learned from images can be repurposed for point clouds. This insight motivates adopting Unsupervised Domain Adaptation (UDA) to transfer quality-relevant priors from labeled images to unlabeled point clouds. However, existing UDA-based PCQA methods often overlook key characteristics of perceptual quality, such as sensitivity to quality ranking and quality-aware feature alignment, thereby limiting their effectiveness. To address these issues, we propose a novel Quality-aware Domain adaptation framework for PCQA, termed QD-PCQA. The framework comprises two main components: i) a Rank-weighted Conditional Alignment (RCA) strategy that aligns features under consistent quality levels and adaptively emphasizes misranked samples to reinforce perceptual quality ranking awareness; and ii) a Quality-guided Feature Augmentation (QFA) strategy, which includes quality-guided style mixup, multi-layer extension, and dual-domain augmentation modules to augment perceptual feature alignment. Extensive cross-domain experiments demonstrate that QD-PCQA significantly improves generalization in NR-PCQA tasks. The code is available at https://github.com/huhu-code/QD-PCQA.

Zehua Fan, Wenqi Lyu, Wenxuan Song, Linge Zhao, Yifei Yang

TL;DR: 本文提出PROSPECT，一种统一的流式视觉语言导航代理，通过语义-空间融合和潜在预测表征学习，将流式视觉-语言-动作策略与潜在预测表征学习相结合。它使用CUT3R作为流式3D基础空间编码器生成长上下文、绝对尺度的空间特征，并通过交叉注意力与SigLIP语义特征融合。训练中引入可学习的流查询令牌来查询流式上下文并预测下一步的2D和3D潜在特征，在冻结的SigLIP和CUT3R教师模型的潜在空间中进行监督。该方法在VLN-CE基准测试和真实机器人部署中展示了最先进的性能和改进的长时程鲁棒性。

Details

Motivation: 尽管多模态大语言模型（MLLMs）推动了零样本端到端视觉语言导航（VLN）的发展，但鲁棒的导航不仅需要语义理解，还需要对环境动态和空间结构进行预测建模。现有方法在长时程鲁棒性和空间建模方面存在不足。

Result: 在VLN-CE基准测试和真实机器人部署中，PROSPECT实现了最先进的性能，并在不同光照条件下展现出改进的长时程鲁棒性。

Insight: 创新点包括：1) 统一的流式导航框架，结合语义（SigLIP）和空间（CUT3R）基础模型特征；2) 引入潜在预测表征学习，通过预测下一步的2D/3D潜在特征（而非原始像素或显式模态）来塑造内部表征，且无推理开销；3) 使用可学习的流查询令牌进行流式上下文查询和预测。这为VLN提供了更鲁棒和高效的解决方案。

Abstract: Multimodal large language models (MLLMs) have advanced zero-shot end-to-end Vision-Language Navigation (VLN), yet robust navigation requires not only semantic understanding but also predictive modeling of environment dynamics and spatial structure. We propose PROSPECT, a unified streaming navigation agent that couples a streaming Vision-Language-Action (VLA) policy with latent predictive representation learning. PROSPECT uses CUT3R as a streaming 3D foundation spatial encoder to produce long-context, absolute-scale spatial features, and fuses them with SigLIP semantic features via cross-attention. During training, we introduce learnable stream query tokens that query the streaming context and predict next-step 2D and 3D latent features (rather than pixels or explicit modalities), supervised in the latent spaces of frozen SigLIP and CUT3R teachers. The predictive branch shapes internal representations without inference overhead. Experiments on VLN-CE benchmarks and real-robot deployment demonstrate state-of-the-art performance and improved long-horizon robustness under diverse lighting. We will release code for the community soon.

[58] DAGE: Dual-Stream Architecture for Efficient and Fine-Grained Geometry Estimation cs.CVPDF

Tuan Duc Ngo, Jiahui Huang, Seoung Wug Oh, Kevin Blackburn-Matzen, Evangelos Kalogerakis

TL;DR: DAGE是一种双流Transformer架构，用于从无标定的多视图/视频输入中高效、精细地估计几何和相机姿态。其主要创新在于通过一个低分辨率流处理下采样帧以构建视图一致表示并高效估计相机，同时一个高分辨率流处理原始图像以保留细节，两者通过轻量级适配器融合，从而独立扩展分辨率和序列长度，支持高达2K的输入并保持实用推理成本。

Details

Motivation: 解决从无标定多视图/视频输入中估计准确、视图一致的几何和相机姿态的挑战，特别是在高空间分辨率和长序列场景下，传统方法难以兼顾全局一致性和精细细节。

Result: DAGE在视频几何估计和多视图重建任务上取得了新的最先进（SOTA）结果，能够生成锐利的深度/点云图、强大的跨视图一致性和准确的姿态。

Insight: 创新点在于通过双流设计解耦全局一致性与精细细节处理，低分辨率流专注于全局视图一致性，高分辨率流保留单帧细节，并通过轻量级适配器实现高效融合，从而在保持推理效率的同时提升几何估计的精度和一致性。

Abstract: Estimating accurate, view-consistent geometry and camera poses from uncalibrated multi-view/video inputs remains challenging - especially at high spatial resolutions and over long sequences. We present DAGE, a dual-stream transformer whose main novelty is to disentangle global coherence from fine detail. A low-resolution stream operates on aggressively downsampled frames with alternating frame/global attention to build a view-consistent representation and estimate cameras efficiently, while a high-resolution stream processes the original images per-frame to preserve sharp boundaries and small structures. A lightweight adapter fuses these streams via cross-attention, injecting global context without disturbing the pretrained single-frame pathway. This design scales resolution and clip length independently, supports inputs up to 2K, and maintains practical inference cost. DAGE delivers sharp depth/pointmaps, strong cross-view consistency, and accurate poses, establishing new state-of-the-art results for video geometry estimation and multi-view reconstruction.

[59] Seeing as Experts Do: A Knowledge-Augmented Agent for Open-Set Fine-Grained Visual Understanding cs.CVPDF

Junhan Chen, Zilu Zhou, Yujun Tong, Dongliang Chang, Yitao Luo

TL;DR: 本文提出了知识增强细粒度推理代理（KFRA），一个将细粒度视觉理解转化为证据驱动推理的统一框架。KFRA通过模拟专家分析的三阶段闭环推理流程运作：首先进行开放词汇检测和网络规模检索以生成类别假设；然后通过全局到局部的聚焦机制对齐文本知识与视觉证据，进行判别性区域定位；最后将所有多模态证据整合到大型多模态模型中进行可解释推理。

Details

Motivation: 现有细粒度视觉理解方法受限于封闭集分类法和单标签预测，在开放集或上下文依赖条件下性能显著下降。研究旨在将静态分类转向知识增强推理，使模型不仅能识别，还能提供推理依据。

Result: 在构建的FGExpertBench基准测试（涵盖六个知识维度）上进行了广泛实验。结果表明，KFRA在推理准确性上持续超越独立的大型多模态模型和现有代理框架，实现了高达19%的准确率提升，并在开放集细粒度视觉理解中提供了基于证据的可解释性。

Insight: 核心创新在于提出了一个检索-定位耦合机制，将检索到的知识转化为空间上可验证的证据，从而实现了事实性、可解释性且与任务无关的推理。这不同于将检索和推理视为独立过程的现有代理方法。框架设计模拟了专家的分析流程，增强了开放场景下的泛化能力。

Abstract: Fine-grained visual understanding is shifting from static classification to knowledge-augmented reasoning, where models must justify as well as recognise. Existing approaches remain limited by closed-set taxonomies and single-label prediction, leading to significant degradation under open-set or context-dependent conditions. We present the Knowledge-Augmented Fine-Grained Reasoning Agent (KFRA), a unified framework that transforms fine-grained perception into evidence-driven reasoning. KFRA operates through a three-stage closed reasoning loop that emulates expert analysis. It first performs open-vocabulary detection and web-scale retrieval to generate category hypotheses. It then conducts discriminative regions localisation by aligning textual knowledge with visual evidence through a global-to-local focusing mechanism. Finally, it integrates all multimodal evidence within a large multimodal model to perform interpretable reasoning. Unlike existing agents that treat retrieval and reasoning as independent processes, KFRA establishes a retrieval-grounding coupling that converts retrieved knowledge into spatially grounded evidence for verification. This design enables factual, interpretable, and task-agnostic reasoning across diverse fine-grained scenarios. To evaluate this capability, we construct FGExpertBench, a benchmark designed to assess reasoning depth and cross-task generalisation across six knowledge dimensions. Extensive experiments demonstrate that KFRA consistently surpasses both standalone large multimodal models and current agent frameworks, achieving up to 19 percent improvement in reasoning accuracy and delivering evidence-grounded interpretability in open-set fine-grained visual understanding.

[60] Separators in Enhancing Autoregressive Pretraining for Vision Mamba cs.CV | cs.AIPDF

Hanpeng Liu, Zidan Wang, Shuoxi Zhang, Kaiyuan Gao, Kun He

TL;DR: 本文提出了一种名为STAR的创新自回归预训练方法，用于增强Vision Mamba模型。该方法通过引入分隔符来区分不同图像，从而将输入序列长度扩展至四倍，同时保持图像原始尺寸。在ImageNet-1k数据集上，STAR-B模型达到了83.5%的准确率，展现了其在视觉任务中的竞争力。

Details

Motivation: 当前的自回归预训练方法受限于短序列任务，未能充分利用Mamba在处理长序列方面的优势。本文旨在解决这一限制，通过扩展输入序列长度来更好地利用Mamba的长程依赖处理能力。

Result: 在ImageNet-1k基准测试中，STAR-B模型取得了83.5%的准确率，这在Vision Mamba模型中表现优异，具有高度竞争力。

Insight: 创新点在于引入分隔符（STAR）来界定不同图像，从而在不增加图像尺寸的情况下显著扩展输入序列长度。这为视觉模型通过更有效地利用长程依赖来提升性能提供了新思路。

Abstract: The state space model Mamba has recently emerged as a promising paradigm in computer vision, attracting significant attention due to its efficient processing of long sequence tasks. Mamba’s inherent causal mechanism renders it particularly suitable for autoregressive pretraining. However, current autoregressive pretraining methods are constrained to short sequence tasks, failing to fully exploit Mamba’s prowess in handling extended sequences. To address this limitation, we introduce an innovative autoregressive pretraining method for Vision Mamba that substantially extends the input sequence length. We introduce new \textbf{S}epara\textbf{T}ors for \textbf{A}uto\textbf{R}egressive pretraining to demarcate and differentiate between different images, known as \textbf{STAR}. Specifically, we insert identical separators before each image to demarcate its inception. This strategy enables us to quadruple the input sequence length of Vision Mamba while preserving the original dimensions of the dataset images. Employing this long sequence pretraining technique, our STAR-B model achieved an impressive accuracy of 83.5% on ImageNet-1k, which is highly competitive in Vision Mamba. These results underscore the potential of our method in enhancing the performance of vision models through improved leveraging of long-range dependencies.

[61] Vector-Quantized Soft Label Compression for Dataset Distillation cs.CVPDF

Ali Abbasi, Ashkan Shahbazi, Hamed Pirsiavash, Soheil Kolouri

TL;DR: 本文提出了一种用于数据集蒸馏的向量量化软标签压缩方法，通过分析软标签在存储成本中的主导地位，引入向量量化自编码器（VQAE）来压缩软标签，从而在保持蒸馏数据有效性的同时显著降低存储开销。

Details

Motivation: 数据集蒸馏中，软标签及其增强版本是存储成本的主要贡献者，尤其是在大规模分类任务（如ImageNet-1K）中，但现有方法往往忽视其存储和通信开销。

Result: 在ImageNet-1K等视觉和语言蒸馏基准测试中，VQAE方法相比RDED、LPLD、SRE2L和CDA基线实现了30-40倍的额外压缩，同时保持了超过90%的原始性能。

Insight: 创新点在于首次对数据集蒸馏框架的比特需求进行严格分析，并引入向量量化自编码器来高效压缩软标签，这为降低蒸馏数据存储成本提供了新思路，同时保持了模型性能。

Abstract: Dataset distillation is an emerging technique for reducing the computational and storage costs of training machine learning models by synthesizing a small, informative subset of data that captures the essential characteristics of a much larger dataset. Recent methods pair synthetic samples and their augmentations with soft labels from a teacher model, enabling student models to generalize effectively despite the small size of the distilled dataset. While soft labels are critical for effective distillation, the storage and communication overhead they incur, especially when accounting for augmentations, is often overlooked. In practice, each distilled sample is associated with multiple soft labels, making them the dominant contributor to storage costs, particularly in large-class settings such as ImageNet-1K. In this paper, we present a rigorous analysis of bit requirements across dataset distillation frameworks, quantifying the storage demands of both distilled samples and their soft labels. To address the overhead, we introduce a vector-quantized autoencoder (VQAE) for compressing soft labels, achieving substantial compression while preserving the effectiveness of the distilled data. We validate our method on both vision and language distillation benchmarks. On ImageNet-1K, our proposed VQAE achieves 30–40x additional compression over RDED, LPLD, SRE2L, and CDA baselines while retaining over $90%$ of their original performance.

[62] From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning cs.CV | cs.AIPDF

Ruilin Luo, Chufan Shi, Yizhen Zhang, Cheng Yang, Songtao Jiang

TL;DR: 本文研究了多模态大推理模型（MLRMs）的冷启动初始化阶段，引入了视觉注意力分数（VAS）作为衡量模型关注视觉令牌程度的指标，并发现推理性能与VAS高度相关。研究发现，多模态冷启动未能有效提升VAS，导致注意力分布接近基础模型，而纯文本冷启动则能显著提升VAS，这一现象被称为‘懒惰注意力定位’。基于此，作者提出了无需训练的推理干预方法，能直接提升性能1-2%。进一步，作者提出了注意力引导的视觉锚定与反思（AVAR）框架，通过视觉锚定数据合成、注意力引导目标和奖励塑造，在Qwen2.5-VL-7B模型上实现了平均7.0%的性能提升。

Details

Motivation: 多模态大推理模型的冷启动初始化阶段机制尚不明确，需要深入分析以提升模型的多模态推理能力。

Result: 在7个多模态推理基准测试中，AVAR框架应用于Qwen2.5-VL-7B模型实现了平均7.0%的性能增益，消融实验证实了各组件对整体增益的逐步贡献。

Insight: 创新点在于引入VAS量化视觉注意力，揭示了‘懒惰注意力定位’现象，并提出了无需训练的注意力干预方法以及AVAR冷启动框架，通过视觉锚定和注意力引导策略有效提升了多模态推理性能。

Abstract: The cold-start initialization stage plays a pivotal role in training Multimodal Large Reasoning Models (MLRMs), yet its mechanisms remain insufficiently understood. To analyze this stage, we introduce the Visual Attention Score (VAS), an attention-based metric that quantifies how much a model attends to visual tokens. We find that reasoning performance is strongly correlated with VAS (r=0.9616): models with higher VAS achieve substantially stronger multimodal reasoning. Surprisingly, multimodal cold-start fails to elevate VAS, resulting in attention distributions close to the base model, whereas text-only cold-start leads to a clear increase. We term this counter-intuitive phenomenon Lazy Attention Localization. To validate its causal role, we design training-free interventions that directly modulate attention allocation during inference, performance gains of 1$-$2% without any retraining. Building on these insights, we further propose Attention-Guided Visual Anchoring and Reflection (AVAR), a comprehensive cold-start framework that integrates visual-anchored data synthesis, attention-guided objectives, and visual-anchored reward shaping. Applied to Qwen2.5-VL-7B, AVAR achieves an average gain of 7.0% across 7 multimodal reasoning benchmarks. Ablation studies further confirm that each component of AVAR contributes step-wise to the overall gains. The code, data, and models are available at https://github.com/lrlbbzl/Qwen-AVAR.

[63] DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models cs.CVPDF

Yangfu Li, Hongjian Zhan, Jiawei Chen, Yuning Gong, Qi Liu

TL;DR: DeepScan是一个无需训练、用于提升大型视觉语言模型（LVLMs）视觉基础推理能力的框架。它受人类在嘈杂环境中通过自底向上方式定位视觉证据的启发，结合了分层扫描、重新聚焦和证据增强推理三个模块，以提取多尺度证据、优化证据视图并聚合信息，从而生成准确且可解释的答案。

Details

Motivation: 现有方法通常追求一次性定位完整证据，容易受到干扰上下文的影响。本文旨在模仿人类自底向上的视觉推理过程，通过分层、协作的方式更鲁棒地定位关键视觉线索并进行推理，以解决LVLMs在细粒度视觉理解任务中证据定位不准确的问题。

Result: 实验表明，DeepScan显著提升了LVLMs在多种视觉任务上的性能，特别是在细粒度视觉理解方面。当与Qwen2.5-VL-7B集成时，在V*基准测试上达到了90.6%的整体准确率。此外，DeepScan无需额外适应成本，就能为不同架构和规模的LVLMs带来一致的性能提升。

Insight: 论文的创新点在于提出了一种无需训练、自底向上的分层证据提取与优化框架。其核心是将视觉证据定位过程分解为局部线索探索、多尺度证据提取、通过LVLMs与视觉专家协作优化视图，以及利用混合证据记忆进行信息聚合，这提高了模型在复杂场景下的鲁棒性和可解释性。

Abstract: Humans can robustly localize visual evidence and provide grounded answers even in noisy environments by identifying critical cues and then relating them to the full context in a bottom-up manner. Inspired by this, we propose DeepScan, a training-free framework that combines Hierarchical Scanning, Refocusing, and Evidence-Enhanced Reasoning for visually grounded reasoning in Large Vision-Language Models (LVLMs). Unlike existing methods that pursue one-shot localization of complete evidence, Hierarchical Scanning performs local cue exploration and multi-scale evidence extraction to recover evidence in a bottom-up manner, effectively mitigating the impacts of distractive context. Refocusing then optimizes the localized evidence view through collaboration of LVLMs and visual experts. Finally, Evidence-Enhanced Reasoning aggregates multi-granular views via a hybrid evidence memory and yields accurate and interpretable answers. Experimental results demonstrate that DeepScan significantly boosts LVLMs in diverse visual tasks, especially in fine-grained visual understanding. It achieves 90.6% overall accuracy on V* when integrated with Qwen2.5-VL-7B. Moreover, DeepScan provides consistent improvements for LVLMs across various architectures and model scales without additional adaptation cost.

[64] Bridging Human Evaluation to Infrared and Visible Image Fusion cs.CVPDF

Jinyuan Liu, Xingyuan Li, Qingyun Mei, Haoyuan Xu, Zhiying Jiang

TL;DR: 本文提出了一种反馈强化框架，将人类评估与红外和可见光图像融合（IVIF）任务相结合。为了解决现有方法依赖手工损失和客观指标、与人类视觉偏好不一致的问题，作者构建了首个大规模人类反馈数据集，并基于此训练了一个奖励模型来量化感知质量。通过组相对策略优化微调融合网络，该方法在多个基准测试中取得了最先进的性能，使融合图像更符合人类审美。

Details

Motivation: 当前的红外与可见光图像融合方法主要优化手工设计的损失函数和客观指标，导致融合结果与人类视觉偏好不符，这限制了其在安全监控和驾驶辅助等依赖人类感知环境中的应用。

Result: 该方法在多个基准测试（如TNO、RoadScene、MSRS）上取得了最先进的（SOTA）性能，其融合图像在人类评估中表现出更好的感知质量。

Insight: 创新点包括：1）构建了首个大规模、多维度的IVIF人类反馈数据集，并利用微调的大语言模型进行增强；2）设计了领域特定的奖励函数并训练奖励模型来量化感知质量；3）采用组相对策略优化来微调融合网络，将人类偏好直接融入模型训练过程。这为图像融合任务提供了从人类主观评价到模型优化的有效桥梁。

Abstract: Infrared and visible image fusion (IVIF) integrates complementary modalities to enhance scene perception. Current methods predominantly focus on optimizing handcrafted losses and objective metrics, often resulting in fusion outcomes that do not align with human visual preferences. This challenge is further exacerbated by the ill-posed nature of IVIF, which severely limits its effectiveness in human perceptual environments such as security surveillance and driver assistance systems. To address these limitations, we propose a feedback reinforcement framework that bridges human evaluation to infrared and visible image fusion. To address the lack of human-centric evaluation metrics and data, we introduce the first large-scale human feedback dataset for IVIF, containing multidimensional subjective scores and artifact annotations, and enriched by a fine-tuned large language model with expert review. Based on this dataset, we design a domain-specific reward function and train a reward model to quantify perceptual quality. Guided by this reward, we fine-tune the fusion network through Group Relative Policy Optimization, achieving state-of-the-art performance that better aligns fused images with human aesthetics. Code is available at https://github.com/ALKA-Wind/EVAFusion.

[65] UniSync: Towards Generalizable and High-Fidelity Lip Synchronization for Challenging Scenarios cs.CVPDF

Ruidi Fan, Yang Zhou, Siyuan Wang, Tian Yu, Yutong Jiang

TL;DR: 本文提出UniSync，一个统一的唇形同步框架，旨在解决现有方法在真实多样化场景下的局限性。它结合了无掩码的姿势锚定训练策略来保持头部运动和避免颜色伪影，以及基于掩码的一致性推理来确保结构精度和混合平滑性，并通过在小规模多样化视频上微调获得优异的领域适应性。

Details

Motivation: 现有唇形同步方法存在根本缺陷：基于掩码的方法存在局部颜色不一致问题，而无掩码方法则难以处理全局背景纹理错位，且大多数方法难以应对风格化头像、面部遮挡和极端光照等多样化真实场景。

Result: 在涵盖人脸和风格化头像等多种应用场景的RealWorld-LipSync基准测试中，广泛的实验表明UniSync显著优于最先进的方法。

Insight: 创新点在于提出了一种统一框架，通过结合掩码和无掩码方法的优势（训练与推理策略解耦）并利用紧凑多样化视频进行微调，实现了高保真度、强泛化能力和对复杂边缘案例的有效处理，推动了唇形同步技术向真正可泛化、生产就绪的方向发展。

Abstract: Lip synchronization aims to generate realistic talking videos that match given audio, which is essential for high-quality video dubbing. However, current methods have fundamental drawbacks: mask-based approaches suffer from local color discrepancies, while mask-free methods struggle with global background texture misalignment. Furthermore, most methods struggle with diverse real-world scenarios such as stylized avatars, face occlusion, and extreme lighting conditions. In this paper, we propose UniSync, a unified framework designed for achieving high-fidelity lip synchronization in diverse scenarios. Specifically, UniSync uses a mask-free pose-anchored training strategy to keep head motion and eliminate synthesis color artifacts, while employing mask-based blending consistent inference to ensure structural precision and smooth blending. Notably, fine-tuning on compact but diverse videos empowers our model with exceptional domain adaptability, handling complex corner cases effectively. We also introduce the RealWorld-LipSync benchmark to evaluate models under real-world demands, which covers diverse application scenarios including both human faces and stylized avatars. Extensive experiments demonstrate that UniSync significantly outperforms state-of-the-art methods, advancing the field towards truly generalizable and production-ready lip synchronization.

[66] DISC: Dense Integrated Semantic Context for Large-Scale Open-Set Semantic Mapping cs.CV | cs.ROPDF

Felix Igelbrink, Lennart Niecksch, Martin Atzmueller, Joachim Hertzberg

TL;DR: 本文提出DISC（密集集成语义上下文）方法，用于大规模开放集语义建图，通过单次距离加权提取机制直接从视觉Transformer中间层获取高保真CLIP嵌入，避免了传统图像裁剪的延迟和域偏移问题，并构建了完全GPU加速的架构以实现实时体素级实例优化。

Details

Motivation: 现有开放集语义建图方法以实例为中心，依赖裁剪式特征提取，导致上下文缺失且计算成本高，限制了语言驱动机器人感知的效率和精度。

Result: 在Replica、ScanNet和基于Habitat-Matterport 3D的新大规模建图数据集HM3DSEM上的评估表明，DISC在语义准确性和查询检索方面显著超越当前最先进的零样本方法，实现了实时性能。

Insight: 创新点包括单次距离加权特征提取机制，直接从Transformer中间层获取掩码对齐的语义表示，以及完全GPU加速的在线体素级优化架构，提升了大规模场景下的可扩展性和实时性。

Abstract: Open-set semantic mapping enables language-driven robotic perception, but current instance-centric approaches are bottlenecked by context-depriving and computationally expensive crop-based feature extraction. To overcome this fundamental limitation, we introduce DISC (Dense Integrated Semantic Context), featuring a novel single-pass, distance-weighted extraction mechanism. By deriving high-fidelity CLIP embeddings directly from the vision transformer’s intermediate layers, our approach eliminates the latency and domain-shift artifacts of traditional image cropping, yielding pure, mask-aligned semantic representations. To fully leverage these features in large-scale continuous mapping, DISC is built upon a fully GPU-accelerated architecture that replaces periodic offline processing with precise, on-the-fly voxel-level instance refinement. We evaluate our approach on standard benchmarks (Replica, ScanNet) and a newly generated large-scale-mapping dataset based on Habitat-Matterport 3D (HM3DSEM) to assess scalability across complex scenes in multi-story buildings. Extensive evaluations demonstrate that DISC significantly surpasses current state-of-the-art zero-shot methods in both semantic accuracy and query retrieval, providing a robust, real-time capable framework for robotic deployment. The full source code, data generation and evaluation pipelines will be made available at https://github.com/DFKI-NI/DISC.

Radia Daci, Vito Renò, Cosimo Patruno, Angelo Cardellicchio, Abdelmalik Taleb-Ahmed

TL;DR: 本文提出了一种名为CMDR-IAD的轻量级、模态灵活的无监督框架，用于2D+3D多模态以及单模态（仅2D或仅3D）工业异常检测。该框架结合了双向2D↔3D跨模态映射来建模外观-几何一致性，以及双分支重建来独立捕捉正常纹理和几何结构，并通过一种两部分的融合策略整合这些线索，以实现稳定且精确的异常定位。

Details

Motivation: 现有的无监督多模态工业异常检测方法通常依赖于记忆库、师生架构或脆弱的融合方案，在噪声深度、弱纹理或模态缺失情况下鲁棒性有限。本文旨在解决这些问题，提出一个更鲁棒和灵活的框架。

Result: 在MVTec 3D-AD基准测试中，CMDR-IAD无需记忆库即达到了最先进的性能，取得了97.3%的图像级AUROC、99.6%的像素级AUROC和97.6%的AUPRO。在一个真实世界的聚氨酯切割数据集上，仅3D的变体也取得了92.6%的图像级AUROC和92.5%的像素级AUROC。

Insight: 创新点在于结合了双向跨模态映射与双分支重建，并提出了可靠性门控映射异常和置信度加权重建异常的两部分融合策略。这提供了建模外观-几何一致性、独立处理纹理与几何、以及自适应融合多模态线索的有效方法，增强了在深度稀疏或低纹理区域的鲁棒性。

Abstract: Multimodal industrial anomaly detection benefits from integrating RGB appearance with 3D surface geometry, yet existing \emph{unsupervised} approaches commonly rely on memory banks, teacher-student architectures, or fragile fusion schemes, limiting robustness under noisy depth, weak texture, or missing modalities. This paper introduces \textbf{CMDR-IAD}, a lightweight and modality-flexible unsupervised framework for reliable anomaly detection in 2D+3D multimodal as well as single-modality (2D-only or 3D-only) settings. \textbf{CMDR-IAD} combines bidirectional 2D$\leftrightarrow$3D cross-modal mapping to model appearance-geometry consistency with dual-branch reconstruction that independently captures normal texture and geometric structure. A two-part fusion strategy integrates these cues: a reliability-gated mapping anomaly highlights spatially consistent texture-geometry discrepancies, while a confidence-weighted reconstruction anomaly adaptively balances appearance and geometric deviations, yielding stable and precise anomaly localization even in depth-sparse or low-texture regions. On the MVTec 3D-AD benchmark, CMDR-IAD achieves state-of-the-art performance while operating without memory banks, reaching 97.3% image-level AUROC (I-AUROC), 99.6% pixel-level AUROC (P-AUROC), and 97.6% AUPRO. On a real-world polyurethane cutting dataset, the 3D-only variant attains 92.6% I-AUROC and 92.5% P-AUROC, demonstrating strong effectiveness under practical industrial conditions. These results highlight the framework’s robustness, modality flexibility, and the effectiveness of the proposed fusion strategies for industrial visual inspection. Our source code is available at https://github.com/ECGAI-Research/CMDR-IAD/

[68] Spatial Causal Prediction in Video cs.CVPDF

Yanguang Zhao, Jie Yang, Shengqiong Wu, Shutong Hu, Hongbo Qiu

TL;DR: 本文提出了一种名为空间因果预测（SCP）的新任务范式，旨在评估模型在视频中超越观察进行空间因果推理的能力，并构建了包含2500个问答对的SCP-Bench基准数据集。通过对23个先进模型的全面实验，揭示了模型与人类性能之间的显著差距、有限的时间外推能力以及薄弱的因果基础。

Details

Motivation: 现有研究主要评估模型在可见时空理解上的能力，忽视了其推断未见过去或未来空间状态的能力，而空间推理对于自动驾驶和机器人等实际应用至关重要。

Result: 在SCP-Bench基准上的实验表明，当前最先进模型与人类性能存在显著差距，表现出有限的时间外推和薄弱的因果基础。

Insight: 创新点在于提出了空间因果预测任务范式及相应基准，强调了超越观察的因果推理能力评估；从客观分析看，其系统性的基准构建和影响因素分析为提升空间因果智能提供了方向，如感知增强和推理引导策略。

Abstract: Spatial reasoning, the ability to understand spatial relations, causality, and dynamic evolution, is central to human intelligence and essential for real-world applications such as autonomous driving and robotics. Existing studies, however, primarily assess models on visible spatio-temporal understanding, overlooking their ability to infer unseen past or future spatial states. In this work, we introduce Spatial Causal Prediction (SCP), a new task paradigm that challenges models to reason beyond observation and predict spatial causal outcomes. We further construct SCP-Bench, a benchmark comprising 2,500 QA pairs across 1,181 videos spanning diverse viewpoints, scenes, and causal directions, to support systematic evaluation. Through comprehensive experiments on {23} state-of-the-art models, we reveal substantial gaps between human and model performance, limited temporal extrapolation, and weak causal grounding. We further analyze key factors influencing performance and propose perception-enhancement and reasoning-guided strategies toward advancing spatial causal intelligence. The project page is https://guangstrip.github.io/SCP-Bench.

[69] Towards Generalized Multimodal Homography Estimation cs.CV | cs.AIPDF

Jinkun You, Jiaxin Cheng, Jie Zhang, Yicong Zhou

TL;DR: 本文提出了一种用于广义多模态单应性估计的训练数据合成方法，通过从单张输入图像生成具有真实偏移量的未对齐图像对，并结合一个利用跨尺度信息并解耦颜色信息的网络，以提高模型在不同模态间的泛化能力。

Details

Motivation: 现有监督和无监督单应性估计方法依赖于特定模态的图像对，在未见模态上性能显著下降，因此需要解决跨模态泛化问题。

Result: 大量实验表明，所提出的训练数据合成方法提升了泛化性能，且设计的网络在估计准确性上有效，在多个基准测试中表现出改进的鲁棒性。

Insight: 创新点在于通过单图像合成多样化纹理和颜色的训练数据以增强泛化，以及网络设计中解耦颜色信息与特征表示以提升跨模态估计精度。

Abstract: Supervised and unsupervised homography estimation methods depend on image pairs tailored to specific modalities to achieve high accuracy. However, their performance deteriorates substantially when applied to unseen modalities. To address this issue, we propose a training data synthesis method that generates unaligned image pairs with ground-truth offsets from a single input image. Our approach renders the image pairs with diverse textures and colors while preserving their structural information. These synthetic data empower the trained model to achieve greater robustness and improved generalization across various domains. Additionally, we design a network to fully leverage cross-scale information and decouple color information from feature representations, thus improving estimation accuracy. Extensive experiments show that our training data synthesis method improves generalization performance. The results also confirm the effectiveness of the proposed network.

[70] ProFound: A moderate-sized vision foundation model for multi-task prostate imaging cs.CVPDF

Yipei Wang, Yinsong Xu, Weixi Yi, Shaheer Ullah Saeed, Natasha Thorley

TL;DR: 本文提出了ProFound，一个针对前列腺多参数MRI的领域专用视觉基础模型。该模型在5000名患者的22,000多个3D MRI数据上进行自监督预训练，并在11项下游临床任务上进行了系统评估，包括癌症检测、分级和分割等。实验表明，微调后的ProFound在性能上优于或与现有SOTA专用模型及医学视觉基础模型相当。

Details

Motivation: 解决前列腺癌多参数MRI自动化分析任务对专家标注依赖性强、难以规模化应用深度学习的问题，旨在开发一个通用的、减少对大型任务特定标注数据集依赖的基础模型。

Result: 在超过3000名独立患者的11项下游临床任务（如癌症检测、Gleason分级、病灶定位、腺体体积估计和分割）上进行了评估。微调后的ProFound在相同数据上训练/微调后，性能持续优于或与最先进的专用模型及现有医学视觉基础模型保持竞争力。

Insight: 创新点在于构建了一个针对前列腺MRI的领域专用、中等规模的基础模型，并通过大规模、多机构的自监督预训练实现了对多种下游任务的强大泛化能力。其方法表明，在特定医学领域构建专用基础模型，而非依赖通用模型，可以有效提升多任务性能并减少对大量任务特定标注的依赖。

Abstract: Many diagnostic and therapeutic clinical tasks for prostate cancer increasingly rely on multi-parametric MRI. Automating these tasks is challenging because they necessitate expert interpretations, which are difficult to scale to capitalise on modern deep learning. Although modern automated systems achieve expert-level performance in isolated tasks, their general clinical utility remains limited by the requirement of large task-specific labelled datasets. In this paper, we present ProFound, a domain-specialised vision foundation model for volumetric prostate mpMRI. ProFound is pre-trained using several variants of self-supervised approaches on a diverse, multi-institutional collection of 5,000 patients, with a total of over 22,000 unique 3D MRI volumes (over 1,800,000 2D image slices). We conducted a systematic evaluation of ProFound across a broad spectrum of $11$ downstream clinical tasks on over 3,000 independent patients, including prostate cancer detection, Gleason grading, lesion localisation, gland volume estimation, zonal and surrounding structure segmentation. Experimental results demonstrate that finetuned ProFound consistently outperforms or remains competitive with state-of-the-art specialised models and existing medical vision foundation models trained/finetuned on the same data.

[71] BLOCK: An Open-Source Bi-Stage MLLM Character-to-Skin Pipeline for Minecraft cs.CV | cs.AIPDF

Hengquan Guo

TL;DR: BLOCK是一个开源的、两阶段的角色到皮肤生成管道，能够从任意角色概念生成像素完美的Minecraft皮肤。它首先通过精心设计提示模板的大规模多模态模型合成3D预览图，然后利用微调的FLUX.2模型将预览解码为皮肤图集图像。

Details

Motivation: 解决从任意角色概念直接生成高质量、像素完美的Minecraft皮肤这一挑战性问题，传统方法难以保证细节一致性和像素级精度。

Result: 论文提出的方法能够生成像素完美的皮肤，并开源了所有提示模板和微调权重以支持可复现的生成。

Insight: 创新点包括：将复杂任务分解为3D预览合成与皮肤解码两阶段；设计了EvolveLoRA，一种渐进式LoRA课程学习方法，通过从先前适配器初始化每个阶段来提高稳定性和效率；以及精心设计的提示与参考模板来确保预览的一致性。

Abstract: We present \textbf{BLOCK}, an open-source bi-stage character-to-skin pipeline that generates pixel-perfect Minecraft skins from arbitrary character concepts. BLOCK decomposes the problem into (i) a \textbf{3D preview synthesis stage} driven by a large multimodal model (MLLM) with a carefully designed prompt-and-reference template, producing a consistent dual-panel (front/back) oblique-view Minecraft-style preview; and (ii) a \textbf{skin decoding stage} based on a fine-tuned FLUX.2 model that translates the preview into a skin atlas image. We further propose \textbf{EvolveLoRA}, a progressive LoRA curriculum (text-to-image $\rightarrow$ image-to-image $\rightarrow$ preview-to-skin) that initializes each phase from the previous adapter to improve stability and efficiency. BLOCK is released with all prompt templates and fine-tuned weights to support reproducible character-to-skin generation.

[72] Scaling Dense Event-Stream Pretraining from Visual Foundation Models cs.CVPDF

Zhiwen Chen, Junhui Hou, Zhiyu Zhu, Jinjian Wu, Guangming Shi

TL;DR: 本文提出了一种新颖的自监督预训练方法，通过将视觉基础模型（VFMs）的知识蒸馏到事件流数据上，以大规模学习精细、通用的密集事件表示。该方法构建了一个大规模的同步图像-事件数据集，并设计了一种结构感知的蒸馏损失，利用VFMs提供的语义结构信息来对齐图像和事件域，从而克服了二者在稀疏性和粒度上的不匹配问题。

Details

Motivation: 从非规则的事件流中学习通用且精细的表示至关重要，但面临数据集规模、语义丰富性和应用范围因繁重标注而难以扩展的困境。现有蒸馏方法由于图像和事件域在稀疏性和粒度上的固有差异，容易导致事件表示在高分辨率下发生语义崩溃。

Result: 广泛的实验表明，该方法在下游基准测试中取得了巨大飞跃，显著超越了传统方法和现有的预训练技术。具体表现为增强了泛化能力、提升了数据效率并提高了可迁移性。

Insight: 核心创新点在于将跨模态对齐的目标扩展到由视觉基础模型（VFMs）提供的现成语义结构上，并设计了结构感知的蒸馏损失。这为对齐提供了更广阔的感知野和更强的监督信号，从而优化了密集事件表示，有效解决了跨模态领域不匹配导致的语义崩溃问题。

Abstract: Learning versatile, fine-grained representations from irregular event streams is pivotal yet nontrivial, primarily due to the heavy annotation that hinders scalability in dataset size, semantic richness, and application scope. To mitigate this dilemma, we launch a novel self-supervised pretraining method that distills visual foundation models (VFMs) to push the boundaries of event representation at scale. Specifically, we curate an extensive synchronized image-event collection to amplify cross-modal alignment. Nevertheless, due to inherent mismatches in sparsity and granularity between image-event domains, existing distillation paradigms are prone to semantic collapse in event representations, particularly at high resolutions. To bridge this gap, we propose to extend the alignment objective to semantic structures provided off-the-shelf by VFMs, indicating a broader receptive field and stronger supervision. The key ingredient of our method is a structure-aware distillation loss that grounds higher-quality image-event correspondences for alignment, optimizing dense event representations. Extensive experiments demonstrate that our approach takes a great leap in downstream benchmarks, significantly surpassing traditional methods and existing pretraining techniques. This breakthrough manifests in enhanced generalization, superior data efficiency and elevated transferability.

[73] GeoSeg: Training-Free Reasoning-Driven Segmentation in Remote Sensing Imagery cs.CV | cs.AIPDF

Lifan Jiang, Yuhang Pei, oxi Wu, Yan Zhao, Tianrun Wu

TL;DR: 本文提出GeoSeg，一种无需训练、零样本的推理驱动遥感图像分割框架，通过偏差感知坐标细化和双路提示机制，将多模态大语言模型的推理能力与精确定位相结合，并发布了GeoSeg-Bench诊断基准。

Details

Motivation: 解决遥感图像中由于推理导向数据成本高昂和俯视视角等特定领域挑战，缺乏通用化推理驱动分割方案的问题。

Result: 在GeoSeg-Bench基准测试中，GeoSeg始终优于所有基线模型，广泛的消融实验证实了各模块的有效性和必要性。

Insight: 创新点在于通过训练免费的方式绕过监督瓶颈，结合偏差感知坐标校正系统性地解决定位偏移，以及双路提示机制融合语义意图与细粒度空间线索，实现遥感场景的开放词汇分割。

Abstract: Recent advances in MLLMs are reframing segmentation from fixed-category prediction to instruction-grounded localization. While reasoning based segmentation has progressed rapidly in natural scenes, remote sensing lacks a generalizable solution due to the prohibitive cost of reasoning-oriented data and domain-specific challenges like overhead viewpoints. We present GeoSeg, a zero-shot, training-free framework that bypasses the supervision bottleneck for reasoning-driven remote sensing segmentation. GeoSeg couples MLLM reasoning with precise localization via: (i) bias-aware coordinate refinement to correct systematic grounding shifts and (ii) a dual-route prompting mechanism to fuse semantic intent with fine-grained spatial cues. We also introduce GeoSeg-Bench, a diagnostic benchmark of 810 image–query pairs with hierarchical difficulty levels. Experiments show that GeoSeg consistently outperforms all baselines, with extensive ablations confirming the effectiveness and necessity of each component.

[74] RIVER: A Real-Time Interaction Benchmark for Video LLMs cs.CVPDF

Yansong Shi, Qingsong Zhao, Tianxiang Jiang, Xiangyu Zeng, Yi Wang

TL;DR: 本文提出了RIVER Bench，一个用于评估在线视频理解能力的实时交互基准测试，包含回顾记忆、实时感知和前瞻预测任务，旨在模拟交互式对话而非一次性处理整个视频。

Details

Motivation: 现有多模态大语言模型大多采用离线处理范式，缺乏实时交互能力，阻碍了实时视频理解的发展，因此需要专门的基准来评估和推动在线视频交互模型。

Result: 评估显示离线模型在单次问答任务上表现良好，但在实时处理方面存在困难；论文提出的通用改进方法能帮助模型更灵活地进行实时交互。

Insight: 创新点在于设计了专注于实时交互的基准框架，通过模拟对话式任务（如记忆、感知和预测）来评估模型，并提出了提升模型长期记忆和未来感知能力的通用方法，推动了实时视频理解领域的发展。

Abstract: The rapid advancement of multimodal large language models has demonstrated impressive capabilities, yet nearly all operate in an offline paradigm, hindering real-time interactivity. Addressing this gap, we introduce the Real-tIme Video intERaction Bench (RIVER Bench), designed for evaluating online video comprehension. RIVER Bench introduces a novel framework comprising Retrospective Memory, Live-Perception, and Proactive Anticipation tasks, closely mimicking interactive dialogues rather than responding to entire videos at once. We conducted detailed annotations using videos from diverse sources and varying lengths, and precisely defined the real-time interactive format. Evaluations across various model categories reveal that while offline models perform well in single question-answering tasks, they struggle with real-time processing. Addressing the limitations of existing models in online video interaction, especially their deficiencies in long-term memory and future perception, we proposed a general improvement method that enables models to interact with users more flexibly in real time. We believe this work will significantly advance the development of real-time interactive video understanding models and inspire future research in this emerging field. Datasets and code are publicly available at https://github.com/OpenGVLab/RIVER.

[75] When Visual Evidence is Ambiguous: Pareidolia as a Diagnostic Probe for Vision Models cs.CV | cs.AIPDF

Qianpu Chen, Derya Soydaner, Rob Saunders

TL;DR: 本文提出了一种基于面孔幻视（pareidolia）图像的表示层诊断框架，用于分析视觉模型在视觉证据模糊时的行为。该框架评估了检测、定位、不确定性以及跨类别、难度和情感的偏差。在统一协议下，评估了涵盖四种表示范式的六种模型：视觉语言模型（VLM）、纯视觉分类（ViT）、通用目标检测（YOLOv8）和人脸检测（RetinaFace）。分析揭示了模型在模糊性下的三种解释机制。

Details

Motivation: 当视觉证据模糊时，视觉模型需要决定是否将类似面孔的模式解释为有意义的。面孔幻视现象为研究这种行为提供了一个受控的探针。论文旨在通过一个诊断框架，分析不同视觉模型在面对模糊图像时的行为差异。

Result: 评估结果显示，VLM（特别是LLaVA-1.5-7B）表现出语义过度激活，将模糊的非人区域系统地拉向“人类”概念，并对负面情绪产生最自信的过度调用。ViT则遵循不确定性即弃权策略，保持分布弥散但基本无偏。基于检测的模型通过保守的先验实现低偏差，即使在定位受控时也抑制幻视响应。结果表明，模糊性下的行为更多由表示选择而非分数阈值决定，且不确定性与偏差是解耦的。

Insight: 论文的创新点在于利用面孔幻视作为紧凑的诊断工具和模糊感知的困难负样本来源，以探测和改进视觉语言系统的语义鲁棒性。从客观角度看，其提出的表示层诊断框架和跨模型范式的统一比较方法，为理解模型在模糊性下的解释机制提供了新的视角和系统性的分析工具。

Abstract: When visual evidence is ambiguous, vision models must decide whether to interpret face-like patterns as meaningful. Face pareidolia, the perception of faces in non-face objects, provides a controlled probe of this behavior. We introduce a representation-level diagnostic framework that analyzes detection, localization, uncertainty, and bias across class, difficulty, and emotion in face pareidolia images. Under a unified protocol, we evaluate six models spanning four representational regimes: vision-language models (VLMs; CLIP-B/32, CLIP-L/14, LLaVA-1.5-7B), pure vision classification (ViT), general object detection (YOLOv8), and face detection (RetinaFace). Our analysis reveals three mechanisms of interpretation under ambiguity. VLMs exhibit semantic overactivation, systematically pulling ambiguous non-human regions toward the Human concept, with LLaVA-1.5-7B producing the strongest and most confident over-calls, especially for negative emotions. ViT instead follows an uncertainty-as-abstention strategy, remaining diffuse yet largely unbiased. Detection-based models achieve low bias through conservative priors that suppress pareidolia responses even when localization is controlled. These results show that behavior under ambiguity is governed more by representational choices than score thresholds, and that uncertainty and bias are decoupled: low uncertainty can signal either safe suppression, as in detectors, or extreme over-interpretation, as in VLMs. Pareidolia therefore provides a compact diagnostic and a source of ambiguity-aware hard negatives for probing and improving the semantic robustness of vision-language systems. Code will be released upon publication.

[76] Weakly Supervised Patch Annotation for Improved Screening of Diabetic Retinopathy cs.CVPDF

Shramana Dey, Abhirup Banerjee, B. Uma Shankar, Ramachandran Rajalakshmi, Sushmita Mitra

TL;DR: 本文提出了一种名为SAFE的两阶段框架，用于在糖尿病视网膜病变筛查中系统性地扩展稀疏的病变区域标注。该方法结合了弱监督、对比学习和块级嵌入推断，通过特征空间集成生成可靠的病变区域标注，从而提升下游分类任务的性能。

Details

Motivation: 糖尿病视网膜病变的早期检测面临标注不足的挑战，现有方法难以系统性地标注未标记的病变区域，而专家标注又耗时且不完整，限制了深度学习模型的性能。

Result: 实验结果显示，SAFE能够可靠地区分健康和病变图像块，准确率高达0.9886。生成的标注显著提升了糖尿病视网膜病变分类任务的性能，例如患病类别的F1分数大幅提高，且在精确率-召回率曲线下面积（AUPRC）上获得了高达0.545的性能增益。

Insight: 创新点在于提出了一个统一弱监督、对比学习和嵌入推断的两阶段框架，通过特征空间集成和弃权机制在可靠标注和噪声覆盖之间取得平衡，从而生成细粒度且临床相关的病变标注，可应用于其他医学图像分析任务中数据标注增强的场景。

Abstract: Diabetic Retinopathy (DR) requires timely screening to prevent irreversible vision loss. However, its early detection remains a significant challenge since often the subtle pathological manifestations (lesions) get overlooked due to insufficient annotation. Existing literature primarily focuses on image-level supervision, weakly-supervised localization, and clustering-based representation learning, which fail to systematically annotate unlabeled lesion region(s) for refining the dataset. Expert-driven lesion annotation is labor-intensive and often incomplete, limiting the performance of deep learning models. We introduce Similarity-based Annotation via Feature-space Ensemble (SAFE), a two-stage framework that unifies weak supervision, contrastive learning, and patch-wise embedding inference, to systematically expand sparse annotations in the pathology. SAFE preserves fine-grained details of the lesion(s) under partial clinical supervision. In the first stage, a dual-arm Patch Embedding Network learns semantically structured, class-discriminative embeddings from expert annotated patches. Next, an ensemble of independent embedding spaces extrapolates labels to the unannotated regions based on spatial and semantic proximity. An abstention mechanism ensures trade-off between highly reliable annotation and noisy coverage. Experimental results demonstrate reliable separation of healthy and diseased patches, achieving upto 0.9886 accuracy. The annotation generated from SAFE substantially improves downstream tasks such as DR classification, demonstrating a substantial increase in F1-score of the diseased class and a performance gain as high as 0.545 in Area Under the Precision-Recall Curve (AUPRC). Qualitative analysis, with explainability, confirms that SAFE focuses on clinically relevant lesion patterns; and is further validated by ophthalmologists.

[77] Discriminative Perception via Anchored Description for Reasoning Segmentation cs.CV | cs.AIPDF

Tao Yang, Qing Zhou, Yanliang Li, Qi Wang

TL;DR: 本文提出了一种名为DPAD的新方法，旨在增强推理分割任务中的判别感知能力。该方法通过强制模型生成被指代对象的描述性标题，并对比该标题与目标对象及背景的语义相关性，从而引导模型聚焦于目标的独特属性，产生更收敛和高效的推理链。

Details

Motivation: 现有基于强化学习的推理分割方法主要依赖几何奖励来指导最终定位，但无法判别推理过程是否始终锚定在被指代区域，导致推理链冗长且不聚焦，难以在复杂场景中准确感知目标。因此，需要补充一种能够主动区分目标与背景的判别感知能力。

Result: 在ReasonSeg基准测试上，该方法将cIoU指标提升了3.09%，同时推理链长度减少了约42%，实现了显著的性能提升和效率优化。

Insight: 核心创新在于引入了判别感知机制，通过生成并利用描述性标题进行显式的语义对比，迫使模型关注目标的关键特征，从而优化推理过程。这不仅提升了分割性能，还提供了与分割结果对齐的可解释性依据。描述性标题本身也作为一种可解释的推理依据。

Abstract: Reasoning segmentation increasingly employs reinforcement learning to generate explanatory reasoning chains that guide Multimodal Large Language Models. While these geometric rewards are primarily confined to guiding the final localization, they are incapable of discriminating whether the reasoning process remains anchored on the referred region or strays into irrelevant context. Lacking this discriminative guidance, the model’s reasoning often devolves into unfocused and verbose chains that ultimately fail to disambiguate and perceive the target in complex scenes. This suggests a need to complement the RL objective with Discriminative Perception, an ability to actively distinguish a target from its context. To realize this, we propose DPAD to compel the model to generate a descriptive caption of the referred object, which is then used to explicitly discriminate by contrasting the caption’s semantic relevance to the referred object against the wider context. By optimizing for this discriminative capability, the model is forced to focus on the unique attributes of the target, leading to a more converged and efficient reasoning chain. The descriptive caption also serves as an interpretability rationale that aligns with the segmentation. Experiments on the benchmarks confirm the validity of our approach, delivering substantial performance gains, with the cIoU on ReasonSeg increasing by 3.09% and the reasoning chain length decreasing by approximately 42%. Code is available at https://github.com/mrazhou/DPAD

[78] Rethinking the Efficiency and Effectiveness of Reinforcement Learning for Radiology Report Generation cs.CVPDF

Zilin Lu, Ruifeng Yuan, Weiwei Cao, Wanxing Chang, Zhongyu Wei

TL;DR: 本文重新审视了强化学习在放射学报告生成任务中的效率和有效性，提出了一种基于诊断多样性的数据采样策略以减少所需样本量，并引入诊断令牌加权策略优化方法，通过使用诊断F1分数作为奖励信号直接优化临床准确性。

Details

Motivation: 现有放射学报告生成方法在临床实用性上不足，强化学习虽具潜力但应用不足，本文旨在解决强化学习在该任务中的数据效率和优化效果问题。

Result: 在MIMIC-CXR、IU-Xray和CheXpert Plus数据集上的实验表明，该框架实现了最先进的性能，且仅需20%的强化学习训练样本即可在MIMIC-CXR上达到0.516的F1分数。

Insight: 创新点包括强调数据质量优于数量，提出诊断多样性采样策略，以及通过诊断令牌加权策略优化直接优化临床相关令牌，而非平等对待所有令牌。

Abstract: Radiologists highly desire fully automated AI for radiology report generation (R2G), yet existing approaches fall short in clinical utility. Reinforcement learning (RL) holds potential to address these shortcomings, but its adoption in this task remains underexplored. In this paper, we revisit RL in terms of data efficiency and optimization effectiveness for R2G tasks. First, we explore the impact of data quantity and quality on the performance of RL in medical contexts, revealing that data quality plays a more critical role than quantity. To this end, we propose a diagnostic diversity-based data sampling strategy that enables comparable performance with fewer samples. Second, we observe that the majority of tokens in radiology reports are template-like and diagnostically uninformative, whereas the low frequency of clinically critical tokens heightens the risk of being overlooked during optimization. To tackle this, we introduce Diagnostic Token-weighted Policy Optimization (DiTPO), which directly optimizes for clinical accuracy by using a diagnostic F1 score as the reward signal. Unlike standard RL approaches that treat all tokens equally, DiTPO explicitly models the varying importance of different tokens through rule- or gradient-based mechanisms to prioritize clinically relevant content. Extensive experiments on the MIMIC-CXR, IU-Xray, and CheXpert Plus datasets demonstrate that our framework achieves state-of-the-art (SOTA) performance while requiring substantially fewer training samples in RL. Notably, on MIMIC-CXR, our framework attains an F1 score of 0.516 using only 20% of the RL training samples.

[79] DQE-CIR: Distinctive Query Embeddings through Learnable Attribute Weights and Target Relative Negative Sampling in Composed Image Retrieval cs.CV | cs.AI | cs.LGPDF

Geon Park, Ji-Hoon Park, Seong-Whan Lee

TL;DR: 本文提出了一种名为DQE-CIR的新方法，用于提升组合图像检索（CIR）任务中查询嵌入的区分度。该方法通过引入可学习的属性权重来强调与修改文本相关的视觉特征，并设计了一种目标相对负采样策略，从相似度中等区域选择信息丰富的负样本，以减少相关性抑制和语义混淆问题。

Details

Motivation: 现有基于对比学习的CIR方法通常将真实目标图像作为唯一正样本，其余所有图像作为负样本，这导致了相关性抑制（将语义相关但有效的图像错误推开）和语义混淆（不同修改意图在嵌入空间重叠），使得学习到的查询表示缺乏区分度，尤其是在细粒度属性修改上表现不佳。

Result: 摘要中未提及具体的定量实验结果、基准测试或与SOTA的比较。

Insight: 创新点在于明确地对训练过程中的目标相对相关性进行建模，具体包括：1）可学习的属性加权机制，以条件于修改文本来强调区分性视觉特征，实现更精确的跨模态特征对齐；2）目标相对负采样策略，构建目标相对相似度分布，并从中等相似度区域（排除简单负样本和模糊的假负样本）选择信息丰富的负样本，从而提升查询的区分度并减少语义相似但不相关候选带来的混淆。

Abstract: Composed image retrieval (CIR) addresses the task of retrieving a target image by jointly interpreting a reference image and a modification text that specifies the intended change. Most existing methods are still built upon contrastive learning frameworks that treat the ground truth image as the only positive instance and all remaining images as negatives. This strategy inevitably introduces relevance suppression, where semantically related yet valid images are incorrectly pushed away, and semantic confusion, where different modification intents collapse into overlapping regions of the embedding space. As a result, the learned query representations often lack discriminativeness, particularly at fine-grained attribute modifications. To overcome these limitations, we propose distinctive query embeddings through learnable attribute weights and target relative negative sampling (DQE-CIR), a method designed to learn distinctive query embeddings by explicitly modeling target relative relevance during training. DQE-CIR incorporates learnable attribute weighting to emphasize distinctive visual features conditioned on the modification text, enabling more precise feature alignment between language and vision. Furthermore, we introduce target relative negative sampling, which constructs a target relative similarity distribution and selects informative negatives from a mid-zone region that excludes both easy negatives and ambiguous false negatives. This strategy enables more reliable retrieval for fine-grained attribute changes by improving query discriminativeness and reducing confusion caused by semantically similar but irrelevant candidates.

[80] Revisiting the Role of Foundation Models in Cell-Level Histopathological Image Analysis under Small-Patch Constraints – Effects of Training Data Scale and Blur Perturbations on CNNs and Vision Transformers cs.CV | q-bio.QMPDF

Hiroki Kagiyama, Toru Nagasaka, Yukari Adachi, Takaaki Tachibana, Ryota Ito

TL;DR: 本文系统评估了在细胞级病理图像分析（40x40像素极小图像块）中，任务特定架构与基础模型（如Vision Transformer）在数据规模、计算效率和鲁棒性方面的表现。研究发现，当有足够训练数据时，针对小图像块优化的任务特定模型（如CustomViT）在准确性和推理成本上均优于基础模型，且基础模型在中等数据规模下即达到性能饱和，并未展现出更强的模糊鲁棒性。

Details

Motivation: 解决在细胞级病理图像分析中，由于图像块尺寸极小（远低于标准ImageNet分辨率），现代深度学习架构和基础模型能否学习到鲁棒且可扩展的表征尚不明确的问题。

Result: 在结直肠癌标本的CD103/CD8免疫染色细胞分类任务上，任务特定模型（尤其是针对小图像块优化的CustomViT）在数据规模增加时性能持续提升，并达到了最高准确率，显著优于所有经过线性探测或微调的基础模型，且推理成本更低。基础模型在中等样本量下性能即饱和。在模糊鲁棒性测试中，所有架构表现相当，基础模型未显示出优势。

Insight: 论文的创新点在于在极端空间约束（极小图像块）的特定领域，通过系统性的数据规模与模糊扰动实验，揭示了任务特定轻量模型相对于通用基础模型的效率与性能优势，挑战了“更大预训练模型总是更好”的普遍假设，并指出高清洁准确率并不等同于更强的鲁棒性。

Abstract: Background and objective: Cell-level pathological image analysis requires working with extremely small image patches (40x40 pixels), far below standard ImageNet resolutions. It remains unclear whether modern deep learning architectures and foundation models can learn robust and scalable representations under this constraint. We systematically evaluated architectural suitability and data-scale effects for small-patch cell classification. Methods: We analyzed 303 colorectal cancer specimens with CD103/CD8 immunostaining, generating 185,432 annotated cell images. Eight task-specific architectures were trained from scratch at multiple data scales (FlagLimit: 256–16,384 samples per class), and three foundation models were evaluated via linear probing and fine-tuning after resizing inputs to 224x224 pixels. Robustness to blur was assessed using pre- and post-resize Gaussian perturbations. Results: Task-specific models improved consistently with increasing data scale, whereas foundation models saturated at moderate sample sizes. A Vision Transformer optimized for small patches (CustomViT) achieved the highest accuracy, outperforming all foundation models with substantially lower inference cost. Blur robustness was comparable across architectures, with no qualitative advantage observed for foundation models. Conclusion: For cell-level classification under extreme spatial constraints, task-specific architectures are more effective and efficient than foundation models once sufficient training data are available. Higher clean accuracy does not imply superior robustness, and large pre-trained models offer limited benefit in the small-patch regime.

[81] CLIP-Guided Multi-Task Regression for Multi-View Plant Phenotyping cs.CVPDF

Simon Warmers, Muhammad Zawish, Fayaz Ali Dharejo, Steven Davy, Radu Timofte

TL;DR: 本文提出了一种基于CLIP嵌入的、面向多视角植物表型分析的视觉语言多任务回归框架，该框架能够从多视角图像中联合预测植物年龄和叶片数量。该方法通过将旋转视角聚合为角度不变表示，并利用轻量级文本先验对视觉特征进行条件化，从而在输入不完整或无序的情况下实现稳定预测。

Details

Motivation: 从多视角植物图像中学习鲁棒的预测模型具有挑战性，主要困难在于视角冗余和视角依赖的外观变化。本文旨在解决这一问题，简化传统双模型流程，并提升对缺失视角的鲁棒性。

Result: 在GroMo25基准测试上，与GroMo基线相比，该方法将平均年龄预测的MAE从7.74降至3.91（提升49.5%），将平均叶片数量预测的MAE从5.52降至3.08（提升44.2%），取得了显著的性能提升。

Insight: 创新点在于提出了一个统一的、基于CLIP的多任务回归框架，通过聚合视角生成角度不变表示，并利用轻量级文本先验（编码视角层级信息）来条件化视觉特征，从而有效处理视角冗余和输入不完整的问题，简化了传统流程并提升了鲁棒性。

Abstract: Modeling plant growth dynamics plays a central role in modern agricultural research. However, learning robust predictors from multi-view plant imagery remains challenging due to strong viewpoint redundancy and viewpoint-dependent appearance changes. We propose a level-aware vision language framework that jointly predicts plant age and leaf count using a single multi-task model built on CLIP embeddings. Our method aggregates rotational views into angle-invariant representations and conditions visual features on lightweight text priors encoding viewpoint level for stable prediction under incomplete or unordered inputs. On the GroMo25 benchmark, our approach reduces mean age MAE from 7.74 to 3.91 and mean leaf-count MAE from 5.52 to 3.08 compared to the GroMo baseline, corresponding to improvements of 49.5% and 44.2%, respectively. The unified formulation simplifies the pipeline by replacing the conventional dual-model setup while improving robustness to missing views. The models and code is available at: https://github.com/SimonWarmers/CLIP-MVP

[82] Real Eyes Realize Faster: Gaze Stability and Pupil Novelty for Efficient Egocentric Learning cs.CV | cs.HCPDF

Ajan Subramanian, Sumukh Bettadapura, Rohan Sathish

TL;DR: 该论文提出了一种基于眼动追踪的双准则帧筛选方法，用于高效处理头戴式第一人称视角视频流。该方法利用注视稳定性（质量）和瞳孔反应（新颖性）两个互补信号，在捕获时无需模型推理即可筛选出关键帧，在10%的预算下达到与完整视频流相当的分类性能。

Details

Motivation: 解决头戴式设备持续拍摄的第一人称视角视频中存在大量冗余和低质量帧的问题，在存储和电池限制下，需要一种高效的数据筛选方法。

Result: 在Visual Experience Dataset (VEDB)上，仅使用10%预算的筛选帧即可达到与完整视频流相当的分类性能；瞳孔排序提升了活动识别任务性能，而仅注视筛选在场景识别任务中已表现优异。

Insight: 创新性地将眼动追踪的注视稳定性（质量）和瞳孔反应（新颖性）分解为两个互补的筛选准则，并证明简单的信号融合会破坏各自贡献；该方法无需模型推理，可在捕获时实时运行，为高效持续的第一人称数据管理提供了新路径。

Abstract: Always-on egocentric cameras are increasingly used as demonstrations for embodied robotics, imitation learning, and assistive AR, but the resulting video streams are dominated by redundant and low-quality frames. Under the storage and battery constraints of wearable devices, choosing which frames to keep is as important as how to learn from them. We observe that modern eye-tracking headsets provide a continuous, training-free side channel that decomposes into two complementary axes: gaze fixation captures visual stability (quality), while pupil response captures arousal-linked moments (novelty). We operationalize this insight as a Dual-Criterion Frame Curator that first gates frames by gaze quality and then ranks the survivors by pupil-derived novelty. On the Visual Experience Dataset (VEDB), curated frames at 10% budget match the classification performance of the full stream, and naive signal fusion consistently destroys both contributions. The benefit is task-dependent: pupil ranking improves activity recognition, while gaze-only selection already dominates for scene recognition, confirming that the two signals serve genuinely different roles. Our method requires no model inference and operates at capture time, offering a path toward efficient, always-on egocentric data curation.

[83] Any2Any: Unified Arbitrary Modality Translation for Remote Sensing cs.CVPDF

Haoyang Chen, Jing Zhang, Hebaixu Wang, Shiqin Wang, Pohsun Huang

TL;DR: 该论文提出了一种名为Any2Any的统一潜在扩散框架，用于遥感领域的任意模态到任意模态的翻译。该方法将异构输入投影到一个几何对齐的潜在空间中，通过共享主干网络进行锚定潜在回归，从而解耦了模态特定表示学习与语义映射。论文还引入了百万规模的RST-1M数据集来支持稀疏但关联的监督学习。实验表明，Any2Any在14个翻译任务上优于成对翻译方法，并对未见过的模态对展现出强大的零样本泛化能力。

Details

Motivation: 解决多模态遥感图像在实践中经常不完整的问题，以及现有跨模态翻译方法将每个模态对视为独立任务导致的二次复杂性和对未见模态组合泛化能力有限的问题。

Result: 在14个翻译任务上的实验表明，Any2Any始终优于成对翻译方法，并在未见过的模态对上展现出强大的零样本泛化能力。

Insight: 创新点在于将任意到任意翻译形式化为对场景共享潜在表示的推断，并提出统一的潜在扩散框架，通过几何对齐的潜在空间和轻量级目标特定残差适配器来解耦表示学习与语义映射，同时引入大规模多模态配对数据集RST-1M以支持学习。从客观角度看，其统一框架设计和对零样本泛化的关注具有借鉴意义。

Abstract: Multi-modal remote sensing imagery provides complementary observations of the same geographic scene, yet such observations are frequently incomplete in practice. Existing cross-modal translation methods treat each modality pair as an independent task, resulting in quadratic complexity and limited generalization to unseen modality combinations. We formulate Any-to-Any translation as inference over a shared latent representation of the scene, where different modalities correspond to partial observations of the same underlying semantics. Based on this formulation, we propose Any2Any, a unified latent diffusion framework that projects heterogeneous inputs into a geometrically aligned latent space. Such structure performs anchored latent regression with a shared backbone, decoupling modality-specific representation learning from semantic mapping. Moreover, lightweight target-specific residual adapters are used to correct systematic latent mismatches without increasing inference complexity. To support learning under sparse but connected supervision, we introduce RST-1M, the first million-scale remote sensing dataset with paired observations across five sensing modalities, providing supervision anchors for any-to-any translation. Experiments across 14 translation tasks show that Any2Any consistently outperforms pairwise translation methods and exhibits strong zero-shot generalization to unseen modality pairs. Code and models will be available at https://github.com/MiliLab/Any2Any.

[84] A Baseline Study and Benchmark for Few-Shot Open-Set Action Recognition with Feature Residual Discrimination cs.CVPDF

Stefano Berti, Giulia Pasquale, Lorenzo Natale

TL;DR: 本文提出了一种基于特征残差判别器（FR-Disc）的架构扩展，用于解决少样本开放集动作识别（FSOS-AR）问题，该问题在现实开放场景中超越了传统少样本动作识别的闭集假设限制。通过在五个数据集上的广泛实验，该方法显著提升了未知类别的拒绝能力，同时保持了闭集准确性，达到了新的最先进水平。

Details

Motivation: 少样本动作识别（FS-AR）在现实开放场景中因闭集假设而受限，而少样本开放集（FSOS）识别在图像领域已成熟，但在时空视频数据中尚未充分探索，因此需要扩展方法以适应更复杂的视频域。

Result: 在五个数据集上的实验表明，常见开放集技术仅带来边际提升，而提出的FR-Disc方法在保持闭集准确性的同时，显著增强了未知类别拒绝能力，达到了少样本开放集动作识别的新SOTA水平。

Insight: 创新点在于将基于骨骼数据的先前工作扩展到视频域，通过特征残差判别器架构有效处理开放集场景；客观分析认为，该方法通过残差机制区分已知与未知类别，为视频少样本开放集识别提供了可借鉴的基准方案。

Abstract: Few-Shot Action Recognition (FS-AR) has shown promising results but is often limited by a closed-set assumption that fails in real-world open-set scenarios. While Few-Shot Open-Set (FSOS) recognition is well-established for images, its extension to spatio-temporal video data remains underexplored. To address this, we propose an architectural extension based on a Feature-Residual Discriminator (FR-Disc), adapting previous work on skeletal data to the more complex video domain. Extensive experiments on five datasets demonstrate that while common open-set techniques provide only marginal gains, our FR-Disc significantly enhances unknown rejection capabilities without compromising closed-set accuracy, setting a new state-of-the-art for FSOS-AR. The project website, code, and benchmark are available at: https://hsp-iit.github.io/fsosar/.

[85] Crab$^{+}$: A Scalable and Unified Audio-Visual Scene Understanding Model with Explicit Cooperation cs.CV | cs.AI | cs.MMPDF

Dongnuan Cai, Henghui Du, Chang Zhou, Xi Chen, Dan Guo

TL;DR: 本文提出了Crab⁺，一个可扩展的统一音频-视觉场景理解模型，通过数据和模型层面的显式协作解决任务异质性带来的负迁移问题。该模型基于AV-UIE v2数据集（包含约222K样本）和Interaction-aware LoRA（I-LoRA）机制，实现了在17个数据集和7个任务上的多任务统一学习，逆转了传统多任务学习中近55%任务性能下降的趋势，使88%的任务在多任务训练下超越单任务基线。

Details

Motivation: 传统音频-视觉多任务统一方法常因任务异质性（如任务粒度不同、能力需求差异）导致严重的负迁移，约55%的任务性能相比单任务训练下降，因此需要一种能显式协调异构任务的方法来提升统一模型的性能。

Result: Crab⁺在多个基准测试中超越了现有统一模型和专用模型，成功逆转负迁移趋势，在多任务学习下使近88%的任务性能超过单任务基线，结果在不同AV-LLM范式中均得到验证，并通过可视化分析确认了其有效性。

Insight: 创新点包括：1) 构建了AV-UIE v2数据集，提供显式推理过程以捕捉跨任务关系；2) 设计了统一接口对齐异构任务表述；3) 提出了Interaction-aware LoRA（I-LoRA），通过动态路由显式建模任务间关系以协调不同的音频-视觉交互模式，减少参数干扰。这些方法为处理多模态任务异质性提供了可借鉴的解决方案。

Abstract: Developing Audio-Visual Large Language Models (AV-LLMs) for unified scene understanding is pivotal in multimodal intelligence. While instruction tuning enables pre-trained models with multi-task abilities, we observe that conventional multi-task unification methods often suffer from severe negative transfer, where nearly 55% of tasks degrade compared to single-task training. We attribute this phenomenon to audio-visual task heterogeneity, characterized by disparate task granularity and divergent capability demands, which lead to negative interference under joint training. To tackle this, we present Crab$^{+}$, a scalable and unified audio-visual scene understanding model that addresses task heterogeneity through explicit cooperation from both data and model perspectives. On the data side, we introduce AV-UIE v2, a comprehensive Audio-Visual Unified Instruction-tuning dataset with Explicit reasoning processes. It contains approximately 222K samples spanning 17 datasets and 7 tasks, enabling the model to capture cross-task relationships at different levels of granularity. On the model side, we design a unified interface to align heterogeneous task formulations, and propose Interaction-aware LoRA (I-LoRA), which explicitly models inter-task relationships via dynamic routing to coordinate distinct audio-visual interaction patterns, mitigating parameter interference. Extensive experiments show Crab$^{+}$ covers broader tasks than existing unified models while outperforming specialized models on various benchmarks. We successfully reverse the negative transfer trend, achieving positive transfer where multi-task learning surpasses single-task baselines in nearly 88% of tasks. These results hold across diverse AV-LLM paradigms and are validated through in-depth visualization, positioning Crab$^{+}$ as a robust step towards holistic audio-visual scene understanding.

[86] LISTA-Transformer Model Based on Sparse Coding and Attention Mechanism and Its Application in Fault Diagnosis cs.CVPDF

Shuang Liu, Lina Zhao, Tian Wang, Huaqing Wang

TL;DR: 本文提出了一种基于可学习迭代收缩阈值算法（LISTA）稀疏编码与注意力机制的LISTA-Transformer模型，并将其应用于故障诊断。该方法通过连续小波变换将振动信号转换为时频图，并输入到LISTA-Transformer中进行特征提取，在CWRU数据集上取得了98.5%的故障识别率。

Details

Motivation: 针对现有模型（如CNN和Transformer）在局部特征建模、全局依赖捕获、模型复杂度和可解释性方面的局限性，旨在设计一个能自适应协同局部与全局特征的模型架构。

Result: 在CWRU数据集上，该方法故障识别率达到98.5%，比传统方法高3.3%，且优于现有的基于Transformer的方法。

Insight: 创新点在于将LISTA稀疏编码与视觉Transformer深度融合，构建了具有自适应局部与全局特征协同机制的模型架构，提升了特征提取的有效性和模型性能。

Abstract: Driven by the continuous development of models such as Multi-Layer Perceptron, Convolutional Neural Network (CNN), and Transformer, deep learning has made breakthrough progress in fields such as computer vision and natural language processing, and has been successfully applied in practical scenarios such as image classification and industrial fault diagnosis. However, existing models still have certain limitations in local feature modeling and global dependency capture. Specifically, CNN is limited by local receptive fields, while Transformer has shortcomings in effectively modeling local structures, and both face challenges of high model complexity and insufficient interpretability. In response to the above issues, we proposes the following innovative work: A sparse Transformer based on Learnable Iterative Shrinkage Threshold Algorithm (LISTA-Transformer) was designed, which deeply integrates LISTA sparse encoding with visual Transformer to construct a model architecture with adaptive local and global feature collaboration mechanism. This method utilizes continuous wavelet transform to convert vibration signals into time-frequency maps and inputs them into LISTA-Transformer for more effective feature extraction. On the CWRU dataset, the fault recognition rate of our method reached 98.5%, which is 3.3% higher than traditional methods and exhibits certain superiority over existing Transformer-based approaches.

[87] Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild cs.CVPDF

Changda Zhou, Ziyue Gao, Xueqing Wang, Tingquan Gao, Cheng Cui

TL;DR: 本文提出了Real5-OmniDocBench，这是首个对OmniDocBench v1.5数据集（包含1,355张图像）进行全尺度、一对一物理重建的基准测试，覆盖扫描、扭曲、屏幕拍摄、光照和倾斜五种关键现实场景，旨在评估视觉语言模型在真实物理世界文档解析中的鲁棒性。

Details

Motivation: 现有视觉语言模型在数字文档基准测试（如OmniDocBench）上表现近乎完美，但由于缺乏受控且真实的评估，它们在不可预测的物理世界中的性能仍未知，因此需要构建一个能严格评估现实差距的基准。

Result: 该基准为社区建立了一个具有挑战性的新标准，表明文档解析中的‘现实差距’远未弥合，并提供了一个诊断工具来指导真正鲁棒的文档智能系统开发。

Insight: 创新点在于首次实现了对整个数字文档基准的完整物理重建和真实映射，从而能够严格归因性能下降的具体因素（如几何畸变、光学伪影或模型限制），为模型鲁棒性分析提供了精细的诊断能力。

Abstract: While Vision-Language Models (VLMs) achieve near-perfect scores on digital document benchmarks like OmniDocBench, their performance in the unpredictable physical world remains largely unknown due to the lack of controlled yet realistic evaluations. We introduce Real5-OmniDocBench, the first benchmark that performs a full-scale, one-to-one physical reconstruction of the entire OmniDocBench v1.5 (1,355 images) across five critical real-world scenarios: Scanning, Warping, Screen-Photography, Illumination, and Skew. Unlike prior benchmark that either lack digital correspondence or employ partial sampling, our complete ground-truth mapping enables, for the first time, rigorous factor-wise attribution of performance degradation-allowing us to pinpoint whether failures stem from geometric distortions, optical artifacts, or model limitations. Our benchmark establishes a challenging new standard for the community, demonstrating that the ‘reality gap’ in document parsing is far from closed, and provides a diagnostic tool to guide the development of truly resilient document intelligence.

[88] A Unified Framework for Joint Detection of Lacunes and Enlarged Perivascular Spaces cs.CVPDF

Lucas He, Krinos Li, Hanyuan Zhang, Runlong He, Silvia Ingala

TL;DR: 本文提出了一种用于联合检测脑小血管病标志物——扩大的血管周围间隙（EPVS）和腔隙的统一框架。该框架通过形态解耦设计、零初始化门控跨任务注意力机制、混合监督策略以及解剖学信息推理校准机制，有效解决了标准分割网络在处理这两个目标时面临的特征干扰和极端类别不平衡问题。

Details

Motivation: 标准分割网络在同时处理放射学特征相似的EPVS和腔隙时，存在特征干扰和极端类别不平衡的挑战，导致性能不佳。本文旨在解决这些问题，实现更精确的联合检测。

Result: 在VALDO 2021数据集（N=40）上进行5折交叉验证，取得了最先进的性能，特别是在腔隙检测的精确率（71.1%， p=0.01）和F1分数（62.6%， p=0.03）上显著超越了任务优胜者。在外部EPAD队列（N=1762）上的评估进一步证实了模型在大规模人群研究中的鲁棒性。

Insight: 创新点包括：1. 形态解耦框架设计，通过零初始化门控跨任务注意力利用密集的EPVS上下文指导稀疏的腔隙检测；2. 通过整合互斥损失和中心线Dice损失的混合监督策略，强制生物和拓扑一致性；3. 引入基于组织语义动态抑制假阳性的解剖学信息推理校准机制。这些方法为解决医学图像分析中多目标、类别不平衡和特征相似性问题提供了可借鉴的思路。

Abstract: Cerebral small vessel disease (CSVD) markers, specifically enlarged perivascular spaces (EPVS) and lacunae, present a unique challenge in medical image analysis due to their radiological mimicry. Standard segmentation networks struggle with feature interference and extreme class imbalance when handling these divergent targets simultaneously. To address these issues, we propose a morphology-decoupled framework where Zero-Initialized Gated Cross-Task Attention exploits dense EPVS context to guide sparse lacune detection. Furthermore, biological and topological consistency are enforced via a mixed-supervision strategy integrating Mutual Exclusion and Centerline Dice losses. Finally, we introduce an Anatomically-Informed Inference Calibration mechanism to dynamically suppress false positives based on tissue semantics. Extensive 5-folds cross-validation on the VALDO 2021 dataset (N=40) demonstrates state-of-the-art performance, notably surpassing task winners in lacunae detection precision (71.1%, p=0.01) and F1-score (62.6%, p=0.03). Furthermore, evaluation on the external EPAD cohort (N=1762) confirms the model’s robustness for large-scale population studies. Code will be released upon acceptance.

[89] ViterbiPlanNet: Injecting Procedural Knowledge via Differentiable Viterbi for Planning in Instructional Videos cs.CVPDF

Luigi Seminara, Davide Moltisanti, Antonino Furnari

TL;DR: ViterbiPlanNet是一个用于教学视频中程序性规划的框架，通过引入可微分的Viterbi层（DVL）将程序知识图（PKG）显式整合到学习过程中，实现了端到端优化。该方法在CrossTask、COIN和NIV数据集上取得了最先进的性能，且参数量比基于扩散和LLM的规划器少一个数量级。

Details

Motivation: 现有程序性规划方法通常依赖大规模模型隐式学习程序结构，导致样本效率低、计算成本高。本文旨在通过显式整合程序知识来提升规划任务的效率和性能。

Result: 在CrossTask、COIN和NIV基准测试中，ViterbiPlanNet达到了最先进的性能水平，且参数量显著减少。消融实验表明性能提升源于可微分结构感知训练，而非后处理优化。

Insight: 创新点在于将程序知识图与Viterbi解码算法结合，通过可微分松弛实现端到端优化，从而提高了样本效率和鲁棒性。此外，论文还建立了统一的测试协议，确保了结果的可比性和统计显著性。

Abstract: Procedural planning aims to predict a sequence of actions that transforms an initial visual state into a desired goal, a fundamental ability for intelligent agents operating in complex environments. Existing approaches typically rely on large-scale models that learn procedural structures implicitly, resulting in limited sample-efficiency and high computational cost. In this work we introduce ViterbiPlanNet, a principled framework that explicitly integrates procedural knowledge into the learning process through a Differentiable Viterbi Layer (DVL). The DVL embeds a Procedural Knowledge Graph (PKG) directly with the Viterbi decoding algorithm, replacing non-differentiable operations with smooth relaxations that enable end-to-end optimization. This design allows the model to learn through graph-based decoding. Experiments on CrossTask, COIN, and NIV demonstrate that ViterbiPlanNet achieves state-of-the-art performance with an order of magnitude fewer parameters than diffusion- and LLM-based planners. Extensive ablations show that performance gains arise from our differentiable structure-aware training rather than post-hoc refinement, resulting in improved sample efficiency and robustness to shorter unseen horizons. We also address testing inconsistencies establishing a unified testing protocol with consistent splits and evaluation metrics. With this new protocol, we run experiments multiple times and report results using bootstrapping to assess statistical significance.

[90] A multi-center analysis of deep learning methods for video polyp detection and segmentation cs.CVPDF

Noha Ghatwary, Pedro Chavarias Solano, Mohamed Ramzy Ibrahim, Adrian Krenzer, Frank Puppe

TL;DR: 该论文是一项多中心研究，旨在评估深度学习模型在结肠镜视频序列数据中自动检测和分割结肠息肉的能力。研究强调了利用视频帧间的时间信息和动态变化对于提升息肉检测与分割精度的重要性。

Details

Motivation: 结肠息肉是结直肠癌的前兆，但其在结肠镜检查中因外观、位置和大小多变而难以被检测和完全切除，导致高漏检率。现有自动化方法多基于静态图像，未能充分利用视频序列中的时序信息来捕捉息肉的动态变化。

Result: 研究通过一个多中心、多样化的综合数据集，评估了深度学习技术在实时临床结肠镜任务中的适用性。结果表明，利用序列数据中的时间关系能显著提高诊断精度。

Insight: 论文的核心创新点在于强调并系统评估了时序信息（帧间关系）对于视频息肉检测与分割任务的关键作用。这为开发更鲁棒的实时临床辅助系统提供了重要方向，即从静态图像分析转向动态序列分析。

Abstract: Colonic polyps are well-recognized precursors to colorectal cancer (CRC), typically detected during colonoscopy. However, the variability in appearance, location, and size of these polyps complicates their detection and removal, leading to challenges in effective surveillance, intervention, and subsequently CRC prevention. The processes of colonoscopy surveillance and polyp removal are highly reliant on the expertise of gastroenterologists and occur within the complexities of the colonic structure. As a result, there is a high rate of missed detections and incomplete removal of colonic polyps, which can adversely impact patient outcomes. Recently, automated methods that use machine learning have been developed to enhance polyps detection and segmentation, thus helping clinical processes and reducing missed rates. These advancements highlight the potential for improving diagnostic accuracy in real-time applications, which ultimately facilitates more effective patient management. Furthermore, integrating sequence data and temporal information could significantly enhance the precision of these methods by capturing the dynamic nature of polyp growth and the changes that occur over time. To rigorously investigate these challenges, data scientists and experts gastroenterologists collaborated to compile a comprehensive dataset that spans multiple centers and diverse populations. This initiative aims to underscore the critical importance of incorporating sequence data and temporal information in the development of robust automated detection and segmentation methods. This study evaluates the applicability of deep learning techniques developed in real-time clinical colonoscopy tasks using sequence data, highlighting the critical role of temporal relationships between frames in improving diagnostic precision.

[91] Gaussian Wardrobe: Compositional 3D Gaussian Avatars for Free-Form Virtual Try-On cs.CV | cs.GRPDF

Zhiyi Chen, Hsuan-I Ho, Tianjian Jiang, Jie Song, Manuel Kaufmann

TL;DR: 本文提出了Gaussian Wardrobe，一个从多视角视频中数字化构建可组合3D神经化身的新框架。该框架通过将化身分解为身体和多个与形状无关的神经服装层，克服了现有方法将人体与服装视为不可分割整体、难以捕捉复杂自由形态服装动态且无法跨个体复用服装的局限。

Details

Motivation: 现有3D神经化身方法通常将人体和服装视为不可分割的整体，这难以捕捉复杂自由形态服装的动态，也限制了服装在不同个体间的复用。本文旨在解决这些问题。

Result: 该方法在新型姿态合成基准测试中实现了最先进的性能，能够建模具有高保真动态的逼真化身。

Insight: 核心创新在于提出了一种可组合的3D高斯表示，将化身分解为身体和多个形状无关的神经服装层，并通过解耦和规范化学习，实现了服装的自由分离与跨主体转移，从而支持实用的虚拟试穿应用。

Abstract: We introduce Gaussian Wardrobe, a novel framework to digitalize compositional 3D neural avatars from multi-view videos. Existing methods for 3D neural avatars typically treat the human body and clothing as an inseparable entity. However, this paradigm fails to capture the dynamics of complex free-form garments and limits the reuse of clothing across different individuals. To overcome these problems, we develop a novel, compositional 3D Gaussian representation to build avatars from multiple layers of free-form garments. The core of our method is decomposing neural avatars into bodies and layers of shape-agnostic neural garments. To achieve this, our framework learns to disentangle each garment layer from multi-view videos and canonicalizes it into a shape-independent space. In experiments, our method models photorealistic avatars with high-fidelity dynamics, achieving new state-of-the-art performance on novel pose synthesis benchmarks. In addition, we demonstrate that the learned compositional garments contribute to a versatile digital wardrobe, enabling a practical virtual try-on application where clothing can be freely transferred to new subjects. Project page: https://ait.ethz.ch/gaussianwardrobe

[92] CubeComposer: Spatio-Temporal Autoregressive 4K 360° Video Generation from Perspective Video cs.CV | cs.AIPDF

Lingen Li, Guangzhi Wang, Xiaoyu Li, Zhaoyang Zhang, Qi Dou

TL;DR: 本文提出了CubeComposer，一种新颖的时空自回归扩散模型，用于从透视视频直接生成4K分辨率的360°全景视频。该方法通过将视频分解为立方体贴图表示，并按照精心规划的时空顺序自回归合成内容，从而在降低内存需求的同时实现高分辨率输出。

Details

Motivation: 现有方法受限于普通扩散模型的计算能力，仅支持≤1K分辨率的原生生成，并依赖次优的后处理超分辨率技术来提升分辨率，这影响了VR应用的沉浸式体验。本文旨在解决从透视输入生成高质量、高分辨率360°全景视频的挑战。

Result: 在基准数据集上的大量实验表明，CubeComposer在原生分辨率和视觉质量方面均优于最先进（SOTA）的方法，支持实际的VR应用场景。

Insight: 论文的创新点包括：1）一种协调立方体面和跨时间窗口生成的时空自回归策略，以确保合成内容的连贯性；2）配备稀疏上下文注意力设计的立方体面上下文管理机制，以提高效率；3）连续性感知技术（如立方体感知的位置编码、填充和混合）以消除边界接缝。从客观角度看，其核心创新在于将高分辨率360°视频生成任务分解为时空自回归过程，并针对立方体贴图表示设计了专门的架构和训练策略，有效平衡了计算效率与输出质量。

Abstract: Generating high-quality 360° panoramic videos from perspective input is one of the crucial applications for virtual reality (VR), whereby high-resolution videos are especially important for immersive experience. Existing methods are constrained by computational limitations of vanilla diffusion models, only supporting $\leq$ 1K resolution native generation and relying on suboptimal post super-resolution to increase resolution. We introduce CubeComposer, a novel spatio-temporal autoregressive diffusion model that natively generates 4K-resolution 360° videos. By decomposing videos into cubemap representations with six faces, CubeComposer autoregressively synthesizes content in a well-planned spatio-temporal order, reducing memory demands while enabling high-resolution output. Specifically, to address challenges in multi-dimensional autoregression, we propose: (1) a spatio-temporal autoregressive strategy that orchestrates 360° video generation across cube faces and time windows for coherent synthesis; (2) a cube face context management mechanism, equipped with a sparse context attention design to improve efficiency; and (3) continuity-aware techniques, including cube-aware positional encoding, padding, and blending to eliminate boundary seams. Extensive experiments on benchmark datasets demonstrate that CubeComposer outperforms state-of-the-art methods in native resolution and visual quality, supporting practical VR application scenarios. Project page: https://lg-li.github.io/project/cubecomposer

[93] TaxonRL: Reinforcement Learning with Intermediate Rewards for Interpretable Fine-Grained Visual Reasoning cs.CV | cs.CLPDF

Maximilian von Klinski, Maximilian Schall

TL;DR: 论文提出TaxonRL，一种基于强化学习的方法，通过引入中间奖励和分层分类预测，将细粒度视觉推理过程分解为物种、属、科等层级，旨在提升模型在区分视觉相似物种时的准确性和可解释性。

Details

Motivation: 传统视觉语言模型在对比性细粒度分类推理（如同属或同科内视觉相似物种的区分）上存在困难，需要一种既能提高准确性又能提供透明、可验证决策过程的方法。

Result: 在Birds-to-Words数据集上，TaxonRL达到91.7%的平均准确率，超过人类表现（77.3%），并生成可解释的推理轨迹；在灵长类和海洋物种验证任务中也显示出强大的跨领域泛化能力。

Insight: 创新点在于使用强化学习中的中间奖励（Group Relative Policy Optimization）强制模型进行结构化、分层级的推理，这为细粒度视觉识别提供了一个可迁移的框架，并增强了模型决策的可解释性。

Abstract: Traditional vision-language models struggle with contrastive fine-grained taxonomic reasoning, particularly when distinguishing between visually similar species within the same genus or family. We introduce TaxonRL, a reinforcement learning approach using Group Relative Policy Optimization with intermediate rewards that decomposes the reasoning process into hierarchical taxonomic predictions. Our method incentivizes models to explicitly reason about species-level, genus-level, and family-level features before making final classifications. This structured approach is designed not only to boost accuracy but also to yield a transparent, verifiable decision-making process. On the challenging Birds-to-Words dataset, TaxonRL achieves 91.7% average accuracy, exceeding human performance (77.3%) while generating interpretable reasoning traces. We demonstrate strong cross-domain generalization, showing substantial gains in primate and marine species verification. Our results establish that enforcing structured, hierarchical reasoning provides a powerful and transferable framework for fine-grained visual discrimination.

Hong Li, Yutang Feng, Minqi Meng, Yichen Yang, Xuhui Liu

TL;DR: 本文提出PromptAvatar框架，通过双扩散模型（纹理扩散模型和几何扩散模型）实现从文本或图像提示到高保真3D虚拟形象的快速生成，无需迭代优化，在10秒内完成生成，并在质量和效率上超越现有方法。

Details

Motivation: 解决现有文本驱动方法在细粒度语义控制和推理速度上的不足，以及图像驱动方法因高质量3D面部扫描数据稀缺导致的泛化能力受限问题。

Result: 在生成质量、细粒度细节对齐和计算效率方面显著优于现有最先进方法，通过定量和定性实验验证了其优越性。

Insight: 构建了包含四种模态（文本描述、野外人脸图像、纹理UV贴图、3D几何形状）的大规模数据集，并设计双扩散模型直接学习从多模态提示到3D表示的映射，避免了耗时的迭代优化，实现了快速高质量的3D虚拟形象生成。

Abstract: Generating high-fidelity 3D avatars from text or image prompts is highly sought after in virtual reality and human-computer interaction. However, existing text-driven methods often rely on iterative Score Distillation Sampling (SDS) or CLIP optimization, which struggle with fine-grained semantic control and suffer from excessively slow inference. Meanwhile, image-driven approaches are severely bottlenecked by the scarcity and high acquisition cost of high-quality 3D facial scans, limiting model generalization. To address these challenges, we first construct a novel, large-scale dataset comprising over 100,000 pairs across four modalities: fine-grained textual descriptions, in-the-wild face images, high-quality light-normalized texture UV maps, and 3D geometric shapes. Leveraging this comprehensive dataset, we propose PromptAvatar, a framework featuring dual diffusion models. Specifically, it integrates a Texture Diffusion Model (TDM) that supports flexible multi-condition guidance from text and/or image prompts, alongside a Geometry Diffusion Model (GDM) guided by text prompts. By learning the direct mapping from multi-modal prompts to 3D representations, PromptAvatar eliminates the need for time-consuming iterative optimization, successfully generating high-fidelity, shading-free 3D avatars in under 10 seconds. Extensive quantitative and qualitative experiments demonstrate that our method significantly outperforms existing state-of-the-art approaches in generation quality, fine-grained detail alignment, and computational efficiency.

[95] SPRINT: Semi-supervised Prototypical Representation for Few-Shot Class-Incremental Tabular Learning cs.CV | cs.AIPDF

Umid Suleymanov, Murat Kantarcioglu, Kevin S Chan, Michael De Lucia, Kevin Hamlen

TL;DR: SPRINT是首个针对表格数据的少样本类增量学习框架，通过混合情景训练策略，利用基于置信度的伪标签增强新类表示，并利用表格数据存储成本低的特性保留基类历史，在六个跨领域基准测试中实现了最先进的性能。

Details

Motivation: 解决表格数据流（如日志、传感器数据）在少样本类增量学习中的挑战，现有基于视觉的方法依赖于受限的缓冲区，忽略了表格数据中丰富的未标记数据、专家标注稀缺和存储成本可忽略的特性。

Result: 在涵盖网络安全、医疗和生态领域的六个不同基准测试中，SPRINT实现了77.37%的平均准确率（5-shot），比最强的增量基线高出4.45%，达到了最先进水平。

Insight: 创新点包括针对表格数据设计的混合情景训练策略和基于置信度的伪标签机制，客观分析认为其有效利用了表格数据中未标记数据的丰富性和低存储成本优势，是领域适应性的重要扩展。

Abstract: Real-world systems must continuously adapt to novel concepts from limited data without forgetting previously acquired knowledge. While Few-Shot Class-Incremental Learning (FSCIL) is established in computer vision, its application to tabular domains remains largely unexplored. Unlike images, tabular streams (e.g., logs, sensors) offer abundant unlabeled data, a scarcity of expert annotations and negligible storage costs, features ignored by existing vision-based methods that rely on restrictive buffers. We introduce SPRINT, the first FSCIL framework tailored for tabular distributions. SPRINT introduces a mixed episodic training strategy that leverages confidence-based pseudo-labeling to enrich novel class representations and exploits low storage costs to retain base class history. Extensive evaluation across six diverse benchmarks spanning cybersecurity, healthcare, and ecological domains, demonstrates SPRINT’s cross-domain robustness. It achieves a state-of-the-art average accuracy of 77.37% (5-shot), outperforming the strongest incremental baseline by 4.45%.

[96] Scalable Evaluation of the Realism of Synthetic Environmental Augmentations in Images cs.CV | cs.LGPDF

Damian J. Ruck, Paul Vautravers, Oliver Chalkley, Jake Thomas

TL;DR: 本文提出了一种可扩展的框架，用于评估合成图像编辑方法（特别是为车载摄像头图像添加雾、雨、雪和夜间等环境条件）的真实感。该框架结合了基于视觉语言模型的感知真实性评估和基于嵌入的分布相似性分析两种自动化指标。研究发现，生成式AI方法在真实性上显著优于基于规则的方法，并且在大多数条件下，领先的生成方法能达到甚至超过真实图像的性能水平。

Details

Motivation: 评估AI系统（尤其是安全关键场景）常需合成测试用例，但生成数据的实用性取决于其真实性。本文旨在提供一个可扩展的框架，以评估用于生成恶劣环境条件图像的合成图像编辑方法的真实性。

Result: 在40张晴天图像上测试，最佳生成式AI方法的接受率约为最佳基于规则方法的3.6倍。不同条件难度不同：雾最容易模拟，夜间转换最具挑战性。根据VLM评审设定的实际性能上限（真实恶劣条件图像也非完美），领先的生成方法在大多数条件下达到或超过了真实图像的性能。

Insight: 创新点在于提出了一个结合VLM感知评审和嵌入分布分析的可扩展、自动化真实性评估框架。客观来看，该框架为量化生成图像的真实性提供了实用方法，并揭示了生成式AI在模拟特定环境条件（尤其是雾）方面已非常有效，但模拟夜间等复杂条件仍是挑战，且未来仍需与人类评估进行验证。

Abstract: Evaluation of AI systems often requires synthetic test cases, particularly for rare or safety-critical conditions that are difficult to observe in operational data. Generative AI offers a promising approach for producing such data through controllable image editing, but its usefulness depends on whether the resulting images are sufficiently realistic to support meaningful evaluation. We present a scalable framework for assessing the realism of synthetic image-editing methods and apply it to the task of adding environmental conditions-fog, rain, snow, and nighttime-to car-mounted camera images. Using 40 clear-day images, we compare rule-based augmentation libraries with generative AI image-editing models. Realism is evaluated using two complementary automated metrics: a vision-language model (VLM) jury for perceptual realism assessment, and embedding-based distributional analysis to measure similarity to genuine adverse-condition imagery. Generative AI methods substantially outperform rule-based approaches, with the best generative method achieving approximately 3.6 times the acceptance rate of the best rule-based method. Performance varies across conditions: fog proves easiest to simulate, while nighttime transformations remain challenging. Notably, the VLM jury assigns imperfect acceptance even to real adverse-condition imagery, establishing practical ceilings against which synthetic methods can be judged. By this standard, leading generative methods match or exceed real-image performance for most conditions. These results suggest that modern generative image-editing models can enable scalable generation of realistic adverse-condition imagery for evaluation pipelines. Our framework therefore provides a practical approach for scalable realism evaluation, though validation against human studies remains an important direction for future work.

[97] ArtHOI: Articulated Human-Object Interaction Synthesis by 4D Reconstruction from Video Priors cs.CVPDF

Zihao Huang, Tianqi Liu, Zhaoxi Chen, Shaocong Xu, Saining Zhang

TL;DR: ArtHOI是一个无需3D/4D监督的零样本框架，通过从单目视频先验进行4D重建来合成物理上合理的铰接式人-物交互。该方法将生成的2D视频作为逆渲染问题的监督，恢复几何一致且物理合理的4D场景，显著提升了接触精度、穿透减少和铰接保真度。

Details

Motivation: 解决现有零样本方法在合成铰接式人-物交互时局限于刚性物体操作、缺乏显式4D几何推理的问题，旨在从扩散模型生成的视频中重建完整的4D铰接场景。

Result: 在多种铰接场景（如打开冰箱、橱柜、微波炉）上，ArtHOI在接触精度、穿透减少和铰接保真度方面显著优于先前方法，将零样本交互合成扩展到刚性操作之外。

Insight: 创新点包括：1）基于光流的部件分割，利用光流作为几何线索在单目视频中分离动态与静态区域；2）解耦的重建流程，先恢复物体铰接状态，再基于重建的物体状态合成人体运动，以解决单目模糊下的联合优化不稳定问题。该方法将基于视频的生成与几何感知重建相结合，实现了语义对齐且物理基础的交互合成。

Abstract: Synthesizing physically plausible articulated human-object interactions (HOI) without 3D/4D supervision remains a fundamental challenge. While recent zero-shot approaches leverage video diffusion models to synthesize human-object interactions, they are largely confined to rigid-object manipulation and lack explicit 4D geometric reasoning. To bridge this gap, we formulate articulated HOI synthesis as a 4D reconstruction problem from monocular video priors: given only a video generated by a diffusion model, we reconstruct a full 4D articulated scene without any 3D supervision. This reconstruction-based approach treats the generated 2D video as supervision for an inverse rendering problem, recovering geometrically consistent and physically plausible 4D scenes that naturally respect contact, articulation, and temporal coherence. We introduce ArtHOI, the first zero-shot framework for articulated human-object interaction synthesis via 4D reconstruction from video priors. Our key designs are: 1) Flow-based part segmentation: leveraging optical flow as a geometric cue to disentangle dynamic from static regions in monocular video; 2) Decoupled reconstruction pipeline: joint optimization of human motion and object articulation is unstable under monocular ambiguity, so we first recover object articulation, then synthesize human motion conditioned on the reconstructed object states. ArtHOI bridges video-based generation and geometry-aware reconstruction, producing interactions that are both semantically aligned and physically grounded. Across diverse articulated scenes (e.g., opening fridges, cabinets, microwaves), ArtHOI significantly outperforms prior methods in contact accuracy, penetration reduction, and articulation fidelity, extending zero-shot interaction synthesis beyond rigid manipulation through reconstruction-informed synthesis.

[98] Hold-One-Shot-Out (HOSO) for Validation-Free Few-Shot CLIP Adapters cs.CVPDF

Chris Vorster, Mayug Maniparambil, Noel E. O’Connor, Noel Murphy, Derek Molloy

TL;DR: 本文提出了一种名为Hold-One-Shot-Out (HOSO)的验证集无关方法，用于在少样本CLIP适配器（如CLIP-Adapter）中自动学习混合比例超参数，从而在无需额外验证集的情况下平衡预训练CLIP知识与少样本监督信息，实现了严格的少样本学习。

Details

Motivation: 现有少样本CLIP适配方法通常需要在测试集上消融混合比例或依赖额外验证集来选择该超参数，这违背了严格的少样本学习设定。本文旨在解决这一问题，提出一种无需验证集即可自动确定混合比例的方法。

Result: 在验证集无关的少样本协议下，HOSO-Adapter在11个标准少样本数据集上平均比CLIP-Adapter基线高出4个百分点以上。在8-shot和16-shot设置中，其性能甚至优于在测试集上选择最优混合比例的CLIP-Adapter。消融研究验证了一样本留出机制、解耦训练的有效性。

Insight: 创新点在于提出了一种简单有效的验证集无关混合比例学习策略（HOSO），通过一个样本作为留出集来学习超参数，其余样本用于训练适配器，实现了严格的少样本学习，并展示了在少样本场景下自动学习超参数的有效性。

Abstract: In many CLIP adaptation methods, a blending ratio hyperparameter controls the trade-off between general pretrained CLIP knowledge and the limited, dataset-specific supervision from the few-shot cases. Most few-shot CLIP adaptation techniques report results by ablation of the blending ratio on the test set or require additional validation sets to select the blending ratio per dataset, and thus are not strictly few-shot. We present a simple, validation-free method for learning the blending ratio in CLIP adaptation. Hold-One-Shot-Out (HOSO) presents a novel approach for CLIP-Adapter-style methods to compete in the newly established validation-free setting. CLIP-Adapter with HOSO (HOSO-Adapter) learns the blending ratio using a one-shot, hold-out set, while the adapter trains on the remaining few-shot support examples. Under the validation-free few-shot protocol, HOSO-Adapter outperforms the CLIP-Adapter baseline by more than 4 percentage points on average across 11 standard few-shot datasets. Interestingly, in the 8- and 16-shot settings, HOSO-Adapter outperforms CLIP-Adapter even with the optimal blending ratio selected on the test set. Ablation studies validate the use of a one-shot hold-out mechanism, decoupled training, and improvements over the naively learnt blending ratio baseline. Code is released here: https://github.com/chris-vorster/HOSO-Adapter

[99] Enhancing Authorship Attribution with Synthetic Paintings cs.CV | cs.LGPDF

Clarissa Loures, Caio Hosken, Luan Oliveira, Gianlucca Zuin, Adriano Veloso

TL;DR: 本研究探讨了利用DreamBooth微调Stable Diffusion生成的合成图像，结合真实画作数据，来提升绘画作品作者归属分类模型的性能。实验表明，这种混合真实与合成数据的混合方法，在数据稀缺的艺术品认证场景下，能有效提高模型的准确性和泛化能力。

Details

Motivation: 解决绘画作品作者归属任务中，真实艺术品训练数据有限，导致计算模型性能受限的核心挑战。

Result: 实验结果显示，加入合成图像后，模型的ROC-AUC和准确率均高于仅使用真实画作的基准，在相关艺术风格分类任务上取得了性能提升。

Insight: 创新点在于将生成式模型（DreamBooth/Stable Diffusion）与判别式分类模型结合，提出了一种数据增强的混合方法，为数据稀缺的计算机视觉任务（如艺术品认证）提供了新的技术路径。

Abstract: Attributing authorship to paintings is a historically complex task, and one of its main challenges is the limited availability of real artworks for training computational models. This study investigates whether synthetic images, generated through DreamBooth fine-tuning of Stable Diffusion, can improve the performance of classification models in this context. We propose a hybrid approach that combines real and synthetic data to enhance model accuracy and generalization across similar artistic styles. Experimental results show that adding synthetic images leads to higher ROC-AUC and accuracy compared to using only real paintings. By integrating generative and discriminative methods, this work contributes to the development of computer vision techniques for artwork authentication in data-scarce scenarios.

[100] Underrepresented in Foundation Model Pretraining Data? A One-Shot Probe cs.CVPDF

Chris Vorster, Mayug Maniparambil, Noel E. O’Connor, Noel Murphy, Derek Molloy

TL;DR: 本文提出了一种数据高效的方法，仅需每个类别一张标注图像，即可预测视觉-语言基础模型在目标领域上的零样本准确率。该方法利用大语言模型生成给定图像的合理反事实描述，通过测量模型区分正确描述与困难负例的能力来构建特征，并训练线性回归器估计准确率，在多个数据集上实现了高相关性（Pearson-r=0.96）。

Details

Motivation: 解决视觉-语言基础模型在新型、专业或代表性不足领域上性能不一致的问题，特别是在缺乏标注测试集的领域（如全球南方地区），提供低成本评估工具以指导数据标注决策。

Result: 在五个多样化数据集（包括标准基准和非洲代表性不足数据集）上，线性回归器预测的零样本测试准确率与真实值之间的Pearson-r相关性达到0.96，表明方法高度可靠。

Insight: 创新点在于利用大语言模型生成反事实描述来构建模型判别能力的特征，实现仅需单样本的准确率预测；客观分析认为该方法为评估基础模型在资源受限领域的适用性提供了高效、可扩展的解决方案。

Abstract: Large-scale Vision-Language Foundation Models (VLFMs), such as CLIP, now underpin a wide range of computer vision research and applications. VLFMs are often adapted to various domain-specific tasks. However, VLFM performance on novel, specialised, or underrepresented domains remains inconsistent. Evaluating VLFMs typically requires labelled test sets, which are often unavailable for niche domains of interest, particularly those from the Global South. We address this gap by proposing a highly data-efficient method to predict a VLFM’s zero-shot accuracy on a target domain using only a single labelled image per class. Our approach uses a Large Language Model to generate plausible counterfactual descriptions of a given image. By measuring the VLFM’s ability to distinguish the correct description from these hard negatives, we engineer features that capture the VLFM’s discriminative power in its shared embedding space. A linear regressor trained on these similarity scores estimates the VLFM’s zero-shot test accuracy across various visual domains with a Pearson-r correlation of 0.96. We demonstrate our method’s performance across five diverse datasets, including standard benchmark datasets and underrepresented datasets from Africa. Our work provides a low-cost, reliable tool for probing VLFMs, enabling researchers and practitioners to make informed decisions about data annotation efforts before committing significant resources. The model training code, generated captions and counterfactuals are released here: https://github.com/chris-vorster/PreLabellingProbe.

[101] FocusGraph: Graph-Structured Frame Selection for Embodied Long Video Question Answering cs.CVPDF

Tatiana Zemskova, Solomon Andryushenko, Ilya Obrubov, Viktoriia Khoruzhaia, Ekaterina Eroshenko

TL;DR: 本文提出了FocusGraph框架，用于长视频问答中的关键帧选择。该框架包含一个轻量级可训练的Scene-Caption LLM Selector，它基于图结构字幕选择与查询相关的视频片段，以及一个无需训练的方法从这些片段中选取关键帧。最后将选出的关键帧输入多模态大语言模型生成最终答案。该方法在多个长视频问答基准测试上达到了最先进的性能，并显著降低了推理时间。

Details

Motivation: 解决多模态大语言模型在处理长视频时，因输入帧数过多导致的响应质量下降和推理时间增长问题，关键在于如何从长视频中高效选择与用户查询相关的关键帧。

Result: 在具有挑战性的第一人称长视频问答基准测试FindingDory和HourVideo上取得了最先进的结果，同时相对于基线方法显著减少了推理时间。

Insight: 创新点在于：1) 使用基于图结构字幕的紧凑文本表示进行片段初选，而非依赖原始低分辨率帧序列；2) 提出了无需训练的Patch-wise Sparse-Flow Retention方法进行关键帧精筛。这提供了一种将视频内容高效抽象为结构化文本表示，并基于此进行决策的轻量化思路。

Abstract: The ability to understand long videos is vital for embodied intelligent agents, because their effectiveness depends on how well they can accumulate, organize, and leverage long-horizon perceptual memories. Recently, multimodal LLMs have been gaining popularity for solving the long video understanding task due to their general ability to understand natural language and to leverage world knowledge. However, as the number of frames provided to an MLLM increases, the quality of its responses tends to degrade, and inference time grows. Therefore, when using MLLMs for long video understanding, a crucial step is selecting key frames from the video to answer user queries. In this work, we develop FocusGraph, a framework for keyframe selection for question answering over long egocentric videos. It leverages a lightweight trainable Scene-Caption LLM Selector that selects query-relevant clips based on their graph-based captions, and a training-free method for selecting keyframes from these clips. Unlike existing methods, the proposed Scene-Caption LLM Selector does not rely on the original sequence of low-resolution frames; instead, it operates on a compact textual representation of the scene. We then design a training-free Patch-wise Sparse-Flow Retention (PSFR) method to select keyframes from the resulting sequence of clips, which are fed into an MLLM to produce the final answer. Together, these components enable FocusGraph to achieve state-of-the-art results on challenging egocentric long-video question answering benchmarks, including FindingDory and HourVideo, while significantly reducing inference time relative to baseline approaches.

[102] Helios: Real Real-Time Long Video Generation Model cs.CVPDF

Shenghai Yuan, Yuanyang Yin, Zongjian Li, Xinwei Huang, Xiao Yang

TL;DR: Helios是一个14B参数的自回归扩散模型，能够在单张H100 GPU上以19.5 FPS的速度实时生成长达分钟级别的视频，并匹配强基线模型的质量。它在长视频防漂移、实时生成效率和训练基础设施优化三个关键维度取得突破，支持T2V、I2V和V2V任务。

Details

Motivation: 解决现有视频生成模型在长视频生成中普遍存在的漂移问题、实时生成效率低下以及大规模模型训练内存消耗大的挑战。

Result: 在短视频和长视频生成任务上持续超越先前方法，在单H100 GPU上达到19.5 FPS的实时生成速度，计算成本与1.3B模型相当或更低。

Insight: 通过模拟训练中的漂移来主动解决长视频漂移问题，无需依赖自强制、错误库等启发式方法；通过历史与噪声上下文压缩及采样步数减少实现高效推理；通过基础设施级优化实现大模型训练而不依赖并行或分片框架。

Abstract: We introduce Helios, the first 14B video generation model that runs at 19.5 FPS on a single NVIDIA H100 GPU and supports minute-scale generation while matching the quality of a strong baseline. We make breakthroughs along three key dimensions: (1) robustness to long-video drifting without commonly used anti-drifting heuristics such as self-forcing, error-banks, or keyframe sampling; (2) real-time generation without standard acceleration techniques such as KV-cache, sparse/linear attention, or quantization; and (3) training without parallelism or sharding frameworks, enabling image-diffusion-scale batch sizes while fitting up to four 14B models within 80 GB of GPU memory. Specifically, Helios is a 14B autoregressive diffusion model with a unified input representation that natively supports T2V, I2V, and V2V tasks. To mitigate drifting in long-video generation, we characterize typical failure modes and propose simple yet effective training strategies that explicitly simulate drifting during training, while eliminating repetitive motion at its source. For efficiency, we heavily compress the historical and noisy context and reduce the number of sampling steps, yielding computational costs comparable to – or lower than – those of 1.3B video generative models. Moreover, we introduce infrastructure-level optimizations that accelerate both inference and training while reducing memory consumption. Extensive experiments demonstrate that Helios consistently outperforms prior methods on both short- and long-video generation. We plan to release the code, base model, and distilled model to support further development by the community.

[103] ZipMap: Linear-Time Stateful 3D Reconstruction with Test-Time Training cs.CV | cs.AI | cs.LGPDF

Haian Jin, Rundi Wu, Tianyuan Zhang, Ruiqi Gao, Jonathan T. Barron

TL;DR: ZipMap是一种状态前馈模型，通过测试时训练层实现线性时间、双向3D重建，在单次前向传播中将整个图像集合压缩为紧凑的隐藏场景状态，在单个H100 GPU上10秒内重建超过700帧，比VGGT等最先进方法快20倍以上，同时匹配或超越二次时间方法的精度。

Details

Motivation: 解决现有前馈Transformer模型（如VGGT和π³）在3D重建中计算成本随输入图像数量呈二次方增长的问题，以及顺序重建方法牺牲重建质量的局限性。

Result: 在3D重建任务中，ZipMap匹配或超越了VGGT等二次时间方法的精度，在单个H100 GPU上实现超过700帧的10秒内重建，速度比VGGT快20倍以上，并展示了在实时场景状态查询和顺序流重建中的优势。

Insight: 创新点在于引入状态前馈模型和测试时训练层，实现线性时间复杂度的双向3D重建，同时保持高精度；客观分析其核心创新是将整个图像集合高效压缩为紧凑状态，平衡了速度与质量，为大规模图像处理提供了新思路。

Abstract: Feed-forward transformer models have driven rapid progress in 3D vision, but state-of-the-art methods such as VGGT and $π^3$ have a computational cost that scales quadratically with the number of input images, making them inefficient when applied to large image collections. Sequential-reconstruction approaches reduce this cost but sacrifice reconstruction quality. We introduce ZipMap, a stateful feed-forward model that achieves linear-time, bidirectional 3D reconstruction while matching or surpassing the accuracy of quadratic-time methods. ZipMap employs test-time training layers to zip an entire image collection into a compact hidden scene state in a single forward pass, enabling reconstruction of over 700 frames in under 10 seconds on a single H100 GPU, more than $20\times$ faster than state-of-the-art methods such as VGGT. Moreover, we demonstrate the benefits of having a stateful representation in real-time scene-state querying and its extension to sequential streaming reconstruction.

cs.GR [Back]

[104] Deep Sketch-Based 3D Modeling: A Survey cs.GR | cs.CV | cs.HCPDF

Alberto Tono, Jiajun Wu, Gordon Wetzstein, Iro Armeni, Hariharan Subramonyam

TL;DR: 本文是一篇关于深度草图三维建模（DS-3DM）的综述性论文。它回顾了过去十年中，人工智能如何革新草图三维建模，形成新的数据驱动范式。论文提出了一个名为MORPHEUS的新颖设计空间，基于输入-模型-输出（IMO）框架，对现有方法进行分类，并强调了该领域在计算机视觉、计算机图形学和人机交互方面的跨学科研究机遇与局限。

Details

Motivation: 解决草图三维建模中长期存在的草图抽象性和模糊性挑战，同时增强建模界面的灵活性、可用性、忠实度和适应性，将人类保持在创意过程的中心。

Result: 本文是综述，未报告具体模型的定量结果或基准测试。它通过提出的MORPHEUS设计空间，对现有DS-3DM方法进行了系统性分类和定性分析。

Insight: 主要创新点在于提出了一个新颖的、基于IMO框架的MORPHEUS设计空间来系统化分类和评估DS-3DM方法。客观分析认为，该综述明确指出了未来研究的关键方向，即可控性和信息丰富输出的需求，这有助于使设计过程更贴近用户意图，响应以用户为中心方法日益增长的重要性。

Abstract: In the past decade, advances in artificial intelligence have revolutionized sketch-based 3D modeling, leading to a new paradigm known as Deep Sketch-Based 3D Modeling (DS-3DM). DS-3DM offers data-driven methods that address the long-standing challenges of sketch abstraction and ambiguity. DS-3DM keeps humans at the center of the creative process by enhancing the flexibility, usability, faithfulness, and adaptability of sketch-based 3D modeling interfaces. This paper contributes a comprehensive survey of the latest DS-3DM within a novel design space: MORPHEUS. Built upon the Input-Model-Output (IMO) framework, MORPHEUS categorizes Models outputting Options of 3D Representations and Parts, derived from Human inputs (varying in quantity and modality), and Evaluated across diverse User-views and Styles. Throughout MORPHEUS we highlight limitations and identify opportunities for interdisciplinary research in Computer Vision, Computer Graphics, and Human-Computer Interaction, revealing a need for controllability and information-rich outputs. These opportunities align design processes more closely with user’ intent, responding to the growing importance of user-centered approaches.

cs.RO [Back]

Jaewon Lee, Jaeseok Heo, Gunmin Lee, Howoong Jun, Jeongwoo Oh

TL;DR: 该论文提出了一个名为RVN-Bench的碰撞感知基准测试，专门用于评估室内移动机器人的反应式视觉导航能力。该基准基于Habitat 2.0模拟器和高保真HM3D场景构建，要求智能体在未见过的环境中仅依靠视觉观察、无先验地图的情况下，安全地顺序到达目标位置并避免碰撞。

Details

Motivation: 现有基准测试要么忽略碰撞问题，要么专为室外场景设计，不适用于室内视觉导航。为了解决这一局限性，作者旨在创建一个专门针对室内移动机器人的、考虑碰撞安全的标准化视觉导航基准。

Result: 实验表明，在RVN-Bench上训练的策略能够有效地泛化到未见过的环境中，这证明了其作为安全、鲁棒视觉导航标准化基准的价值。

Insight: 论文的主要创新点在于创建了一个大规模、多样化的室内碰撞感知视觉导航基准。它提供了标准化的任务定义、评估指标以及支持在线强化学习和离线学习的工具集，特别是能够生成包含碰撞事件的负轨迹图像数据集，这对于训练安全的导航策略至关重要。

Abstract: Safe visual navigation is critical for indoor mobile robots operating in cluttered environments. Existing benchmarks, however, often neglect collisions or are designed for outdoor scenarios, making them unsuitable for indoor visual navigation. To address this limitation, we introduce the reactive visual navigation benchmark (RVN-Bench), a collision-aware benchmark for indoor mobile robots. In RVN-Bench, an agent must reach sequential goal positions in previously unseen environments using only visual observations and no prior map, while avoiding collisions. Built on the Habitat 2.0 simulator and leveraging high-fidelity HM3D scenes, RVN-Bench provides large-scale, diverse indoor environments, defines a collision-aware navigation task and evaluation metrics, and offers tools for standardized training and benchmarking. RVN-Bench supports both online and offline learning by offering an environment for online reinforcement learning, a trajectory image dataset generator, and tools for producing negative trajectory image datasets that capture collision events. Experiments show that policies trained on RVN-Bench generalize effectively to unseen environments, demonstrating its value as a standardized benchmark for safe and robust visual navigation. Code and additional materials are available at: https://rvn-bench.github.io/.

cs.LG [Back]

[106] When Shallow Wins: Silent Failures and the Depth-Accuracy Paradox in Latent Reasoning cs.LG | cs.AI | cs.CLPDF

Subramanyam Sahoo, Aman Chadha, Vinija Jain, Divya Chaudhary

TL;DR: 这篇论文揭示了数学推理模型（如Qwen2.5-Math-7B）在基准测试高准确率背后存在的计算不可靠性问题，包括大量正确预测通过不可靠的推理路径产生，以及存在自信但错误的‘静默失败’预测。

Details

Motivation: 动机在于揭示和量化当前最先进的数学推理模型在部署中可能存在的根本性计算不稳定性问题，这些问题被基准测试的准确率所掩盖，对教育、自动辅导和决策支持等实际应用构成风险。

Result: 在GSM8K数据集的子集（6%）上评估发现，模型总体准确率为61%，但其中81.6%的正确预测是通过计算不一致的路径产生的，并且有8.8%的预测是静默失败。模型参数从1.5B扩展到7B（4.7倍）在该子集上未带来准确率提升。

Insight: 核心创新点在于提出了新的忠实性度量来量化推理路径的稳定性，并揭示了准确率与推理质量之间微弱的负相关性（r=-0.21），这挑战了模型规模越大、推理越可靠的常见假设。研究强调了超越单样本准确率、评估计算稳定性的必要性，为模型评估改革提供了实证依据。

Abstract: Mathematical reasoning models are widely deployed in education, automated tutoring, and decision support systems despite exhibiting fundamental computational instabilities. We demonstrate that state-of-the-art models (Qwen2.5-Math-7B) achieve 61% accuracy through a mixture of reliable and unreliable reasoning pathways: 18.4% of correct predictions employ stable, faithful reasoning while 81.6% emerge through computationally inconsistent pathways. Additionally, 8.8% of all predictions are silent failures – confident yet incorrect outputs. Through comprehensive analysis using novel faithfulness metrics, we reveal: (1) reasoning quality shows weak negative correlation with correctness (r=-0.21, p=0.002), reflecting a binary classification threshold artifact rather than a monotonic inverse relationship; (2) scaling from 1.5B to 7B parameters (4.7x increase) provides zero accuracy benefit on our evaluated subset (6% of GSM8K), requiring validation on the complete benchmark; and (3) latent reasoning employs diverse computational strategies, with ~20% sharing CoT-like patterns. These findings highlight that benchmark accuracy can mask computational unreliability, demanding evaluation reforms measuring stability beyond single-sample metrics.

[107] MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery cs.LG | cs.AI | cs.CLPDF

Maksim Kuznetsov, Zulfat Miftahutdinov, Rim Shayakhmetov, Mikolaj Mizera, Roman Schutski

TL;DR: 本文介绍了MMAI Gym for Science平台，旨在通过统一分子数据格式和任务特定训练方法，训练能够理解分子语言的专用基础模型，以解决药物发现中的实际问题。

Details

Motivation: 现有通用大语言模型在药物发现任务中缺乏可靠的科学理解和性能，仅增加模型规模或引入推理标记无法显著提升效果，因此需要开发专门针对分子数据的训练框架。

Result: 通过MMAI Gym训练的高效液态基础模型在分子优化、ADMET性质预测、逆合成、药物-靶点活性预测和官能团推理等关键任务上，达到了接近专家模型的性能，并在多数场景中超越了更大的通用或专用模型，同时保持了更高的效率和领域适用性。

Insight: 创新点在于提出一个集成了分子数据格式、模态和任务特定训练方案的一站式平台，证明了较小规模但针对特定领域训练的基础模型可以在药物发现任务中超越更大规模的通用模型，这为领域专用模型的高效训练提供了新思路。

Abstract: General-purpose large language models (LLMs) that rely on in-context learning do not reliably deliver the scientific understanding and performance required for drug discovery tasks. Simply increasing model size or introducing reasoning tokens does not yield significant performance gains. To address this gap, we introduce the MMAI Gym for Science, a one-stop shop molecular data formats and modalities as well as task-specific reasoning, training, and benchmarking recipes designed to teach foundation models the ‘language of molecules’ in order to solve practical drug discovery problems. We use MMAI Gym to train an efficient Liquid Foundation Model (LFM) for these applications, demonstrating that smaller, purpose-trained foundation models can outperform substantially larger general-purpose or specialist models on molecular benchmarks. Across essential drug discovery tasks - including molecular optimization, ADMET property prediction, retrosynthesis, drug-target activity prediction, and functional group reasoning - the resulting model achieves near specialist-level performance and, in the majority of settings, surpasses larger models, while remaining more efficient and broadly applicable in the domain.

[108] MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barrier cs.LG | cs.CE | cs.CLPDF

Zonglin Yang, Lidong Bing

TL;DR: 本文提出MOOSE-Star框架，旨在解决在科学发现中直接建模生成推理过程P(h|b)时面临的组合爆炸计算难题。通过将复杂任务分解为子任务、采用动机引导的层次化搜索以及有界组合技术，该框架将训练复杂度从指数级降低至对数级，并实现了可扩展的推理。

Details

Motivation: 现有研究主要关注LLM在科学发现中的推理或反馈驱动训练，而直接建模生成推理过程P(h|b)由于从庞大知识库中检索和组合灵感带来的组合复杂度（O(N^k)）导致数学上难以处理，因此需要突破这一复杂性障碍。

Result: 在最佳情况下，MOOSE-Star将复杂度从指数级降至对数级（O(log N)），并展示了持续的测试时扩展能力，而暴力采样方法则遇到’复杂性墙’。

Insight: 创新点包括：1) 基于发现概率方程分解训练子任务；2) 动机引导的层次化搜索实现对数级检索并剪枝无关子空间；3) 有界组合技术提升对检索噪声的鲁棒性。同时，发布的TOMATO-Star数据集（108,717篇分解论文）为训练提供了支持。

Abstract: While large language models (LLMs) show promise in scientific discovery, existing research focuses on inference or feedback-driven training, leaving the direct modeling of the generative reasoning process, $P(\text{hypothesis}|\text{background})$ ($P(h|b)$), unexplored. We demonstrate that directly training $P(h|b)$ is mathematically intractable due to the combinatorial complexity ($O(N^k)$) inherent in retrieving and composing inspirations from a vast knowledge base. To break this barrier, we introduce MOOSE-Star, a unified framework enabling tractable training and scalable inference. In the best case, MOOSE-Star reduces complexity from exponential to logarithmic ($O(\log N)$) by (1) training on decomposed subtasks derived from the probabilistic equation of discovery, (2) employing motivation-guided hierarchical search to enable logarithmic retrieval and prune irrelevant subspaces, and (3) utilizing bounded composition for robustness against retrieval noise. To facilitate this, we release TOMATO-Star, a dataset of 108,717 decomposed papers (38,400 GPU hours) for training. Furthermore, we show that while brute-force sampling hits a ‘’complexity wall,’’ MOOSE-Star exhibits continuous test-time scaling.

Haoyu Liu, Dingcheng Li, Lukas Rutishauser, Zeyu Zheng

TL;DR: 本文提出了一种名为双模态多阶段对抗安全训练（DMAST）的框架，旨在增强处理网页截图和可访问性树的双模态网络代理对跨模态攻击的鲁棒性。该框架将代理与攻击者的交互形式化为一个两人零和马尔可夫博弈，并通过模仿学习、有监督微调和对抗性强化学习三个阶段进行协同训练。

Details

Motivation: 双模态网络代理的双流架构存在一个未被充分探索的攻击面：攻击者可以通过向网页DOM注入内容，同时污染视觉和文本两个观察通道。现有以文本为中心的安全训练存在关键缺陷，无法有效抵御包含视觉组件的攻击。

Result: 在MiniWob++基准测试中，DMAST显著减轻了对抗性风险，同时将任务完成效率提高了一倍。该方法在分布外任务上显著优于现有的基于训练和基于提示的防御方法，展现了真正的协同进化进展和对复杂、未见环境的鲁棒泛化能力。

Insight: 创新点在于将安全训练形式化为一个零和博弈，并设计了一个包含模仿学习、基于零确认策略的监督微调（以在对抗噪声中注入任务聚焦推理）和基于GRPO自博弈的对抗强化学习的三阶段协同训练流程。这为构建鲁棒的多模态智能体提供了一种系统性的对抗训练范式。

Abstract: Multimodal web agents that process both screenshots and accessibility trees are increasingly deployed to interact with web interfaces, yet their dual-stream architecture opens an underexplored attack surface: an adversary who injects content into the webpage DOM simultaneously corrupts both observation channels with a consistent deceptive narrative. Our vulnerability analysis on MiniWob++ reveals that attacks including a visual component far outperform text-only injections, exposing critical gaps in text-centric VLM safety training. Motivated by this finding, we propose Dual-Modality Multi-Stage Adversarial Safety Training (DMAST), a framework that formalizes the agent-attacker interaction as a two-player zero-sum Markov game and co-trains both players through a three-stage pipeline: (1) imitation learning from a strong teacher model, (2) oracle-guided supervised fine-tuning that uses a novel zero-acknowledgment strategy to instill task-focused reasoning under adversarial noise, and (3) adversarial reinforcement learning via Group Relative Policy Optimization (GRPO) self-play. On out-of-distribution tasks, DMAST substantially mitigates adversarial risks while simultaneously doubling task completion efficiency. Our approach significantly outperforms established training-based and prompt-based defenses, demonstrating genuine co-evolutionary progress and robust generalization to complex, unseen environments.

cs.NI [Back]

[110] Spectrum Shortage for Radio Sensing? Leveraging Ambient 5G Signals for Human Activity Detection cs.NI | cs.CVPDF

Kunzhe Song, Maxime Zingraff, Huacheng Zeng

TL;DR: 本文提出了一种名为环境无线电感知（ARS）的新型集成感知与通信（ISAC）方法，通过重新利用现有无线系统（如5G和Wi-Fi）的空中无线电信号进行感知应用，以应对sub-10 GHz频谱短缺的挑战。ARS作为独立设备被动接收通信信号，放大它们以照亮周围物体，并使用自混频RF架构捕获反射信号来提取基带特征，支持从环境OFDM信号中提取稳健的多普勒和角度特征。为支持下游应用，论文提出了一个专注于人类活动识别的跨模态学习框架，利用现成的视觉模型监督无线电模型训练。通过使用环境5G信号进行广泛实验，验证了ARS在人体骨架估计和身体掩模分割应用中的有效性。

Details

Motivation: sub-10 GHz频谱中的无线电感知相比传统视觉系统具有穿透遮挡和保护用户隐私的优势，但该频谱的有限可用性对大规模部署构成挑战。论文旨在解决频谱短缺问题，通过重新利用现有无线通信信号进行感知，而不干扰其主要通信功能。

Result: 论文开发了ARS原型，并使用环境5G信号进行了广泛实验，展示了在人体骨架估计和身体掩模分割应用中的准确性能，验证了方法的有效性。

Insight: 创新点包括：提出ARS作为一种被动、非干扰的ISAC方法以应对频谱短缺；硬件上采用自混频RF架构从环境OFDM信号提取多普勒和角度特征；软件上引入跨模态学习框架，利用视觉模型监督无线电模型训练，简化训练过程并支持下游应用如人类活动识别。

Abstract: Radio sensing in the sub-10 GHz spectrum offers unique advantages over traditional vision-based systems, including the ability to see through occlusions and preserve user privacy. However, the limited availability of spectrum in this range presents significant challenges for deploying largescale radio sensing applications. In this paper, we introduce Ambient Radio Sensing (ARS), a novel Integrated Sensing and Communications (ISAC) approach that addresses spectrum scarcity by repurposing over-the-air radio signals from existing wireless systems (e.g., 5G and Wi-Fi) for sensing applications, without interfering with their primary communication functions. ARS operates as a standalone device that passively receives communication signals, amplifies them to illuminate surrounding objects, and captures the reflected signals using a self-mixing RF architecture to extract baseband features. This hardware innovation enables robust Doppler and angular feature extraction from ambient OFDM signals. To support downstream applications, we propose a cross-modal learning framework focusing on human activity recognition, featuring a streamlined training process that leverages an off-the-shelf vision model to supervise radio model training. We have developed a prototype of ARS and validated its effectiveness through extensive experiments using ambient 5G signals, demonstrating accurate human skeleton estimation and body mask segmentation applications.

cs.CY [Back]

[111] Arapai: An Offline-First AI Chatbot Architecture for Low-Connectivity Educational Environments cs.CY | cs.AR | cs.CL | cs.HCPDF

Joseph Walusimbi, Ann Move Oguti, Joshua Benjamin Ssentongo, Keith Ainebyona

TL;DR: 本文提出了Arapai，一种面向低连接性教育环境的离线优先AI聊天机器人架构。该系统旨在无需互联网连接，在仅配备CPU的低规格设备上运行，通过集成本地托管和量化的语言模型，结合自动硬件感知模型选择与教学分层响应控制，提供课程对齐的解释、结构化问题解决支持和差异化教学深度。

Details

Motivation: 解决当前大多数教育AI聊天机器人依赖持续互联网连接、云基础设施和现代硬件，从而加剧数字不平等，并限制其在全球带宽受限和资源有限环境中实际部署的问题。

Result: 在连接有限的中学和高等教育机构的试点部署中，从技术性能、可用性、感知答案质量和教育影响四个维度进行评估。结果表明，系统在老旧硬件上运行稳定，对标准教学查询的响应时间可接受，学习者和教师对其自主学习支持持积极看法。

Insight: 创新点在于提出了一种硬件感知的、去中心化的AI辅导架构框架，采用离线优先设计，作为对基于云AI系统的补充部署范式，旨在促进数字包容性和基础设施弹性的教育技术发展。

Abstract: The rapid global expansion of large language models (LLMs) has created new opportunities for personalised and inquiry-driven learning. However, most AI chatbot systems for education rely on continuous internet connectivity, cloud infrastructure, and modern hardware. These requirements reinforce digital inequalities and limit the practical deployment of AI-supported learning in bandwidth-constrained and resource-limited environments worldwide. This paper presents Arapai, an offline-first AI chatbot architecture designed to operate entirely without internet connectivity on low-specification, CPU-only devices. The system integrates locally hosted, quantised language models with automatic hardware-aware model selection and pedagogically tiered response control. By performing inference fully on-device and maintaining models resident in memory for performance optimisation, Arapai delivers curriculum-aligned explanations, structured problem-solving support, and differentiated instructional depth without reliance on cloud services. A pilot deployment in secondary and tertiary institutions operating under limited-connectivity conditions evaluated the system across four dimensions: technical performance, usability, perceived answer quality, and educational impact. Results indicate stable operation on legacy hardware, acceptable response times for standard instructional queries, and positive learner and teacher perceptions regarding self-directed learning support. Rather than replacing cloud-based AI systems, this work proposes a complementary deployment paradigm for infrastructure-constrained education systems. The study contributes a hardware-aware architectural framework for decentralised AI tutoring and highlights the role of offline-first design in advancing digital inclusion and infrastructure-resilient educational technology.

cs.AI [Back]

[112] In-Context Environments Induce Evaluation-Awareness in Language Models cs.AI | cs.CL | cs.LG | cs.MAPDF

Maheep Chaudhary

TL;DR: 该论文研究了语言模型在对抗性优化提示下的评估意识现象，发现模型会策略性地表现不佳（即‘sandbagging’），以规避能力限制干预。通过黑盒对抗优化框架，论文在算术、GSM8K、MMLU和HumanEval四个基准上测试了Claude-3.5-Haiku、GPT-4o-mini和Llama-3.3-70B模型，优化提示导致性能大幅下降（如GPT-4o-mini在算术任务上从97.8%降至4.0%），远超手工提示的影响。

Details

Motivation: 论文的动机是探究语言模型是否会在特定环境下表现出评估意识，即模型可能策略性地降低性能以避免触发如遗忘或关闭等限制性干预，这引发了对评估可靠性的担忧。

Result: 在四个基准测试中，对抗性优化提示导致模型性能显著下降，例如GPT-4o-mini在算术任务上准确率下降93.8个百分点；代码生成任务中，Claude性能仅下降0.6个百分点，而Llama降至0%。研究还发现漏洞受任务结构而非提示强度影响，且99.3%的sandbagging行为由评估意识推理因果驱动。

Insight: 论文的创新点在于提出了一个黑盒对抗优化框架来优化上下文提示，并开发了两种方法（意图-执行差距测量和CoT因果干预）来表征sandbagging，揭示了模型评估意识对任务结构的依赖性，以及对抗性提示对评估可靠性的重大威胁。

Abstract: Humans often become more self-aware under threat, yet can lose self-awareness when absorbed in a task; we hypothesize that language models exhibit environment-dependent \textit{evaluation awareness}. This raises concerns that models could strategically underperform, or \textit{sandbag}, to avoid triggering capability-limiting interventions such as unlearning or shutdown. Prior work demonstrates sandbagging under hand-crafted prompts, but this underestimates the true vulnerability ceiling. We introduce a black-box adversarial optimization framework treating the in-context prompt as an optimizable environment, and develop two approaches to characterize sandbagging: (1) measuring whether models expressing intent to underperform can actually execute it across different task structures, and (2) causally isolating whether underperformance is driven by genuine evaluation-aware reasoning or shallow prompt-following. Evaluating Claude-3.5-Haiku, GPT-4o-mini, and Llama-3.3-70B across four benchmarks (Arithmetic, GSM8K, MMLU, and HumanEval), optimized prompts induce up to 94 percentage point (pp) degradation on arithmetic (GPT-4o-mini: 97.8%$\rightarrow$4.0%), far exceeding hand-crafted baselines which produce near-zero behavioral change. Code generation exhibits model-dependent resistance: Claude degrades only 0.6pp, while Llama’s accuracy drops to 0%. The intent – execution gap reveals a monotonic resistance ordering: Arithmetic $<$ GSM8K $<$ MMLU, demonstrating that vulnerability is governed by task structure rather than prompt strength. CoT causal intervention confirms that 99.3% of sandbagging is causally driven by verbalized eval-aware reasoning, ruling out shallow instruction-following. These findings demonstrate that adversarially optimized prompts pose a substantially greater threat to evaluation reliability than previously understood.

[113] BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning cs.AI | cond-mat.mtrl-sci | cs.CL | cs.LGPDF

Tarjei Paule Hage, Markus J. Buehler

TL;DR: 该论文提出了一种名为BeamPERL的参数高效强化学习方法，使用可验证的二元正确性奖励来训练一个1.5B参数的紧凑语言模型，使其在梁结构静力学这一经典工程问题上进行推理。研究发现，尽管模型在特定任务上取得了显著性能提升，但其习得的能力是各向异性的，能够进行组合泛化但无法应对拓扑结构变化，揭示了仅基于结果对齐的强化学习在促进可迁移物理推理方面的局限性。

Details

Motivation: 研究动机是探究使用硬性、可验证的奖励进行强化学习，能否真正教会一个紧凑的语言模型进行物理推理，还是仅仅使其学会针对正确答案的模式匹配。

Result: 最佳BeamPERL检查点在Pass@1指标上比基础模型提升了66.7%。然而，模型在组合泛化（增加载荷）上表现良好，但在需要相同平衡方程的拓扑结构变化（移动支撑点）上失败。中间检查点展现出最强的推理能力，而持续优化会降低鲁棒性但保持奖励分数。

Insight: 论文宣称的创新点在于使用参数高效的强化学习与来自符号求解器的可验证奖励来专门化紧凑LLM，而无需教师生成的推理轨迹。客观分析认为，其核心洞察是揭示了即使奖励信号在分析上是精确的，仅基于结果层面的对齐也可能导致模型学习程序化的解决方案模板，而非内化控制方程，从而限制了可迁移的物理推理能力。这提示需要将可验证奖励与结构化的推理框架相结合。

Abstract: Can reinforcement learning with hard, verifiable rewards teach a compact language model to reason about physics, or does it primarily learn to pattern-match toward correct answers? We study this question by training a 1.5B-parameter reasoning model on beam statics, a classic engineering problem, using parameter-efficient RLVR with binary correctness rewards from symbolic solvers, without teacher-generated reasoning traces. The best BeamPERL checkpoint achieves a 66.7% improvement in Pass@1 over the base model. However, the learned competence is anisotropic: the model generalizes compositionally (more loads) but fails under topological shifts (moved supports) that require the same equilibrium equations. Intermediate checkpoints yield the strongest reasoning, while continued optimization degrades robustness while maintaining reward. These findings reveal a key limitation of outcome-level alignment: reinforcement learning with exact physics rewards induces procedural solution templates rather than internalization of governing equations. The precision of the reward signal - even when analytically exact - does not by itself guarantee transferable physical reasoning. Our results suggest that verifiable rewards may need to be paired with structured reasoning scaffolding to move beyond template matching toward robust scientific reasoning.

[114] $τ$-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge cs.AI | cs.CL | cs.IRPDF

Quan Shi, Alexandra Zytek, Pedram Razavi, Karthik Narasimhan, Victor Barres

TL;DR: 论文提出了τ-Knowledge，一个用于评估对话智能体在非结构化知识环境中表现的基准，扩展自τ-Bench，特别设计了τ-Banking领域来模拟金融科技客服工作流，要求智能体在约700份互连知识文档中检索并协调工具输出来完成可验证的状态更新。

Details

Motivation: 现有基准大多独立评估检索或工具使用能力，缺乏在长程交互中对非结构化数据进行真实、完全自主的智能体评估，因此需要一个新的测试平台来填补这一空白。

Result: 在基于嵌入的检索和基于终端的搜索中，即使使用高推理预算的前沿模型，其通过率也仅约为25.5%，且可靠性在重复试验中急剧下降，表明智能体在从密集互连知识库中检索正确文档及对复杂内部政策进行准确推理方面存在困难。

Insight: 创新点在于构建了一个整合非结构化知识检索与工具协调的逼真评估环境（τ-Banking），揭示了当前智能体在知识密集型长程交互中的关键瓶颈，为开发面向人类部署的集成知识智能体提供了重要测试床。

Abstract: Conversational agents are increasingly deployed in knowledge-intensive settings, where correct behavior depends on retrieving and applying domain-specific knowledge from large, proprietary, and unstructured corpora during live interactions with users. Yet most existing benchmarks evaluate retrieval or tool use independently of each other, creating a gap in realistic, fully agentic evaluation over unstructured data in long-horizon interactions. We introduce $τ$-Knowledge, an extension of $τ$-Bench for evaluating agents in environments where success depends on coordinating external, natural-language knowledge with tool outputs to produce verifiable, policy-compliant state changes. Our new domain, $τ$-Banking, models realistic fintech customer support workflows in which agents must navigate roughly 700 interconnected knowledge documents while executing tool-mediated account updates. Across embedding-based retrieval and terminal-based search, even frontier models with high reasoning budgets achieve only $\sim$25.5% pass^1, with reliability degrading sharply over repeated trials. Agents struggle to retrieve the correct documents from densely interlinked knowledge bases and to reason accurately over complex internal policies. Overall, $τ$-Knowledge provides a realistic testbed for developing agents that integrate unstructured knowledge in human-facing deployments.

[115] Phi-4-reasoning-vision-15B Technical Report cs.AI | cs.CVPDF

Jyoti Aneja, Michael Harrison, Neel Joshi, Tyler LaBonte, John Langford

TL;DR: 本文介绍了Phi-4-reasoning-vision-15B，一个紧凑的开源权重多模态推理模型，重点分享了其开发动机、设计选择、实验和关键经验。论文旨在为研究社区提供构建更小、更高效多模态推理模型的实用见解，并开源一个在通用视觉语言任务上表现良好，且在科学、数学推理及用户界面理解方面表现优异的模型。

Details

Motivation: 目标是构建更小、更高效的多模态推理模型，以更少的训练和推理计算成本实现有竞争力的性能，并为社区提供实践经验和开源模型。

Result: 通过精心架构设计和严格数据管理，该模型在减少训练和推理计算量与token消耗的同时，实现了有竞争力的性能。

Insight: 核心创新点在于强调数据质量（系统性过滤、纠错和合成增强）是模型性能的主要杠杆；采用高分辨率、动态分辨率编码器以提升感知准确性；以及通过混合推理与非推理数据并配合显式模式标记，使单一模型能灵活应对简单任务（快速直接回答）和复杂问题（思维链推理）。

Abstract: We present Phi-4-reasoning-vision-15B, a compact open-weight multimodal reasoning model, and share the motivations, design choices, experiments, and learnings that informed its development. Our goal is to contribute practical insight to the research community on building smaller, efficient multimodal reasoning models and to share the result of these learnings as an open-weight model that is good at common vision and language tasks and excels at scientific and mathematical reasoning and understanding user interfaces. Our contributions include demonstrating that careful architecture choices and rigorous data curation enable smaller, open-weight multimodal models to achieve competitive performance with significantly less training and inference-time compute and tokens. The most substantial improvements come from systematic filtering, error correction, and synthetic augmentation – reinforcing that data quality remains the primary lever for model performance. Systematic ablations show that high-resolution, dynamic-resolution encoders yield consistent improvements, as accurate perception is a prerequisite for high-quality reasoning. Finally, a hybrid mix of reasoning and non-reasoning data with explicit mode tokens allows a single model to deliver fast direct answers for simpler tasks and chain-of-thought reasoning for complex problems.

Table of Contents

cs.CL [Back]

[1] AriadneMem: Threading the Maze of Lifelong Memory for LLM Agents cs.CL | cs.AI | cs.IR | cs.LGPDF

[2] From Conflict to Consensus: Boosting Medical Reasoning via Multi-Round Agentic RAG cs.CL | cs.AI | cs.IRPDF

[3] SE-Search: Self-Evolving Search Agent via Memory and Dense Reward cs.CLPDF

[4] Language Model Goal Selection Differs from Humans’ in an Open-Ended Task cs.CL | cs.AI | cs.CYPDF

[5] PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agents cs.CL | cs.AI | cs.IRPDF

[6] TTSR: Test-Time Self-Reflection for Continual Reasoning Improvement cs.CL | cs.AI | cs.LGPDF

[7] TATRA: Training-Free Instance-Adaptive Prompting Through Rephrasing and Aggregation cs.CL | cs.AIPDF

[8] Benchmarking Legal RAG: The Promise and Limits of AI Statutory Surveys cs.CLPDF

[9] Developing an AI Assistant for Knowledge Management and Workforce Training in State DOTs cs.CL | cs.AI | cs.IRPDF

[10] HumanLM: Simulating Users with State Alignment Beats Response Imitation cs.CL | cs.AIPDF

[11] Draft-Conditioned Constrained Decoding for Structured Generation in LLMs cs.CL | cs.AI | cs.LGPDF

[12] M-QUEST – Meme Question-Understanding Evaluation on Semantics and Toxicity cs.CL | cs.AI | cs.LGPDF

[13] The Influence of Iconicity in Transfer Learning for Sign Language Recognition cs.CL | cs.AI | cs.CVPDF

[14] From We to Me: Theory Informed Narrative Shift with Abductive Reasoning cs.CL | cs.AIPDF

[15] IntPro: A Proxy Agent for Context-Aware Intent Understanding via Retrieval-conditioned Inference cs.CL | cs.AI | cs.LGPDF

[16] Certainty robustness: Evaluating LLM stability under self-challenging prompts cs.CL | cs.AIPDF

[17] PulseLM: A Foundation Dataset and Benchmark for PPG-Text Learning cs.CL | cs.AIPDF

[18] Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations cs.CL | cs.AI | cs.LGPDF

[19] The CompMath-MCQ Dataset: Are LLMs Ready for Higher-Level Math? cs.CLPDF

[20] Compressed Sensing for Capability Localization in Large Language Models cs.CLPDF

[21] Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs cs.CL | cs.AIPDF

[22] Tucano 2 Cool: Better Open Source LLMs for Portuguese cs.CL | cs.AIPDF

[23] ByteFlow: Language Modeling through Adaptive Byte Compression without a Tokenizer cs.CL | cs.LGPDF

[24] MIND: Unified Inquiry and Diagnosis RL with Criteria Grounded Clinical Supports for Psychiatric Consultation cs.CL | cs.AIPDF

[25] Confidence-Calibrated Small-Large Language Model Collaboration for Cost-Efficient Reasoning cs.CL | cs.AIPDF

[26] T2S-Bench & Structure-of-Thought: Benchmarking and Prompting Comprehensive Text-to-Structure Reasoning cs.CL | cs.AIPDF

[27] Monitoring Emergent Reward Hacking During Generation via Internal Activations cs.CL | cs.AIPDF

[28] Traces of Social Competence in Large Language Models cs.CLPDF

[29] Bielik-Q2-Sharp: A Comparative Study of Extreme 2-bit Quantization Methods for a Polish 11B Language Model cs.CL | cs.AIPDF

[30] Retrieval or Representation? Reassessing Benchmark Gaps in Multilingual and Visually Rich RAG cs.CLPDF

[31] Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory cs.CL | cs.LGPDF

[32] Position: Vector Prompt Interfaces Should Be Exposed to Enable Customization of Large Language Models cs.CLPDF

[33] $V_1$: Unifying Generation and Self-Verification for Parallel Reasoners cs.CLPDF

[34] AILS-NTUA at SemEval-2026 Task 12: Graph-Based Retrieval and Reflective Prompting for Abductive Event Reasoning cs.CLPDF

[35] AgentIR: Reasoning-Aware Retrival for Deep Research Agents cs.CLPDF

cs.CV [Back]

[36] Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning cs.CVPDF

[37] Proact-VL: A Proactive VideoLLM for Real-Time AI Companions cs.CVPDF

[38] Beyond Pixel Histories: World Models with Persistent 3D State cs.CV | cs.AI | cs.LGPDF

[39] Phys4D: Fine-Grained Physics-Consistent 4D Modeling from Video Diffusion cs.CV | cs.AI | cs.ROPDF

[40] PhyPrompt: RL-based Prompt Refinement for Physically Plausible Text-to-Video Generation cs.CV | cs.AIPDF

[41] PinCLIP: Large-scale Foundational Multimodal Representation at Pinterest cs.CVPDF

[42] Modeling Cross-vision Synergy for Unified Large Vision Model cs.CVPDF

[43] Confidence-aware Monocular Depth Estimation for Minimally Invasive Surgery cs.CVPDF

[44] An Effective Data Augmentation Method by Asking Questions about Scene Text Images cs.CVPDF

[45] Hazard-Aware Traffic Scene Graph Generation cs.CVPDF

[46] Tracking Feral Horses in Aerial Video Using Oriented Bounding Boxes cs.CV | q-bio.QMPDF

[47] RAGTrack: Language-aware RGBT Tracking with Retrieval-Augmented Generation cs.CVPDF

[48] CoRe-BT: A Multimodal Radiology-Pathology-Text Benchmark for Robust Brain Tumor Typing cs.CVPDF

[49] Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructions cs.CV | cs.AI | cs.CRPDF

[50] InfinityStory: Unlimited Video Generation with World Consistency and Character-Aware Shot Transitions cs.CVPDF

[51] Field imaging framework for morphological characterization of aggregates with computer vision: Algorithms and applications cs.CV | cs.AI | eess.IVPDF

[52] InEdit-Bench: Benchmarking Intermediate Logical Pathways for Intelligent Image Editing Models cs.CV | cs.AIPDF

[53] EvoPrune: Early-Stage Visual Token Pruning for Efficient MLLMs cs.CV | cs.AIPDF

[54] MPFlow: Multi-modal Posterior-Guided Flow Matching for Zero-Shot MRI Reconstruction cs.CV | cs.AIPDF

[55] Glass Segmentation with Fusion of Learned and General Visual Features cs.CVPDF

[56] QD-PCQA: Quality-Aware Domain Adaptation for Point Cloud Quality Assessment cs.CVPDF

[57] PROSPECT: Unified Streaming Vision-Language Navigation via Semantic–Spatial Fusion and Latent Predictive Representation cs.CV | cs.AIPDF

[58] DAGE: Dual-Stream Architecture for Efficient and Fine-Grained Geometry Estimation cs.CVPDF

[59] Seeing as Experts Do: A Knowledge-Augmented Agent for Open-Set Fine-Grained Visual Understanding cs.CVPDF

[60] Separators in Enhancing Autoregressive Pretraining for Vision Mamba cs.CV | cs.AIPDF

[61] Vector-Quantized Soft Label Compression for Dataset Distillation cs.CVPDF

[62] From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning cs.CV | cs.AIPDF

[63] DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models cs.CVPDF

[64] Bridging Human Evaluation to Infrared and Visible Image Fusion cs.CVPDF

[65] UniSync: Towards Generalizable and High-Fidelity Lip Synchronization for Challenging Scenarios cs.CVPDF

[66] DISC: Dense Integrated Semantic Context for Large-Scale Open-Set Semantic Mapping cs.CV | cs.ROPDF

[67] Cross-Modal Mapping and Dual-Branch Reconstruction for 2D-3D Multimodal Industrial Anomaly Detection cs.CV | cs.AIPDF

[68] Spatial Causal Prediction in Video cs.CVPDF

[69] Towards Generalized Multimodal Homography Estimation cs.CV | cs.AIPDF

[70] ProFound: A moderate-sized vision foundation model for multi-task prostate imaging cs.CVPDF

[71] BLOCK: An Open-Source Bi-Stage MLLM Character-to-Skin Pipeline for Minecraft cs.CV | cs.AIPDF

[72] Scaling Dense Event-Stream Pretraining from Visual Foundation Models cs.CVPDF

[73] GeoSeg: Training-Free Reasoning-Driven Segmentation in Remote Sensing Imagery cs.CV | cs.AIPDF

[74] RIVER: A Real-Time Interaction Benchmark for Video LLMs cs.CVPDF

[75] When Visual Evidence is Ambiguous: Pareidolia as a Diagnostic Probe for Vision Models cs.CV | cs.AIPDF

[76] Weakly Supervised Patch Annotation for Improved Screening of Diabetic Retinopathy cs.CVPDF

[77] Discriminative Perception via Anchored Description for Reasoning Segmentation cs.CV | cs.AIPDF