cs.CL [Total: 30]
cs.CV [Total: 97]
cs.AI [Total: 6]
physics.geo-ph [Total: 1]
cs.SE [Total: 1]
cs.MM [Total: 1]
cs.CR [Total: 1]
cs.LG [Total: 14]
eess.IV [Total: 1]
cs.IR [Total: 2]
cs.RO [Total: 5]

cs.CL [Back]

[1] ClinicalBench: Stress-Testing Assertion-Aware Retrieval for Cross-Admission Clinical QA on MIMIC-IV cs.CL | cs.AI | cs.IRPDF

Alex Stinard

TL;DR: 该论文提出了ClinicalBench，一个用于评估临床问答系统中检索性能的基准测试，重点关注真实电子健康记录（EHR）中的否定、时序性和归属关系等断言敏感问题。论文还提出了EpiKG方法，通过构建带有断言标签和时序标签的患者知识图谱，并根据问题意图进行检索路由，以提升检索质量。

Details

Motivation: 现有临床推理基准通常在干净的输入上评估性能，而忽略了真实EHR笔记中存在的否定、时序性和患者/家属归属等复杂断言问题，这些问题可能导致检索错误并影响最终答案的正确性。因此，需要一个新的基准来专门评估和提升在真实、复杂临床数据上的检索能力。

Result: 在包含400个问题、涉及43名MIMIC-IV患者的ClinicalBench基准上，EpiKG方法相比密集检索基线（Contriever）在主要终点（50个一致严格项目）上带来了+22.0个百分点的显著提升（p=0.0192）。其架构创新（意图感知的KG-RAG）相比基线带来了+8.84个百分点的提升（p=1.79e-3），在理想意图下可达+12.43个百分点。此外，研究发现随着LLM自身基线性能的提高，EpiKG带来的增益会减小。

Insight: 论文的核心创新在于提出了一个专门针对临床断言敏感问题的检索基准（ClinicalBench）和一种结合了断言标签、时序标签及问题意图路由的知识图谱增强检索方法（EpiKG）。一个重要的方法论发现是，自动生成的参考答案有56%存在缺陷，这强调了在临床QA基准构建中引入医生人工裁决的必要性。从客观角度看，将细粒度的临床语义（如断言、时序）结构化并融入检索过程，是针对领域特定挑战的有效策略。

Abstract: Reasoning benchmarks measure clinical performance on clean inputs. We evaluate the step before reasoning: retrieval over real EHR notes, where negation, temporality, and family-versus-patient attribution can flip a correct answer to a wrong one. EpiKG carries an assertion label and a temporality tag with every fact in a patient knowledge graph, then routes retrieval by question intent. ClinicalBench is a 400-question test over 43 MIMIC-IV patients across 9 assertion-sensitive categories. A 7-condition ablation tests each piece of EpiKG across six LLMs (Claude Opus 4.6, GPT-OSS 20B, MedGemma 27B, Gemma 4 31B, MedGemma 1.5 4B, Qwen 3.5 35B). Three physicians blindly adjudicated 100 paired items. The author-blind primary endpoint, leave-author-out paired exact McNemar on 50 unanimous-strict items rated by two external physicians, yields +22.0 percentage points (95 percent Newcombe CI [+5.1, +31.5], p=0.0192). The architectural novelty, intent-aware KG-RAG over a Contriever dense-RAG baseline (C2b to C4g_kw on the change-excluded n=362 endpoint), is +8.84 percentage points (paired McNemar p=1.79e-3); +12.43 percentage points under oracle intent. Sensitivities agree directionally: three-rater physician majority +24.0 percentage points (subject to single-author circularity); deterministic keyword reproducibility proxy +39.5 percentage points. Across the six models, the gain shrinks as the LLM-alone baseline rises (beta=-1.123, r=-0.921, p=0.009). With n=6 this looks more like regression to the mean than encoding substituting for model size. Physician adjudication identified 56 percent of auto-generated reference answers as defective, a methodological finding indicating that NLP-pipeline clinical-QA benchmarks require physician adjudication to be usable. ClinicalBench, the frozen evaluator, three-rater adjudication data, and the EpiKG output stack are publicly released.

[2] The Bicameral Model: Bidirectional Hidden-State Coupling Between Parallel Language Models cs.CL | cs.AI | cs.LGPDF

Cedric Flamant, Udaya Ghai, Kanna Shimizu

TL;DR: 本文提出了一种名为“两院制模型”的新架构，它通过一个可训练的神经接口，将两个冻结的预训练语言模型在中间隐藏状态层面进行双向耦合，实现连续、并发的通信，从而协同完成任务，如驱动任务、操作工具、解决约束或执行代码。

Details

Motivation: 现有多模型和工具增强系统通常通过生成文本进行通信，所有交换都通过输出词汇序列化。本文旨在探索两个预训练语言模型是否可以通过连续、并发的通道进行协调，以更高效地利用模型能力。

Result: 在算术任务中，耦合两个0.5B模型与计算器可将准确率从36%提升至96%；在逻辑网格谜题（ZebraLogic）上，耦合两个0.6B模型与Z3求解器达到未增强基线的1.7倍；在数学推理任务中，耦合Python沙箱使辅助模型仅通过隐藏状态信号就能生成特定问题代码，而无需看到问题文本。

Insight: 创新点在于通过可训练的神经接口（包括翻译网络和学习抑制门）实现两个冻结语言模型在隐藏状态层面的双向耦合，学习选择性通信协议，无需预设格式，仅从任务损失中学习，这为多模型协作提供了新的连续通信范式。

Abstract: Existing multi-model and tool-augmented systems communicate by generating text, serializing every exchange through the output vocabulary. Can two pretrained language models instead coordinate through a continuous, concurrent channel? The Bicameral Model couples two frozen language models through a trainable neural interface on their intermediate hidden states. At every generation step, both models run in lockstep: a primary model drives the task while an auxiliary model operates tools, solves constraints, or executes code, with both conditioning on each other’s activations through a translation network and a learned suppression gate ($\sim$1% of combined parameters). The gate learns a selective communication protocol from task loss alone, without a prescribed format. We demonstrate the mechanism across three tool backends. On arithmetic, coupling two 0.5B models with a calculator raises accuracy from 36% to 96%. On logic grid puzzles, coupling two 0.6B models with a Z3 solver achieves $1.7\times$ the unaugmented baseline on ZebraLogic. On mathematical reasoning, coupling with a Python sandbox enables the auxiliary to generate problem-specific code from hidden-state signals alone, without ever seeing the problem text.

[3] ReVision: Scaling Computer-Use Agents via Temporal Visual Redundancy Reduction cs.CLPDF

Amirhossein Abaskohi, Yuhang He, Peter West, Giuseppe Carenini, Pranit Chawla

TL;DR: 本文提出ReVision方法，通过减少计算机使用代理（CUAs）中视觉观察的时间冗余来提升效率。该方法训练一个补丁选择器，在连续屏幕截图间比较并移除冗余视觉补丁，同时保留模型所需的空间结构，从而显著降低视觉令牌消耗。

Details

Motivation: 计算机使用代理依赖图形用户界面的视觉观察，每个屏幕截图编码为大量视觉令牌，导致随着交互轨迹增长令牌成本急剧增加，限制了在固定上下文和计算预算下可纳入的历史信息量，从而阻碍了性能提升。

Result: 在OSWorld、WebTailBench和AgentNetBench三个基准测试中，使用Qwen2.5-VL-7B模型处理包含5个历史屏幕截图的轨迹时，ReVision平均减少约46%的令牌使用，同时成功率比无丢弃基线提高3%，实现了效率增益。

Insight: 创新点在于通过学习的补丁选择器动态减少视觉冗余，使代理能处理更长轨迹；客观分析表明，视觉历史性能饱和并非源于过去信息无用，而是低效令牌表示所致，冗余移除后性能随历史增加持续提升。

Abstract: Computer-use agents~(CUAs) rely on visual observations of graphical user interfaces, where each screenshot is encoded into a large number of visual tokens. As interaction trajectories grow, the token cost increases rapidly, limiting the amount of history that can be incorporated under fixed context and compute budgets. This has resulted in no or very limited improvement in the performance when using history unlike other domains. We address this inefficiency by introducing ReVision, which is used to train multimodal language models on trajectories where redundant visual patches are removed using a learned patch selector that compares patch representations across consecutive screenshots while preserving spatial structure required by the model. Across three benchmarks, OSWorld, WebTailBench, and AgentNetBench, when processing trajectories with 5 history screenshots using Qwen2.5-VL-7B, ReVision reduces token usage by approximately 46% on average while improving success rate by 3% over the no drop baseline. This establishes a clear efficiency gain, enabling agents to process longer trajectories with fewer tokens. With this improved efficiency, we revisit the role of history in CUAs and find that performance continues to improve as more past observations are incorporated when redundancy is removed. This suggests that the commonly observed saturation in visual history is not due to limited usefulness of past information, but rather a consequence of inefficient token representations.

[4] HEBATRON: A Hebrew-Specialized Open-Weight Mixture-of-Experts Language Model cs.CLPDF

Noam Kayzer, Dan Revital, Ori Bar Joseph, Smadar Arvatz, Or Levi

TL;DR: Hebatron是一个基于NVIDIA Nemotron-3稀疏专家混合架构的希伯来语专用开源大语言模型。它采用三阶段由易到难的课程学习策略，并结合持续抗遗忘锚定技术进行训练，随后在200万希伯来语-英语双语样本上进行监督微调。该模型在希伯来语推理任务上平均达到73.8%的准确率，超越了DictaLM-3.0-24B-Thinking，并与Gemma-3-27B-IT在GSM8K-HE和以色列常识问答任务上表现相当，同时每次前向传播仅激活30B总参数中的3B参数，在长达65,536个token的上下文长度下实现了约9倍的推理吞吐量提升。

Details

Motivation: 解决希伯来语及闪米特语族自然语言处理研究中缺乏专用、开源且支持长上下文的大语言模型的问题，旨在为这些语言提供高性能的AI工具。

Result: 在希伯来语推理任务上平均准确率为73.8%，超越了DictaLM-3.0-24B-Thinking（68.9%），并在GSM8K-HE和Israeli Trivia基准测试上与Gemma-3-27B-IT保持竞争力。模型每次前向传播仅激活3B参数，在长达65,536 token的上下文下，推理吞吐量提升约9倍。

Insight: 创新点包括：首次将Nemotron-3稀疏MoE架构针对特定语言（希伯来语）进行适配；采用了三阶段由易到难的课程学习与持续抗遗忘锚定相结合的训练策略，仅此策略就带来了3个百分点的基准测试提升；是首个开源、支持原生长上下文的希伯来语专用MoE模型，在保持高性能的同时显著提升了推理效率。

Abstract: We present Hebatron, a Hebrew-specialized open-weight large language model built on the NVIDIA Nemotron-3 sparse Mixture-of-Experts architecture. Training employs a three-phase easy-to-hard curriculum with continuous anti-forgetting anchoring, followed by supervised fine-tuning on 2 million bilingual Hebrew–English samples. The curriculum ordering alone yields a 3-point aggregate benchmark gain over the reversed configuration. Hebatron achieves a Hebrew reasoning average of 73.8%, outperforming DictaLM-3.0-24B-Thinking (68.9%) and remaining competitive with Gemma-3-27B-IT on GSM8K-HE and Israeli Trivia, while activating only 3B parameters per forward pass across a 30B-parameter model, delivering approximately 9 times higher inference throughput at native context lengths up to 65,536 tokens. To our knowledge, this is the first language-specific adaptation of the Nemotron-3 architecture for any target language, and the first open-weight Hebrew-specialized MoE model with native long-context support. Model weights are released openly to support further research in Hebrew and Semitic-language NLP.

[5] ReAD: Reinforcement-Guided Capability Distillation for Large Language Models cs.CL | cs.AIPDF

Xueqi Cheng, Xugui Zhou, Tyler Derr, Yushun Dong

TL;DR: 本文提出ReAD框架，一种基于强化学习引导的能力蒸馏方法，用于在固定token预算下将大语言模型压缩为小模型。该方法通过识别任务关键能力、动态生成能力监督信号，并利用不确定性感知的上下文赌博机自适应分配蒸馏预算，以优化下游任务效用并减少有害的能力溢出。

Details

Motivation: 现有能力蒸馏方法通常将不同能力视为独立的训练目标，忽略了能力间的相互依赖关系，导致在固定token预算下，提升某一能力可能对其他有用能力产生负面影响，且额外预算带来的任务相关收益有限。

Result: 在多个基准测试上的广泛实验表明，ReAD在相同token预算下相比强基线方法提升了下游任务效用，同时减少了有害的能力溢出和蒸馏努力浪费。

Insight: 创新点在于首次系统研究了固定预算下能力蒸馏中的跨能力转移现象，并提出了一个显式建模能力相互依赖性的强化学习引导框架，通过动态生成监督和自适应预算分配来优化整体能力轮廓。

Abstract: Capability distillation applies knowledge distillation to selected model capabilities, aiming to compress a large language model (LLM) into a smaller one while preserving the abilities needed for a downstream task. However, most existing methods treat capabilities as independent training targets and overlook how improving one capability can reshape the student’s broader capability profile, especially when multiple abilities jointly determine task success. We study capability distillation under a fixed token budget and identify two consistent patterns: distillation induces systematic, budget-dependent cross-capability transfer, and additional budget often brings limited task-relevant gains while sometimes degrading other useful abilities. Building on these insights, we propose ReAD, a Reinforcement-guided cApability Distillation framework that explicitly accounts for capability interdependence. ReAD first infers task-essential capabilities, then generates capability-targeted supervision on the fly, and finally uses an uncertainty-aware contextual bandit to adaptively allocate the distillation budget based on expected utility gains. Extensive experiments show that ReAD improves downstream utility under the same token budget while reducing harmful spillover and wasted distillation effort compared to strong baselines. Our code is publicly available at https://github.com/LabRAI/ReAD.

[6] An Empirical Study of Automating Agent Evaluation cs.CLPDF

Kang Zhou, Sangmin Woo, Haibo Ding, Kiran Ramnath, Subramanian Chidambaram

TL;DR: 该论文研究了如何利用前沿编码助手自动化评估智能体（agent）的可行性，发现单纯提示编码助手效果不佳，因此提出了EvalAgent系统，该系统通过编码评估领域知识作为评估技能，构建端到端的自动化评估流程，显著提升了评估代码的一次执行成功率。

Details

Motivation: 智能体评估涉及复杂的多步骤行为、工具使用和中间推理，过程成本高昂且依赖专家知识，因此研究能否利用前沿编码助手可靠地自动化这一过程。

Result: 在提出的AgentEvalBench基准（包含20个智能体）上，EvalAgent将评估代码首次执行即成功且有意义的指标Eval@1从基线（单纯使用编码助手）的17.5%提升至65%，并在79.5%的情况下获得人类专家偏好。

Insight: 核心创新在于将评估领域知识（如程序指令、可复用代码模板、动态检索的API文档）编码为可组合的“评估技能”，并构建了基于执行轨迹的自动化评估流水线；同时提出了Eval@1这一衡量评估代码实用性的新指标以及配套的元评估框架。

Abstract: Agent evaluation requires assessing complex multi-step behaviors involving tool use and intermediate reasoning, making it costly and expertise-intensive. A natural question arises: can frontier coding assistants reliably automate this evaluation process? Our study shows that simply prompting coding assistants is insufficient for this task. Without domain-specific evaluation knowledge, frontier coding assistants achieve only a 30% execution success rate and produce over-engineered evaluations averaging 12+ metrics per agent, indicating that strong coding ability does not automatically translate to reliable agent evaluation. We introduce EvalAgent, an AI assistant that automates the end-to-end agent evaluation pipeline. EvalAgent encodes evaluation domain expertise as evaluation skills (procedural instructions, reusable code and templates, and dynamically retrieved API documentation) that compose into a trace-based pipeline producing complete evaluation artifacts including metrics, executable code, and reports. To systematically assess generated evaluations, we introduce a meta-evaluation framework alongside AgentEvalBench, a benchmark comprising 20 agents, each paired with evaluation requirements and test scenarios. We further propose the Eval@1 metric to measure whether generated evaluation code both executes and yields meaningful results on the first run. Our experiments show that EvalAgent produces focused evaluations, improving Eval@1 from 17.5% to 65%, and achieving 79.5% human expert preference over baseline approaches. Further ablation studies show that evaluation skills are critical for handling complex evaluation: removing them causes Eval@1 to drop significantly from 65% to 30%.

[7] Deep Reasoning in General Purpose Agents via Structured Meta-Cognition cs.CL | cs.AIPDF

Dean Light, Michael Theologitis, Kshitish Ghate, Shuyue Stella Li, Benjamin Newman

TL;DR: 本文提出了一种名为Deep Reasoning的推理时方法，通过结构化元认知为通用智能体（DOLORES）构建任务特定的推理支架。该方法使用一种形式化语言将元推理表示为在联想推理、形式计算和递归子问题求解上的可执行分解，使分解原则能作为上下文示例指导测试时支架构建。DOLORES将复杂任务分配到多个更可控的推理线程中，在四个困难基准测试上超越了现有最先进的支架方法。

Details

Motivation: 解决当前LLM智能体缺乏人类灵活切换推理模式（如计划、执行、修订、联想判断、形式化程序）的能力的问题。现有智能体的推理支架是预先硬编码的，当任务需要适应推理结构本身时显得脆弱。

Result: 在四个困难基准（多跳推理、长链问答、长上下文聚合、深度研究式信息寻求）上，DOLORES在三个模型规模和两个模型系列上均优于所有评估的支架基线，平均比最强基线提升24.8%。其优势甚至能弥补规模差距，一个8B版本在超过一半的设置中超越了同系列所有评估的32B基线。

Insight: 核心创新在于将推理支架的构建从预先硬编码转变为推理时自适应生成，通过形式化语言和元推理实现任务特定的、即时的结构化分解。这减少了过早终止和幻觉，通过将认知分配到结构化、低负载的推理线程中，提升了复杂问题解决的鲁棒性和灵活性。

Abstract: Humans intuitively solve complex problems by flexibly shifting among reasoning modes: they plan, execute, revise intermediate goals, resolve ambiguity through associative judgment, and apply formal procedures to well-specified subproblems. Current LLM agents lack this flexibility, as their scaffolds hard-code such reasoning decisions in advance. These scaffolds are effective when their prescribed structure matches the task, but brittle when solving the task requires adapting the structure of reasoning itself. We introduce Deep Reasoning – an inference-time approach for constructing task-specific scaffolds through structured meta-reasoning. Deep Reasoning uses a formal language that represents meta-reasoning as executable decompositions over associative inference, formal computation, and recursive subproblem solving, enabling decomposition principles to be encoded as in-context examples that guide test-time scaffold construction. We instantiate this approach in a general-purpose agent (DOLORES) that distributes complex tasks across more controlled reasoning threads. We evaluate it against state-of-the-art scaffolding methods across four hard benchmarks: multi-hop reasoning, long-chain question answering, long-context aggregation, and deep research-style information seeking. DOLORES outperforms all evaluated scaffolds across three model sizes and two model families, improving over the strongest evaluated scaffold baseline by 24.8% on average. DOLORES distributes cognition across structured, lower-load reasoning threads, thereby reducing premature termination and hallucinations. This advantage can even bridge the scaling gap, with an 8B version surpassing all evaluated 32B baselines from the same family in more than half the settings. These results point toward future agentic systems that treat scaffolding as adaptive reasoning, constructing the structure each task requires just-in-time.

[8] Agent-BRACE: Decoupling Beliefs from Actions in Long-Horizon Tasks via Verbalized State Uncertainty cs.CL | cs.AIPDF

Joykirat Singh, Zaid Khan, Archiki Prasad, Justin Chih-Yao Chen, Akshay Nambi

TL;DR: 本文提出了Agent-BRACE方法，旨在解决大型语言模型在部分可观测环境中执行长程任务时面临的两个挑战：部分可观测性带来的状态不确定性，以及长交互历史导致的上下文无限增长。该方法将LLM智能体解耦为信念状态模型和策略模型，信念状态模型生成一组带有口语化确定性标签的自然语言声明来近似信念分布，策略模型基于此紧凑的信念表示而非完整历史来选择动作。

Details

Motivation: 动机是解决LLM在部分可观测、长程任务中因无法有效表示和跟踪环境状态不确定性，以及历史上下文无限膨胀导致任务相关信息被稀释的问题。

Result: 在长程、部分可观测的具身语言环境中，Agent-BRACE相比强化学习基线取得了显著提升，使用Qwen2.5-3B-Instruct和Qwen3-4B-Instruct模型分别实现了平均+14.5%和+5.3%的绝对性能改进，同时保持了几乎恒定的上下文窗口，且学习到的信念随证据积累逐渐校准。

Insight: 创新点在于将信念状态与动作策略解耦，并引入一种结构化的近似信念表示方法：用带有序数化口语确定性标签的自然语言原子声明集合来近似后验分布。这为LLM智能体提供了一种紧凑、可解释且能显式处理不确定性的状态表示机制。

Abstract: Large language models (LLMs) are increasingly deployed on long-horizon tasks in partially observable environments, where they must act while inferring and tracking a complex environment state over many steps. This leads to two challenges: partial observability requires maintaining uncertainty over unobserved world attributes, and long interaction history causes context to grow without bound, diluting task-relevant information. A principled solution to both challenges is a belief state: a posterior distribution over environment states given past observations and actions, which compactly encodes history for decision making regardless of episode length. In LLM agents, however, the open-ended nature of text makes it unclear how to represent such a distribution. Therefore, we introduce Agent-BRACE: Agent Belief state Representation via Abstraction and Confidence Estimation, a method that decouples an LLM agent into a belief state model and a policy model, jointly optimized via reinforcement learning. The belief state model produces a structured approximation of the belief distribution: a set of atomic natural language claims about the environment, each annotated with an ordinal verbalized certainty label ranging from certain to unknown. The policy model conditions on this compact, structured approximate belief rather than the full history, learning to select actions under explicit uncertainty. Across long-horizon, partially observable embodied language environments, Agent-BRACE achieves an average absolute improvement of +14.5% (Qwen2.5-3B-Instruct) and +5.3% (Qwen3-4B-Instruct), outperforming strong RL baselines while maintaining a near-constant context window independent of episode length. Further analysis shows that the learned belief becomes increasingly calibrated over the course of an episode as evidence accumulates.

[9] Checkup2Action: A Multimodal Clinical Check-up Report Dataset for Patient-Oriented Action Card Generation cs.CL | cs.CVPDF

Sike Xiang, Shuang Chen, Kevin Qinghong Lin, Jialin Yu, Yijia Sun

TL;DR: 本文介绍了Checkup2Action数据集，这是一个用于从多模态临床体检报告生成结构化行动卡的多模态数据集和基准。该数据集包含2000份真实世界的去标识化体检报告，旨在评估大语言模型在生成安全、优先排序且以患者为导向的随访行动方面的能力。

Details

Motivation: 临床体检报告包含布局、表格、生物标志物、异常标志、影像发现和领域术语等多模态异构信息，普通人难以解读并转化为具体的随访行动。现有大语言模型在从多模态体检报告生成安全、优先排序且以患者为导向的行动方面的能力尚未得到充分评估。

Result: 实验使用通用和医学领域的大语言模型进行，揭示了在问题覆盖率、行动正确性、简洁性和安全性对齐之间存在明显的权衡关系。

Insight: 创新点在于提出了一个专门用于评估以患者为导向的临床体检报告推理的多模态基准，将体检报告到行动生成任务形式化为一个受约束的结构化生成任务，并引入了一个涵盖问题覆盖率与精确度、优先级一致性、科室与时间推荐准确性、行动复杂性、有用性、可读性和安全性合规性的综合评估协议。

Abstract: Clinical check-up reports are multimodal documents that combine page layouts, tables, numerical biomarkers, abnormality flags, imaging findings, and domain-specific terminology. Such heterogeneous evidence is difficult for laypersons to interpret and translate into concrete follow-up actions. Although large language models show promise in medical summarisation and triage support, their ability to generate safe, prioritised, and patient-oriented actions from multimodal check-up reports remains under-benchmarked. We present \textbf{Checkup2Action}, a multimodal clinical check-up report dataset and benchmark for structured \textit{Action Card} generation. Each card describes one clinically relevant issue and specifies its priority, recommended department, follow-up time window, patient-facing explanation, and questions for clinicians, while avoiding diagnostic or treatment-prescriptive claims. The dataset contains 2,000 de-identified real-world check-up reports covering demographic information, physical examinations, laboratory tests, cardiovascular assessments, imaging-related evidence, and physician summaries. We formulate checkup-to-action generation as a constrained structured generation task and introduce an evaluation protocol covering issue coverage and precision, priority consistency, department and time recommendation accuracy, action complexity, usefulness, readability, and safety compliance. Experiments with general-purpose and medical large language models reveal clear trade-offs between issue coverage, action correctness, conciseness, and safety alignment. Checkup2Action provides a new multimodal benchmark for evaluating patient-oriented reasoning over clinical check-up reports.

[10] Taming Extreme Tokens: Covariance-Aware GRPO with Gaussian-Kernel Advantage Reweighting cs.CL | cs.AI | cs.LGPDF

Cheng Wang, Qin Liu, Wenxuan Zhou, Muhao Chen

TL;DR: 本文提出了一种名为Covariance-Aware GRPO with Gaussian-Kernel Advantage Reweighting的方法，旨在改进Group Relative Policy Optimization（GRPO）在训练大型语言模型推理能力时面临的探索与利用权衡问题。该方法通过基于协方差的权重调整和高斯核重加权，动态抑制极端令牌级别的更新，从而稳定训练过程并提升性能。

Details

Motivation: GRPO在训练中难以有效平衡探索与利用，常导致次优性能；受理论启发，即熵的变化由令牌概率与其对应优势的协方差控制，本文旨在设计一种无需超参数的方法来动态调整更新权重，以缓解不稳定性。

Result: 在多个推理基准测试中，该方法相比GRPO提升了下游性能，并有效稳定了训练过程中的熵变化，表明其具有更好的稳定性和效果。

Insight: 创新点在于利用协方差理论指导权重设计，引入高斯核进行优势重加权，实现超参数自由的动态优化；从客观角度看，该方法通过数学原理直接关联探索-利用权衡与更新稳定性，为策略优化提供了可解释且高效的改进思路。

Abstract: Group Relative Policy Optimization (GRPO) has emerged as a promising approach for improving the reasoning capabilities of large language models. However, it struggles to effectively balance the tradeoff between exploration and exploitation during training, often resulting in suboptimal performance. Motivated by the theoretical insight that changes in entropy are governed by the covariance between token probabilities and their corresponding advantages, we propose a hyperparameter-free, covariance-weighted optimization method that dynamically down-weights extreme token-level updates via a Gaussian kernel. This approach automatically reduces the instability caused by exploration-exploitation trade-off while preserving informative learning signals. Extensive empirical evaluations show that our approach improves downstream performance across reasoning benchmarks compared with GRPO, and effectively stablizes entropy as training progresses.

[11] OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models cs.CLPDF

Yuanhao Yue, Chengyu Wang, Yuanjie Lyu, Lei Shen, Jun Huang

TL;DR: 本文提出了OmniThoughtVis，一个可扩展的数据蒸馏流水线，用于将大型多模态大语言模型的推理能力迁移到更小、面向部署的模型中。该流水线从多样化开源种子数据出发，生成结构化思维链轨迹，并联合标注推理难度、答案质量和语义任务标签，通过规则过滤、难度感知选择和标签多样性采样，构建了包含180万样本的高质量数据集。

Details

Motivation: 现有大型多模态模型虽具备强大的思维链推理能力，但受限于延迟和资源消耗，难以直接部署；而小型模型又缺乏大规模高质量的多模态思维链监督数据，导致推理性能瓶颈。

Result: 在九个多模态推理基准测试中，蒸馏后的Qwen3-VL模型（2B至8B参数）均取得一致性能提升，其中4B模型在MathVerse上提升16.8分，在MMMU-Pro上提升5.6分；蒸馏后的4B模型在多项任务上达到或超越了未蒸馏的8B基线模型。

Insight: 创新点在于提出了一套可扩展的、结合结构化思维链生成、多维度标注（难度、质量、任务标签）及可控数据筛选的蒸馏流水线，实现了从大型教师模型到小型部署模型的高效推理能力迁移，为资源受限场景下的多模态推理模型部署提供了实用解决方案。

Abstract: Recent multimodal large language models (MLLMs) have shown strong chain-of-thought (CoT) reasoning ability on vision-language tasks, but their direct deployment in real-world systems is often limited by latency and resource constraints. In practice, smaller MLLMs are preferred for online serving, yet their reasoning performance is bottlenecked by the lack of large-scale, high-quality multimodal CoT supervision. In this paper, we present OmniThoughtVis, a scalable data curation and distillation pipeline for transferring multimodal reasoning capabilities from high-capacity teacher models to smaller, deployment-oriented MLLMs. Starting from a diverse open-source seed pool, our pipeline generates structured CoT traces and performs joint annotation of reasoning difficulty, answer quality, and semantic task tags. To maintain data quality at scale, we combine rule-based filtering, difficulty-aware selection, and tag-based diversity sampling, resulting in a curated corpus of 1.8M samples that supports controllable subset construction for downstream training. We use OmniThoughtVis to distill Qwen3-VL models from 2B to 8B parameters and evaluate them on nine multimodal reasoning benchmarks. The resulting distilled models show consistent gains across model scales, including improvements of up to +16.8 points on MathVerse and +5.6 points on MMMU-Pro for the 4B model. Notably, the distilled 4B model matches or surpasses the undistilled 8B baseline on several tasks, highlighting the practical value of scalable reasoning distillation for deployment-oriented MLLMs.

[12] Human-Grounded Multimodal Benchmark with 900K-Scale Aggregated Student Response Distributions from Japan’s National Assessment of Academic Ability cs.CLPDF

Kyosuke Takami, Yuka Tateisi, Satoshi Sekine, Yusuke Miyao

TL;DR: 本文提出了一个基于日本全国学力测试的多模态数据集，包含中学科学、数学和日语科目的真实试题、图表和文本，并附有约90万学生作答的聚合分布。该数据集用于评估多模态大语言模型在真实教育场景下的表现，支持自动评分与人类表现的直接比较。

Details

Motivation: 现有基于日本K-12评估的多模态基准稀缺，而真实学校考试为评估多模态大语言模型提供了高有效性测试平台。本文旨在构建一个保留真实考试布局、图表和日语教育文本的数据集，以支持对模型在教育推理任务中的性能进行可靠评估。

Result: 在科学、数学和日语科目上，使用精确匹配准确率和字符级F1分数评估了近期多模态大语言模型，发现不同科目间性能差异显著，且模型对视觉推理需求表现出强烈敏感性。人类评估和LLM-as-judge分析进一步验证了自动评分的可靠性。

Insight: 创新点在于利用大规模真实学生作答分布（约90万）构建了可复现的、以人类表现为基准的多模态教育推理评估框架。该数据集支持未来在真实评估场景下的评估、反馈生成和可解释AI研究，弥补了日语教育多模态基准的空白。

Abstract: Authentic school examinations provide a high-validity test bed for evaluating multimodal large language models (MLLMs), yet benchmarks grounded in Japanese K-12 assessments remain scarce. We present a multimodal dataset constructed from Japan’s National Assessment of Academic Ability, comprising officially released middle-school items in Science, Mathematics, and Japanese Language. Unlike existing benchmarks based on synthetic or curated data, our dataset preserves real exam layouts, diagrams, and Japanese educational text, together with nationwide aggregated student response distributions (N $\approx$ 900{,}000). These features enable direct comparison between human and model performance under a unified evaluation framework. We benchmark recent multimodal LLMs using exact-match accuracy and character-level F1 for open-ended responses, observing substantial variation across subjects and strong sensitivity to visual reasoning demands. Human evaluation and LLM-as-judge analyses further assess the reliability of automatic scoring. Our dataset establishes a reproducible, human-grounded benchmark for multimodal educational reasoning and supports future research on evaluation, feedback generation, and explainable AI in authentic assessment contexts. Our dataset is available at: https://github.com/KyosukeTakami/gakucho-benchmark

[13] Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation cs.CLPDF

Yuchen Cai, Ding Cao, Liang Lin, Chunxi Luo, Xin Xu

TL;DR: 本文揭示了在线策略蒸馏（OPD）高效性的参数动态机制，提出OPD的‘预见性’体现在模块分配和更新方向两个层面，并基于此设计了无需额外可训练模块的加速方法EffOPD，实现了3倍训练加速且保持性能。

Details

Motivation: 现有研究将OPD的高效性归因于更密集稳定的监督，但对其参数层面的机制理解不足，本文旨在从参数动态角度揭示OPD效率的内在原因。

Result: 提出的EffOPD方法在保持最终性能相当的情况下，平均实现了3倍训练加速，且无需额外可训练模块或复杂超参数调优。

Insight: 创新点在于从模块分配（聚焦关键推理模块）和更新方向（低秩集中且与最终更新子空间早期对齐）两个层面量化了OPD的‘预见性’机制，并据此设计了简单有效的加速方法。

Abstract: On-policy distillation (OPD) has emerged as an efficient post-training paradigm for large language models. However, existing studies largely attribute this advantage to denser and more stable supervision, while the parameter-level mechanisms underlying OPD’s efficiency remain poorly understood. In this work, we argue that OPD’s efficiency stems from a form of ``foresight’’: it establishes a stable update trajectory toward the final model early in training. This foresight manifests in two aspects. First, at the \textbf{Module-Allocation Level}, OPD identifies regions with low marginal utility and concentrates updates on modules that are more critical to reasoning. Second, at the \textbf{Update-Direction Level}, OPD exhibits stronger low-rank concentration, with its dominant subspaces aligning closely with the final update subspace early in training. Building on these findings, we propose \textbf{EffOPD}, a plug-and-play acceleration method that speeds up OPD by adaptively selecting an extrapolation step size and moving along the current update direction. EffOPD requires no additional trainable modules or complex hyperparameter tuning, and achieves an average training acceleration of $3\times$ while maintaining comparable final performance. Overall, our findings provide a parameter-dynamics perspective for understanding the efficiency of OPD and offer practical insights for designing more efficient post-training methods for large language models.

[14] Probabilistic Calibration Is a Trainable Capability in Language Models cs.CLPDF

Davide Baldelli, Sruthi Kuriakose, Maryam Hashemzadeh, Amal Zouaq, Sarath Chandar

TL;DR: 该论文研究了语言模型通过微调提升概率校准能力的可行性，提出了两种校准微调方法（软目标和硬目标），在多种模型上验证了其能有效改善模型对数学分布的采样忠实度，并分析了两种方法在不同随机生成任务上的性能差异。

Details

Motivation: 语言模型在需要满足用户指定随机性约束的场景中应用日益增多，但其生成概率往往与目标分布校准不佳，论文旨在探索是否可以通过微调直接提升这种概率校准能力。

Result: 在涵盖四个系列的12个模型上，两种方法均在未见过的分布族和参数设置上显著提升了结构化采样的忠实度，表明概率校准是可训练的能力。硬目标微调在结构化数值采样上通常更强，而软目标微调在更广泛的随机生成基准（如开放式随机生成、多项选择答案位置平衡和NoveltyBench）上表现更好，但提升有时会损害下游能力（尤其是算术推理）。

Insight: 论文的创新点在于系统性地验证了概率校准作为一种可通过微调习得的能力，并对比了两种微调策略（基于目标分布转换的软目标和基于采样完成的硬目标）在不同任务上的优劣，为在语言模型中实现可控随机性提供了实用的训练方法。

Abstract: Language models are increasingly used in settings where outputs must satisfy user-specified randomness constraints, yet their generation probabilities are often poorly calibrated to those targets. We study whether this capability can be improved directly through fine-tuning. Concretely, we fine-tune language models on synthetic prompts that require sampling from mathematical distributions, and compare two Calibration Fine-Tuning variants: a soft-target method that converts the desired output distribution into trie-derived next-token targets, and a hard-target method that trains on sampled completions from the same target distribution. Across 12 models spanning four families, both methods substantially improve structured-sampling fidelity on held-out distribution families and unseen parameter settings, showing that probabilistic calibration is a trainable capability. Under our selected training configurations, the two methods exhibit different empirical profiles: hard-target fine-tuning is often strongest on structured numeric sampling, while soft-target fine-tuning performs better on broader stochastic generation benchmarks, including open-ended random generation, multiple-choice answer-position balancing, and NoveltyBench. The gains sometimes reduce downstream capability, especially arithmetic reasoning, with costs varying by model. Overall, our results show that probabilistic calibration can be improved through fine-tuning, with our hard-target configuration favoring exact numeric fidelity and our soft-target configuration favoring broader stochastic transfer. Code is available at https://github.com/chandar-lab/calibration-finetuning.

[15] Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Models cs.CL | cs.LGPDF

Boyi Deng, Xu Wang, Yaoning Wang, Yu Wan, Yubo Ma

TL;DR: 本文介绍了Qwen-Scope，一个基于Qwen模型系列的开源稀疏自编码器套件，包含7个模型变体的14组SAE。论文展示了SAE不仅可用于事后分析，还能作为模型开发的实用接口，应用于推理时引导、评估分析、以数据为中心的工作流以及训练后优化，从而连接模型内部表示与下游行为。

Details

Motivation: 大语言模型的内部决策过程不透明，限制了对其检查、控制和系统性改进的能力。稀疏自编码器是分解模型激活为稀疏、可解释特征表示的有前景工具，但需要更实用的开发接口。

Result: 论文展示了SAE在四个方向的应用效果：实现了无需修改权重的推理时引导；为基准冗余和能力覆盖提供了表示层代理；支持了多语言毒性分类和安全数据合成；并在监督微调和强化学习目标中融入了SAE信号，缓解了代码切换和重复等不良行为。

Insight: 将稀疏自编码器从单纯的分析工具提升为可重用的表示层接口，用于诊断、控制、评估和改进大语言模型，为模型内部机制与下游应用之间建立了实用的桥梁。

Abstract: Large language models have achieved remarkable capabilities across diverse tasks, yet their internal decision-making processes remain largely opaque, limiting our ability to inspect, control, and systematically improve them. This opacity motivates a growing body of research in mechanistic interpretability, with sparse autoencoders (SAEs) emerging as one of the most promising tools for decomposing model activations into sparse, interpretable feature representations. We introduce Qwen-Scope, an open-source suite of SAEs built on the Qwen model family, comprising 14 groups of SAEs across 7 model variants from the Qwen3 and Qwen3.5 series, covering both dense and mixture-of-expert architectures. Built on top of these SAEs, we show that SAEs can go beyond post-hoc analysis to serve as practical interfaces for model development along four directions: (i) inference-time steering, where SAE feature directions control language, concepts, and preferences without modifying model weights; (ii) evaluation analysis, where activated SAE features provide a representation-level proxy for benchmark redundancy and capability coverage; (iii) data-centric workflows, where SAE features support multilingual toxicity classification and safety-oriented data synthesis; and (iv) post-training optimization, where SAE-derived signals are incorporated into supervised fine-tuning and reinforcement learning objectives to mitigate undesirable behaviors such as code-switching and repetition. Together, these results demonstrate that SAEs can serve not only as post-hoc analysis tools, but also as reusable representation-level interfaces for diagnosing, controlling, evaluating, and improving large language models. By open-sourcing Qwen-Scope, we aim to support mechanistic research and accelerate practical workflows that connect model internals to downstream behavior.

[16] YFPO: A Preliminary Study of Yoked Feature Preference Optimization with Neuron-Guided Rewards for Mathematical Reasoning cs.CLPDF

Yifan Le

TL;DR: 本文提出了一种名为YFPO的神经元引导偏好优化框架，用于提升大语言模型的数学推理能力。该方法通过AttnLRP识别与数学相关的神经元，并利用其在偏好与不偏好响应间的激活差异构建辅助奖励，从而将模型内部表征能力信息与外部偏好数据结合进行训练。

Details

Motivation: 现有偏好优化方法主要依赖外部构建的偏好数据作为样本级监督，但很少显式利用模型内部表征中包含的与能力相关的信息。对于数学推理任务，某些神经元组的激活模式可能与数学知识、符号操作或逻辑推理相关，这些内部信号可作为模型是否运用数学相关能力的粗略指示。

Result: 在GSM8K基准上对一个小规模语言模型进行的初步实验表明，神经元级信号可以与偏好优化交互，并偶尔提升推理性能。

Insight: 创新点在于将模型内部神经元激活模式作为辅助奖励信号引入偏好优化过程，为更细粒度、可解释的面向推理的后训练提供了有前景的方向。这实现了外部行为信号与内部能力表征的协同利用。

Abstract: Preference optimization has become an important post-training paradigm for improving the reasoning abilities of large language models. Existing methods typically rely on externally constructed preference data, using preferred and dispreferred responses as sample-level supervision. However, such external signals rarely make explicit use of capability-related information contained in the model’s internal representations. For mathematical reasoning, certain neuron groups may exhibit activation patterns associated with mathematical knowledge, symbolic manipulation, or logical reasoning. Similar to reflexive behavioral signals, these internal activations may provide a coarse indication of whether the model is engaging math-related capabilities.We introduce YFPO, short for Yoked Feature Preference Optimization, a preliminary neuron-guided preference optimization framework for mathematical reasoning. YFPO first uses AttnLRP to identify math-related neurons, and then constructs an auxiliary reward from their activation margin between preferred and dispreferred responses. This design augments external preference learning with internal neuron-level signals. We conduct preliminary experiments on a small-scale language model using GSM8K as the main benchmark. Results suggest that neuron-level signals can interact with preference optimization and occasionally improve reasoning performance, offering a promising direction for more fine-grained and interpretable reasoning-oriented post-training.

[17] On Predicting the Post-training Potential of Pre-trained LLMs cs.CLPDF

Xiaoyuan Li, Yubo Ma, Kexin Yang, Moxin Li, Keqin Bao

TL;DR: 该论文提出了一种名为RuDE（基于量规的判别性评估）的新框架，用于预测预训练大语言模型（LLMs）在后续训练（如微调或强化学习）前的潜力，旨在解决传统基准（如MMLU）在评估模型于复杂开放场景中可塑性方面的不足。

Details

Motivation: 传统基准测试无法有效反映基础模型在复杂开放场景中的可塑性，导致模型选择效率低下，因此需要一种能够预测模型后训练潜力的新方法。

Result: 大量实验表明，RuDE的评估结果与模型后训练性能的相关性超过90%；通过强化学习验证，RuDE能有效识别出潜力高的小模型，使其性能超越更大的模型。

Insight: 创新点在于提出了预测后训练潜力的新任务，并设计了基于系统化4C分类法的RuDE框架，通过构建细粒度量规违规的对比对来绕过基础模型的生成差距，实现判别性评估，为高效的基础模型开发提供了计算高效的机制。

Abstract: The performance of Large Language Models (LLMs) on downstream tasks is fundamentally constrained by the capabilities acquired during pre-training. However, traditional benchmarks like MMLU often fail to reflect a base model’s plasticity in complex open-ended scenarios, leading to inefficient model selection. We address this by introducing a new task of predicting post-training potential - forecasting a base model’s performance before post-training. We propose RuDE (Rubric-based Discriminative Evaluation), a unified framework that bypasses the generation gap of base models by leveraging response discrimination. Guided by our systematic 4C Taxonomy, RuDE constructs controlled contrastive pairs across diverse domains by fine-grained rubric violations. Extensive experiments demonstrate a correlation greater than 90% with post-training performance. Crucially, validation via Reinforcement Learning (RL) confirms that RuDE effectively identifies high-potential smaller models that outperform larger counterparts, offering a compute-efficient mechanism for foundation model development.

[18] Towards Visually-Guided Movie Subtitle Translation for Indic Languages cs.CLPDF

Tarun Chintada, Kshetrimayum Boynao Singh, Asif Ekbal

TL;DR: 本文针对低资源印度语言（印地语、孟加拉语、泰卢固语、泰米尔语和卡纳达语）的电影字幕翻译任务，研究了视觉引导的翻译方法。通过分析五部完整电影，比较了两种轻量级视觉基础策略：基于5分钟滑动窗口的结构化属性摘要和字幕间视觉间隙的自由文本摘要。研究发现，字幕与视频帧的时间错位是长视频翻译的主要障碍，但选择性视觉增强（仅替换基线中质量最低的20-30%片段）能有效提升翻译质量，同时减少视觉处理需求。

Details

Motivation: 电影字幕翻译本质上是多模态任务，但现有文本系统常忽略传达情感、动作和社会细微差别所需的视觉线索，尤其在低资源印度语言翻译中问题更为突出。

Result: 在五部电影的实验中，选择性视觉增强（替换基线质量最低的20-30%片段）相比纯文本基线持续提升了COMET评分，且所需视觉处理更少。其中基于粗粒度属性的视觉上下文摘要方法更鲁棒，能捕捉场景级情感和文本常忽略的上下文细微线索。

Insight: 创新点在于提出两种轻量级视觉基础策略，并揭示时间错位对长视频视觉引导翻译的影响；客观分析表明，选择性视觉增强（而非全盘使用）在提升翻译质量与计算效率间取得了平衡，为多模态低资源翻译提供了实用方案。

Abstract: Movie subtitle translation is inherently multimodal, yet text-only systems often miss visual cues needed to convey emotion, action, and social nuance, especially for low-resource Indic languages (English to Hindi, Bengali, Telugu, Tamil and Kannada). We present a case study on five full-length films and compare two lightweight visual grounding strategies: structured attribute summaries from a 5-minute sliding window and free-text summaries of inter-subtitle visual gaps. Our analysis shows that temporal misalignment between subtitles and frames is a major obstacle in long-form video, often rendering indiscriminate visual grounding ineffective. However, oracle selective grounding, which replaces only the lowest-quality 20-30% of baseline segments with visual-enhanced outputs, consistently improves COMET over the text-only baseline while requiring far less visual processing. Among the two approaches, coarse attribute-based visual context summarization is more robust, capturing scene-level emotion and contextual subtle cues that text alone often misses

[19] Learning Agentic Policy from Action Guidance cs.CLPDF

Yuxiang Ji, Zengbin Wang, Yong Wang, Shidong Yang, Ziyu Ma

TL;DR: 本文提出了一种名为ActGuide-RL的新方法，用于提升大型语言模型（LLMs）在智能体强化学习（RL）中的探索能力。该方法的核心是利用日常人类交互中产生的丰富动作数据作为计划式参考指导，帮助智能体策略克服难以到达奖励状态的障碍。通过混合策略训练，将指导性探索的收益内化到无指导策略中，并采用最小干预原则，仅在必要时自适应地调用指导，以平衡收益与风险。

Details

Motivation: 动机在于解决智能体强化学习中基础策略探索能力不足的问题。当基础策略无法到达奖励状态时，训练信号缺失，传统方法依赖成本高昂的迭代监督微调（SFT）。本文旨在利用可扩展的动作数据作为指导，减少对大量SFT数据的依赖，从而恢复有效的学习信号。

Result: 在搜索智能体基准测试（GAIA和XBench）上，使用Qwen3-4B模型，ActGuide-RL相比零RL基线显著提升了性能（在GAIA上提升10.7个百分点，在XBench上提升19个百分点），并且与需要冷启动的SFT+RL流程性能相当，达到了先进水平（on par with SOTA pipeline）。

Insight: 创新点在于引入动作数据作为计划式参考指导来突破探索瓶颈，并通过混合策略训练和最小干预的自适应回退机制，将外部指导高效内化。这为智能体RL提供了一种新范式，即用可扩展的动作指导替代对大量监督微调数据的依赖，降低了训练成本。

Abstract: Agentic reinforcement learning (RL) for Large Language Models (LLMs) critically depends on the exploration capability of the base policy, as training signals emerge only within its in-capability region. For tasks where the base policy cannot reach reward states, additional training or external guidance is needed to recover effective learning signals. Rather than relying on costly iterative supervised fine tuning (SFT), we exploit the abundant action data generated in everyday human interactions. We propose \textsc{ActGuide-RL}, which injects action data as plan-style reference guidance, enabling the agentic policy to overcome reachability barriers to reward states. Guided and unguided rollouts are then jointly optimized via mixed-policy training, internalizing the exploration gains back into the unguided policy. Motivated by a theoretical and empirical analysis of the benefit-risk trade-off, we adopt a minimal intervention principle that invokes guidance only as an adaptive fallback, matching task difficulty while minimizing off-policy risk. On search-agent benchmarks, \textsc{ActGuide-RL} substantially improves over zero RL (+10.7 pp on GAIA and +19 pp on XBench with Qwen3-4B), and performs on par with the SFT+RL pipeline without any cold start. This suggests a new paradigm for agentic RL that reduces the reliance on heavy SFT data by using scalable action guidance instead.

[20] SAGE: Scalable Automated Robustness Augmentation for LLM Knowledge Evaluation cs.CLPDF

Xiaoyuan Li, Yuzhe Wang, Moxin Li, Keqin Bao, Rui Men

TL;DR: 本文提出SAGE框架，用于大规模自动化增强知识评估基准的鲁棒性，通过微调的小模型生成和验证问题变体，以低成本构建高质量的大规模鲁棒性增强基准。

Details

Motivation: 当前LLM在标准知识评估基准上表现良好，但在不同形式的问题变体下知识能力仍显脆弱，而现有LLM辅助的生成-验证流程成本高、难以扩展，因此需要可扩展的鲁棒性增强方法。

Result: 在HellaSwag上的实验表明，SAGE构建的大规模鲁棒性增强基准质量与人工标注的HellaSwag-Pro相当，但成本显著降低；微调模型进一步泛化到MMLU基准，无需特定微调。

Insight: 创新点包括基于规则的验证器VariantQual和强化学习优化的生成器VariantGen，利用微调小模型替代大模型实现可扩展的自动化基准增强，降低了成本并保持了质量。

Abstract: Large Language Models (LLMs) achieve strong performance on standard knowledge evaluation benchmarks, yet recent work shows that their knowledge capabilities remain brittle under question variants that test the same knowledge in different forms. Robustness augmentation of existing knowledge evaluation benchmarks is therefore necessary, but current LLM-assisted generate-then-verify pipelines are costly and difficult to scale due to low-yield variant generation and unreliable variant verification. We propose SAGE (Scalable Automated Generation of Robustness BEnchmarks), a framework for scalable robustness augmentation of knowledge evaluation benchmarks using fine-tuned smaller models. SAGE consists of VariantQual, a rubric-based verifier trained on human-labeled seed data, and VariantGen, a variant generator initialized with supervised fine-tuning and further optimized with reinforcement learning using VariantQual as the reward model. Experiments on HellaSwag show that SAGE constructs a large-scale robustness-augmented benchmark with quality comparable to the human-annotated HellaSwag-Pro at substantially lower cost, while the fine-tuned models further generalize to MMLU without benchmark-specific fine-tuning.

[21] SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphs cs.CLPDF

Xiaoyuan Li, Moxin Li, Keqin Bao, Yubo Ma, Wenjie Wang

TL;DR: 本文提出了SKILLGRAPH框架，通过将技能表示为有向图中的节点，并利用类型化边编码技能间的依赖关系，来解决现有技能库中技能孤立存储、仅通过语义相似性检索的问题。该框架能够为新任务检索有序的技能子图以指导多步决策，并通过智能体轨迹和强化学习反馈持续更新图结构，实现技能库与智能体策略的共同优化。

Details

Motivation: 现有技能库将技能存储为孤立条目且仅通过语义相似性检索，这导致在组合任务中难以识别技能间的依赖关系，并使得技能库的维护（如合并、拆分或移除技能）变得困难。

Result: 在ALFWorld、WebShop和七个搜索增强QA任务上的实验表明，SKILLGRAPH在性能上超越了现有的记忆增强强化学习方法，达到了最先进水平，尤其是在需要组合多个技能的复杂任务上取得了显著提升。

Insight: 核心创新在于将技能库组织为动态演化的有向图结构，利用图关系（先决条件、增强、共现）显式建模技能间的依赖与组合逻辑，从而支持更复杂的任务规划与技能库的自动化维护。

Abstract: Skill libraries enable large language model agents to reuse experience from past interactions, but most existing libraries store skills as isolated entries and retrieve them only by semantic similarity. This leads to two key challenges for compositional tasks. Firstly, an agent must identify not only relevant skills but also how they depend on and build upon each other. Secondly, it also makes library maintenance difficult, since the system lacks structural cues for deciding when skills should be merged, split, or removed. We propose SKILLGRAPH, a framework that represents reusable skills as nodes in a directed graph, with typed edges encoding prerequisite, enhancement, and co-occurrence relations. Given a new task, SKILLGRAPH retrieves not just individual skills, but an ordered skill subgraph that can guide multi-step decision making. The graph is continuously updated from agent trajectories and reinforcement learning feedback, allowing both the skill library and the agent policy to improve together. Experiments on ALFWorld, WebShop, and seven search-augmented QA tasks show that SKILLGRAPH achieves state-of-the-art performance against memory-augmented RL methods, with especially large gains on complex tasks that require composing multiple skills.

[22] Latent Causal Void: Explicit Missing-Context Reconstruction for Misinformation Detection cs.CL | cs.SIPDF

Hui Li, Zhongquan Jian, Jinsong Su, Junfeng Yao

TL;DR: 本文提出了一种名为Latent Causal Void (LCV)的检索引导检测器，用于处理因信息省略而产生的虚假信息检测问题。该方法通过检索时间对齐的背景文章，利用冻结的指令调优大语言模型为每个目标句子-文章对生成简短的缺失上下文描述，并将生成的文本关系输入到由目标句子和背景文章构成的异质图中进行推理，从而显式地重建缺失事实。

Details

Motivation: 当前虚假信息检测方法在文章明确陈述的欺骗内容上表现良好，但对于那些局部连贯、仅在与提供被省略背景事实的同期报道对比时才显误导性的信息（即省略相关场景）处理不足。现有方法通常仅将检索到的上下文作为辅助证据或推断一个分类的省略信号，而未显式处理具体的缺失事实。

Result: 在Sheng等人提出的双语基准测试中，LCV在英文和中文子集上的macro-F1分数分别比最强的省略感知基线提高了2.56和2.84点，表明该方法在省略感知的虚假信息检测任务上达到了新的SOTA水平。

Insight: 论文的创新点在于显式地重建缺失的跨源事实本身，并将其作为文本关系用于图推理，而不是仅仅附加检索证据或预测省略信号。这为处理依赖上下文省略的虚假信息提供了一种更有效的表示方法，强调了建模具体缺失内容的重要性。

Abstract: Automatic misinformation detection performs well when deception is visible in what an article explicitly states. However, some misinformation articles remain locally coherent and only become misleading once compared with contemporaneous reports that supply background facts the article omits. We study this omission-relevant setting and observe that current omission-aware approaches typically either attach retrieved context as auxiliary evidence or infer a categorical omission signal, leaving the specific missing fact implicit. We propose \emph{Latent Causal Void} (LCV), a retrieval-guided detector that explicitly reconstructs the missing fact for each target sentence and uses it as a textual cross-source relation in graph reasoning. Concretely, LCV retrieves temporally aligned context articles, asks a frozen instruction-tuned large language model to generate a short missing-context description for each sentence–article pair, and feeds the resulting relation text into a heterograph over target sentences and context articles. On the bilingual benchmark of Sheng et al., LCV improves over the strongest omission-aware baseline by $2.56$ and $2.84$ macro-F1 points on the English and Chinese splits, respectively. The results indicate that modeling the missing cross-source fact itself, rather than only attaching retrieved evidence or predicting an omission signal, is a useful representation for omission-aware misinformation detection.

[23] Combining On-Policy Optimization and Distillation for Long-Context Reasoning in Large Language Models cs.CLPDF

Miguel Moura Ramos, Duarte M. Alves, André F. T. Martins

TL;DR: 本文提出了一种名为Distilled Group Relative Policy Optimization (dGRPO)的方法，用于增强大型语言模型在长上下文推理任务中的性能。该方法结合了基于策略的强化学习（GRPO）和策略蒸馏（OPD），通过引入更强教师的密集指导来改进训练稳定性和样本效率。同时，作者还创建了一个名为LongBlocks的合成长上下文数据集，用于评估模型在多跳推理、上下文基础和长文本生成等任务上的表现。实验表明，dGRPO在长上下文对齐方面提供了更稳定有效的路径，同时保持了短上下文能力。

Details

Motivation: 现有方法在适应长上下文任务时存在局限：监督微调（SFT）和知识蒸馏（KD）等离策略方法存在暴露偏差和长序列错误恢复能力不足的问题；基于策略的强化学习方法（如GRPO）虽能更好对齐模型生成状态，但奖励稀疏导致不稳定和样本效率低；策略蒸馏（OPD）提供密集的令牌级指导，但无法直接优化任意奖励信号。因此，需要一种结合两者优势的方法来提升长上下文推理的准确性和连贯性。

Result: 通过广泛的实验和消融研究，比较了离策略训练、稀疏奖励GRPO和提出的组合方法（dGRPO）。结果表明，结合基于结果的策略优化和知识蒸馏的单一目标，在长上下文对齐方面提供了更稳定有效的路径，同时保持了短上下文能力。具体结果在LongBlocks数据集上进行了评估，涵盖了多跳推理、上下文基础和长文本生成等任务。

Insight: 创新点在于将基于策略的强化学习（GRPO）与策略蒸馏（OPD）相结合，通过引入更强教师的密集指导来缓解稀疏奖励问题，从而提升训练稳定性和样本效率。从客观角度看，该方法提供了一种统一的训练框架，能够同时优化奖励信号和利用教师模型的密集监督，为长上下文推理任务的有效对齐提供了新思路。LongBlocks数据集的引入也为长上下文任务的评估提供了标准化基准。

Abstract: Adapting large language models (LLMs) to long-context tasks requires post-training methods that remain accurate and coherent over thousands of tokens. Existing approaches are limited in several ways: 1) off-policy methods such as supervised fine-tuning (SFT) and knowledge distillation (KD) suffer from exposure bias and limited recovery from model-generated errors over long horizons; 2) on-policy reinforcement learning methods such as Group Relative Policy Optimization (GRPO) better align training with model-generated states, but are unstable and sample-inefficient due to sparse rewards; 3) on-policy distillation (OPD) provides dense token-level guidance, but does not directly optimize arbitrary reward signals. In this paper, we propose Distilled Group Relative Policy Optimization (dGRPO), a method for long-context reasoning that augments GRPO with dense guidance from a stronger teacher via OPD. We also introduce LongBlocks, a synthetic long-context dataset spanning multi-hop reasoning, contextual grounding, and long-form generation. We conduct extensive experiments and ablations comparing off-policy training, sparse-reward GRPO, and our combined approach, leading to an improved recipe for long-context alignment. Overall, our results show that combining outcome-based policy optimization with knowledge distillation in a single objective provides a more stable and effective path to long-context reasoning, while preserving short-context capabilities.

[24] Overview of the MedHopQA track at BioCreative IX: track description, participation and evaluation of systems for multi-hop medical question answering cs.CL | cs.IRPDF

Rezarta Islamaj, Joey Chan, Robert Leaman, Jongmyung Jung, Hyeongsoon Hwang

TL;DR: 本文介绍了BioCreative IX MedHopQA共享任务，该任务旨在评估大型语言模型在生物医学领域多跳问答中的性能。作者构建了一个包含1000个复杂问答对的新数据集，重点关注罕见疾病，每个问题都需要整合两个不同维基百科页面的信息进行两跳推理。任务吸引了13个团队的48份提交，评估指标包括表面字符串匹配和概念准确性（MedCPT分数）。结果显示，增强系统（如检索增强生成）显著优于基线模型。

Details

Motivation: 解决生物医学领域中多跳问答的挑战，该任务需要系统整合多个来源的信息来回答复杂问题，尤其是在罕见疾病方面。

Result: 在MedHopQA数据集上，最佳提交的F1分数达到89.30%（MedCPT指标），精确匹配分数为87.30%，而零样本基线的分数分别为67.40%和60.20%，显示出增强系统（如RAG）的显著优势。

Insight: 论文的创新点包括构建了一个专注于罕见疾病、需要两跳推理的生物医学多跳问答数据集，并强调了检索增强生成（RAG）等检索策略对提升性能的关键作用，同时引入概念级评估（MedCPT分数）以更准确地评估答案。

Abstract: Multi-hop question answering (QA) remains a significant challenge in the biomedical domain, requiring systems to integrate information across multiple sources to answer complex questions. To address this problem, the BioCreative IX MedHopQA shared task was designed to benchmark in multi-hop reasoning for large language models (LLMs). We developed a novel dataset of 1,000 challenging QA pairs spanning diseases, genes, and chemicals, with particular emphasis on rare diseases. Each question was constructed to require two-hop reasoning through the integration of information from two distinct Wikipedia pages. The challenge attracted 48 submissions from 13 teams. Systems were evaluated using both surface string comparison and conceptual accuracy (MedCPT score). The results showed a substantial performance gap between baseline LLMs and enhanced systems. The top-ranked submission achieved an 89.30% F1 score on the MedCPT metric and an 87.30% exact match (EM) score, compared with 67.40% and 60.20%, respectively, for the zero-shot baseline. A central finding of the challenge was that retrieval-augmented generation (RAG) and related retrieval-based strategies were critical for strong performance. In addition, concept-level evaluation improved answer assessment when correct responses differed in surface form. The MedHopQA dataset is publicly available to support continued progress in this important area. Challenge materials: https://www.ncbi.nlm.nih.gov/research/bionlp/medhopqa and benchmark https://www.codabench.org/competitions/7609/

[25] MedHopQA: A Disease-Centered Multi-Hop Reasoning Benchmark and Evaluation Framework for LLM-Based Biomedical Question Answering cs.CL | cs.AI | cs.IRPDF

Rezarta Islamaj, Robert Leaman, Joey Chan, Nicholas Wan, Qiao Jin

TL;DR: MedHopQA是一个以疾病为中心的多跳推理基准数据集，包含1000个专家标注的问答对，旨在评估大型语言模型在生物医学领域的推理能力。该基准要求模型整合来自两个不同维基百科文章的信息进行开放文本回答，并通过本体论同义词集支持词汇和概念级评估。

Details

Motivation: 现有生物医学问答基准存在局限性，如多项选择题可能让模型通过答案排除而非推理成功，且易受性能饱和和训练数据污染影响。多跳推理是临床诊断支持等任务的核心能力，但在当前基准中代表性不足。

Result: MedHopQA在BioCreative IX中作为共享任务发布，包含1000个评分问题，嵌入在可公开下载的10000个问题集中以降低排行榜操纵和污染风险。

Insight: 创新点在于构建了一个强调组合推理、抗饱和和抗污染的生物医学QA基准框架，通过结构化人工标注与LLM作为裁判验证相结合，并利用本体论资源增强评估的鲁棒性。

Abstract: Evaluating large language models (LLMs) in the biomedical domain requires benchmarks that can distinguish reasoning from pattern matching and remain discriminative as model capabilities improve. Existing biomedical question answering (QA) benchmarks are limited in this respect. Multiple-choice formats can allow models to succeed through answer elimination rather than inference, while widely circulated exam-style datasets are increasingly vulnerable to performance saturation and training data contamination. Multi-hop reasoning, defined as the ability to integrate information across multiple sources to derive an answer, is central to clinically meaningful tasks such as diagnostic support, literature-based discovery, and hypothesis generation, yet remains underrepresented in current biomedical QA benchmarks. MedHopQA is a disease-centered multi-hop reasoning benchmark consisting of 1,000 expert-curated question-answer pairs introduced as a shared task at BioCreative IX. Each question requires synthesis of information across two distinct Wikipedia articles, and answers are provided in an open-ended free-text format. Gold annotations are augmented with ontology-grounded synonym sets from MONDO, NCBI Gene, and NCBI Taxonomy to support both lexical and concept-level evaluation. MedHopQA was constructed through a structured process combining human annotation, triage, iterative verification, and LLM-as-a-judge validation. To reduce leaderboard gaming and contamination risk, the 1,000 scored questions are embedded within a publicly downloadable set of 10,000 questions, with answers withheld, on a CodaBench leaderboard. MedHopQA provides both a benchmark and a reusable framework for constructing future biomedical QA datasets that prioritize compositional reasoning, saturation resistance, and contamination resistance as core design constraints.

[26] Context Convergence Improves Answering Inferential Questions cs.CL | cs.IRPDF

Jamshid Mozafari, Bhawna Piryani, Adam Jatowt

TL;DR: 本文研究了大型语言模型在推理型问答任务中的表现，重点探讨了段落结构和质量对模型性能的影响。通过引入收敛性作为衡量句子消除错误答案能力的指标，构建不同收敛水平的段落，并在TriviaHG数据集上评估了六种不同规模和架构的LLM。结果表明，高收敛性句子构建的段落能显著提升答案准确率，优于基于余弦相似性的选择方法，且按收敛性降序排列句子可略微改善性能，揭示了LLM倾向于优先处理信息丰富的早期线索。

Details

Motivation: 尽管大型语言模型在开放域问答中广泛应用，但其处理需要推导而非直接检索答案的推理型问题的能力仍未被充分探索，本文旨在研究段落结构和质量如何影响LLM在此类问题上的表现。

Result: 在TriviaHG数据集子集上的实验显示，基于高收敛性句子构建的段落比基于余弦相似性选择的段落显著提高了答案准确率，且按收敛性降序排列句子带来轻微性能提升，这验证了收敛性作为推理相关性的有效信号。

Insight: 论文的创新点在于提出收敛性作为构建推理型问答段落的新标准，能够捕捉句子消除错误答案的能力，从而优化LLM的推理性能；从客观角度看，该方法为分析LLM的推理行为提供了实用工具，并揭示了模型对信息呈现顺序的敏感性，可借鉴于提升检索增强生成系统的段落构建策略。

Abstract: While Large Language Models (LLMs) are widely used in open-domain Question Answering (QA), their ability to handle inferential questions-where answers must be derived rather than directly retrieved-remains still underexplored. This study investigates how the structure and quality of passages influence LLM performance on such questions. We focus on convergence, a measure of how effectively sentences (hints) eliminate incorrect answers, as a criterion for constructing passages. Using subsets of the TriviaHG dataset, we form passages by combining sentences with varying convergence levels and evaluate six LLMs of different sizes and architectures. Our results show that passages built from higher convergence sentences lead to substantially better answer accuracy than those selected by cosine similarity, indicating that convergence captures meaningful relevance for inferential reasoning. Additionally, ordering sentences by descending convergence slightly improves performance, suggesting that LLMs tend to prioritize earlier, information-rich cues. These findings highlight convergence as a practical signal for guiding passage construction and analyzing inferential reasoning behavior in LLMs.

[27] Scalable Token-Level Hallucination Detection in Large Language Models cs.CL | cs.AI | cs.LGPDF

Rui Min, Tianyu Pang, Chao Du, Minhao Cheng, Yi R. Fung

TL;DR: 本文提出了TokenHD，一个用于训练词元级幻觉检测器的整体流程，旨在解决大语言模型在推理密集型任务中产生的、难以检测的幻觉问题。该方法通过可扩展的数据引擎合成大规模幻觉标注，并采用重要性加权策略进行鲁棒训练，无需预定义步骤分割或额外文本格式化。

Details

Motivation: 大语言模型虽然能力出众，但经常产生幻觉，尤其在推理任务中，内容看似连贯但包含逻辑缺陷和不可靠中间结果等错误。现有的步骤级分析方法因依赖步骤分割而存在粒度有限和可扩展性差的问题。

Result: 实验表明，即使是一个小型检测器（0.6B）在训练后也能实现显著的性能提升，超越了更大的推理模型（如QwQ-32B），并且检测性能在0.6B到8B的模型规模上持续提升。检测器在多样化的实际场景中表现出良好的泛化能力。

Insight: 创新点在于提出了一个可扩展的词元级幻觉检测训练流程，通过合成数据和重要性加权策略实现细粒度、无需步骤分割的检测，并建立了严格的评估协议。其方法展示了小模型通过针对性训练可以在特定任务上超越大模型，且性能随模型规模可扩展。

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities, but they still frequently produce hallucinations. These hallucinations are difficult to detect in reasoning-intensive tasks, where the content appears coherent but contains errors like logical flaws and unreliable intermediate results. While step-level analysis is commonly used to detect internal hallucinations, it suffers from limited granularity and poor scalability due to its reliance on step segmentation. To address these limitations, we propose TokenHD, a holistic pipeline for training token-level hallucination detectors. Specifically, TokenHD consists of a scalable data engine for synthesizing large-scale hallucination annotations along with a training recipe featuring an importance-weighted strategy for robust model training. To systematically assess the detection performance, we also provide a rigorous evaluation protocol. Through training within TokenHD, our detector operates directly on free-form text to identify hallucinations, eliminating the need for predefined step segmentation or additional text reformatting. Our experiments show that even a small detector (0.6B) achieves substantial performance gains after training, surpassing much larger reasoning models (e.g., QwQ-32B), and detection performance scales consistently with model size from 0.6B to 8B. Finally, we show that our detector can generalize well across diverse practical scenarios and explore strategies to further enhance its cross-domain generalization capability.

[28] Question Difficulty Estimation for Large Language Models via Answer Plausibility Scoring cs.CL | cs.IRPDF

Jamshid Mozafari, Bhawna Piryani, Adam Jatowt

TL;DR: 本文提出了一种名为Q-DAPS的新方法，通过计算候选答案的合理性分数的熵来估计问题难度，以更好地评估和改进大型语言模型在问答任务中的表现。该方法在多个QA数据集上进行了系统评估，并显示出优于基线方法的性能。

Details

Motivation: 现有问题难度估计方法（如可读性公式、基于检索的信号或流行度统计）可能无法充分捕捉现代大型语言模型所面临的推理挑战，因此需要一种更有效的方法来估计问题难度。

Result: 在TriviaQA、NQ、MuSiQue和QASC四个QA数据集上的实验表明，Q-DAPS方法在问题难度估计上一致优于基线方法，并显示出对超参数变化和问题类型的强鲁棒性。人类评估进一步证实了Q-DAPS的难度估计与人类判断高度一致。

Insight: 创新点在于利用候选答案的合理性分数的熵作为难度估计指标，这种方法具有可解释性、可扩展性和对偏差的强鲁棒性，为现代问答系统中的问题难度估计提供了一种新范式。

Abstract: Estimating question difficulty is a critical component in evaluating and improving large language models (LLMs) for question answering (QA). Existing approaches often rely on readability formulas, retrieval-based signals, or popularity statistics, which may not fully capture the reasoning challenges posed to modern LLMs. In this paper, we introduce Q-DAPS (Question Difficulty based on Answer Plausibility Scores) method, a novel approach that estimates question difficulty by computing the entropy of plausibility scores over candidate answers. We systematically evaluate Q-DAPS across four prominent QA datasets-TriviaQA, NQ, MuSiQue, and QASC-demonstrating that it consistently outperforms baselines. Moreover, Q-DAPS shows strong robustness across hyperparameter variations and question types. Extensive ablation studies further show that Q-DAPS remains robust across different plausibility estimation paradigms, model sizes, and realistic settings. Human evaluations further confirm strong alignment between Q-DAPS’s difficulty estimates and human judgments of question difficulty. Overall, Q-DAPS provides an interpretable, scalable, and bias-resilient approach to question difficulty estimation in modern QA systems.

[29] ORBIT: Preserving Foundational Language Capabilities in GenRetrieval via Origin-Regulated Merging cs.CL | cs.IR | cs.LGPDF

Neha Verma, Nikhil Mehta, Shao-Chuan Wang, Naijing Zhang, Alicia Tsai

TL;DR: 本文提出ORBIT方法，通过动态追踪微调模型与原始模型参数的距离，并在距离超过阈值时采用权重平均策略来约束模型漂移，从而在生成式检索任务微调中有效缓解大语言模型基础语言能力的灾难性遗忘问题。

Details

Motivation: 解决大语言模型在针对特定任务（如生成式检索）进行微调时，其通用语言推理能力会快速发生灾难性遗忘的问题。

Result: 在生成式检索任务上，ORBIT方法在文本和检索性能上均优于常见的持续学习基线方法和相关的正则化方法（这些方法也采用了权重平均）。

Insight: 创新点在于主动监控微调过程中的模型参数漂移，并基于模型间距离阈值动态应用权重平均来保留基础能力；其核心洞察是模型遗忘的严重性与微调模型和原始模型的参数距离相关。

Abstract: Despite the rapid advancements in large language model (LLM) development, fine-tuning them for specific tasks often results in the catastrophic forgetting of their general, language-based reasoning abilities. This work investigates and addresses this challenge in the context of the Generative Retrieval (GenRetrieval) task. During GenRetrieval fine-tuning, we find this forgetting occurs rapidly and correlates with the distance between the fine-tuned and original model parameters. Given these observations, we propose ORBIT, a novel approach that actively tracks the distance between fine-tuned and initial model weights, and uses a weight averaging strategy to constrain model drift during GenRetrieval fine-tuning when this inter-model distance exceeds a maximum threshold. Our results show that ORBIT retains substantial text and retrieval performance by outperforming both common continual learning baselines and related regularization methods that also employ weight averaging.

[30] A Causal Language Modeling Detour Improves Encoder Continued Pretraining cs.CL | cs.AIPDF

Rian Touchent, Eric de la Clergerie

TL;DR: 本文提出了一种改进编码器持续预训练的方法，即在适应新领域时，暂时从掩码语言建模（MLM）切换到因果语言建模（CLM），再进行短暂的MLM衰减训练，以提高下游任务性能。在生物医学文本上使用ModernBERT模型进行实验，该方法在8个法语和11个英语生物医学任务上均优于仅使用MLM的基线模型。

Details

Motivation: 标准方法在将编码器适应新领域时通常仅使用MLM进行持续训练，本文旨在探索通过引入CLM训练阶段是否能进一步提升模型在下游任务中的表现。

Result: 在生物医学任务上，该方法相比纯MLM基线在法语任务上提升1.2-2.8个百分点，在英语任务上提升0.3-0.8个百分点，具体提升幅度取决于模型大小。作者发布了ModernCamemBERT-bio和ModernBERT-bio模型，在Base和Large尺寸上达到了生物医学编码器的SOTA水平。

Insight: 创新点在于提出了CLM绕道训练策略，并发现CLM的密集监督主要影响Transformer的低层（0-7层），且这种表示变化在后续MLM衰减阶段得以保持。客观分析表明，该方法通过改变训练目标顺序有效利用了不同预训练任务的优势，提升了模型对新领域数据的适应性。

Abstract: When adapting an encoder to a new domain, the standard approach is to continue training with Masked Language Modeling (MLM). We show that temporarily switching to Causal Language Modeling (CLM) followed by a short MLM decay improves downstream performance. On biomedical texts with ModernBERT, this CLM detour outperforms MLM baselines trained on identical data and compute across 8 French and 11 English biomedical tasks, by +1.2-2.8pp and +0.3-0.8pp respectively, depending on model size. We investigate the reasons for these gains. We find that CLM’s dense supervision impacts low transformer layers (0-7) far more than MLM does. Freezing low layers during CLM eliminates the downstream benefit; freezing mid layers preserves it. The representational changes persist through the MLM decay phase, even when it matches the CLM phase in length, and they scale with model capacity. We release ModernCamemBERT-bio and ModernBERT-bio as state-of-the-art biomedical encoders in Base and Large sizes.

cs.CV [Back]

[31] Principle-Guided Supervision for Interpretable Uncertainty in Medical Image Segmentation cs.CVPDF

An Sui, Yuzhu Li, Gunter Schumann, Fuping Wu, Xiahai Zhuang

TL;DR: 本文提出了一种原则引导的监督框架（PriUS），用于提升医学图像分割中不确定性估计的可解释性。该框架基于证据学习，通过三个与人类感知对齐的原则来监督不确定性的空间分布，使其能反映图像对比度、图像损坏程度和解剖结构的几何复杂性。

Details

Motivation: 现有不确定性量化方法通常将不确定性简化为标量置信度估计，导致其空间分布缺乏语义约束，难以解释。本文旨在提升不确定性估计的可解释性，使其行为与人类理解的模糊性来源一致。

Result: 在ACDC、ISIC和WHS数据集上的实验表明，PriUS在保持竞争力的分割性能的同时，相比SOTA方法能产生更一致的不确定性估计。

Insight: 创新点在于提出了三个感知对齐原则来显式监督不确定性的空间分布，并设计了相应的量化指标。这为构建可解释、语义明确的不确定性估计提供了新思路，尤其适用于需要高可靠性决策的医疗领域。

Abstract: Uncertainty quantification complements model predictions by characterizing their reliability, which is essential for high-stakes decision making such as medical image segmentation. However, most existing methods reduce uncertainty to a scalar confidence estimate, leaving its spatial distribution semantically underconstrained. In this work, we focus on uncertainty interpretability, namely, whether estimated uncertainty behaves in a human-understandable manner with respect to sources of ambiguity. We identify three perception-aligned principles requiring the spatial distribution of uncertainty to reflect: (1) image contrast between structures, (2) severity of image corruption, and (3) geometric complexity in anatomical structures. Accordingly, we develop a principle-guided uncertainty supervision framework (PriUS) based on evidential learning, in which the corresponding supervision objectives are explicitly enforced during training. We further introduce quantitative metrics to measure the consistency between predicted uncertainty and image attributes that induce ambiguity. Experiments on ACDC, ISIC, and WHS datasets showed that, compared with state-of-the-art methods, PriUS produced more consistent uncertainty estimates while maintaining competitive segmentation performance.

[32] HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer cs.CV | cs.MMPDF

Qi Cai, Jingwen Chen, Chengmin Gao, Zijian Gong, Yehao Li

TL;DR: 本文提出了HiDream-O1-Image，一种基于像素空间扩散变换器（Diffusion Transformer）的原生统一生成基础模型。它通过将原始图像像素、文本标记和任务特定条件映射到单一的共享标记空间，在统一变换器（UiT）架构内实现了多模态输入的结构统一。该模型无需依赖外部VAE或独立的预训练文本编码器，能够将多样化的生成和编辑任务视为一致的情境推理过程。实验表明，该模型在文本到图像生成、基于指令的编辑和主题驱动的个性化等任务上表现出色，其8B参数版本性能与或超越参数量大得多的现有SOTA模型（如27B Qwen-Image），而超过200B参数的Pro版本则展现了前所未有的生成能力和SOTA性能。

Details

Motivation: 解决视觉生成模型长期依赖碎片化架构（如独立的文本编码器和外部VAE）的问题，旨在实现从模块化架构到端到端情境视觉生成引擎的范式转变。

Result: 在多种生成任务上表现出色。8B参数版本在性能上达到或超越了参数量显著更大的现有SOTA模型（如27B Qwen-Image）。成功将架构扩展到超过200B参数（HiDream-O1-Image-Pro），该版本解锁了前所未有的生成能力，并建立了新的SOTA基准。

Insight: 核心创新在于提出了一个原生统一的多模态编码范式，通过单一共享标记空间和统一变换器（UiT）架构，实现了对原始像素、文本和任务条件的端到端统一处理，从而消除了对独立VAE和文本编码器的依赖。这为构建高度可扩展的下一代多模态AI指明了方向。

Abstract: The evolution of visual generative models has long been constrained by fragmented architectures relying on disjoint text encoders and external VAEs. In this report, we present HiDream-O1-Image, a natively unified generative foundation model via pixel-space Diffusion Transformer, that pioneers a paradigm shift from modular architectures to an end-to-end in-context visual generation engine. By mapping raw image pixels, text tokens, and task-specific conditions into a single shared token space, HiDream-O1-Image achieves a structural unification of multimodal inputs within an Unified Transformer (UiT) architecture. This native encoding paradigm eliminates the need for separate VAEs or disjoint pre-trained text encoders, allowing the model to treat diverse generation and editing tasks as a consistent in-context reasoning process. Extensive experiments show that HiDream-O1-Image excels across various generation tasks, including text-to-image generation, instruction-based editing, and subject-driven personalization. Notably, with only 8B parameters, HiDream-O1-Image (8B) achieves performance parity with or even surpasses established state-of-the-art models with significantly larger parameters (e.g., 27B Qwen-Image). Crucially, to validate the immense scalability of this paradigm, we successfully scale the architecture up to over 200B parameters. Experimental results demonstrate that this massive-scale version HiDream-O1-Image-Pro (200B+) unlocks unprecedented generative capabilities and superior performance, establishing new state-of-the-art benchmarks. Ultimately, HiDream-O1-Image highlights the immense potential of natively unified architectures and charts a highly scalable path toward next-generation multimodal AI.

[33] Birds of a Feather Flock Together: Background-Invariant Representations via Linear Structure in VLMs cs.CV | cs.AIPDF

Youssef Zaazou, Mark Thomas

TL;DR: 本文提出了一种利用视觉语言模型（VLM）嵌入空间高度线性可加性的特性，将场景表征分解为前景和背景成分的方法，并基于此设计了一种使用合成数据进行预训练的方法，以构建对背景不变的鲁棒表征。该方法在Waterbirds数据集上实现了超过90%的最差组准确率，并展示了良好的仿真到真实迁移能力。

Details

Motivation: 解决当前广泛使用的视觉语言模型（如CLIP、SigLIP）的视觉编码器容易受到系统性偏见（特别是前景物体与其背景之间的虚假相关性）影响，从而损害模型鲁棒性的问题。

Result: 在Waterbirds数据集上，在训练数据存在完美（100%）虚假相关性（即训练数据中没有少数群体样本）的极端条件下，该方法实现了首个最差组准确率超过90%的结果，达到了SOTA水平。

Insight: 核心创新点在于重新审视并利用了VLM嵌入空间高度线性可加性的已知特性，将其形式化为一种将场景表征分解为前景和背景分量的方法，并据此设计了一种仅需合成数据、无需真实世界去偏数据的实用预训练策略，以实现背景不变的鲁棒表征学习。

Abstract: Vision-language models (VLMs), such as CLIP and SigLIP 2, are widely used for image classification, yet their vision encoders remain vulnerable to systematic biases that undermine robustness. In particular, correlations between foreground objects and their backgrounds constitute a salient and practically important class of spurious dependencies. In this work, we revisit the well-known property of high linear additivity in VLM embedding spaces and show that it enables a decomposition of scene representations into foreground and background components. Leveraging this insight, we introduce a pre-training approach that exploits this property to construct background-invariant representations using synthetic data. Our method achieves, to our knowledge, the first worst-group accuracy exceeding $90%$ on Waterbirds under perfect ($100%$) spurious correlation (i.e., no minority-group examples in the training data). Furthermore, it demonstrates strong sim-to-real transfer and requires no access to real-world debiased data, making it practical for real-world deployment.

[34] USEMA: a Scalable Efficient Mamba Like Attention for Medical Image Segmentation cs.CVPDF

Elisha Dayag, Nhat Thanh Tran, Jack Xin

TL;DR: 本文提出USEMA，一种结合卷积神经网络局部特征提取能力与SEMA注意力的混合UNet架构，用于医学图像分割。SEMA是一种可扩展、高效的类Mamba注意力机制，通过局部窗口注意力进行令牌定位以避免注意力分散，并辅以理论一致的算术平均来捕获注意力的全局方面。实验表明，USEMA在多种模态和图像尺寸上，相比基于完整自注意力的Transformer模型具有更高的计算效率，并且相对于纯卷积和基于Mamba的模型具有更优的分割性能。

Details

Motivation: 医学图像分割需要融合局部和全局信息。虽然视觉Transformer能够使用标准自注意力捕获全局交互，但其在输入尺寸上的二次计算复杂度对于医学图像分割任务仍然是一个挑战。受标准自注意力分散特性以及近期Mamba形式注意力发展的启发，本文旨在设计一种更高效、可扩展的注意力机制来解决此问题。

Result: 在多种模态和图像尺寸上的实验表明，USEMA相比基于完整自注意力的Transformer模型具有更高的计算效率，并且分割性能优于纯卷积模型和基于Mamba的模型。

Insight: 主要创新点在于提出了SEMA注意力机制，它结合了局部窗口注意力（用于令牌定位和避免注意力分散）与理论一致的算术平均（用于捕获全局信息）。此外，将SEMA与CNN的局部特征提取能力在UNet架构中结合，形成了一个高效的混合模型。从客观角度看，这种结合高效注意力机制与经典CNN架构的思路，为在资源受限场景（如医学图像分析）下设计高性能模型提供了借鉴。

Abstract: Accurate medical image segmentation is an integral part of the medical image analysis pipeline that requires the ability to merge local and global information. While vision transformers are able to capture global interactions using vanilla self-attention, their quadratic computational complexity in the input size remains a struggle for medical image segmentation tasks. Motivated by the dispersion property of vanilla self-attention and recent development of Mamba form of attention, Scalable and Efficient Mamba like Attention (SEMA) utilizes token localization via local window attention to avoid dispersion and maintain focusing, complemented by theoretically consistent arithmetic averaging to capture global aspect of attention. In this work, we present USEMA, a hybrid UNet architecture that merges the local feature extraction ability of convolutional neural networks (CNNs) with SEMA attention. We conduct experiments with USEMA across a variety of modalities and image sizes, demonstrating improved computational efficiency compared to transformer based models using full self-attention, and superior segmentation performance relative to purely convolution and Mamba-based models.

Elena Sirotkina

TL;DR: 本文开发了Perspectivist Visual Political Sentiment (PVPS)分类器，旨在解决现有计算工具忽略社会身份对政治信息感知影响的问题。该模型基于约82,000份美国成年人的评估数据，能够预测不同政治和社会身份群体对同一图像的评价差异，而非输出单一的平均分数。

Details

Motivation: 现有计算工具常为文本、图像或视频生成单一的情感或立场分数，忽视了政治和社会身份如何结构化人们对政治信息的评估这一数十年来的政治学发现。本文旨在解决这一差距，证明同一政治图像对不同身份群体的含义存在显著且重要的差异。

Result: PVPS模型应用于多项视觉情感研究，结果表明，一旦考虑受众身份，抗议图像中感知到的暴力程度以及参与抗议图像的情感机制都会发生实质性变化。模型通过保留系统性分歧，提供了记录不同身份群体间赞同与分歧的评价档案。

Insight: 核心创新在于提出了一个视角主义的（perspectivist）分类框架PVPS，它不将分歧平均化，而是建模并输出基于受众身份的评价分布。这揭示了政治图像的涵义是一个’移动的目标’，其测量必须关联特定的受众群体，为计算政治分析引入了身份敏感的维度。

Abstract: Political and social identities structure how people evaluate political information, a finding decades deep in political science and routinely discarded by computational tools that often produce single scores that treat a piece of text, an image, or a video as if it means the same thing to everyone. This paper shows that it does not, and that the difference is consequential. To address this problem, I develop the Perspectivist Visual Political Sentiment (PVPS) classifier, which learns from approximately 82,000 evaluations by 5,575 U.S. adults to predict how audiences defined by political and social identities will evaluate the same image. Unlike standard tools that average systematic disagreement away, PVPS preserves it, returning an evaluative profile that records who agrees, who diverges, and along which identity lines. Applied to several influential studies of visual sentiment, PVPS shows that perceived violence in protest imagery and the emotional mechanisms behind protest image engagement both change substantively once audience identity is taken into account. It follows that what a political image conveys is a moving target, and measuring it requires knowing whom it is moving.

[36] Hi-GaTA: Hierarchical Gated Temporal Aggregation Adapter for Surgical Video Report Generation cs.CVPDF

Kedi Sun, Chaohui Dang, Yue Feng, James Glasbey, Theodoros N. Arvanitis

TL;DR: 本文提出了Hi-GaTA（分层门控时序聚合适配器），用于手术视频报告生成。该方法基于一个包含214个高质量模拟手术视频与外科医生撰写评估报告的新基准数据集，并采用感知-对齐-推理框架。核心创新是Hi-GaTA轻量级时序适配器，它能通过短到长程时序聚合将长视频序列高效压缩为紧凑的、与大语言模型兼容的视觉前缀令牌。此外，研究还预训练了Sur40k手术专用视频编码器以获取细粒度时空先验，并利用LoRA微调LLM生成连贯、风格一致的手术报告。

Details

Motivation: 动机是解决手术视频自动生成临床级评估报告的挑战，这些挑战包括密集时空视频表征与基于语言推理的对齐困难，以及高质量、保护隐私的数据集稀缺。

Result: 实验表明，该方法在建立的基准上取得了最佳整体性能，相对于强大的多模态大语言模型基线模型获得了持续的性能提升。消融研究进一步验证了每个提出组件的有效性。

Insight: 宣称的创新点包括：1) 建立了高质量的手术视频报告生成基准数据集；2) 提出了新颖的Hi-GaTA轻量级时序适配器，通过文本条件双交叉注意力的时序金字塔、跨层级门控融合和深度递增策略实现高效的多尺度时序聚合与一致性；3) 预训练了手术领域的专用视频编码器Sur40k。从客观角度看，其将长视频高效压缩为LLM兼容令牌的适配器设计，以及在有限监督下利用LoRA微调LLM生成风格一致报告的方法，具有借鉴价值。

Abstract: Automated, clinician-grade assessment reports for surgical procedures could reduce documentation burden and provide objective feedback, yet remain challenging due to the difficulty of aligning dense spatio-temporal video representations with language-based reasoning and the scarcity of high-quality, privacy-preserving datasets. To address this gap, we establish a benchmark comprising 214 high-quality simulated surgical videos paired with surgeon-authored evaluation reports. Building on this resource, we propose a Perception-Alignment-Reasoning framework for surgical video report generation, featuring Hi-GaTA, a novel lightweight temporal adapter that efficiently compresses long video sequences into compact, LLM-compatible visual prefix tokens through short-to-long-range temporal aggregation. For robust visual perception, we pretrain Sur40k, a surgical-specific ViViT-style video encoder on 40,000 minutes of public surgical videos to capture fine-grained spatio-temporal procedural priors. Hi-GaTA employs a temporal pyramid with text-conditioned dual cross-attention, and improves multi-scale consistency through cross-level gated fusion and an increasing-depth strategy. Finally, we fine-tune the LLM backbone using LoRA to enable coherent and stylistically consistent surgical report generation under limited supervision. Experiments show our approach achieves the best overall performance, with consistent gains over strong Multimodal Large Language Model (MLLM) baselines. Ablation studies further validate the effectiveness of each proposed component.

[37] ABRA: Agent Benchmark for Radiology Applications cs.CV | cs.AIPDF

Bulat Maksudov, Vladislav Kurenkov, Kathleen M. Curran, Alessandra Mileo

TL;DR: 本文提出了ABRA，一个用于放射学应用的智能体基准测试，通过模拟OHIF查看器和Orthanc DICOM服务器环境，提供21个函数调用工具，涵盖切片导航、窗宽窗位调整、序列选择、像素坐标标注和结构化报告等功能。该基准包含655个程序生成的任务，分为三个难度等级和八种类型，并基于特定任务的自动评分器从规划、执行和结果三个维度进行评估。

Details

Motivation: 现有医学智能体基准测试通常将影像作为预选样本提供，而非需要智能体主动导航的环境，这限制了智能体在实际放射学工作流中的评估。ABRA旨在填补这一空白，通过模拟真实放射学操作环境，全面评估智能体在复杂任务中的表现。

Result: 在评估的十个当前模型（五个闭源权重和五个开源权重）中，所有模型在真实标注任务上的执行分数至少达到89%，但结果分数仅为0-25%；而在配对的神谕变体任务中（模拟检测器提供发现），相同任务的结果分数提升至69-100%，表明瓶颈主要在于感知能力而非工具协调。

Insight: 创新点在于将放射学任务环境化，通过程序化生成多样化任务和自动评分机制，系统性地评估智能体的规划、执行和结果能力。客观分析显示，该基准有效揭示了当前模型在感知与工具使用协调中的关键瓶颈，为未来医学AI智能体的开发提供了重要方向。

Abstract: Existing medical-agent benchmarks deliver imaging as pre-selected samples, never as an environment the agent must navigate. We introduce ABRA, a radiology-agent benchmark in which the agent operates an OHIF viewer and an Orthanc DICOM server through twenty-one function-calling tools that span slice navigation, windowing, series selection, pixel-coordinate annotation, and structured reporting. ABRA contains 655 programmatically generated tasks across three difficulty tiers and eight types (viewer control, metadata QA, vision probe, annotation, longitudinal comparison, BI-RADS reporting, and oracle variants of annotation and BI-RADS reporting), drawn from LIDC-IDRI, Duke Breast Cancer MRI, and NLST New-Lesion LongCT. Each episode is scored along Planning, Execution, and Outcome (Bluethgen et al., 2025) by task-type-specific automatic scorers. Ten current models, five closed-weight and five open-weight, reach at least 89% Execution on real annotation but only 0-25% Outcome; on the paired oracle variant where a simulated detector supplies the finding, Outcome on the same task reaches 69-100% across the models evaluated, localising the bottleneck to perception rather than tool orchestration. Code, task generators, and scorers are released at https://github.com/Luab/ABRA

[38] DenseTRF: Texture-Aware Unsupervised Representation Adaptation for Surgical Scene Dense Prediction cs.CV | cs.AI | cs.LGPDF

Guiqiu Liao, Matjaž Jogan, Daniel A. Hashimoto

TL;DR: 本文提出DenseTRF，一种基于纹理中心注意力的自监督表示适应框架，用于提升手术场景密集预测任务（如分割和手术区域预测）对分布偏移的鲁棒性。该方法利用槽注意力学习纹理感知表示，并通过无监督方式将其适应到目标分布，从而显著改善模型的泛化能力。

Details

Motivation: 手术计算机视觉中的密集预测模型常因训练数据分布与部署环境存在差异（分布偏移）而导致泛化性能下降，需要一种无监督方法来适应目标域以提升鲁棒性。

Result: 在多个手术流程的实验表明，DenseTRF在密集预测任务上相比最先进的分割模型和测试分布适应方法，实现了更好的跨分布泛化性能。

Insight: 创新点在于结合槽注意力学习纹理感知的表示，并通过模型合并策略实现无监督表示适应，这为处理手术场景中的域偏移问题提供了一种新的纹理中心自监督适应思路。

Abstract: Dense prediction tasks in surgical computer vision, such as segmentation and surgical zone prediction, can provide valuable guidance for laparoscopic and robotic surgery. However, these models often suffer from distribution shifts, as training datasets rarely cover the variability encountered during deployment, leading to poor generalization. We propose DenseTRF, a self-supervised representation adaptation framework based on texture-centric attention. Our method leverages slot attention to learn texture-aware representations that capture invariant visual structures. By adapting these representations to the target distribution without supervision, DenseTRF significantly improves robustness to domain shifts. The framework is implemented through conditioning dense prediction on slot attention and model merging strategies. Experiments across multiple surgical procedures demonstrate improved cross-distribution generalization in comparison to state-of-the-art segmentation models and test-distribution adaptation methods for dense prediction tasks.

[39] PG-3DGS: Optimizing 3D Gaussian Splatting to Satisfy Physics Objectives cs.CV | cs.GR | cs.LGPDF

Zachary Lee, Maxwell Jacobson, Yexiang Xue

TL;DR: 本文提出PG-3DGS框架，将可微分物理模拟与3D高斯泼溅表示相结合，优化生成既满足视觉保真度又具备物理功能性的3D结构。该方法通过物理目标引导形状优化，使生成物体（如可倾倒的茶壶、能产生升力的飞机）在保持视觉质量的同时实现物理功能。实验在倾倒和空气动力学升力任务中验证了其有效性，并通过3D打印模型在真实气流测试中展示了优于视觉匹配基线的物理性能。

Details

Motivation: 现有3D高斯泼溅方法仅关注视觉重建，缺乏对物体在物理世界中行为功能的理解。本文旨在将物理功能目标融入3D形状优化过程，生成既视觉逼真又物理可用的3D结构。

Result: 在倾倒和空气动力学升力模拟任务中，PG-3DGS在保持视觉质量的同时提升了物理功能。对3D打印的飞机模型（Cessna、B-2 Spirit、纸飞机）进行真实气流台架测试，在所有案例中，PG-3DGS生成的结构均比视觉匹配基线产生了更高的实测升力。

Insight: 创新点在于将可微分物理模拟与3D高斯表示进行端到端耦合，使物理目标与视觉损失共同指导优化。这为3D重建领域引入了物理推理能力，实现了外观重建与物理功能性的统一优化框架。

Abstract: Recent advances in Gaussian Splatting have enabled fast, high-fidelity 3D scene generation, yet these methods remain purely visual and lack an understanding of how shapes behave in the physical world. We introduce Physics-Guided 3D Gaussian Splatting (PG-3DGS), a framework that couples differentiable physics simulation with 3D Gaussian representations to generate 3D structures satisfying physics functionalities. By allowing physical objectives to guide the shape optimization process alongside visual losses, our approach produces geometries that are not only photometrically accurate but also physically functional. The model learns to adjust shapes so that the generated objects exhibit physically meaningful behaviors, for example, teapots that can pour and airplanes that can generate lift, without sacrificing visual quality. Experiments on pouring and aerodynamic lift tasks show that PG-3DGS improves physical functionality while preserving visual quality. In addition to simulation gains, bench-top physical lift tests with 3D-printed aircraft (Cessna, B-2 Spirit, and paper plane) under identical airflow conditions show higher scale-measured lift for PG-3DGS, generated structures than an appearance-matching baseline in all three cases. Our unified framework connects appearance-based reconstruction with physics-based reasoning, enabling end-to-end generation of 3D structures that both look realistic and function correctly.

[40] Real-Scale Island Area and Coastline Estimation using Only its Place Name or Coordinates cs.CVPDF

Quanyun Wu, Kyle Gao, Wentao Sun, Hongjie He, Yuhao Chen

TL;DR: 本文提出了一种基于纯单目视觉的几何一致、真实尺度岛屿测量框架，仅需输入目标区域的地理坐标或名称即可获取低空环绕图像序列，通过轻量级轨迹对齐算法恢复全局物理尺度，并在二维栅格化平面上实现高精度面积和周长提取。

Details

Motivation: 传统岛屿面积和海岸线长度测量方法依赖正射影像、昂贵传感器或密集地面控制点，成本高、效率低，难以在广阔海域应用。本文旨在克服这些限制，摆脱对人工实地勘探的依赖，实现低成本、高效率的自动化测量。

Result: 在四种不同地形特征的岛屿（包括自然地貌和复杂人工设施岛屿）上验证，系统最终测量误差稳定在10%左右，表现出优秀的准确性和鲁棒性；推理速度突出，单张高分辨率图像处理及点云生成仅需70毫秒。

Insight: 创新点在于结合纯单目视觉与轻量级轨迹对齐算法（Umeyama）实现真实尺度恢复，无需先验GIS数据，为大规模海洋和海岸线监测提供了高效实用的新范式。

Abstract: Accurate measurement of island area and coastline length is crucial for coastal zone monitoring and oceanographic analysis. However, traditional measurement and mapping methods usually rely heavily on orthophotos, expensive airborne depth sensors, or dense ground control points, which face serious limitations of high labor costs, time-consuming efforts, and low operational efficiency in vast and inaccessible open sea environments. To overcome these challenges and break away from the reliance on manual field exploration, this paper proposes a geometrically consistent, real-scale island measurement framework based on pure monocular vision. This project significantly reduces the mapping cost through a fully automated process and achieves high-efficiency measurement without prior GIS data. In our system pipeline, only the geographical coordinates or names of the target area need to be input to obtain a low-altitude surrounding image sequence. After obtaining the point clouds, a lightweight trajectory alignment algorithm (Umeyama) is used to restore the global physical scale, and the scaled model is orthorectified, enabling high-precision area and perimeter extraction directly on the 2D rasterized plane. We have fully verified this pipeline on four islands with different terrain features (covering natural landform islands and islands with complex artificial facilities). The experimental results show that the final measurement error of the system is stable at around 10%, demonstrating excellent accuracy and robustness. Moreover, this framework has outstanding inference speed, requiring only 70 ms to process a single high-resolution image and generate point clouds, providing a highly practical new paradigm for large-scale marine and coastline

[41] Can Graphs Help Vision SSMs See Better? cs.CVPDF

Dhruv Parikh, Anvitha Ramachandran, Haoyang Fan, Mustafa Munir, Rajgopal Kannan

TL;DR: 本文提出GraphScan，一种基于图结构的动态扫描算子，用于增强视觉状态空间模型（Vision SSMs）。GraphScan通过构建局部图、学习特征条件亲和力并进行消息传递，实现语义邻居间的信息交换，再结合选择性SSM进行全局聚合。集成到分层骨干网络中的GraphScan-Mamba在图像分类、目标检测、实例分割和语义分割任务上达到了视觉SSMs中的最先进性能。

Details

Motivation: 现有视觉SSMs的扫描算子主要关注几何遍历或动态采样位置调整，但未能显式建模局部图像块之间在全局状态空间混合前应如何交换信息。本文旨在探索图结构是否能帮助视觉SSMs更好地理解图像，通过引入语义路由来改进信息传递。

Result: GraphScan-Mamba在图像分类、目标检测、实例分割和语义分割等多个基准测试中取得了视觉SSMs中的SOTA性能，且计算开销适中。分析表明，GraphScan能在令牌网格上产生可解释的位移场，提供语义和空间基础化的动态扫描视图。

Insight: 创新点在于将扫描操作从几何序列化转变为学习局部语义路由，通过图结构实现特征条件亲和力学习和消息传递，从而在全局状态空间建模前进行局部语义信息交换。这为未来视觉SSMs的设计提供了新思路，即扫描应作为局部语义路由而非单纯几何处理。

Abstract: Vision state space models inherit the efficiency and long-range modeling ability of Mamba-style selective scans. However, their performance depends critically on the representation of two-dimensional visual features as one-dimensional token sequences. Existing scan operators range from predefined geometric traversals to dynamic coordinate-based samplers that reroute tokens through predicted offsets and interpolation. While effective, these mechanisms primarily adapt paths or sampling locations, rather than explicitly modeling which local patches should exchange information before global state-space mixing. This motivates a simple question: \emph{can graphs help vision state space models see better?} We introduce \textbf{GraphScan}, a graph-induced dynamic scanning operator for Vision SSMs. For each token, GraphScan constructs a spatially bounded local graph, learns feature-conditioned affinities with relative positional bias, and produces the output token by one-step message passing over its semantic neighborhood. The resulting tokens are locally grounded before being processed by the selective SSM for global aggregation. GraphScan preserves token count and linear scaling in image size, while replacing coordinate-conditioned interpolation with feature-conditioned semantic routing. Integrated into a hierarchical backbone, \textbf{GraphScan-Mamba} achieves state-of-the-art performance among Vision SSMs across image classification, object detection, instance segmentation, and semantic segmentation, with modest computational overhead. Our analysis further shows that GraphScan induces interpretable displacement fields over the token lattice, providing a semantic and spatially grounded view of dynamic scanning. These results suggest that future Vision SSMs should treat scanning not merely as geometric serialization, but as learned local semantic routing before global state-space modeling.

[42] CheXTemporal: A Dataset for Temporally-Grounded Reasoning in Chest Radiography cs.CVPDF

Eva Prakash, Yunhe Gao, Chong Wang, Justin Xu, Neal Prakash

TL;DR: 本文介绍了CheXTemporal数据集，这是一个用于胸部X光影像时序推理的数据集，包含成对的先前与当前胸部X光影像，并带有病灶级别的时序和空间标注。数据集提供了五类进展分类、病灶空间监督、跨研究时空对齐以及多源覆盖用于跨域评估。此外，还构建了一个包含28万对样本的银标准数据集，用于大规模弱监督评估。通过评估多个先进视觉语言模型在零样本设置下的表现，发现当前模型在空间定位、细粒度时序推理和分布偏移鲁棒性方面存在局限。

Details

Motivation: 胸部X光影像解读需要基于先前和当前研究的时序推理，但大多数视觉语言模型在静态图像-报告对上训练，缺乏对纵向变化建模的显式监督。

Result: 在零样本设置下评估多个先进视觉语言CXR模型，发现模型在空间定位、细粒度时序推理和分布偏移鲁棒性方面存在一致局限，尤其在显著进展类别（如恶化）上表现优于时序细微状态（如稳定和消退）。

Insight: 创新点在于构建了首个用于胸部X光时序推理的标注数据集，提供细粒度的时空监督，并揭示了当前模型在建模纵向疾病演变方面的不足，为未来研究提供了基准和方向。

Abstract: Chest radiograph interpretation requires temporal reasoning over prior and current studies, yet most vision-language models are trained on static image-report pairs and lack explicit supervision for modeling longitudinal change. We introduce CheXTemporal, a dataset for temporally grounded reasoning in chest radiography consisting of paired prior-current chest X-rays (CXR) with finding-level temporal and spatial annotations. The dataset includes a five-class progression taxonomy (new, worse, stable, improved, resolved), localized spatial supervision of pathology, explicit spatial-temporal alignment across paired studies, and multi-source coverage for cross-domain evaluation. We additionally construct a 280K-pair silver dataset with automatically derived temporal and anatomical supervision for large-scale evaluation under weaker supervision. Using these resources, we evaluate multiple state-of-the-art vision-language CXR models on grounding and progression-classification tasks in a zero-shot setting. Across both gold and silver evaluations, current models exhibit consistent limitations in spatial grounding, fine-grained temporal reasoning, and robustness under distribution shift. In particular, models perform substantially better on salient progression categories such as worse than on temporally subtle states such as stable and resolved, suggesting limited modeling of longitudinal disease evolution in chest radiography.

[43] Vision2Code: A Multi-Domain Benchmark for Evaluating Image-to-Code Generation cs.CV | cs.LGPDF

Ajay Vikram Periasami, Junlin Wang, Bhuwan Dhingra

TL;DR: 本文提出了Vision2Code，一个用于评估图像到代码生成的多领域基准测试和评估框架。该基准包含来自15个源数据集的2,169个测试样例，涵盖图表、几何、图形、科学图像、文档和3D空间场景。通过使用带有特定领域评分细则的视觉语言模型评估器，对模型生成的代码进行渲染和评分，以衡量其恢复图像结构并生成可执行代码的能力。

Details

Motivation: 现有基准测试要么专注于狭窄的视觉领域，要么依赖于配对的参考代码，或使用通用评分标准而忽略了特定领域的重建错误。因此，需要一个新的、无需参考代码的、多领域的评估框架来更全面地衡量图像到代码生成能力。

Result: 在九个开源和专有模型上的评估表明，图像到代码的性能是领域依赖的：领先模型在常规图表类视觉上表现良好，但在空间场景、化学、文档和电路图等领域仍然较弱。人类验证显示，该评估协议比通用视觉评分标准或嵌入相似性基线更符合人类判断。此外，使用评估器过滤的模型输出作为训练数据可以提升能力，例如Qwen3.5-9B的得分从1.60提升到了1.86。

Insight: 论文的创新点在于提出了一个无需参考代码、多领域的图像到代码生成基准测试Vision2Code，并设计了一个结合特定领域评分细则和确定性护栏的VLM评估器。这为衡量、诊断和改进图像到代码生成提供了一个可复现的测试平台，其评估方法更贴近人类判断，且生成的代码可直接用于模型训练以提升性能。

Abstract: Image-to-code generation tests whether a vision-language model (VLM) can recover the structure of an image enough to express it as executable code. Existing benchmarks either focus on narrow visual domains, depend on paired executable reference code, or rely on generic rubrics that miss domain-specific reconstruction errors. We introduce Vision2Code, a reference-code-free benchmark and evaluation framework for multi-domain image-to-code generation. Vision2Code contains 2,169 test examples from 15 source datasets that span charts and plots, geometry, graphs, scientific imagery, documents, and 3D spatial scenes. Models generate executable programs, which we render and score against the source image using a VLM rater with dataset-specific rubrics and deterministic guardrails for severe semantic failures. We report render-success diagnostics that separate code execution failures from reconstruction quality. Human validation shows that this evaluation protocol aligns better with human judgments than either a generic visual rubric or embedding-similarity baselines. Across nine open-weight and proprietary models, we find that image-to-code performance is domain-dependent: leading models perform well on regular chart- and graph-like visuals but remain weak on spatial scenes, chemistry, documents, and circuit-style diagrams. Finally, we show that evaluator-filtered model outputs can serve as training data to improve image-to-code capability, with Qwen3.5-9B improving from 1.60 to 1.86 on the benchmark without paired source programs. Vision2Code provides a reproducible testbed for measuring, diagnosing, and improving image-to-code generation. Our code and data are publicly available at https://image2code.github.io/vision2code/.

[44] Quantifying Rodda and Graham Gait Classification from 3D Makerless Kinematics derived from a Single-view Video in a Heterogeneous Pediatric Clinical Cohort cs.CV | cs.AIPDF

Lauhitya Reddy, Seth Donahue, Jeremy Bauer, Susan Sienko, Anita Bagley

TL;DR: 该论文提出了一种基于单视角临床步态视频的无标记步态分析流程，用于量化Rodda和 Graham分类系统中的膝关节和踝关节z分数，旨在为脑瘫等儿科患者提供一种低成本、可扩展的客观步态评估方法。

Details

Motivation: 解决传统3D仪器步态分析昂贵且仅限于专业中心，而观察性评估者间一致性一般的问题，为资源有限的临床环境提供可扩展的客观步态评估方案。

Result: 在包含152名儿童（共1058个肢体样本）的异质性儿科临床队列中，模型预测膝关节z分数与3D-IGA相比达到R²=0.80±0.02和CCC=0.89±0.02，踝关节z分数达到R²=0.57±0.02和CCC=0.72±0.02；膝关节过度屈曲的二元筛查AUROC=0.88，正确识别83%的受影响儿童；应用Rodda和 Graham规则进行7分类的准确率为43±1%，宏观AUROC=0.78±0.01。

Insight: 创新点在于直接从单视角视频量化Rodda和 Graham步态分类的z分数，实现了基于视频的z分数估计、过度屈曲筛查和纵向轨迹跟踪，为监测疾病进展和治疗反应提供了定量基础，踝关节预测误差仍是主要瓶颈。

Abstract: Cerebral Palsy (CP) is a neurological disorder of movement and the most common cause of lifelong physical disability in childhood. Approximately 75% of children with CP are ambulatory, and accurate gait assessment is central to preserving walking function, which deteriorates by mid-adulthood in a quarter to half of adults with CP. The Rodda and Graham classification system quantifies sagittal-plane gait deviations using ankle and knee z-scores derived from 3D Instrumented Gait Analysis (3D-IGA), but 3D-IGA is expensive and limited to specialized centers, while observational assessment shows only moderate inter-rater agreement. We developed a markerless gait analysis pipeline that quantifies Rodda and Graham knee and ankle z-scores directly from single-view clinical gait videos. Across 1,058 bilateral limb samples from 529 trials of 152 children (88 male, 63 female; age 12.1 $\pm$ 4.0 years; 60 distinct primary diagnoses, cerebral palsy the most common at $n=54$), the sagittal-view model achieved $R^2 = 0.80 \pm 0.02$ and CCC $= 0.89 \pm 0.02$ for knee z-scores and $R^2 = 0.57 \pm 0.02$ and CCC $= 0.72 \pm 0.02$ for ankle z-scores against 3D-IGA. Binary screening for excess knee flexion achieves AUROC $= 0.88$, correctly identifying 83% of affected children, and applying Rodda and Graham rules yields $43 \pm 1$% 7-class accuracy with macro-AUROC $= 0.78 \pm 0.01$, ankle prediction error remaining the primary bottleneck. Beyond cross-sectional screening, continuous z-scores support longitudinal trajectory tracking across visits, providing a quantitative substrate for monitoring disease progression and treatment response unavailable from observational scales. These results demonstrate the feasibility of video-based z-score estimation, excess-flexion screening, and longitudinal trajectory tracking as a path toward scalable, objective gait assessment in low-resource clinical settings.

[45] PresentAgent-2: Towards Generalist Multimodal Presentation Agents cs.CV | cs.CLPDF

Wei Wu, Ziyang Xu, Zeyu Zhang, Yang Zhao, Hao Tang

TL;DR: 本文介绍了PresentAgent-2，一个用于从用户查询生成演示视频的智能体框架。它首先将开放式查询总结为聚焦主题，并从演示友好型来源进行深度研究以收集多模态资源（文本、图像、GIF、视频），然后构建幻灯片、生成特定模式的脚本，并将幻灯片、音频和动态媒体组合成完整的演示视频。该框架支持三种独立的演示模式：单人演示、讨论（多角色对话）和交互（回答观众问题）。

Details

Motivation: 演示生成正从静态幻灯片创建向端到端的演示视频生成发展，需要融合研究基础、多模态媒体和交互式交付。本文旨在解决从开放式查询直接生成具有研究支撑、多模态媒体和交互能力的演示视频这一更广泛的问题。

Result: 为评估其能力，作者构建了一个涵盖单人演示、讨论和交互场景的多模态演示基准，并制定了针对内容质量、媒体相关性、动态媒体使用、对话自然度和交互基础等任务特定评估标准。论文未在摘要中提供具体的定量比较结果或SOTA声明。

Insight: 主要创新点在于提出了一个统一的智能体框架，将演示生成从依赖文档的幻灯片创建扩展到由查询驱动、基于研究的、融合多模态媒体、对话和交互的演示视频生成。其框架设计支持三种不同的演示模式，增强了通用性和实用性。

Abstract: Presentation generation is moving beyond static slide creation toward end-to-end presentation video generation with research grounding, multimodal media, and interactive delivery. We introduce PresentAgent-2, an agentic framework for generating presentation videos from user queries. Given an open-ended user query and a selected presentation mode, PresentAgent-2 first summarizes the query into a focused topic and performs deep research over presentation-friendly sources to collect multimodal resources, including relevant text, images, GIFs, and videos. It then constructs presentation slides, generates mode-specific scripts, and composes slides, audio, and dynamic media into a complete presentation video. PresentAgent-2 supports three independent presentation modes within a unified framework: Single Presentation, which generates a single-speaker narrated presentation video; Discussion, which creates a multi-speaker presentation with structured speaker roles, such as for asking guiding questions, explaining concepts, clarifying details, and summarizing key points; and Interaction, which independently supports answering audience questions grounded in the generated slides, scripts, retrieved evidence, and presentation context. To evaluate these capabilities, we build a multimodal presentation benchmark covering single presentation, discussion, and interaction scenarios, with task-specific evaluation criteria for content quality, media relevance, dynamic media use, dialogue naturalness, and interaction grounding. Overall, PresentAgent-2 extends presentation generation from document-dependent slide creation to query-driven, research-grounded presentation video generation with multimodal media, dialogue, and interaction. Code: https://github.com/AIGeeksGroup/PresentAgent-2. Website: https://aigeeksgroup.github.io/PresentAgent-2.

[46] VidSplat: Gaussian Splatting Reconstruction with Geometry-Guided Video Diffusion Priors cs.CVPDF

Jimin Tang, Wenyuan Zhang, Junsheng Zhou, Zian Huang, Kanle Shi

TL;DR: VidSplat是一个无需训练的生成式重建框架，旨在解决稀疏视角下高斯泼溅（Gaussian Splatting）重建效果不佳的问题。它利用视频扩散先验迭代合成新视角以补偿缺失的输入覆盖，从而从稀疏输入中恢复完整的3D场景。

Details

Motivation: 高斯泼溅在多视角表面重建中取得显著进展，但在输入视角极少时性能会显著下降。现有方法通过增强多视角一致性来生成合理表面，但难以推断输入覆盖范围之外未被观察、遮挡或弱约束的区域。

Result: 在广泛使用的基准测试上的大量实验表明，VidSplat在稀疏视角场景重建中表现出优越性能，能够鲁棒地处理稀疏输入甚至单张图像。

Insight: 主要创新点包括：1) 一种无需训练、分阶段的去噪策略，利用渲染的RGB和掩码图像自适应地引导去噪方向朝向底层几何结构，以实现3D一致的生成；2) 一种迭代机制，通过采样相机轨迹、探索未观测区域、合成新视角以及通过置信度加权细化进行补充训练，从而增强重建效果。

Abstract: Gaussian Splatting has achieved remarkable progress in multi-view surface reconstruction, yet it exhibits notable degradation when only few views are available. Although recent efforts alleviate this issue by enhancing multi-view consistency to produce plausible surfaces, they struggle to infer unseen, occluded, or weakly constrained regions beyond the input coverage. To address this limitation, we present VidSplat, a training-free generative reconstruction framework that leverages powerful video diffusion priors to iteratively synthesize novel views that compensate for missing input coverage, and thereby recover complete 3D scenes from sparse inputs. Specifically, we tackle two key challenges that enable the effective integration of generation and reconstruction. First, for 3D consistent generation, we elaborate a training-free, stage-wise denoising strategy that adaptively guides the denoising direction toward the underlying geometry using the rendered RGB and mask images. Second, to enhance the reconstruction, we develop an iterative mechanism that samples camera trajectories, explores unobserved regions, synthesizes novel views, and supplements training through confidence weighted refinement. VidSplat performs robustly to sparse input and even a single image. Extensive experiments on widely used benchmarks demonstrate our superior performance in sparse-view scene reconstruction.

[47] ZeroIDIR: Zero-Reference Illumination Degradation Image Restoration with Perturbed Consistency Diffusion Models cs.CVPDF

Hai Jiang, Zhen Liu, Yinjie Lei, Songchen Han, Bing Zeng

TL;DR: 本文提出了一种名为ZeroIDIR的零参考扩散框架，用于光照退化图像恢复。该方法将恢复过程解耦为自适应光照校正和基于扩散的重建，仅使用低质量退化图像进行训练。具体包括自适应伽马校正模块和扰动一致性扩散模型，以校正光照并重建细节。

Details

Motivation: 解决光照退化图像恢复问题，特别是在无参考图像监督的情况下，实现高质量恢复并提高对不同场景的泛化能力。

Result: 在公开基准测试中，该方法优于最先进的无监督方法，并与有监督方法相当，且对各种场景具有更好的泛化性。

Insight: 创新点包括将恢复过程解耦为光照校正和扩散重建，引入直方图引导的光照校正损失和扰动扩散一致性损失，以在无监督下提升恢复保真度和稳定性。

Abstract: In this paper, we propose a zero-reference diffusion-based framework, named ZeroIDIR, for illumination degradation image restoration, which decouples the restoration process into adaptive illumination correction and diffusion-based reconstruction while being trained solely on low-quality degraded images. Specifically, we design an adaptive gamma correction module that performs spatially varying exposure correction to generate illumination-corrected only representations to mitigate exposure bias and serve as reliable inputs for subsequent diffusion processes, where a histogram-guided illumination correction loss is introduced to regularize the corrected illumination distribution toward that of natural scenes. Subsequently, the illumination-corrected image is treated as an intermediate noisy state for the proposed perturbed consistency diffusion model to reconstruct details and suppress noise. Moreover, a perturbed diffusion consistency loss is proposed to constrain the forward diffusion trajectory of the final restored image to remain consistent with the perturbed state, thus improving restoration fidelity and stability in the absence of supervision. Extensive experiments on publicly available benchmarks show that the proposed method outperforms state-of-the-art unsupervised competitors and is comparable to supervised methods while being more generalizable to various scenes. Code is available at https://github.com/JianghaiSCU/ZeroIDIR.

[48] Instruct-ICL: Instruction-Guided In-Context Learning for Post-Disaster Damage Assessment cs.CV | cs.LGPDF

Armin Zarbaft, Ehsan Karimi, Nhut Le, Maryam Rahnemoonfar

TL;DR: 本文提出了一种名为Instruct-ICL的方法，旨在通过指令引导的上下文学习来提升预训练多模态大语言模型（MLLMs）在灾后视觉问答（VQA）任务中的可靠性。该方法利用一个MLLM生成任务特定的指令，作为第二个MLLM的思维链（CoT）指导，并结合不同程度的上下文学习来生成答案。

Details

Motivation: 解决在时间紧迫的灾后评估场景中，训练专用模型耗时耗力，而现有预训练MLLMs虽具备多模态推理能力，但其回答对提示词敏感、可靠性受限的问题。

Result: 在FloodNet数据集上的评估表明，与零样本基线相比，集成指令驱动的CoT推理方法能持续提升答案准确性。

Insight: 创新点在于将指令生成与思维链引导相结合，通过一个模型为另一个模型提供结构化推理策略，从而增强MLLMs在现实世界灾难评估中的稳定性和性能。这为提升预训练模型在特定领域任务中的可靠性提供了一种可借鉴的提示工程范式。

Abstract: Rapid and accurate situational awareness is essential for effective response during natural disasters, where delays in analysis can significantly hinder decision-making. Training task-specific models for post-disaster assessment is often time-consuming and computationally expensive, making such approaches impractical in time-critical scenarios. Consequently, pretrained multimodal large language models (MLLMs) have emerged as a promising alternative for post-disaster visual question answering (VQA), a task that aims to answer structured questions about visual scenes by jointly reasoning over images and text. While these models demonstrate strong multimodal reasoning capabilities, their responses can be sensitive to prompt formulation, which can limit their reliability in real-world disaster assessment scenarios. In this paper, we investigate whether structured reasoning strategies can improve the reliability of pretrained MLLMs for post-disaster VQA. Specifically, we explore multiple prompting paradigms in which one MLLM is used to generate task-specific instructions that serve as Chain-of-Thought (CoT) guidance for a second MLLM. These instructions are incorporated during answer generation with varying degrees of in-context learning (ICL), enabling the model to leverage both explicit reasoning guidance and contextual examples. We conduct our evaluation on the FloodNet dataset and compare these approaches against a zero-shot baseline. Our results demonstrate that integrating instruction-driven CoT reasoning consistently improves answer accuracy.

[49] Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts cs.CVPDF

Eunho Lee, Youngbae Hwang, Rei Kawakami

TL;DR: 本文提出了一种基于多模态大语言模型（MLLM）引导的图像修复框架，通过MLLM引导的融合块和混合频率专家模块，利用多模态嵌入作为指导来增强对退化感知的表征，从而在一个统一模型中处理多种未知的图像退化问题。

Details

Motivation: 现有的一体化图像修复方法通常将退化视为离散类别，难以建模复合退化中存在的连续关系结构，因此需要一种能利用连续语义关系指导修复过程的新方法。

Result: 在多个基准测试上的广泛实验表明，该方法在多种修复设置下均表现出色，并在具有挑战性的CDD11数据集上达到了新的最先进水平（SOTA），比先前方法性能提升高达1.35 dB。

Insight: 创新点在于利用MLLM的多模态嵌入作为连续语义指导来增强低级视觉修复，并设计了MLLM引导的混合频率专家模块和路由器，通过关系对齐损失确保路由模式与退化输入在嵌入空间的关系保持一致，从而更好地处理复合退化。

Abstract: All-in-one image restoration seeks to recover clean images from inputs affected by diverse and unknown degradations using a unified framework. Recent methods have shown strong performance by identifying degradation characteristics to guide the restoration process. However, many of them treat degradations as discrete categories, which limits their ability to model the continuous relational structure that arises in composite degradations. To address this issue, we propose a multimodal large language model (MLLM)-guided image restoration framework that exploits multimodal embeddings as guidance for low-level restoration. Specifically, MLLM-derived features are injected into an encoder-decoder architecture through an MLLM-guided fusion block (MGFB) to enhance degradation-aware representations. In addition, we incorporate a mixture-of-frequency-experts (MoFE) module that adaptively combines frequency experts using MLLM-guided contextual cues. To further improve expert routing, we design an MLLM-guided router with a relational alignment loss that encourages routing patterns consistent with the embedding-space relationships of degraded inputs. Extensive experiments on multiple benchmarks show that the proposed method achieves strong performance across diverse restoration settings and establishes a new state of the art on the challenging CDD11 dataset, outperforming previous methods by up to 1.35 dB.

[50] SpatialForge: Bootstrapping 3D-Aware Spatial Reasoning from Open-World 2D Images cs.CV | cs.AIPDF

Zishan Liu, Ruoxi Zang, Yanglin Zhang, Wei Liu, Yin Zhang

TL;DR: 本文提出SpatialForge，一种可扩展的数据合成流程，将开放世界2D图像转化为空间推理监督信号，以解决大视觉语言模型在空间推理（如深度排序和坐标定位）方面的不足。该方法将空间推理分解为感知和关系，构建涵盖深度、布局和视角相关推理的结构化监督，并通过自动验证确保数据质量。基于此流程构建了包含1000万个空间问答对的大规模数据集SpatialForge-10M，实验表明，在该数据集上训练能显著提升标准视觉语言模型在多个空间推理基准上的性能。

Details

Motivation: 大视觉语言模型在语义理解方面表现出色，但在空间推理（如基本几何任务）上存在困难；现有方法依赖场景中心数据集（如多视图扫描或室内视频），但受限于场景数量少、规模和多样性不足，远小于网络规模的2D图像集合。

Result: 在多个空间推理基准上的广泛实验表明，使用SpatialForge-10M数据集训练能显著提升标准视觉语言模型的空间推理能力，证明了利用2D数据扩展进行3D感知空间推理的有效性。

Insight: 创新点包括：将空间推理分解为感知和关系两个层面，构建结构化监督信号（深度、布局、视角相关推理），并通过自动验证确保合成数据质量；从客观角度看，该方法利用丰富的2D图像资源生成大规模、多样化的空间推理监督，避免了依赖有限3D场景数据的瓶颈，为提升模型空间能力提供了可扩展的解决方案。

Abstract: Recent advancements in Large Vision-Language Models (VLMs) have demonstrated exceptional semantic understanding, yet these models consistently struggle with spatial reasoning, often failing at fundamental geometric tasks such as depth ordering and precise coordinate grounding. Recent efforts introduce spatial supervision from scene-centric datasets (e.g., multi-view scans or indoor video), but are constrained by the limited number of underlying scenes. As a result, the scale and diversity of such data remain significantly smaller than those of web-scale 2D image collections. To address this limitation, we propose SpatialForge, a scalable data synthesis pipeline that transforms in-the-wild 2D images into spatial reasoning supervision. Our approach decomposes spatial reasoning into perception and relation, and constructs structured supervision signals covering depth, layout, and viewpoint-dependent reasoning, with automatic verification to ensure data quality. Based on this pipeline, we build SpatialForge-10M, a large-scale dataset containing 10 million spatial QA pairs. Extensive experiments across multiple spatial reasoning benchmarks demonstrate that training on SpatialForge-10M significantly improves the spatial reasoning ability of standard VLMs, highlighting the effectiveness of scaling 2D data for 3D-aware spatial reasoning.

[51] LDDR: Linear-DPP-Based Dynamic-Resolution Frame Sampling for Video MLLMs cs.CVPDF

Jingfeng Chen, Jiawen Qian, Wendi Deng, Yinuo Guo, Jiaqi Yu

TL;DR: 本文提出了一种名为LDDR（基于线性DPP的动态分辨率）的视频帧采样框架，用于多模态大语言模型中的视频理解。该框架无需训练、即插即用，并在预算约束下，通过查询感知的Determinantal Point Process（DPP）在任务条件特征空间中选择帧，同时引入组DPP重要性度量来动态分配分辨率，为信息丰富且非冗余的帧分配更多token，而对信息较少的帧进行下采样或剪枝。

Details

Motivation: 解决在有限的视觉token预算下，从长且冗余的视频中选择信息丰富帧的挑战。现有方法（如均匀采样、逐点相关性评分、分块选择或代理探索）要么忽略了全局依赖，要么引入了大量开销。

Result: 在涵盖短、中、长视频的四个视频基准测试中，LDDR始终优于次优基线方法，在预算受限设置下提升2.5个百分点，在高预算场景下提升1.6个百分点。这些改进在多个开源和闭源的MLLM骨干模型上均得到一致验证。

Insight: 创新点在于将查询感知的DPP选择与动态分辨率分配相结合，通过组DPP重要性度量指导帧保留和token分配，实现了全局依赖建模与高效运行（相比标准DPP基线提速3倍）。这是一种无需训练、预算感知的通用视频帧采样方案。

Abstract: Video understanding in multimodal large language models requires selecting informative frames from long, redundant videos under limited visual-token budgets. Existing methods often rely on uniform sampling, point-wise relevance scoring, chunk-wise selection, or agentic exploration, which either miss global dependencies or introduce substantial overhead. We propose LDDR (Linear DPP-Based Dynamic Resolution), a training-free, plug-and-play, and budget-aware video frame sampling framework. LDDR performs query-aware Determinantal Point Process (DPP) frame selection in a task-conditioned feature space, achieving a 3x runtime speedup over standard DPP baselines. It further introduces a Group DPP importance metric to guide frame retention and dynamic resolution allocation, assigning more tokens to informative, non-redundant frames while downscaling or pruning less useful ones. Across four video benchmarks spanning short-, medium-, and long-range videos, LDDR consistently outperforms the next-best baselines, achieving gains of 2.5 points under budget-constrained settings and 1.6 points in high-budget scenarios. These improvements are consistently observed across multiple MLLM backbones, including both open- and closed-source models. Qualitative analysis confirms that relevant frames are selected and allocated a higher budget, facilitating improved video understanding.

[52] PoseBridge: Bridging the Skeletonization Gap for Zero-Shot Skeleton-Based Action Recognition cs.CVPDF

Sanghyeon Lee, Jinwoo Kim, Jong Taek Lee

TL;DR: PoseBridge是一个用于零样本骨架动作识别（ZSSAR）的框架，它通过桥接人体姿态估计（HPE）的中间表示与骨架-文本对齐，来解决上游语义丢失问题，从而提升对未见动作的分类性能。

Details

Motivation: 现有ZSSAR方法通常将骨架序列与文本对齐，但骨架作为HPE的压缩输出，可能丢失了人-物交互和姿态相关的视觉线索，导致上游语义丢失，限制了识别效果。

Result: 在NTU-RGB+D 60/120、PKU-MMD和Kinetics-200/400数据集上，PoseBridge提升了ZSSAR性能；特别是在Kinetics-200/400 PURLS基准测试中，它在所有八个分割上比最强基线提升了13.3-17.4个百分点，显示出最清晰的分离效果。

Insight: 创新点在于提出HPE感知的ZSSAR框架，从生成骨架的同一HPE过程中提取姿态锚定的语义线索，并通过骨架条件桥接和语义原型适应进行传递，避免了额外添加RGB分支或物体检测器，有效弥补了骨架化过程中的语义损失。

Abstract: Zero-shot skeleton-based action recognition (ZSSAR) is typically treated as a skeleton-text alignment problem: encode joint-coordinate sequences, align them with language, and classify unseen actions. We argue that this alignment is often too late. Skeletons are not complete action observations, but compressed outputs of human pose estimation (HPE); by the time alignment begins, human-object interactions and pose-relative visual cues may no longer be explicit. We call this upstream semantic loss. To address it, we propose PoseBridge, an HPE-aware ZSSAR framework that bridges intermediate HPE representations to skeleton-text alignment. Rather than adding an RGB action branch or object detector, PoseBridge extracts pose-anchored semantic cues from the same HPE process that produces skeletons, then transfers them through skeleton-conditioned bridging and semantic prototype adaptation. Across NTU-RGB+D 60/120, PKU-MMD, and Kinetics-200/400, PoseBridge improves ZSSAR performance under the evaluated protocols. On the Kinetics-200/400 PURLS benchmark, which contains in-the-wild videos with diverse scenes and action contexts, PoseBridge shows the clearest separation, improving the strongest compared baseline by 13.3-17.4 points across all eight splits. Our code will be publicly released.

[53] LiBrA-Net: Lie-Algebraic Bilateral Affine Fields for Real-Time 4K Video Dehazing cs.CVPDF

Yongcong Wang, Chengchao Shen, Guangwei Gao, Wei Wang, Pengwen Dai

TL;DR: 本文提出了LiBrA-Net，一种用于实时4K视频去雾的高效方法，并发布了首个带深度、透射率和光流标注的配对4K视频去雾基准数据集UHV-4K。该方法将时空仿射场分解为空间-颜色和时序双边子网格，在$mathfrak{gl}(3)$李代数下融合系数，通过Cayley参数化映射到可逆GL(3)变换，并利用轻量级输入引导分支恢复高频细节。

Details

Motivation: 当前超高清晰度（UHD）视频去雾领域缺乏评估基准，且现有方法在处理连续UHD序列时无法在消费级GPU上实时运行。本文旨在同时解决这两个问题。

Result: 在UHV-4K、REVIDE和HazeWorld基准测试中，LiBrA-Net在视频去雾方法中达到了新的最先进水平（SOTA），在单GPU上以25 FPS实时处理原生4K视频，参数量仅为6.12 M。

Insight: 核心创新点在于将去雾问题建模为受低频深度场控制的逐像素仿射变换，并通过双边网格进行紧凑编码，使预测成本与输出分辨率解耦。此外，在$mathfrak{gl}(3)$李代数下进行群论正则化的系数融合以及Cayley参数化的使用，是方法高效且数学严谨的关键。发布的UHV-4K基准填补了领域空白。

Abstract: Currently, there is a gap in the field of ultra-high-definition (UHD) video dehazing due to the lack of a benchmark for evaluation. Furthermore, existing video dehazing methods cannot run on consumer-grade GPUs when processing continuous UHD sequences of 3–5 frames at a time. In this paper, we address both issues with a new benchmark and an efficient method. Our key observation is that atmospheric dehazing reduces to a per-pixel affine transform governed by the low-frequency depth field, which can be compactly encoded in bilateral grids whose prediction cost is decoupled from the output resolution. Building on this, we propose LiBrA-Net, which factorizes the spatiotemporal affine field into a spatial–color and a temporal bilateral sub-grid predicted at a fixed low resolution, fuses their coefficients in the $\mathfrak{gl}(3)$ Lie algebra under group-theoretic regularization, maps the result to invertible GL(3) transforms via a Cayley parameterization, and restores high-frequency detail through a lightweight input-guided branch. We further release UHV-4K, the first paired 4K video dehazing benchmark with depth, transmission, and optical-flow annotations on every frame. Across UHV-4K, REVIDE, and HazeWorld, LiBrA-Net sets a new state of the art among compared video dehazing methods while running native 4K at 25 FPS on a single GPU with only 6.12 M parameters. Code and data are available at https://anonymous.4open.science/r/LiBrA-Net-42B8.

[54] GeoR-Bench: Evaluating Geoscience Visual Reasoning cs.CVPDF

Yushuo Zheng, Zicheng Zhang, Huiyu Duan, Chunyi Li, Zijian Chen

TL;DR: 本文提出了GeoR-Bench，一个用于评估地球科学视觉推理能力的基准测试，它通过基于推理的视觉编辑任务来测试模型对地球观测图像、地图和图表等科学数据的理解与推理能力。该基准包含440个样本，涵盖6个地球科学类别和24种任务类型。对21个闭源和开源多模态模型的评估表明，地球科学推理仍是当前AI系统的关键瓶颈，最高性能模型仅达到42.7%的严格准确率。

Details

Motivation: 现有地球科学AI研究多针对特定任务，缺乏能评估开放世界真实地球科学问题的基准，因此不清楚当前AI系统离实现真正的地球科学智能还有多远。

Result: 在GeoR-Bench上评估了21个模型，最佳模型总体严格准确率为42.7%，最佳开源模型仅为10.3%。输出结果的视觉一致性和图像质量常高于其科学准确性。

Insight: 创新点在于构建了一个专注于地球科学视觉推理的综合性基准，通过视觉编辑任务评估模型对底层地球科学过程的理解。客观来看，该工作揭示了当前多模态模型在生成表面合理结果与掌握科学原理之间的显著差距，为领域评估提供了新方向。

Abstract: Geoscience intelligence is expected to understand, reason about, and predict earth system changes to support human decision-making in critical domains such as disaster response, climate adaptation and environmental protection. Although current research has shown promising progress on specific geoscience tasks, such as remote sensing interpretation, geographic question-answering, existing benchmarks remain largely task-specific which failing to capture the open-ended real world geoscience problems. As a result, it remains unclear how far current AI systems are from achieving genuine geoscience intelligence. To address this gap, we present \textbf{GeoR-Bench}, a \underline{Bench}mark for evaluating \underline{Geo}science visual \underline{R}easoning through reasoning informed visual editing tasks. GeoR-Bench contains 440 curated samples spanning 6 geoscience categories and 24 task types, covering earth observation imagery and structured scientific representations such as maps and diagrams. We evaluate outputs along three dimensions, including reasoning, consistency, and quality. Benchmark results of 21 closed- and open-source multimodal models reveal that geoscience reasoning remains a critical bottleneck. The highest-performing model achieves 42.7% overall strict accuracy, while the best open-source models only get 10.3%. Notably, the visual consistency and image quality of the outputs frequently surpass their scientific accuracy. Ultimately, these findings indicate that current models generate superficially plausible results but fail to capture underlying earth science processes.

[55] When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs cs.CV | cs.AIPDF

Fanpu Cao, Xin Zou, Xuming Hu, Hui Xiong

TL;DR: 本文提出了一种名为LaSCD（Laplacian-Spectral Contrastive Decoding）的无训练解码策略，通过分析多模态大语言模型（MLLM）视觉注意力结构的高频成分（由层间拉普拉斯能量测量）来检测和缓解视觉幻觉问题。该方法能识别幻觉偏好出现的层以及正确答案短暂恢复的层，并据此重映射下一个token的对数概率，从而在减少幻觉的同时保持模型的通用能力。

Details

Motivation: 多模态大语言模型在视觉推理和基于图像的问答中易产生视觉幻觉，即生成与图像内容矛盾或提及不存在对象的回答。核心挑战在于幻觉并非总是由简单的视觉注意力缺失引起，即使模型分配了显著的注意力到图像token，内部仍可能漂移至错误答案。

Result: 在幻觉和通用多模态基准测试上的实验表明，LaSCD能持续减少幻觉，同时保持通用能力，突显了其作为一种可靠解码范式的潜力。

Insight: 创新点在于利用视觉注意力结构的高频成分（层间拉普拉斯能量）作为诊断工具，揭示了幻觉产生和正确答案恢复的特定层，并基于此设计了一种无需训练、闭式解的对数概率重映射解码策略，为缓解MLLM幻觉问题提供了新的视角和方法。

Abstract: Multimodal large language models (MLLMs) have become a key interface for visual reasoning and grounded question answering, yet they remain vulnerable to visual hallucinations, where generated responses contradict image content or mention nonexistent objects. A central challenge is that hallucination is not always caused by a simple lack of visual attention: the model may still assign substantial attention mass to image tokens while internally drifting toward an incorrect answer. In this paper, we show that the high-frequency structure of visual attention, measured by layer-wise Laplacian energy, reveals both the layer where hallucinated preferences emerge and the layer where the ground-truth answer transiently recovers. Building on this finding, we propose LaSCD (Laplacian-Spectral Contrastive Decoding), a training-free decoding strategy that selects informative layers via Laplacian energy and remaps next-token logits in closed form. Experiments on hallucination and general multimodal benchmarks show that LaSCD consistently reduces hallucination while preserving general capabilities, highlighting its potential as a faithful decoding paradigm. The code is available at https://github.com/macovaseas/LaSCD.

[56] TCP-SSM: Efficient Vision State Space Models with Token-Conditioned Poles cs.CV | cs.AIPDF

Sara Shoouri, Morteza Tavakoli Taba, Hun-Seok Kim

TL;DR: TCP-SSM是一种高效的结构化选择性状态空间模型框架，通过引入稳定的极点来显式控制循环动态，以替代注意力机制处理长距离视觉任务。该方法利用实极点和复共轭极点分别建模单调衰减和阻尼振荡响应，并通过有界半径和角度调制将共享基极点转换为令牌依赖的极点，从而在保持线性时间扫描复杂度的同时提升效率。

Details

Motivation: 现有高效SSM变体主要通过修改扫描路径、分辨率或遍历模式来降低计算成本，但循环动态通常隐式且难以控制，尤其在紧凑骨干网络中长扫描路径可能超出有效记忆范围，因此需要一种能显式控制状态依赖记忆行为的方法。

Result: 在图像分类、语义分割和目标检测任务中，TCP-SSM在Vision Mamba风格模型上将SSM计算复杂度降低了高达44%，同时保持或超越了基线准确率。

Insight: 创新点在于通过结构化极点设计（实极点与复共轭极点）显式建模循环动态，并引入令牌条件调制实现自适应记忆行为；从客观角度看，其将控制理论与SSM结合，通过极点稳定性保证和轻量级低秩输入路径，在提升可解释性的同时维持了计算效率。

Abstract: State Space Models (SSMs) have emerged as a compelling alternative to attention models for long-range vision tasks, offering input-dependent recurrence with linear complexity. However, most efficient SSM variants reduce computation cost by modifying scan routes, resolutions, or traversal patterns, while largely leaving the recurrent dynamics implicit. Consequently, the model’s state-dependent memory behavior is difficult to control, particularly in compact backbones where long scan paths can exceed the effective memory horizon. We propose Token-Conditioned Poles SSM (TCP-SSM), a structured selective SSM framework that improves efficiency while making recurrence dynamics explicit and interpretable through stable poles. TCP-SSM builds each scan operator with 1) real poles that model monotone or sign-alternating decay, and 2) complex-conjugate poles that capture damped oscillatory responses. Using bounded radius and angle modulation, TCP-SSM converts shared base poles into token-dependent poles, allowing each scan step to adapt its memory behavior to the current visual token while preserving pole stability. For practical scalability, we integrate grouped pole sharing with a lightweight low-rank input pathway, yielding an efficient scan operator that preserves linear-time scan complexity. Across image classification, semantic segmentation, and object detection, TCP-SSM reduces SSM computation complexity up to 44% in Vision Mamba-style models while maintaining or surpassing baseline accuracy.

[57] Dynamic Execution Commitment of Vision-Language-Action Models cs.CVPDF

Feng Chen, Xianghui Wang, Yuxuan Chen, Boying Li, Yefei He

TL;DR: 本文提出了一种名为A3的自适应动作接受机制，用于改进视觉-语言-动作（VLA）模型在动态执行中的动作块预测与提交策略。传统方法采用固定的执行范围来平衡成功率和推理效率，而A3通过将动态执行承诺重新定义为自推测前缀验证问题，利用群体采样计算动作的轨迹一致性分数，并基于共识排序的条件不变性和前缀封闭的顺序一致性来验证和选择可执行的最长动作前缀，从而自动确定执行范围，无需手动调整。

Details

Motivation: 解决VLA模型中动作分块预测方法因采用固定执行范围而忽略预测可靠性的状态依赖性，导致在动态或分布外环境中性能脆弱的问题。

Result: 在多种VLA模型和基准测试上的实验表明，A3无需手动调整执行范围，并在执行鲁棒性和推理吞吐量之间实现了更优的权衡。

Insight: 创新点在于将动态执行承诺重新定义为自推测前缀验证问题，并引入共识排序的条件不变性和前缀封闭的顺序一致性作为验证准则，从而自适应地确定可靠执行范围，提升了VLA模型在动态环境中的鲁棒性和效率。

Abstract: Vision-Language-Action (VLA) models predominantly adopt action chunking, i.e., predicting and committing to a short horizon of consecutive low-level actions in a single forward pass, to amortize the inference cost of large-scale backbones and reduce per-step latency. However, committing these multi-step predictions to real-world execution requires balancing success rate against inference efficiency, a decision typically governed by fixed execution horizons tuned per task. Such heuristics ignore the state-dependent nature of predictive reliability, leading to brittle performance in dynamic or out-of-distribution settings. In this paper, we introduce A3, an Adaptive Action Acceptance mechanism that reframes dynamic execution commitment as a self-speculative prefix verification problem. A3 first computes a trajectory-wise consensus score of actions via group sampling, then selects a representative draft and prioritizes downstream verification. Specifically, it enforces: (1) consensus-ordered conditional invariance, which validates low-consensus actions by judging whether they remain consistent when re-decoded conditioned on high-consensus actions; and (2) prefix-closed sequential consistency, which guarantees physical rollout integrity by accepting only the longest continuous sequence of verified actions starting from the beginning. Consequently, the execution horizon emerges as the longest verifiable prefix satisfying both internal model logic and sequential execution constraints. Experiments across diverse VLA models and benchmarks demonstrate that A3 eliminates the need for manual horizon tuning while achieving a superior trade-off between execution robustness and inference throughput.

[58] Logit-Attention Divergence: Mitigating Position Bias in Multi-Image Retrieval via Attention-Guided Calibration cs.CVPDF

Mingtao Xian, Yifeng Yang, Qinying Gu, Xinbing Wang, Nanyang Ye

TL;DR: 本文针对多模态大语言模型在多图像检索任务中存在的位置偏差问题，提出了一种基于注意力引导的校准方法。通过观察发现，模型输出logits存在偏差，而内部注意力图仍与视觉证据对齐，据此提出无需训练的推理时校正框架，显著提升了排列不变性和检索精度。

Details

Motivation: 解决多模态大语言模型在多图像检索中因输入顺序而非语义相关性主导预测的严重位置偏差问题。

Result: 在基于MS-COCO的基准测试中，该方法大幅提升了排列不变性，并达到SOTA性能，相比基线准确率提升超过40%。

Insight: 创新点在于揭示了Logit-Attention Divergence现象，并利用内部注意力信号进行实例级校正，实现无需训练、低开销的推理时去偏。

Abstract: Multimodal Large Language Models (MLLMs) have shown strong performance in multi-image cross-modal retrieval, yet suffer from severe position bias, where predictions are dominated by input order rather than semantic relevance. Through empirical analysis, we identify a phenomenon termed Logit-Attention Divergence, in which output logits are heavily biased while internal attention maps remain well-aligned with relevant visual evidence. This observation reveals a fundamental limitation of existing logit-level calibration methods such as PriDe. Based on this insight, we propose a training-free, attention-guided debiasing framework that leverages intrinsic attention signals for instance-level correction at inference time, requiring only a minimal calibration set with negligible computational overhead. Experiments on MS-COCO-based benchmarks show that our method substantially improves permutation invariance and achieves state-of-the-art performance, enhancing accuracy by over 40% compared to baselines. Code is available at https://github.com/brightXian/LAD.

[59] HorizonDrive: Self-Corrective Autoregressive World Model for Long-horizon Driving Simulation cs.CVPDF

Conglang Zhang, Yifan Zhan, Qingjie Wang, Zhanpeng Ouyang, Yu Li

TL;DR: HorizonDrive提出了一种自校正的自回归世界模型，用于长时域驾驶模拟。通过使教师模型具备自回归推演能力，提供无界时域监督，并结合学生模型进行高效实时部署，显著提升了驾驶模拟的长期稳定性和生成质量。

Details

Motivation: 现有自回归蒸馏方法在驾驶模拟中因快速自我运动和场景变化而表现不佳，且受限于教师模型的单次输出长度，无法提供长时域监督。本文旨在通过扩展教师模型的自回归推演能力，解决监督时域有限和模型漂移问题。

Result: 在nuScenes数据集上，HorizonDrive将FID降低52%，FVD降低37%，ARE和DTW分别降低21%和9%，优于最强的长时域流式基线，同时与单次通过的驾驶视频生成器保持竞争力。

Insight: 创新点包括：1) 计划推演恢复（SRR）训练基础模型从预测污染的历史中重建真实未来片段，确保教师模型在长自回归推演中稳定；2) 教师推演分布匹配蒸馏（TRD）使学生模型在有限内存下对齐教师的长时域监督，实现高效实时部署。

Abstract: Closed-loop driving simulation requires real-time interaction beyond short offline clips, pushing current driving world models toward autoregressive (AR) rollout. Existing AR distillation approaches typically rely on frame sinks or student-side degradation training. The former transfers poorly to driving due to fast ego-motion and rapid scene changes, while the latter remains bounded by the teacher’s single-pass output length and thus provides only a limited supervision horizon. A natural question is: can the teacher itself be extended via AR rollout to provide unbounded-horizon supervision at bounded memory cost? The key difficulty is that a standard teacher drifts under its own predictions, contaminating the supervision it provides. Our key insight is to make the teacher rollout-capable, ensuring reliable supervision from its own AR rollouts. This is instantiated as HorizonDrive, an anti-drifting training-and-distillation framework for AR driving simulation. First, scheduled rollout recovery (SRR) trains the base model to reconstruct ground-truth future clips from prediction-corrupted histories, yielding a teacher that remains stable across long AR rollouts. Second, the rollout-capable teacher is extended via AR rollout, providing long-horizon distribution-matching supervision under bounded memory, while a short-window student aligns to it with teacher rollout DMD (TRD) for efficient real-time deployment. HorizonDrive natively supports minute-scale AR rollout under bounded memory; on nuScenes, HorizonDrive reduces FID by 52% and FVD by 37%, and lowers ARE and DTW by 21% and 9% relative to the strongest long-horizon streaming baselines, while remaining competitive with single-pass driving video generators.

[60] Keep What Audio Cannot Say: Context-Preserving Token Pruning for Omni-LLMs cs.CV | cs.AIPDF

Chaeyoung Jung, Kyeongha Rho, Joon Son Chung

TL;DR: 本文提出了一种名为ContextGuard的推理时令牌剪枝框架，旨在解决Omnimodal大语言模型（Omni-LLMs）因处理大量多模态输入令牌而产生的高计算开销问题。该方法通过保留音频无法表达的视觉上下文信息，并去除跨模态冗余，实现了高效的令牌压缩。

Details

Motivation: 现有Omni-LLM剪枝方法通常根据当前查询的重要性或跨模态线索的对齐程度来选择令牌，这可能会丢弃对理解整体上下文或回答不同问题所必需的证据。本文旨在解决这一局限，将令牌减少问题重新定义为在保留广泛视听上下文的同时去除冗余。

Result: 在Qwen2.5-Omni和Video-SALMONN2+的3B和7B模型上，于六个视听基准测试中，ContextGuard在剪枝更多令牌的同时，性能优于先前的推理时剪枝方法。特别是在Qwen2.5-Omni 7B上，它在六个基准中的五个上达到了全令牌级别的性能，同时剪枝了55%的输入令牌。

Insight: 核心创新在于将令牌剪枝重新定义为保留音频无法表达的视觉上下文（如局部化视觉细节），而非仅依赖跨模态对齐或查询重要性。该方法无需下游LLM微调，仅使用独立训练的轻量级预测器，通过从音频预测粗略视觉语义来指导视频令牌剪枝，并合并时间上相似的视频令牌以实现进一步压缩。

Abstract: Omnimodal Large Language Models (Omni-LLMs) incur substantial computational overhead due to the large number of multimodal input tokens they process, making token reduction essential for real-world deployment. Existing Omni-LLM pruning methods typically reduce this cost by selecting tokens that are important for the current query or strongly aligned with cross-modal cues. However, such strategies can discard evidence that falls outside these criteria, even when needed for different questions or for understanding context beyond aligned audio-visual cues. To address this limitation, we reframe Omni-LLM token reduction as preserving broad audio-visual context while removing cross-modal redundancy. We propose ContextGuard, an inference-time token pruning framework built on this principle. ContextGuard predicts coarse visual semantics from audio and prunes video tokens whose coarse semantics are likely recoverable from audio, while retaining additional video tokens to preserve localized visual details that audio alone cannot specify. For further compression, our method merges temporally similar video tokens. The framework requires no downstream LLM fine-tuning and uses only an independently trained lightweight predictor. On Qwen2.5-Omni and Video-SALMONN2+ at 3B and 7B scales across six audio-visual benchmarks, ContextGuard outperforms prior inference-time pruning methods while pruning more tokens. Notably, on Qwen2.5-Omni 7B, ContextGuard achieves full-token-level performance on five of six benchmarks while pruning 55% of input tokens.

[61] Grounding by Remembering: Cross-Scene and In-Scene Memory for 3D Functional Affordances cs.CVPDF

Qirui Wang, Jingyi He, Yining Pan, Xulei Yang, Shijie Li

TL;DR: 本文提出AFFORDMEM框架，通过构建跨场景功能可供性记忆库和场景内空间记忆图，在无需模型微调或目标场景标注的情况下，实现3D场景中功能可供性的精准定位。该方法在SceneFun3D基准测试中显著超越了现有无训练方法。

Details

Motivation: 解决现有免训练视觉语言管道在3D功能可供性定位中的不足，即可操作区域通常较小、视觉模糊且在场景中重复出现，导致纯文本提示难以准确定位。

Result: 在SceneFun3D数据集上，AP50指标相比先前最先进的免训练方法在Split 0和Split 1上分别提升了3.23和3.7，达到新的SOTA水平。消融实验表明跨场景记忆提升了细粒度定位能力，场景内空间记忆则在空间限定查询上贡献更大。

Insight: 创新点在于引入双层记忆机制：跨场景可供性记忆库通过渲染覆盖图引导VLM关注细小子区域；场景内空间记忆图则利用3D空间关系结构化组织候选实例，使语言模型能解析复杂空间指代。该方法无需微调，仅依赖可复用的源场景记忆库，具有较好的泛化性和实用性。

Abstract: Functional affordance grounding requires more than recognizing an object: an agent must localize the specific region that supports an interaction, such as the handle to pull or the button to press. This is difficult for training-free vision-language pipelines because actionable regions are often small, visually ambiguous, and repeated across multiple same-category instances in a scene. We propose AFFORDMEM, a framework that grounds 3D functional affordances by remembering geometry at two levels. The first is cross-scene affordance memory: the agent maintains a category-level memory bank of RGB images with affordance regions rendered as overlays, and recalls the most informative examples at query time to guide a frozen VLM toward small operable subregions that text-only prompting consistently misses. The second is in-scene spatial memory: as the agent processes the scene, it organizes candidate instances and their 3D spatial relations into a structured scene graph, enabling the language model to resolve references over distant or currently unobserved candidates such as “the second handle from the top.” AFFORDMEM requires no model fine-tuning and no target-scene annotation, using a reusable memory bank built from source scenes. On SceneFun3D, our method improves AP50 over the prior training-free state of the art by 3.23 on Split 0 and 3.7 on Split 1. Ablation studies support complementary benefits: cross-scene affordance memory improves fine-grained localization, while in-scene spatial memory provides the larger gain on spatially qualified queries. The project homepage is available at the project page.

[62] Single-Shot HDR Recovery via a Video Diffusion Prior cs.CVPDF

Chinmay Talegaonkar, Jinshi He, Christopher McKenna, Nicholas Antipa

TL;DR: 本文提出了一种基于视频扩散先验的单次高动态范围（HDR）图像重建方法。该方法将单次HDR重建重新定义为条件视频生成任务，通过微调视频扩散模型，以低动态范围（LDR）图像为条件生成一个曝光序列（曝光包围），然后利用一个轻量级UNet预测的逐像素权重融合这些帧，最终得到HDR图像。

Details

Motivation: 解决现有单次HDR生成方法在保持输入图像保真度方面的不足，这些方法通常需要多个模型分别处理高光和阴影，或直接预测最终HDR图像而牺牲了可解释性。

Result: 在定量基准测试中，该方法在多个重建指标上超越了具有可比模型容量的最先进（SOTA）生成基线方法。在人工评估的成对比较中，72%的参与者更偏好该方法的结果。

Insight: 创新点在于将HDR重建重新构建为条件视频生成（生成曝光序列）与融合的框架，而非直接生成HDR图像，这使得过程更简单、可解释且有效。该框架具有通用性，可扩展至其他图像重建任务，如从单张散焦模糊图像恢复全焦点图像。

Abstract: Recent generative methods for single-shot high dynamic range (HDR) image reconstruction show promising results, but often struggle with preserving fidelity to the input image. They require separate models to handle highlights and shadows, or sacrifice interpretability by directly predicting the final HDR image. We address these limitations by re-casting single-shot HDR reconstruction as conditional video generation and fusing the generated frames into an HDR image. We finetune a video diffusion model to generate an exposure bracket, conditioned on a low dynamic range (LDR) input. We fuse this image bracket using per-pixel weights predicted by a light-weight UNet. This formulation is simple, interpretable, and effective. Rather than directly hallucinating an HDR image, it explicitly reconstructs the intermediate exposure stack and fuses it into the final output. Our method eliminates the need for separate models across exposure regimes and produces HDR reconstructions with high input fidelity. On quantitative benchmarks, we outperform state-of-the-art generative baselines with comparable model capacity on several reconstruction metrics. Human evaluators further prefer our results in 72% of pairwise comparisons against existing methods. Finally, we show that this input-conditioned sequence generation and fusion framework extends beyond HDR to other image reconstruction tasks, such as all-in-focus image recovery from a single defocus-blurred input.

[63] Unlocking UML Class Diagram Understanding in Vision Language Models cs.CV | cs.AIPDF

Artem Naboichenko, René Peinl

TL;DR: 本文针对视觉语言模型在理解UML类图方面的不足，提出了一个基于UML类图的视觉问答基准和一个包含1.6万个图像-问题-答案三元组的大规模训练数据集。研究表明，通过基于LoRA的微调方法，可以轻松超越在许多其他基准测试中表现优异的Qwen 3.5 27B模型。

Details

Motivation: 尽管视觉语言模型在各种用例中取得了巨大进展，但在回答关于图表（尤其是计算机科学领域的UML类图等特定类型图表）的问题时，其性能仍落后于处理照片的能力。现有研究多集中于条形图、折线图等，对UML类图等专业图表的关注较少。

Result: 论文构建的UML类图视觉问答基准具有挑战性但可管理。通过基于LoRA的微调方法，在提出的基准上轻松超越了近期在其他多个基准中表现优异的Qwen 3.5 27B模型，展示了方法的有效性。

Insight: 创新点在于首次针对UML类图这一特定且重要的计算机科学领域图表类型，构建了专门的视觉问答基准和训练数据集。从客观角度看，这为评估和提升VLMs在专业领域图表理解能力提供了新的方向，并且证明了即使是轻量级的微调（如LoRA）也能在特定任务上显著超越通用的大型VLMs。

Abstract: Although Vision Language Models (VLMs) have seen tremendous progress across all kinds of use cases, they still fall behind in answering questions regard-ing diagrams compared to photos. Although progress has been made in the area of bar charts, line charts and other diagrams like that there is still few research concerned with other types of diagrams, e.g. in the computer science domain. Our work presents a benchmark for visual question answering based on UML class diagrams which is both challenging and manageable. We further construct a large-scale training dataset with 16.000 image-question-answer triples and show that a LoRA-based finetune easily outperforms Qwen 3.5 27B, which is a recent and well-performing VLM in many other benchmarks.

[64] Hide to See: Reasoning-prefix Masking for Visual-anchored Thinking in VLM Distillation cs.CV | cs.AI | cs.CLPDF

Seonghoon Yu, Dongjun Nam, Byung-Kwan Lee, Jeany Son

TL;DR: 本文提出了一种新颖的视觉语言模型（VLM）蒸馏框架，旨在将大型‘思维-回答’VLM（如Qwen3-VL-Thinking）的推理能力高效地迁移到紧凑模型中。核心方法是引入‘推理前缀掩码’，通过掩码学生模型推理过程中的关键文本线索，迫使其更多地依赖视觉证据进行锚定思考，从而提升其利用视觉信息进行推理的能力。

Details

Motivation: 现有‘思维-回答’VLM通过中间推理步骤提升了性能，但计算成本高昂，难以实际部署。因此，需要一种蒸馏方法，将这种能力迁移到紧凑模型中，并特别提升学生在整个推理过程中利用视觉证据的能力。

Result: 实验结果表明，该方法在多模态推理基准测试中，超越了近期的开源VLM、VLM蒸馏和自蒸馏方法。进一步分析证实了学生模型在思维过程中对视觉信息的利用得到了增强。

Insight: 创新点在于提出了‘推理前缀掩码’机制，包括针对每个下一个词预测的‘词元级显著推理前缀掩码’，以及根据师生分布差异动态调整掩码规模的‘自步调掩码预算调度’。这迫使模型在文本线索被部分屏蔽的情况下，转向视觉信息进行推理，从而强化了视觉锚定思维，这是一种新颖的蒸馏监督信号设计。

Abstract: Recent think-answer approaches in VLMs, such as Qwen3-VL-Thinking, boost reasoning performance by leveraging intermediate thinking steps before the final answer, but their high computational cost limits real-world deployment. To distill such capabilities into compact think-answer VLMs, a primary objective is to improve the student’s ability to utilize visual evidence throughout its reasoning trace. To this end, we introduce a novel think-answer distillation framework that encourages the student to anchor its thinking on visual information by masking the student’s salient reasoning prefixes. To compensate for such masked textual cues, the student is encouraged to rely more on visual evidence as an alternative source of information during distillation. Our masking strategies include: 1) token-wise salient reasoning-prefix masking, which masks high-influence reasoning prefixes selectively for each next-token prediction, and 2) self-paced masking budget scheduling, which gradually increases the masking scale according to distillation difficulty, {measured by discrepancy between teacher–student distributions. In the distillation phase, the student is guided by our salient reasoning-prefix mask, which blocks both future tokens and salient reasoning cues, in place of the standard causal mask used for auto-regressive language modeling. Experimental results show that our approach outperforms recent open-source VLMs, VLM distillation, and self-distillation methods on multimodal reasoning benchmarks, while further analyses confirm enhanced visual utilization along the student thinking process.

[65] Weather-Robust Cross-View Geo-Localization via Prototype-Based Semantic Part Discovery cs.CV | cs.AI | cs.ROPDF

Chi-Nguyen Tran, Dao Sy Duy Minh, Huynh Trung Kiet, Nguyen Lam Phu Quy, Phu-Hoa Pham

TL;DR: 本文提出了一种名为SkyPart的轻量级可交换头部模块，用于基于补丁的视觉变换器（ViTs），旨在解决跨视角地理定位（CVGL）中的三个主要限制：全局描述符设计未分离布局与纹理、嵌入中保留高度相关的尺度变化以及多目标训练中损失权重的手动调优问题。SkyPart通过可学习原型竞争分配、高度条件线性调制、图注意力读出和Kendall不确定性加权多目标损失，实现了在SUES-200、University-1652和DenseUAV数据集上的SOTA性能，并在恶劣天气条件下表现出更强的鲁棒性。

Details

Motivation: 解决跨视角地理定位（CVGL）中存在的三个关键问题：全局描述符设计无法分离布局与纹理、嵌入中保留高度相关的尺度变化以及多目标训练中损失权重的手动调优，以提高无人机在GNSS信号不可用时的自主导航能力。

Result: 在SUES-200、University-1652和DenseUAV数据集上，采用单次推理、无重排序、无测试时增强协议，SkyPart以26.95M参数和22.14 GFLOPs成为性能最佳方法中最小的模型，并达到SOTA水平；在WeatherPrompt恶劣天气基准测试中，其相对于最强基线的优势进一步扩大。

Insight: 创新点包括：通过可学习原型实现显式部分分组以分离布局与纹理；引入高度条件线性调制使推理时嵌入与高度无关；使用图注意力读出增强表示；以及Kendall不确定性加权多目标损失自动优化损失权重，提升模型鲁棒性和效率。

Abstract: Cross-view geo-localization (CVGL), which matches an oblique drone view to a geo-referenced satellite tile, has emerged as a key alternative for autonomous drone navigation when GNSS signals are jammed, spoofed, or unavailable. Despite strong recent progress, three limitations persist: (1) global-descriptor designs compress the patch grid into a single vector without separating layout from texture across the view gap; (2) altitude-related scale variation is retained in the learned embedding rather than marginalized; and (3) multi-objective training relies on hand-tuned scalars over losses on incompatible gradient scales. We propose SkyPart, a lightweight swappable head for patch-based vision transformers (ViTs) that institutes explicit part grouping over the patch grid. SkyPart has four theory-grounded components: (i) learnable prototypes competing for patch tokens via single-pass cosine assignment; (ii) altitude-conditioned linear modulation applied only during training, making the retrieval embedding altitude-free at inference; (iii) a graph-attention readout over active prototypes; and (iv) a Kendall uncertainty-weighted multi-objective loss whose stationary points are Pareto-stationary. At 26.95M parameters and 22.14 GFLOPs, SkyPart is the smallest among top-performing methods and sets a new state of the art on SUES-200, University-1652, and DenseUAV under a single-pass, no-re-ranking, no-TTA protocol. Its advantage over the strongest baseline widens under the ten-condition WeatherPrompt corruption benchmark.

[66] Reviving In-domain Fine-tuning Methods for Source-Free Cross-domain Few-shot Learning cs.CV | cs.AIPDF

Yaze Zhao, Yicong Liu, Yixiong Zou, Yuhua Li, Ruixuan Li

TL;DR: 本文研究了跨域少样本学习（CDFSL）中视觉语言模型（如CLIP）的微调方法。通过建立多个微调基线，发现基于适配器的方法（如LoRA）在CDFSL中优于基于提示的方法（如MaPLe），这与域内场景相反。分析表明，LoRA的优势源于其能纠正视觉CLS令牌的注意力崩溃，通过关注与文本相关的视觉区域来增强模态对齐和类别分离。基于此，作者提出了一个即插即用的注意力纠正框架——Semantic Probe，用于提升适配器和提示方法的性能。在四个CDFSL基准测试上的广泛实验验证了该框架的有效性，实现了最先进的性能。

Details

Motivation: 解决跨域少样本学习（CDFSL）中，视觉语言模型（如CLIP）的微调方法未被充分探索的问题，特别是如何使在域内场景中有效的微调方法（如基于提示的方法）在CDFSL中重新具有竞争力。

Result: 在四个CDFSL基准测试上进行了广泛实验，验证了提出的Semantic Probe框架的有效性，实现了最先进的（SOTA）性能，并提升了基于适配器和基于提示的两种微调范式的表现。

Insight: 创新点包括：发现LoRA在CDFSL中优于提示方法的原因是其能纠正视觉CLS令牌的注意力崩溃；提出使用文本EOS令牌以更好地关注视觉样本；指出CLIP的标准对比损失对模态对齐的约束较弱；设计了一个即插即用的注意力纠正框架Semantic Probe，可同时提升适配器和提示方法，增强模态对齐和类别分离。

Abstract: Cross-Domain Few-Shot Learning (CDFSL) aims to adapt large-scale pretrained models to specialized target domains with limited samples, yet the few-shot fine-tuning of vision-language models like CLIP remains underexplored. By establishing multiple fine-tuning baselines of CLIP for CDFSL, we find adapter-based methods (e.g., LoRA) consistently outperform prompt-based ones (e.g., MaPLe), contrary to in-domain scenarios. To make those effective in-domain methods competitive again in CDFSL, we analyze this phenomenon and discover LoRA’s superiority stems from rectifying the collapsed attention of visual CLS token, enhancing modality alignment and class separation by focusing on text-related visual regions. Further, we find textual EOS token exhibit much better attention to visual samples, and CLIP’s standard contrastive loss weakly constrains modality alignment. Based on these insights, we propose Semantic Probe, a plug-and-play attention rectification framework for both adapter- and prompt-based methods. Extensive experiments on four CDFSL benchmarks validate our rationale, achieving state-of-the-art performance and benefiting both fine-tuning paradigms. Codes will be released.

[67] ShapeCodeBench: A Renewable Benchmark for Perception-to-Program Reconstruction of Synthetic Shape Scenes cs.CVPDF

Shivam Kumar

TL;DR: 本文提出了ShapeCodeBench，这是一个用于感知到程序重建的可再生合成基准测试：给定渲染的栅格图像，模型必须输出一个可执行的绘图程序，由确定性评估器重新渲染并与目标图像比较。v1版本在512x512黑白画布上使用四种图元，每个实例由种子RNG生成，因此可以创建新的保留集以减少实例污染。作者发布了包含150个样本的固定评估集，涵盖简单、中等和困难三个级别，并通过精确匹配、像素精度、前景IoU、解析成功率和执行成功率进行评分。

Details

Motivation: 为了解决现有基准测试中存在的实例污染问题，并推动从感知图像到可执行程序的重建任务的研究，作者设计了一个可再生的合成基准测试，旨在评估模型在理解视觉场景并生成对应绘图程序方面的能力。

Result: 在ShapeCodeBench v1的评估中，经典计算机视觉启发式方法在简单场景上表现尚可但在组件重叠时失效；最强的多模态配置（Claude Opus 4.7和GPT-5.5）能保留大部分前景结构，但由于小的参数误差仍无法达到精确匹配。整体最佳精确匹配率仍然很低，表明该基准测试远未饱和。

Insight: 论文的创新点在于提出了一个基于种子随机数生成的可再生基准测试框架，能够动态生成新的测试集以避免数据污染，并引入了从图像到可执行程序的重建这一具有挑战性的任务。从客观角度看，其设计的确定性评估流程和综合评分指标为未来研究提供了可复现和可扩展的评估标准。

Abstract: We introduce ShapeCodeBench, a synthetic benchmark for perception-to-program reconstruction: given a rendered raster image, a model must emit an executable drawing program that a deterministic evaluator re-renders and compares with the target. The v1 DSL has four primitives on a 512 x 512 black-on-white canvas, but every instance is generated from a seeded RNG, so fresh held-out sets can be created to reduce exact-instance contamination. We release a frozen eval_v1 split with 150 samples across easy, medium, and hard tiers, scored by exact match, pixel accuracy, foreground IoU, parse success, and execution success. We evaluate an empty-program floor, a classical computer-vision heuristic, Claude Opus 4.7 at high and max effort, and GPT-5.5 at medium and extra_high reasoning effort. The heuristic is competitive on easy scenes but collapses when overlaps fuse components; the strongest multimodal configuration preserves much of the foreground structure but still misses exact match because of small parameter errors. Best overall exact match remains low, so ShapeCodeBench is far from saturated. The benchmark code, frozen dataset, run artifacts, and paper sources are released to support independent replication and extension.

[68] DORA: Dynamic Online Reinforcement Agent for Token Merging in Vision Transformers cs.CVPDF

Kaixuan He, Song Chen, Yi Kang

TL;DR: 本文提出了DORA（动态在线强化学习代理），一种基于强化学习的在线推理框架，用于动态合并视觉Transformer（ViT）中的冗余token，以降低计算开销。该方法将token合并过程建模为马尔可夫决策过程，通过轻量级RL代理根据当前特征状态和层上下文自适应地决定合并策略，在保持精度的同时显著提升计算效率。

Details

Motivation: 现有ViT的token缩减方法多依赖固定启发式指标、预定义比率或静态离线掩码，缺乏在推理时根据输入动态捕捉冗余的自适应性，导致计算效率与特征保真度之间的权衡不足。

Result: 在多个ViT规模（Tiny到Large）上的评估表明，DORA在精度-效率帕累托前沿上优于现有基线。在精度下降可忽略（≤0.05%）的约束下，DORA实现了最高12.66%的token合并率，相比最高效基线有569.7%的相对提升；在ImageNet-1K上，在同等精度约束下，DORA相比SOTA方法实现了最高76%的相对计算节省提升；在OOD基准（如ImageNet-A和ImageNet-C）上，DORA获得了超过430%的相对效率优势。

Insight: 创新点包括：首次将强化学习应用于ViT在线推理中的动态token合并，通过序列MDP建模实现输入自适应性；设计了结合非线性蒸馏惩罚的密集奖励函数以平衡效率与保真度；采用非对称Actor-Critic架构，离线训练时使用高容量Critic确保稳定性，在线推理时仅需轻量Actor头以降低开销。该方法为动态模型压缩提供了可借鉴的强化学习驱动框架。

Abstract: Vision Transformers (ViTs) incur significant computational overhead due to the quadratic complexity of self-attention relative to the token sequence length. While existing token reduction methods mitigate this issue, they predominantly rely on fixed heuristic metrics, predefined ratios, or static offline masks, which lack the adaptability to capture input-dependent redundancy during inference. In this paper, we propose DORA (Dynamic Online Reinforcement Agent), the first reinforcement learning (RL)-driven online inference framework for dynamic token merging in ViTs. We formulate the merging process as a sequential Markov Decision Process (MDP), where a lightweight RL agent determines the merging strategy for each Transformer block based on the current feature state and layer-specific context. To balance computational efficiency and feature fidelity, the agent is optimized via a dense reward function incorporating a non-linear distillation-based penalty. We implement an asymmetric Actor-Critic architecture that utilizes a high-capacity Critic for stable offline training while retaining a minimal Actor head for low-computation online inference. Evaluations across multiple ViT scales (Tiny to Large) demonstrate that DORA improves the accuracy-efficiency Pareto front compared to current baselines. Under strict negligible accuracy-drop constraints (<= 0.05%), DORA achieves up to a 12.66% token merging rate, and delivers up to a 569.7% relative improvement over the most efficient baseline. On ImageNet-1K, under aligned accuracy constraints, DORA achieves up to a 76% relative improvement in computational savings compared to state-of-the-art methods. Furthermore, on out-of-distribution (OOD) benchmarks such as ImageNet-A and ImageNet-C, DORA attains a relative efficiency advantage of over 430%.

[69] CAST: Collapse-Aware multi-Scale Topology Fusion for Multimodal Coreset Selection cs.CVPDF

Boran Zhao, Hetian Liu, Zhenxian Hu, Yuqing Yuan, Yu Yan

TL;DR: 本文提出了一种名为CAST的新型多模态核心集选择框架，旨在解决大规模多模态模型训练中数据选择的问题。该方法通过构建图像和文本模态的拓扑结构，进行局部塌陷感知的细化和跨模态融合，生成统一拓扑，并在扩散小波域引入多尺度分布匹配准则，同时采用局部软关系覆盖机制来减少密集区域的冗余选择。

Details

Motivation: 现有核心集选择方法存在两个关键局限：一是单模态主导的采样方法忽略了多模态数据中固有的细粒度跨模态信息不平衡，导致另一模态的语义损失；二是基于粗粒度样本评分的采样方法容易偏向评分模型，难以保证核心集与原始数据集之间的分布等价性。现有方法未能同时考虑全局语义结构、局部细粒度细节以及密集区域的冗余感知覆盖。

Result: 在Flickr30K和MS-COCO数据集上的大量实验表明，CAST优于现有的数据集选择基线方法，并在跨架构泛化和能源效率方面展现出相对于最先进多模态合成方法的显著优势。

Insight: 创新点包括：通过局部塌陷感知的细化和跨模态融合构建统一拓扑来整合多模态信息；在扩散小波域引入多尺度分布匹配准则以在多个尺度上逼近原始数据集分布；以及提出局部软关系覆盖机制，将纯几何覆盖扩展为关系感知的间接覆盖，以惩罚密集聚类中的冗余选择。这些方法共同解决了现有方法在全局结构、局部细节和冗余覆盖方面的不足。

Abstract: The training of large multimodal models fundamentally relies on massive image-text datasets, which inevitably incur prohibitive computational overhead. Dataset selection offers a promising paradigm by identifying a highly informative coreset. However, existing approaches suffer from two critical limitations: (i) single-modality-dominated sampling methods, which ignore the fine-grained cross-modal information imbalance inherent in multimodal datasets and thus lead to semantic loss in the other modality; and (ii) coarse-grained sample-scoring-based sampling methods, where the selected coreset tends to be biased toward the scoring model, making it difficult to guarantee distributional equivalence between the coreset and the original dataset. Meanwhile, existing distribution matching and discrete sampling strategies often fail to jointly account for global semantic structure, local fine-grained details, and redundancy-aware coverage in dense regions. To this end, we propose CAST, a Collapse-Aware multi-Scale Topology fusion framework for multimodal coreset selection. We first construct image- and text-modality topologies, and derive a unified topology via local-collapse-aware refinement and cross-modal fusion. We then introduce a multi-scale distribution matching criterion in the diffusion wavelet domain, encouraging the coreset to approximate the original dataset at multiple scales. Finally, we introduce a local soft relational coverage mechanism that extends pure geometric coverage to relation-aware indirect coverage, penalizing redundant selections in dense clusters. Extensive experiments on Flickr30K and MS-COCO show that CAST outperforms existing dataset selection baselines, showcasing great superiority in cross-architecture generalization and energy efficiency over state-of-the-art multimodal synthesis methods.

[70] EPIC: Efficient Predicate-Guided Inference-Time Control for Compositional Text-to-Image Generation cs.CV | cs.LGPDF

Sunung Mun, Sunghyun Cho, Jungseul Ok

TL;DR: EPIC是一种无需训练、在推理时进行优化的框架，用于提升组合式文本到图像生成的准确性。它将优化过程视为基于谓词的搜索：将原始提示解析为包含对象变量和类型化谓词的固定视觉程序，并利用从图像中提取的视觉证据验证生成的图像是否满足所有谓词条件。若不满足，则根据失败的谓词进行局部编辑或重新采样。

Details

Motivation: 解决现有文本到图像生成器在处理涉及多个对象、数量、属性和关系的组合式提示时表现不佳的问题。

Result: 在GenEval2基准测试中，EPIC将基础生成器的提示级准确率从34.16%提升至71.46%。在相同的生成器/编辑器设置和最大图像模型执行预算下，EPIC比先前最强的优化基线高出19.23个百分点，同时将每个提示的图像模型执行成本降低31%、MLLM调用降低72%、MLLM令牌降低81%。

Insight: 创新点在于将组合式提示解析为固定的视觉程序，并通过谓词引导的搜索进行推理时优化，无需额外训练，实现了高效的成本控制和准确性提升。从客观角度看，其模块化验证和针对性编辑策略为组合式生成任务提供了可扩展的解决方案。

Abstract: Recent text-to-image (T2I) generators can synthesize realistic images, but still struggle with compositional prompts involving multiple objects, counts, attributes, and relations. We introduce EPIC (Efficient Predicate-Guided Inference-Time Control), a training-free inference-time refinement framework for compositional T2I generation. EPIC casts refinement as predicate-guided search: it parses the original prompt once into a fixed visual program of object variables and typed predicates, covering checkable conditions such as object presence, counts, attributes, and relations. Each generated or edited image is verified against this program using visual evidence extracted from that image. An image is judged to satisfy the prompt only when all predicates are satisfied; otherwise, failed predicates decide the next step, routing local failures to targeted editing and global failures to resampling while the fixed visual program remains unchanged. On GenEval2, EPIC improves prompt-level accuracy from 34.16% for single-pass generation with the base generator to 71.46%. Under the same generator/editor setting and maximum image-model execution budget, EPIC outperforms the strongest prior refinement baseline by 19.23 points while reducing realized cost by 31% in image-model executions, 72% in MLLM calls, and 81% in MLLM tokens per prompt.

[71] CaC: Advancing Video Reward Models via Hierarchical Spatiotemporal Concentrating cs.CV | cs.AIPDF

Jiyuan Wang, Huan Ouyang, Jiuzhou Lin, Chunyu Lin, Dewen Fan

TL;DR: 本文提出了一种名为CaC的粗到细异常奖励模型，基于视觉语言模型，通过全局时间扫描定位异常时间窗口，然后在局部区间内进行细粒度空间定位，最后通过结构化时空思维链推理得出稳健判断。

Details

Motivation: 动机是解决视频异常检测中难以稳定关注细微异常的问题，通过构建首个大规模生成视频异常数据集，并设计渐进式训练范式，以提升模型的时空推理能力。

Result: 在细粒度异常基准测试中，CaC实现了25.7%的准确率提升；作为奖励信号使用时，能将生成视频的异常减少11.7%，同时提高整体视频质量，达到SOTA水平。

Insight: 创新点包括构建带逐帧边界框标注、时间异常窗口和细粒度归因标签的大规模生成视频异常数据集，以及引入时空IoU奖励来监督中间定位过程，从而引导模型进行更接地气和可解释的时空推理。

Abstract: In this paper, we propose Concentrate and Concentrate (CaC), a coarse-to-fine anomaly reward model based on Vision-Language Models. During inference, it first conducts a global temporal scan to anchor anomalous time windows, then performs fine-grained spatial grounding within the localized interval, and finally derives robust judgments via structured spatiotemporal Chain-of-Thought reasoning. To equip the model with these capabilities, we construct the first large-scale generated video anomaly dataset with per-frame bounding-box annotations, temporal anomaly windows, and fine-grained attribution labels. Building on this dataset, we design a three-stage progressive training paradigm. The model initially learns spatial and temporal anchoring through single- and multi-frame supervised fine-tuning, and then is optimized by a reinforcement learning strategy based on two-turn Group Relative Policy Optimization (GRPO). Beyond conventional accuracy rewards, we introduce Temporal and Spatial IoU rewards to supervise the intermediate localization process, effectively guiding the model toward more grounded and interpretable spatiotemporal reasoning. Extensive experiments demonstrate that CaC can stably concentrate on subtle anomalies, achieving a 25.7% accuracy improvement on fine-grained anomaly benchmarks and, when used as a reward signal, CaC reduces generated-video anomalies by 11.7% while improving overall video quality.

[72] WorldComp2D: Spatio-semantic Representations of Object Identity and Location from Local Views cs.CV | cs.LGPDF

SeongMin Jin, Doo Seok Jeong

TL;DR: 论文提出WorldComp2D，一种新颖的轻量级表示学习框架，通过多尺度局部感受野显式地根据物体身份和空间邻近性构建潜在空间几何结构，以捕获语义和空间信息。该框架包含一个邻近性依赖编码器和一个定位器，并以人脸关键点定位作为概念验证，展示了其在减少参数和计算量的同时保持实时性能。

Details

Motivation: 现有方法通常依赖隐式潜在结构结合密集特征图或任务特定头，限制了计算效率和灵活性，因此需要一种能显式构建空间-语义潜在表示的高效框架。

Result: 在人脸关键点定位任务上，与最先进的轻量级模型相比，WorldComp2D将参数数量和FLOPs分别减少了最多4.0倍和2.2倍，同时在CPU上保持实时性能。

Insight: 创新点在于显式地根据物体身份和空间邻近性结构化潜在空间，使用多尺度局部感受野来提升计算效率和灵活性，为空间-语义推理提供了高效通用基础。

Abstract: Learning latent representations that capture both semantic and spatial information is central to efficient spatio-semantic reasoning. However, many existing approaches rely on implicit latent structures combined with dense feature maps or task-specific heads, limiting computational efficiency and flexibility. We propose WorldComp2D, a novel lightweight representation learning framework that explicitly structures latent space geometry according to object identity and spatial proximity using multiscale local receptive fields. This framework consists of (i) a proximity-dependent encoder that maps a given observation into a spatio-semantic latent space and (ii) a localizer that infers the coordinates of objects in the input from the resulting spatio-semantic representation. Using facial landmark localization as a proof-of-concept, we show that, compared to SoTA lightweight models, WorldComp2D reduces the numbers of parameters and FLOPs by up to 4.0X and 2.2X, respectively, while maintaining real-time performance on CPU. These results demonstrate that explicitly structured latent spaces provide an efficient and general foundation for spatio-semantic reasoning. This framework is open-sourced at https://github.com/JinSeongmin/WorldComp2D.

[73] BronchoLumen: Analysis of recent YOLO-based architectures for real-time bronchial orifice detection in video bronchoscopy cs.CVPDF

Yongchao Li, Marian Himstedt

TL;DR: 本文介绍了BronchoLumen，一个基于YOLO的实时支气管孔检测系统，用于辅助视频支气管镜导航。研究评估了YOLOv8和YOLOv12两种架构在公开数据集上的性能，重点关注其在域内和跨域场景下的鲁棒性。

Details

Motivation: 支气管镜检查中，在复杂的呼吸道分支中导航具有挑战性。论文旨在探索能否利用最先进的目标检测技术和有限的公开图像数据，在不同图像域中鲁棒地检测支气管孔，以辅助导航和计算机辅助诊断系统。

Result: 在公开数据集上，YOLOv8在域内测试集上mAP@0.5为0.91，跨域为0.68；YOLOv12分别为0.84和0.68。YOLOv12在定位精度（mAP@0.5:0.9）上略优（0.48 vs 0.45，0.26 vs 0.25），但精度稍低。系统在多数场景下表现出鲁棒性。

Insight: 创新点在于提出了一个开源的、基于YOLO的实时支气管孔检测解决方案，并在多个图像域上验证了其高精度和效率。客观分析表明，较新的YOLOv12通过集成注意力模块提升了空间推理和定位精度，但在特定精度指标上可能与传统YOLOv8各有优劣，为支气管镜导航研究提供了可复现的基准模型。

Abstract: Bronchoscopy is routinely conducted in pulmonary clinics and intensive care units, but navigating the complex branching of the respiratory tract remains challenging. This paper introduces BronchoLumen, a real-time YOLO-based system for detecting bronchial orifices in video bronchoscopy, aiming to assist navigation and CAD systems. The paper investigates if bronchial orifices can be robustly detected across image domains using state-of-the-art object detection and a limited set of public image data. The study includes the description and comparison of YOLOv8, a widely adopted architecture, and YOLOv12, a more recent architecture integrating attention-based modules to improve spatial reasoning. Both models are trained and tested solely on publicly available datasets comprising different image domains. A comparison of both models is conducted based on the common metrics mAP@0.5 and mAP@0.5:0.9 with the latter emphasizing localization accuracy. For YOLOv8 we obtained a mAP@0.5 of 0.91 on an in-domain and 0.68 on a cross-domain test set. YOLOv12 achieved 0.84 and 0.68 respectively with slightly better localization accuracy with mAP@0.5:0.9 of 0.48 and 0.26 compared to YOLOv8 with 0.45 and 0.25. Challenges like motion blur and low contrast occasionally entailed uncertainties but the system demonstrated overall robustness in most scenarios. BronchoLumen is an open-weight, YOLO-based solution for bronchial orifice detection offering high accuracy and efficiency across multiple image domains. While the more recent YOLOv12 achieves better localization accuracy, we observed a slightly worse precision. The models have been made publicly available to foster further research in bronchoscopy navigation.

Jiyuan Liu, Jia Lin, Xiaofei Zhou, Runmin Cong, Deyang Liu

TL;DR: 该论文提出了M⁴-SAM模型，通过为Segment Anything Model 2 (SAM2) 引入模态感知的MoE-LoRA微调、门控多级特征融合和无提示记忆初始化三大模块，解决了将SAM2基础模型应用于RGB-D视频显著目标检测任务时面临的三个关键挑战，并在三个公开数据集上实现了最先进的性能。

Details

Motivation: 将强大的通用分割基础模型SAM2直接应用于RGB-D视频显著目标检测任务时，面临三个挑战：线性LoRA的空间建模能力有限、未能充分利用SAM的多尺度特征、以及模型初始化依赖于显式提示。

Result: 在三个公开的RGB-D VSOD数据集（DVS、DAVIS-3D、DUT-RGBD-VS）上进行了广泛实验，M⁴-SAM在所有评估指标上均达到了最先进的性能。

Insight: 主要创新点包括：1) 模态感知的MoE-LoRA，利用卷积专家编码局部空间先验，并通过模态分发器实现高效多模态微调；2) 门控多级特征融合，自适应地聚合多尺度特征以平衡空间细节与语义上下文；3) 伪引导初始化，将粗掩膜作为伪先验来引导记忆库的启动，实现了无需人工提示的零样本VSOD。

Abstract: The Segment Anything Model 2 (SAM2) has emerged as a foundation model for universal segmentation. Owing to its generalizable visual representations, SAM2 has been successfully applied to various downstream tasks. However, extending SAM2 to the RGB-D video salient object detection (RGB-D VSOD) task encounters three challenges including limited spatial modeling of linear LoRA, insufficient employment of SAM’s multi-scale features, and dependence of initialization on explicit prompts. To address the issues, we present Multi-Modal Mixture-of-Experts with Memory-Augmented SAM (M$^4$-SAM), which equips SAM2 with modality-related PEFT, hierarchical feature fusion, and prompt-free memory initialization. Firstly, we inject Modality-Aware MoE-LORA, which employs convolutional experts to encode local spatial priors and introduces a modality dispatcher for efficient multi-modal fine-tuning, into SAM2’s encoder. Secondly, we deploy Gated Multi-Level Feature Fusion, which hierarchically aggregates multi-scale encoder features with an adaptive gating mechanism, to balance spatial details and semantic context. Finally, to conduct zero-shot VSOD without manual prompts, we utilize a Pseudo-Guided Initialization, where a coarse mask is regarded as a pseudo prior and used to bootstrap the memory bank. Extensive experiments demonstrate that M$^4$-SAM achieves the state-of-the-art performance across all evaluation metrics on three public RGB-D VSOD datasets.

[75] Revisiting Shadow Detection from a Vision-Language Perspective cs.CVPDF

Yonghui Wang, Wengang Zhou, Hao Feng, Houqiang Li

TL;DR: 本文提出了一种从视觉-语言角度重新审视阴影检测的方法，通过引入语言作为显式语义参考来消除视觉上模糊的暗区域歧义。

Details

Motivation: 传统阴影检测仅依赖像素级视觉监督，在视觉模糊情况下（如暗影与固有暗表面）不可靠，需要超越视觉线索的语义参考以实现鲁棒预测。

Result: 在多个阴影检测基准测试（包括专门的困难案例评估）上，该方法展现出强大的整体性能，并在视觉模糊条件下提高了鲁棒性。

Insight: 创新点在于将语言作为显式语义参考，通过场景级阴影比例回归目标对齐图像与文本嵌入，并引入全局到局部耦合机制将全局指导传递到密集推理；采用冻结DINOv3编码器，仅训练轻量级模块，实现了参数高效（<1%可训练参数）的设计。

Abstract: Shadow detection is commonly formulated as a vision-driven dense prediction problem, where models rely primarily on pixel-wise visual supervision to distinguish shadows from non-shadow regions. However, this formulation can become unreliable in visually ambiguous cases, where similar dark regions may correspond either to cast shadows or to intrinsically dark surfaces, making visual evidence alone insufficient for establishing a stable decision rule. In this work, we revisit shadow detection from a vision–language perspective and argue that robust prediction benefits from an explicit semantic reference beyond visual cues alone. We propose SVL, a Shadow Vision–Language framework that uses language as an explicit semantic reference to disambiguate shadows from visually similar dark regions. SVL aligns the global image representation with shadow-related text embeddings through a scene-level shadow ratio regression objective, thereby providing image-level guidance on the overall extent of shadows. To transfer this global guidance to dense inference, SVL introduces a global-to-local coupling mechanism that enforces consistency between image-level guidance and patch-level predictions. In parallel, SVL applies local patch-level constraints with text embeddings to improve fine-grained discrimination under challenging appearance conditions. Built on a frozen DINOv3 image encoder, the framework learns only lightweight projection and decoding modules, yielding a parameter-efficient design with less than $1%$ trainable parameters. Extensive experiments on multiple shadow detection benchmarks, including dedicated hard-case evaluations, suggest strong overall performance and improved robustness under visually ambiguous conditions.

Antoni Valls, Jordi Sanchez-Riera

TL;DR: 本文提出了一种基于视觉问答的事件地图框架，利用视觉语言模型为低视力人群提供城市风险感知导航。该方法通过三层层次化查询结构实现细粒度场景理解，无需任务特定重训练，并将模型响应聚合为加权风险评分系统，生成可导航的风险感知事件地图用于路径规划。

Details

Motivation: 解决全球数亿视觉障碍者在城市环境中安全独立导航的挑战，现有可穿戴辅助设备依赖缺乏灵活性和泛化性的任务特定视觉流程，需要更通用的解决方案。

Result: 在涵盖六大洲20个城市的多样化数据集上评估了四种VQA架构，发现生成式多模态大语言模型显著优于基于分类的方法，其中Qwen-VL在精确率和召回率上取得了最佳平衡。

Insight: 创新点在于利用视觉语言模型的通用场景描述和危险识别能力，通过层次化查询结构实现细粒度理解，并构建加权风险评分系统生成事件地图；客观分析认为将MLLMs作为辅助导航系统的灵活可泛化基础是可行的新方向。

Abstract: Visual impairment affects hundreds of millions of people worldwide, severely limiting their ability to navigate urban environments safely and independently. While wearable assistive devices offer a promising platform for real-time hazard detection, existing approaches rely on task-specific vision pipelines that lack flexibility and generalizability. In this work, we propose an event map framework based on visual question answering that leverages Vision-Language Models (VLMs) for pedestrian scene description and hazard identification across diverse real-world environments, using a three-level hierarchical query structure to enable fine-grained scene understanding without task-specific retraining. Model responses are aggregated into a weighted risk scoring system that maps street segments into four discrete safety categories, producing navigable risk-aware event maps for route planning. To support evaluation and future research, we introduce a geographically diverse dataset spanning 20 cities across six continents, comprising over 800 annotated images and 18,000 answered questions. We benchmark four VQA architectures -ViLT, LLaVA, InstructBLIP, and Qwen-VL- and find that generative Multimodal Large Language Models (MLLMs) substantially outperform classification-based approaches, with Qwen-VL achieving the best overall balance of precision and recall. These results demonstrate the viability of MLLMs as a flexible and generalizable foundation for assistive navigation systems for visually impaired people.

[77] SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions cs.CVPDF

Markus Essl, Marta Moscati, Mubashir Noman, Muhammad Zaigham Zaheer, Usman Naseem

TL;DR: 本文提出SB-BEVFusion，一种针对自动驾驶3D目标检测的框架无关多模态融合模块，旨在提升系统在相机或激光雷达数据缺失、损坏或噪声情况下的鲁棒性。该模块被实例化于BEVFusion框架中，并在MultiCorrupt数据集上验证了其在多种传感器退化场景下的有效性。

Details

Motivation: 现有基于统一BEV表示的多模态融合方法在理想条件下表现优异，但当相机或LiDAR数据缺失、损坏或存在噪声时，性能会显著下降，因此需要增强系统对传感器故障和损坏的鲁棒性。

Result: 在MultiCorrupt数据集上的定量实验表明，该模块在模态缺失和损坏场景下取得了显著的性能提升，在广泛的传感器退化场景中大幅优于现有统一表示方法，并在极端天气条件和传感器故障导致的模态损坏场景中达到了最先进的性能水平。

Insight: 创新点在于提出了一种框架无关的融合模块，能够处理单一模态缺失或损坏的情况，增强了多模态融合系统的容错能力和实际部署的鲁棒性，为解决传感器不可靠问题提供了通用解决方案。

Abstract: Multimodal sensor fusion has demonstrated remarkable performance improvements over unimodal approaches in 3D object detection for autonomous vehicles. Typically, existing methods transform multimodal data from independent sensors, such as camera and LiDAR, into a unified bird’s-eye view (BEV) representation for fusion. Although effective in ideal conditions, this strategy suffers from substantial performance deterioration when camera or LiDAR data are missing, corrupted, or noisy. To address this vulnerability, we develop a framework-agnostic fusion module for camera and LiDAR data that allows for handling cases when one of the two modalities is missing or corrupted. To demonstrate the effectiveness of our module, we instantiate it in BEVFusion [1], a well-established framework to combine camera and LiDAR data for 3D object detection. By means of quantitative experiments on the MultiCorrupt dataset, we demonstrate that our module achieves favorable performance improvements under scenarios of missing and corrupted modalities, substantially outperforming existing unified representation approaches across a wide range of sensor deterioration scenarios and reaching state-of-the-art performance in scenarios of corrupted modality due to extreme weather conditions and sensor failure.

[78] OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Models cs.CV | cs.AIPDF

Minseok Kang, Minhyeok Lee, Jungho Lee, Minjung Kim, Donghyeong Kim

TL;DR: 本文提出了OTT-Vid，一种基于最优传输（Optimal Transport）的时域令牌压缩框架，用于视频大语言模型（Video-LLMs）。该方法通过两阶段处理：首先在每帧内进行空间剪枝以识别代表性内容，然后利用非均匀令牌质量和局部感知成本的最优传输来估计相邻帧间的可压缩性，并据此动态分配压缩预算。实验表明，在仅保留10%令牌的情况下，该方法在多个视频问答和时序定位基准上保持了高性能。

Details

Motivation: 随着视频大语言模型处理更长、更复杂的视频，跨帧积累的大量视觉令牌导致推理成本急剧增加。现有的无训练时域压缩方法主要依赖跨帧令牌相似性或分割启发式方法，忽略了令牌在帧内的语义角色，且未能根据每对帧的可压缩性自适应调整压缩强度。

Result: 在六个涵盖视频问答（VQA）和时序定位（VTG）的基准测试中，OTT-Vid在仅保留10%令牌的情况下，保持了95.8%的VQA性能和73.9%的VTG性能，持续优于现有的最先进（SOTA）无训练压缩方法。

Insight: 创新点在于将最优传输理论引入时域令牌压缩，通过非均匀令牌质量保护语义重要令牌，并使用局部感知成本同时考虑特征和空间差异。该方法能根据传输难度动态分配压缩预算，实现了更精细的自适应压缩，平衡了令牌重要性与匹配成本。

Abstract: As Video Large Language Models (Video-LLMs) scale to longer and more complex videos, their inference cost grows rapidly due to the large volume of visual tokens accumulated across frames. Training-free token compression has emerged as a practical solution to this bottleneck. However, existing temporal compression methods rely primarily on cross-frame token similarity or segmentation heuristics, overlooking each token’s semantic role within its frame and failing to adapt compression strength to the compressibility of each frame pair. In this work, we propose OTT-Vid, a transport-derived allocation framework for temporal token compression. Our approach consists of two stages: spatial pruning identifies representative content within each frame, and optimal transport (OT) is then solved between neighboring frames to estimate temporal compressibility. We formulate this OT with non-uniform token mass, which protects semantically important tokens from aggressive compression, and a locality-aware cost that captures both feature and spatial disparities. The resulting transport plan jointly balances token importance and matching cost, while its total cost defines the transport difficulty of each frame pair, which we use to allocate compression budgets dynamically. Experiments on six benchmarks spanning video question answering and temporal grounding show that OTT-Vid preserves 95.8% of VQA and 73.9% of VTG performance while retaining only 10% of tokens, consistently outperforming existing state-of-the-art training-free compression methods.

[79] Mitigating Action-Relation Hallucinations in LVLMs via Relation-aware Visual Enhancement cs.CVPDF

Zhenxin Qin, Qiang Li, Qingzhuo Wang, Ruiyang Qin, Zhihua Wei

TL;DR: 该论文针对大型视觉语言模型（LVLMs）中存在的动作关系幻觉问题，提出了一种关系感知的视觉增强框架。通过定义动作关系敏感度（ARS）分数来定位对动作关系变化最敏感的注意力头，从而识别包含关键视觉线索的动作相关图像区域，并采用关系感知视觉增强（RVE）方法增强模型对这些区域的关注，以缓解幻觉现象。

Details

Motivation: 现有研究主要关注缓解对象幻觉，但往往忽视了更复杂的关系幻觉，特别是涉及对象间交互的动作关系幻觉。论文通过实证观察发现，LVLMs产生动作关系幻觉的主要原因是模型对视觉信息的关注不足。

Result: 大量实验表明，与现有基线方法相比，该方法在缓解动作关系幻觉方面取得了更优的性能，且额外推理成本可忽略不计。此外，该方法还能有效泛化到空间关系幻觉和对象幻觉。

Insight: 论文的创新点在于首次系统性地针对动作关系幻觉问题，提出了通过定位动作相关图像区域并增强模型对其关注的方法。从客观角度看，其定义的ARS分数和RVE方法为理解和缓解LVLMs中复杂的多模态幻觉问题提供了新的技术思路，具有较好的可扩展性。

Abstract: Large Vision-Language Models (LVLMs) have achieved remarkable performance on diverse vision-language tasks. However, LVLMs still suffer from hallucinations, generating text that contradicts the visual input. Existing research has primarily focused on mitigating object hallucinations, but often overlooks more complex relation hallucinations, particularly action relations involving interactions between objects. In this study, we empirically observe that the primary cause of action-relation hallucinations in LVLMs is the insufficient attention allocated to visual information. Thus, we propose a framework to locate action-relevant image regions and enhance the LVLM’s attention to those regions. Specifically, we define the Action-Relation Sensitivity (ARS) score to identify attention heads that are most sensitive to action-relation changes, thereby localizing action-relevant image regions that contain key visual cues. Then, we propose the Relation-aware Visual Enhancement (RVE) method to enhance the LVLM’s attention to these action-relevant image regions. Extensive experiments demonstrate that, compared to existing baselines, our method achieves superior performance in mitigating action-relation hallucinations with negligible additional inference cost. Furthermore, it effectively generalizes to spatial-relation hallucinations and object hallucinations.

[80] REFNet++: Multi-Task Efficient Fusion of Camera and Radar Sensor Data in Bird’s-Eye Polar View cs.CV | cs.AIPDF

Kavin Chandrasekaran, Sorin Grigorescu, Gijs Dubbelman, Pavol Jancura

TL;DR: 本文提出REFNet++，一种多任务高效融合相机和雷达传感器数据的方法，通过将前视相机图像和雷达原始RD频谱统一转换到鸟瞰极坐标域进行对齐，实现车辆检测和自由空间分割任务。

Details

Motivation: 解决在自动驾驶环境感知中，相机和雷达传感器因数据模态差异（相机提供丰富视觉信息但受天气影响，雷达鲁棒性强但输出噪声大且分类能力有限）难以有效融合的问题，同时兼顾精度和计算效率。

Result: 在RADIal数据集上，所提融合方法在车辆检测和自由空间分割任务上达到了最先进（SOTA）水平。

Insight: 创新点在于使用变分编码器-解码器架构将前视相机图像转换到鸟瞰极坐标域，并利用雷达编码器-解码器从RD频谱恢复角度信息生成RA特征，实现两种模态在统一域的对齐，从而促进高效鲁棒的传感器融合。

Abstract: A realistic view of the vehicle’s surroundings is generally offered by camera sensors, which is crucial for environmental perception. Affordable radar sensors, on the other hand, are becoming invaluable due to their robustness in variable weather conditions. However, because of their noisy output and reduced classification capability, they work best when combined with other sensor data. Specifically, we address the challenge of multimodal sensor fusion by aligning radar and camera data in a unified domain, prioritizing not only accuracy, but also computational efficiency. Our work leverages the raw range-Doppler (RD) spectrum from radar and front-view camera images as inputs. To enable effective fusion, we employ a variational encoder-decoder architecture that learns the transformation of front-view camera data into the Bird’s-Eye View (BEV) polar domain. Concurrently, a radar encoder-decoder learns to recover the angle information from the RD data that produce Range-Azimuth (RA) features. This alignment ensures that both modalities are represented in a compatible domain, facilitating robust and efficient sensor fusion. We evaluated our fusion strategy for vehicle detection and free space segmentation against state-of-the-art methods using the RADIal dataset.

[81] Selection, Not Fusion: Radar-Modulated State Space Models for Radar-Camera Depth Estimation cs.CVPDF

Zhangcheng Hou, Tomoaki Ohtsuki

TL;DR: 本文提出了一种用于雷达-相机深度估计的新方法，认为雷达信息应在状态空间模型（如Mamba）的选择机制中注入，而非在序列操作外进行特征融合。作者引入了雷达调制选择（RMS）机制，通过雷达信号调制扫描过程的步长Δ和读出矩阵C，同时保持输入投影B和状态动态A仅由图像决定。该方法在nuScenes数据集上实现了最先进的性能，显著降低了深度估计误差，并具有最低的单帧延迟。

Details

Motivation: 现有雷达-相机深度估计方法通常在主干网络的序列操作之外融合雷达和图像特征，甚至跨模态Mamba变体也保持选择机制为单模态。作者认为，选择机制才是雷达信息应该进入的正确位置，以更有效地利用雷达信号进行深度估计。

Result: 在nuScenes数据集上，SemoDepth方法在0-50m、0-70m和0-80m范围内，将平均绝对误差（MAE）分别降低了34.0%、29.9%和29.9%，超越了之前的最佳方法，达到了最先进水平（SOTA），同时实现了最低的单帧延迟（26.8ms）。

Insight: 核心创新点在于提出了“雷达调制选择”（RMS）机制，将雷达信息注入到Mamba模型的选择扫描内部，而非外部融合。这确保了模型在初始化时等同于预训练的纯图像Mamba，雷达仅在能提高精度时产生影响。该方法还实现了线性成本的跨模态耦合，并能在雷达缺失时自然回退到纯图像主干网络，其有效性通过消融实验得到验证。

Abstract: Radar-camera depth estimation must turn an ultra-sparse, all-weather, metric radar signal into a dense per-pixel depth map. Existing methods – concatenation, confidence-aware gating, sparse supervision, graph-based extraction – combine radar and image features outside the backbone’s sequence operator, and even cross-modal Mamba variants leave the selection mechanism itself unimodal. We argue that the selection mechanism is the right place for radar to enter. We introduce Radar-Modulated Selection (RMS), a minimal and principled way to inject radar into Mamba’s selective scan: radar modulates the scan from within, adding zero-initialised perturbations to the step size $Δ$ and readout $\mathbf{C}$ while leaving the input projection $\mathbf{B}$ and state dynamics $\mathbf{A}$ image-only. The construction is exactly equivalent to a pretrained image-only Mamba at initialisation, ensuring radar only influences the model where it improves accuracy. Two further properties follow that out-of-scan fusion cannot offer: linear-cost cross-modal coupling at every recurrence step, and a natural fallback to the image-only backbone when radar is absent. We deploy RMS in a Multi-View Scan Pyramid (MVSP) that matches the fusion operator to radar’s spatial reach at each scale. SemoDepth achieves state-of-the-art performance on nuScenes, reducing MAE by 34.0%, 29.9%, and 29.9% over the previous best at 0–50, 0–70, and 0–80m, while attaining the lowest single-frame latency (26.8ms). A further ablation shows that out-of-scan feature blending adds no accuracy on top of RMS, providing empirical validation that in-scan selection can replace out-of-scan fusion.

[82] UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs cs.CV | cs.CLPDF

Houcheng Jiang, Jiajun Fu, Junfeng Fang, Chen Gao, Xiang Wang

TL;DR: UniVLR提出了一种统一的视觉潜在推理框架，将文本推理和辅助视觉证据整合到一个共享的视觉工作空间中，通过将推理轨迹与辅助图像一起渲染并压缩成紧凑的视觉潜在令牌，使模型在推理时仅通过视觉潜在令牌进行思考并直接解码最终答案，从而提高了多模态大语言模型在视觉推理任务中的效率和性能。

Details

Motivation: 现有视觉潜在推理方法依赖文本思维链与视觉潜在令牌的交错设计，导致效率低下且推理在文本和视觉通道间割裂，UniVLR旨在解决这一问题，实现更统一、高效的视觉思考范式。

Result: 在真实世界感知和视觉推理任务上的实验表明，UniVLR优于先前的视觉潜在推理方法，同时使用了更少的生成推理令牌。

Insight: 创新点在于将文本推理轨迹与视觉证据统一表示为可学习的视觉潜在令牌，避免了推理时的外部工具调用和冗长文本链，实现了端到端的紧凑视觉推理，为多模态大语言模型的视觉思考提供了更高效的统一表征方法。

Abstract: Multimodal large language models are increasingly expected to perform thinking with images, yet existing visual latent reasoning methods still rely on explicit textual chain-of-thought interleaved with visual latent tokens. This interleaved design limits efficiency and keeps reasoning fragmented across separate text and vision channels. We propose UniVLR, a unified visual latent reasoning framework that treats textual reasoning and auxiliary visual evidence as a shared visual workspace. Instead of preserving text CoT as an independent inference-time path, UniVLR renders reasoning traces together with auxiliary images and learns to compress this unified representation into compact visual latent tokens. At inference time, the model reasons only through visual latents and directly decodes the final answer, avoiding both external tool calls and verbose text reasoning. Experiments on real-world perception and visual reasoning tasks show that UniVLR outperforms prior visual latent reasoning methods while using substantially fewer generated reasoning tokens, suggesting a more unified and efficient paradigm for visual thinking in MLLMs.

[83] GATA2Floor: Graph attention for floor counting in street-view facades cs.CV | eess.IVPDF

Ngoc Tan Le, Tzoulio Chamiti, Eirini Papagiannopoulou, Nikos Deligiannis

TL;DR: 本文提出GATA2Floor模型，通过将建筑立面建模为基于窗户/门检测的图结构，并利用多头图注意力机制（GATv2）预测建筑总楼层数，同时通过可学习的交叉注意力查询将元素软分配到潜在楼层槽中，实现可解释的输出和对不规则设计的鲁棒性。此外，该方法通过基于自监督特征和视觉语言评分的轻量级无标签提议机制，解决了标注数据缺乏的问题。

Details

Motivation: 自动化分析街景图像中的建筑立面对于城市分析、能源评估和应急规划具有重要潜力，但需要基于空间排列的元素进行推理，而不仅仅是孤立检测。

Result: 该方法在立面理解任务中展示了基于图注意力的关系推理的价值，但摘要未提及具体基准测试或定量结果。

Insight: 创新点包括将立面建模为具有垂直先验的图结构、使用GATv2进行全局楼层预测和元素分配，以及引入无标签提议机制以减少对标注数据的依赖，为立面分析提供了可解释且鲁棒的解决方案。

Abstract: Automated analysis of building facades from street-level imagery has great potential for urban analytics, energy assessment, and emergency planning. However, it requires reasoning over spatially arranged elements rather than solely isolated detections. In this work, we model each facade as a graph over window/door detections with a vertical prior on edges. Additionally, we introduce GATA2Floor, a multi-head Graph Attention v2 (GATv2) based model that predicts the global floor count of a building and, via learnable cross-attention queries, softly assigns elements to latent floor slots, yielding interpretable outputs and robustness to irregular designs. To mitigate the lack of labeled datasets, we demonstrate that the proposed graph-based reasoning can be applied without annotations by leveraging a lightweight label-free proposal mechanism based on self-supervised features and vision-language scoring. Our approach demonstrates the value of graph-attention-based relational reasoning for facade understanding.

[84] When Brains Disagree: Biological Ambiguity Underlies the Challenge of Amyloid PET Synthesis from Structural MRI cs.CVPDF

Louise E. G. Baron, Ross Callaghan, David M. Cash, Philip S. Weston, Hojjat Azadbakht

TL;DR: 该论文研究了从结构MRI合成淀粉样蛋白PET图像这一任务中性能不一致的根本原因。作者通过实验证明，这种不一致性源于阿尔茨海默病中神经退行性变（MRI捕捉）与淀粉样蛋白病理（PET测量）在时间上的解耦，导致MRI到PET的映射存在固有的生物学模糊性。研究进一步表明，引入血浆生物标志物等多模态信息可以解决这种模糊性，从而实现稳定且更有意义的性能提升。

Details

Motivation: 解决现有研究中结构MRI到淀粉样蛋白PET合成模型性能差异巨大且复杂架构未能带来一致提升的问题，并验证其根本原因在于MRI与PET所反映的生物学过程（神经退行与淀粉样蛋白沉积）在时间上的解耦所导致的固有数据模糊性。

Result: 在受控实验中，当数据无模糊性时，标准合成模型可学习；引入数据模糊性后性能崩溃。引入血浆生物标志物等多模态输入后，合成性能得到提升并恢复稳定。

Insight: 揭示了特定医学图像合成任务性能瓶颈的根本原因可能在于底层生物学过程的解耦所导致的数据分布模糊性，而非模型架构能力不足。这为相关领域的研究提供了关键见解：解决此类问题应优先考虑通过多模态数据整合来消除模糊性，而非一味追求模型复杂化。

Abstract: Structural MRI-to-amyloid PET synthesis has been proposed as a non-invasive alternative for amyloid assessment in Alzheimer’s disease (AD). However, reported performance of identical models varies widely across studies, and increasingly complex architectures have not led to consistent gains. This inconsistency is thought to be caused by a fundamental biological ambiguity: MRI captures neurodegeneration, while PET measures amyloid pathology - two processes that are often temporally decoupled in AD. As a result, similar MRI patterns may correspond to different amyloid states, creating ambiguous one-to-many mappings. MRI-to-amyloid PET synthesis may therefore be intrinsically ill-posed; however, this idea has yet to be tested scientifically. The aim of this work is to test this hypothesis through two controlled experiments. We first control the training distribution by stratifying paired MRI-PET data by amyloid and neurodegeneration status. Using two standard synthesis models under a controlled design, we show that biologically unambiguous mappings are learnable in isolation, but performance collapses when data ambiguity is introduced. This demonstrates that ambiguity in the data distribution, rather than architectural capacity, constrains performance. Second, we show that introducing orthogonal biological information in the form of plasma biomarkers resolves this ambiguity. When multimodal inputs are incorporated, performance improves and stability is restored. Together, these findings suggest that limited and inconsistent performance in MRI-to-amyloid PET synthesis is explained by intrinsic biological ambiguity, and that stable, meaningful progress requires multimodal integration rather than architectural complexity.

[85] FIS-DiT: Breaking the Few-Step Video Inference Barrier via Training-Free Frame Interleaved Sparsity cs.CV | cs.LGPDF

Jian Tang, Jiawei Fan, Qingbin Liu, Zheng Wei

TL;DR: 本文提出了一种名为FIS-DiT的训练无关、算子无关的框架，通过利用潜在帧维度上的帧间稀疏性和结构一致性，在视频扩散模型（DiT）的少步推理场景中，将优化重点从时间轨迹转移到帧维度，从而有效减少每步推理延迟。

Details

Motivation: 现有基于去噪轨迹冗余的加速方法在少步推理场景下收益递减，因为时间状态稀缺导致特征重用或预测建模失效。本文旨在突破这一瓶颈，实现进一步的推理加速。

Result: 在Wan 2.2和HunyuanVideo 1.5模型上的实验表明，FIS-DiT在VBench-Q和CLIP指标上性能下降可忽略不计的情况下，能稳定实现2.11到2.41倍的加速。

Insight: 核心创新点在于发现了潜在帧维度上存在帧间稀疏性（允许减少计算）与结构一致性（每个帧位置对全局时空上下文同等重要）的内在二元性，并据此设计了帧交错稀疏性（FIS）执行策略，通过操作模型层次中的帧子集来更新所有潜在位置，避免了全规模块计算。

Abstract: While the overall inference latency of Video Diffusion Transformers (DiTs) can be substantially reduced through model distillation, per-step inference latency remains a critical bottleneck. Existing acceleration paradigms primarily exploit redundancy across the denoising trajectory; however, we identify a limitation where these step-wise strategies encounter diminishing returns in few-step regimes. In such scenarios, the scarcity of temporal states prevents effective feature reuse or predictive modeling, creating a formidable barrier to further acceleration. To overcome this, we propose Frame Interleaved Sparsity DiT (FIS-DiT), a training-free and operator-agnostic framework that shifts the optimization focus from the temporal trajectory to the latent frame dimension. Our approach is motivated by an intrinsic duality within this dimension: the existence of frame-wise sparsity that permits reduced computation, coupled with a structural consistency where each frame position remains equally vital to the global spatiotemporal context. Leveraging this insight, we implement Frame Interleaved Sparsity (FIS) as an execution strategy that manipulates frame subsets across the model hierarchy, refreshing all latent positions without requiring full-scale block computation. Empirical evaluations on Wan 2.2 and HunyuanVideo 1.5 demonstrate that FIS-DiT consistently achieves 2.11–2.41$\times$ speedup with negligible degradation across VBench-Q and CLIP metrics, providing a scalable and robust pathway toward real-time high-definition video generation.

Yuzhu Wang, Xi Ye, Duo Su, Yangyang Xu, Jun Zhu

TL;DR: 本文提出了一种名为$h$-control的训练自由相机控制方法，用于预训练的流匹配视频生成器。该方法通过结构化的采样器改进，结合了外部硬替换引导步骤与内部块条件伪吉布斯细化，以解决现有方法在轨迹遵循和视觉质量之间的权衡问题，并在RealEstate10K和DAVIS数据集上实现了最佳FVD分数。

Details

Motivation: 解决预训练流匹配视频生成器中训练自由相机控制的难题，这是一个部分观测的逆问题：深度扭曲的引导视频在潜在站点子集上提供噪声证据，采样器必须将其与预训练先验协调。现有方法难以平衡轨迹遵循和视觉质量，且启发式引导强度调整缺乏鲁棒性。

Result: 在RealEstate10K和DAVIS数据集上，$h$-control在所有七个训练自由和基于训练的竞争对手中获得了最佳FVD分数，并在每个报告指标上优于所有训练自由基线，达到了SOTA水平。

Insight: 创新点在于通过结构化的采样器改变，引入块条件伪吉布斯细化来收敛到部分观测条件数据分布，并利用条件局部性对高维视频潜在空间进行分区，通过自适应混合指示器加速收敛，从而在无需训练的情况下实现鲁棒的相机控制。

Abstract: Training-free camera control for pretrained flow-matching video generators is a partial-observation inverse problem: a depth-warped guidance video supplies noisy evidence on a subset of latent sites, which the sampler must reconcile with the pretrained prior. Existing methods struggle to balance the trade-off between trajectory adherence and visual quality and the heuristic guidance-strength tuning lacks robustness. We propose \textbf{$h$-control}, which resolves this dilemma through a structural change to the sampler: each outer hard-replacement guidance step is augmented with an inner-loop \emph{block-conditional pseudo-Gibbs refinement} on the unobserved complement at the same noise level, with provable convergence to the partial-observation conditional data law. To accelerate convergence on high-dimensional video latents, we exploit their conditional locality, partitioning the unobserved complement into 3D patches, each tracked by a custom mixing indicator that adaptively freezes converged patches. On RealEstate10K and DAVIS, \textbf{$h$-control} attains the best FVD against all seven training-free and training-based competitors, outperforming every training-free baseline on every reported metric.

[87] Few-Shot Synthetic Data Generation with Diffusion Models for Downstream Vision Tasks cs.CVPDF

Daniil Dushenev, Nazariy Karpov, Daniil Zinovjev, Alexander Gorin, Konstantin Kulikov

TL;DR: 本文提出了一种基于LoRA适配器和预训练扩散模型的轻量级合成数据增强方法，用于解决视觉识别中的类别不平衡问题。该方法仅需20-50张稀有类别的真实图像进行微调，即可生成合成样本以增强训练数据，并在胸部X光病理分类和工业表面裂纹检测两个不同领域进行了评估。

Details

Motivation: 解决视觉识别中因稀有类别样本收集成本高、代表性不足而导致的类别不平衡问题，特别是在安全关键领域。

Result: 在两个数据集（NIH ChestX-ray14和Magnetic Tile Defect）上，合成数据增强相比仅使用真实数据训练，持续提高了稀有类别的召回率和F1分数；性能在适度合成增强时提升，但随着合成比例增加呈现收益递减。

Insight: 利用LoRA适配器微调预训练扩散模型，提供了一种简单且可扩展的稀有类别数据增强机制，能够在数据稀缺的异构视觉领域中实现有效学习。

Abstract: Class imbalance is a persistent challenge in visual recognition, particularly in safety-critical domains where collecting positive examples is expensive and rare events are inherently underrepresented. We propose a lightweight synthetic data augmentation pipeline that fine-tunes a LoRA adapter on as few as 20-50 real images of a rare class and uses a pretrained diffusion model to generate synthetic samples for training. We systematically vary the synthetic-to-real ratio and evaluate the approach across two structurally different domains: chest X-ray pathology classification (NIH ChestX-ray14) and industrial surface crack detection (Magnetic Tile Defect dataset). All evaluations are performed on held-out sets of real images only. Across both domains, synthetic augmentation consistently improves rare-class recall and F1 compared to training with real data alone. Performance improves with moderate synthetic augmentation and shows diminishing returns as the synthetic ratio increases. These results suggest that LoRA-adapted diffusion models provide a simple and scalable mechanism for augmenting rare classes, enabling effective learning in data-scarce scenarios across heterogeneous visual domains.

[88] Mobile Traffic Camera Calibration from Road Geometry for UAV-Based Traffic Surveillance cs.CVPDF

Alexey Popov, Natalia Trukhina, Vadim Vashkelis

TL;DR: 本文提出了一种轻量级流程，将单目倾斜无人机交通视频转换为局部度量的鸟瞰图表示，利用可见的道路几何信息估计图像坐标到度量地面坐标的单应性变换，从而支持车辆轨迹、速度、方向和三维立方体估计，为可部署的无人机交通摄像头和实时交通数字孪生系统奠定基础。

Details

Motivation: 固定路边摄像头不可用、成本高或不切实际时，无人机可提供灵活的交通监控，但原始无人机视频难以用于交通分析，因为车辆运动是在透视图像坐标中观察的，而非稳定的度量道路坐标系。

Result: 在UAVDT数据集上使用真实标注进行评估，对于序列M1401，从40个采样帧中生成23条轨迹上的632个度量立方体实例，结果表明道路几何校准可将单目无人机视频转换为可解释的交通摄像头式分析。

Insight: 创新点在于利用道路几何（如车道线、路缘、人行横道）估计单应性，实现从图像到度量地面的转换，支持BEV轨迹和同步3D立方体可视化；但远场车辆对单应性误差敏感，手动验证比全自动校准更可靠，单平面假设在非平面或模糊道路区域存在限制。

Abstract: Unmanned aerial vehicles (UAVs) can provide flexible traffic surveillance where fixed roadside cameras are unavailable, costly, or impractical. However, raw UAV video is difficult to use for traffic analytics because vehicle motion is observed in perspective image coordinates rather than in a stable metric road coordinate system. This paper presents a lightweight pipeline for converting monocular oblique UAV traffic video into a local metric bird’s-eye-view (BEV) representation. Visible road geometry, including lane markings, road borders, and crosswalks, is used to estimate a road-plane homography from image coordinates to metric ground-plane coordinates. Vehicle observations from dataset annotations or detectors are then projected to BEV using estimated ground contact points. The resulting trajectories support estimation of vehicle direction, speed, heading, and dynamic 3D cuboids on the road plane. We evaluate the pipeline on UAVDT using ground-truth annotations to isolate calibration and geometric reconstruction from detector and tracker errors. For sequence M1401, 40 sampled frames from img000001-img000196 produce 632 metric cuboid instances across 23 tracks. Results show that road-geometry calibration can transform monocular UAV footage into interpretable traffic-camera-style analytics, including BEV tracks and synchronized 3D cuboid visualizations. They also reveal key limitations: far-field vehicles are sensitive to homography errors, manual validation is currently more reliable than fully automatic calibration, and the single-plane assumption limits performance in non-planar or ambiguous road regions. The proposed pipeline provides a practical foundation for deployable UAV traffic cameras and future real-time traffic digital-twin systems.

[89] Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training cs.CVPDF

Qihuang Zhong, Liang Ding, Wenjie Xuan, Juhua Liu, Bo Du

TL;DR: 本文提出了一种名为VISTA的视觉感知自改进训练框架，旨在解决多模态大语言模型（MLLMs）在自改进训练中存在的两个主要问题：数据不平衡和语言先验偏差。VISTA通过前缀重采样策略高效收集部分正确的推理轨迹，并利用视觉感知注意力分数量化模型对视觉信息的关注度，从而提升MLLMs的多模态推理能力。实验表明，VISTA可应用于监督微调和偏好学习等多种后训练场景，有效提升不同MLLM模型和任务上的性能。

Details

Motivation: 现有MLLMs的自改进训练范式存在数据不平衡（简单样本过训练、关键挑战样本欠训练）和语言先验偏差（过度依赖语言先验而忽视视觉线索）的问题，导致获取高质量推理轨迹成本高昂且耗时。

Result: 在多种后训练场景（如监督微调、偏好学习）和不同MLLM模型（如Qwen2.5-VL-3B-Instruct）及任务上的广泛实验显示，VISTA能有效提升多模态推理性能，例如为Qwen2.5-VL-3B-Instruct带来高达+13.66%的平均性能增益。

Insight: 创新点包括：1）前缀重采样策略，通过重用部分正确的推理轨迹实现高效数据收集；2）视觉感知注意力分数，用于量化模型对视觉信息的关注，缓解语言先验偏差。这为MLLMs的自监督训练提供了数据平衡和视觉感知增强的新思路。

Abstract: Post-training with explicit reasoning traces is common to improve the reasoning capabilities of Multimodal Large Language Models (MLLMs). However, acquiring high-quality reasoning traces is often costly and time-consuming. Hence, the self-improvement paradigm has emerged, enabling MLLMs to self-generate reasoning traces for training without external supervision. Despite its effectiveness, we reveal two shortcomings in the self-improvement training of MLLMs: 1) data imbalance, where simple samples are over-trained, but the challenging yet crucial samples are under-trained; 2) language prior bias, where MLLMs overly rely on linguistic priors while neglecting the visual cues. To this end, we propose VISTA, a vision-aware self-improvement training framework for enhancing the multimodal reasoning of MLLMs. Specifically, VISTA first introduces a prefix resampling strategy to reuse the partial correct reasoning traces for efficient data collection, and then designs a vision-aware attention score to quantify the model’s focus on visual information. Extensive experiments show that VISTA can be applied to various post-training scenarios, i.e., supervised fine-tuning and preference learning, and effectively enhances the multimodal reasoning performance across various MLLMs and tasks, e.g., bringing up to +13.66% average performance gains for Qwen2.5-VL-3B-Instruct.

[90] Cluster-Aware Neural Collapse Prompt Tuning for Long-Tailed Generalization of Vision-Language Models cs.CVPDF

Boyang Guo, Liang Li, Lin Peng, Yuhan Gao, Xichun Sheng

TL;DR: 本文提出了一种名为聚类感知神经坍缩提示调优（CPT）的方法，旨在解决提示学习在适应类别不平衡数据集时尾部类别判别性不足的问题。该方法通过构建聚类不变空间并引入神经坍缩驱动的判别性优化损失，在保持预训练视觉-语言模型全局语义结构的同时，增强尾部类别的区分能力。

Details

Motivation: 当前提示学习方法在适应类别不平衡数据集时，难以保持尾部类别的判别性，导致模型在长尾分布下泛化能力下降。

Result: 在11个多样化数据集上的广泛实验表明，CPT方法优于现有SOTA方法，尤其在长尾类别上表现更强，并对未见类别具有良好的泛化能力。

Insight: 创新点在于结合了聚类不变空间构建与神经坍缩理论驱动的损失设计，通过局部约束和几何优化来提升尾部类别的判别性，而不损害模型的整体泛化性能。

Abstract: Prompt learning has emerged as an efficient alternative to fine-tuning pre-trained vision-language models (VLMs). Despite its promise, current methods still struggle to maintain tail-class discriminability when adapting to class-imbalanced datasets. In this work, we propose cluster-aware neural collapse prompt tuning (CPT), which enhances the discriminability of tail classes in prompt-tuned VLMs without sacrificing their overall generalization. First, we design a cluster-invariant space by mining semantic assignments from the pre-trained VLM and mapping them to prompt-tuned features. This computes cluster-level boundaries and restricts the constraints to local neighborhoods, which reduces interference with the global semantic structure of the pre-trained VLM. Second, we introduce neural-collapse-driven discriminability optimization with three losses: textual Equiangular Tight Frame (ETF) separation loss, class-wise convergence loss, and rotation stabilization loss. These losses work together to shape intra-cluster geometry for better inter-class separation and intra-class alignment. Extensive experiments on 11 diverse datasets demonstrate that CPT outperforms SOTA methods, with stronger performance on long-tail classes and good generalization to unseen classes.

[91] Multimodal Abstractive Summarization of Instructional Videos with Vision-Language Models cs.CV | cs.CLPDF

Maham Nazir, Muhammad Aqeel, Richong Zhang, Francesco Setti

TL;DR: 本文提出ClipSum框架，利用预训练的CLIP视觉语言模型特征，结合显式时序建模和维度自适应融合，用于教学视频的多模态摘要生成。实验表明，该方法在YouCook2数据集上优于传统CNN特征方法，且冻结的CLIP特征比微调版本效果更好。

Details

Motivation: 传统视频摘要方法依赖CNN特征，这些特征基于物体分类训练，将视觉概念表示为离散类别，与自然语言生成不匹配。本文旨在通过CLIP的语义对齐视觉特征来弥合视觉与语言之间的表示鸿沟。

Result: 在YouCook2数据集上，ClipSum达到33.0%的ROUGE-1分数，优于使用ResNet-152特征的30.5%，且特征维度更低（512 vs. 2048）。冻结的CLIP特征（33.0%）也超过了微调的CLIP（32.3%），达到SOTA水平。

Insight: 创新点在于利用CLIP的对比预训练特征实现视觉与语言的语义对齐，并通过时序建模和融合机制提升摘要质量。客观分析表明，语义对齐比特征容量或任务特定适应更重要，这为多模态任务提供了新思路。

Abstract: Multimodal video summarization requires visual features that align semantically with language generation. Traditional approaches rely on CNN features trained for object classification, which represent visual concepts as discrete categories not aligned with natural language. We propose ClipSum, a framework that leverages frozen CLIP vision-language features with explicit temporal modeling and dimension-adaptive fusion for instructional video summarization. CLIP’s contrastive pre-training on 400M image-text pairs yields visual features semantically aligned with the linguistic concepts that text decoders generate, bridging the vision-language gap at the representation level. On YouCook2, ClipSum achieves 33.0% ROUGE-1 versus 30.5% for ResNet-152 with 4x lower dimensionality (512 vs. 2048), demonstrating that semantic alignment matters more than feature capacity. Frozen CLIP (33.0%) surpasses fine-tuned CLIP (32.3%), showing that preserving pre-trained alignment is more valuable than task-specific adaptation. https://github.com/aqeeelmirza/clipsum

[92] Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters cs.CVPDF

Gengluo Li, Shangpin Peng, Xingyu Wan, Chengquan Zhang, Hao Feng

TL;DR: 本文介绍了Chronicles-OCR，这是一个专门用于评估视觉大语言模型（VLLMs）在汉字七种书体完整演化轨迹上跨时间视觉感知能力的综合性基准。该数据集包含2800张严格平衡的图像，涵盖从龟甲到纸质书法等多种物理媒介，并提出了阶段自适应标注范式，设计了四个定量任务来隔离视觉感知与语义推理，以暴露当前VLLMs的局限性。

Details

Motivation: 现有古代文本数据集通常只关注孤立的历史时期，未能捕捉跨越数千年的系统性视觉分布变化。为了弥补这一空白并赋能数字人文，需要评估VLLMs在面对历史书写系统持续形态演变时的感知鲁棒性。

Result: 论文提出了Chronicles-OCR基准，并基于此设计了四个严格的定量任务：跨时期字符定位、通过视觉指代的细粒度古文字识别、古代文本解析和书体分类。该基准旨在权威地暴露当前VLLMs的局限性，但摘要中未提及具体模型在该基准上的定量性能结果或是否达到SOTA。

Insight: 主要创新点在于创建了首个覆盖汉字完整演化轨迹（七种书体）的跨时间感知基准，并提出了阶段自适应标注范式来适应不同历史阶段的剧烈形态和拓扑变化。其核心思想是通过精心设计的任务隔离视觉感知与语义推理，为开发对演化敏感、鲁棒的历史文本感知模型提供了新的评估平台和数据基础。

Abstract: Vision Large Language Models (VLLMs) have achieved remarkable success in modern text-rich visual understanding. However, their perceptual robustness in the face of the continuous morphological evolution of historical writing systems remains largely unexplored. Existing ancient text datasets typically focus on isolated historical periods, failing to capture the systematic visual distribution shifts spanning thousands of years. To bridge this gap and empower Digital Humanities, we introduce Chronicles-OCR, the first comprehensive benchmark specifically designed to evaluate the cross-temporal visual perception capabilities of VLLMs across the complete evolutionary trajectory of Chinese characters, known as the Seven Chinese Scripts. Curated in collaboration with top-tier institutional domain experts, the dataset comprises 2,800 strictly balanced images encompassing highly diverse physical media, ranging from tortoise shells to paper-based calligraphy. To accommodate the drastic morphological and topological variations across different historical stages, we propose a novel Stage-Adaptive Annotation Paradigm. Based on this, Chronicles-OCR formulates four rigorous quantitative tasks: cross-period character spotting, fine-grained archaic character recognition via visual referring, ancient text parsing, and script classification. By isolating visual perception from semantic reasoning, Chronicles-OCR provides an authoritative platform to expose the limitations of current VLLMs, paving the way for robust, evolution-aware historical text perception. Chronicles-OCR is publicly available at https://github.com/VirtualLUOUCAS/Chronicles-OCR.

[93] H2G: Hierarchy-Aware Hyperbolic Grouping for 3D Scenes cs.CVPDF

ByungHa Ko, Youngmin Lee, Dong Hwan Kim

TL;DR: 本文提出H2G方法，一种用于三维场景层次化分组的双曲亲和力场。该方法通过解释基础模型的亲和力，利用Dasgupta的基于相似性的层次聚类目标生成语义组织的树状监督，并将此监督蒸馏到单一的洛伦兹双曲特征场中。该几何结构适合树状分支，通过层次感知目标对齐细粒度分配、粗粒度对象结构、紧凑特征簇和最低公共祖先排序，从而在单一特征空间中表示多个分组层次，实现基于2D基础模型知识的语义层次分组。

Details

Motivation: 解决无需语义标签或固定词汇表的情况下，从精细对象部件到完整对象的多粒度三维场景层次化分组问题，核心挑战在于将2D基础模型线索转化为连贯的层次监督并嵌入3D表示。

Result: 未在摘要中明确提及具体定量结果或基准测试，但方法旨在实现基于2D基础模型知识的语义层次分组。

Insight: 创新点包括：利用Dasgupta目标从基础模型亲和力推导树状监督，将层次监督蒸馏到几何上适合树状结构的洛伦兹双曲特征场，以及设计层次感知目标对齐多尺度分组信息，在单一特征空间实现多粒度语义分组。

Abstract: Hierarchical 3D grouping aims to recover scene groups across multiple granularities, from fine object parts to complete objects, without relying on semantic labels or a fixed vocabulary. The main challenge is to transform 2D foundation-model cues into coherent hierarchy supervision and embed that hierarchy in a 3D representation. We propose H2G, a hyperbolic affinity field for hierarchical 3D grouping. Our method derives semantically organized tree supervision by interpreting foundation-model affinities through Dasgupta’s objective for similarity-based hierarchical clustering. This supervision is distilled into a single Lorentz hyperbolic feature field, whose geometry is well suited for tree-like branching structures. A hierarchy-aware objective aligns the field with fine-level assignments, coarse object structure, compact feature clusters, and LCA (Lowest Common Ancestor) ordering. This formulation represents multiple grouping levels in one feature space, enabling semantic hierarchical grouping grounded in 2D foundation-model knowledge.

[94] Robust Promptable Video Object Segmentation cs.CVPDF

Sohyun Lee, Yeho Gwon, Lukas Hoyer, Konrad Schindler, Christos Sakaridis

TL;DR: 该论文首次系统研究了鲁棒性可提示视频目标分割（RobustPVOS）问题，构建了一个包含真实世界和合成数据的新基准测试，并提出了一种名为记忆对象条件门控秩适应（MoGA）的新方法，以应对输入损坏导致的性能下降。

Details

Motivation: 当前可提示视频目标分割（PVOS）模型在输入损坏（如噪声、模糊等）下性能显著下降，限制了其在安全关键领域的应用，因此需要研究鲁棒性PVOS方法。

Result: 在构建的基准测试上，MoGA方法在合成和真实世界数据集上均表现出显著改进，对各种损坏类型具有一致的鲁棒性，为未来研究建立了强基线。

Insight: 创新点在于通过利用跨帧记忆中的对象特定表示来条件化鲁棒化过程，从而有效处理对象特定退化并确保预测的时间一致性，这为视频分割中的鲁棒性研究提供了新思路。

Abstract: The performance of promptable video object segmentation (PVOS) models substantially degrades under input corruptions, which prevents PVOS deployment in safety-critical domains. This paper offers the first comprehensive study on robust PVOS (RobustPVOS). We first construct a new, comprehensive benchmark with two real-world evaluation datasets of 351 video clips and more than 2,500 object masks under real-world adverse conditions. At the same time, we generate synthetic training data by applying diverse and temporally varying corruptions to existing VOS datasets. Moreover, we present a new RobustPVOS method, dubbed Memory-object-conditioned Gated-rank Adaptation (MoGA). The key to successfully performing RobustPVOS is two-fold: effectively handling object-specific degradation and ensuring temporal consistency in predictions. MoGA leverages object-specific representations maintained in memory across frames to condition the robustification process, which allows the model to handle each tracked object differently in a temporally consistent way. Extensive experiments on our benchmark validate MoGA’s efficacy, showing consistent and significant improvements across diverse corruption types on both synthetic and real-world datasets, establishing a strong baseline for future RobustPVOS research. Our benchmark is publicly available at https://sohyun-l.github.io/RobustPVOS_project_page/.

[95] What-Where Transformer: A Slot-Centric Visual Backbone for Concurrent Representation and Localization cs.CVPDF

Ryota Yoshihashi, Masahiro Kada, Satoshi Ikehata, Rei Kawakami, Ikuro Sato

TL;DR: 本文提出了What-Where Transformer（WWT），一种以槽位为中心的新型视觉骨干网络，旨在通过显式分离和并发处理‘是什么’（物体外观）和‘在哪里’（空间位置）信息，来同时支持表征学习和定位任务。该方法将令牌视为‘是什么’的表征，将注意力图视为‘在哪里’的表征，并通过多流槽位架构进行并行前馈处理。

Details

Motivation: 动机在于解决现有以分类为导向的视觉骨干网络（如ViT）倾向于隐式纠缠或抑制空间位置（‘在哪里’）信息的问题，这导致在需要定位的任务（如物体发现、检测、分割）上更为复杂。本文旨在通过引入‘什么-哪里分离’的归纳偏置，使模型能以分解的方式表示物体外观和空间位置。

Result: 在仅使用ImageNet单标签分类监督的情况下，WWT直接从原始注意力图中展现出新兴的多物体发现能力，无需额外的后处理（如令牌聚类）。在零样本物体发现和弱监督语义分割任务上，WWT相比基于ViT的方法取得了更优的性能，并且只需最小修改即可迁移到各种定位设置中。

Insight: 核心创新点在于：1）将令牌和注意力图分别明确地建模为‘是什么’和‘在哪里’的表征，并通过多流槽位架构进行并发处理；2）将最终层的令牌和注意力图直接复用于下游任务，并使其暴露于任务损失梯度，从而促进更有效和显式的定位学习。这为构建能同时胜任识别和定位任务的通用视觉骨干提供了新思路。

Abstract: Many image understanding tasks involve identifying what is present and where it appears. However, tasks that address where, such as object discovery, detection, and segmentation, are often considerably more complex than image classification, which primarily focuses on what. One possible reason is that classification-oriented backbones tend to emphasize semantic information about what, while implicitly entangling or suppressing information about where. In this work, we focus on an inductive bias termed what-where separation, which encourages models to represent object appearance and spatial location in a decomposed manner. To incorporate this bias throughout an attentive backbone in the style of Vision Transformer (ViT), we propose the What-Where Transformer (WWT). Our method introduces two key novel designs: (1) it treats tokens as representations of what and attention maps as representations of where, and processes them in concurrent feed-forward modules via a multi-stream, slot-based architecture; (2) it reuses both the final-layer tokens and attention maps for downstream tasks, and directly exposes them to gradients derived from task losses, thereby facilitating more effective and explicit learning of localization. We demonstrate that even under standard single-label classification-based supervision on ImageNet, WWT exhibits emergent multiple object discovery directly from raw attention maps, rather than via additional postprocessing such as token clustering. Furthermore, WWT achieves superior performance compared to ViT-based methods on zero-shot object discovery and weakly supervised semantic segmentation, and it is transferable to various localization setups with minimal modifications. Code will be published after acceptance.

[96] Spectral Vision Transformer for Efficient Tokenization with Limited Data cs.CV | cs.AI | eess.SPPDF

Alexandra G. Roberts, Maneesh John, Jinwei Zhang, Dominick Romano, Mert Sisman

TL;DR: 本文提出了一种新颖的频谱视觉变换器架构，旨在解决有限数据下的高效标记化问题，并特别关注医学成像应用。该架构利用频谱投影，相比空间视觉变换器降低了计算复杂度，并在参数减少的情况下，在模拟、公开和临床数据上取得了与多种模型相当或更优的性能。

Details

Motivation: 解决在数据有限（尤其是医学成像领域）的情况下，实现高效视觉标记化的挑战，并利用频谱基的选择来获得空间不变性和最优信噪比等理论优势。

Result: 在模拟、公开和临床数据集上，与紧凑/标准视觉变换器、带注意力的卷积神经网络、移位窗口变换器、多层感知机和逻辑回归等多种模型相比，该模型在参数减少的情况下取得了相当或更优的性能。

Insight: 创新点在于将频谱变换引入视觉变换器进行标记化，这带来了计算复杂度的降低和有利的理论特性（如空间不变性）。从客观角度看，这是一种将信号处理思想与Transformer架构结合以应对数据稀缺场景的有效尝试。

Abstract: We propose a novel spectral vision transformer architecture for efficient tokenization in limited data, with an emphasis on medical imaging. We outline convenient theoretical properties arising from the choice of basis including spatial invariance and optimal signal-to-noise ratio. We show reduced complexity arising from the spectral projection compared to spatial vision transformers. We show equitable or superior performance with a reduced number of parameters as compared to a variety of models including compact and standard vision transformers, convolutional neural networks with attention, shifted window transformers, multi-layer perceptrons, and logistic regression. We include simulated, public, and clinical data in our analysis and release our code at: \verb+github.com/agr78/spectralViT+.

[97] 4DVGGT-D: 4D Visual Geometry Transformer with Improved Dynamic Depth Estimation cs.CVPDF

Ying Zang, Xuanyi Liu, Yidong Han, Deyi Ji, Chaotao Ding

TL;DR: 本文提出了一种无需训练的渐进式解耦框架4DVGGT-D，用于从单目视频重建动态4D场景。该方法通过动态掩码引导的姿态解耦模块、拓扑子空间手术机制和信息论置信感知融合策略，以从粗到细的方式分离动态与静态元素，从而解决相机自运动与物体运动在全局注意力机制中的耦合问题。

Details

Motivation: 现有3D基础模型在动态环境中性能显著下降，其根本原因在于相机自运动与物体运动在全局注意力机制中的固有耦合。本文旨在解决这一挑战，实现鲁棒的动态4D场景重建。

Result: 在标准4D重建基准测试上的大量实验表明，该方法在主要点云指标上取得了一致且显著的提升，在无需微调的情况下实现了具有竞争力的鲁棒4D场景重建性能。

Insight: 核心创新在于提出了一种基于数学原理的、从粗到细的动态-静态解耦框架。具体包括：通过姿态解耦获得稳定无运动参考帧；通过拓扑子空间手术正交分解深度流形，在保留动态对象的同时向静态区域注入精细化几何；以及将深度集成公式化为异方差贝叶斯推理问题，实现自适应融合。该方法展示了基于数学基础的动态-静态解耦的潜力。

Abstract: Reconstructing dynamic 4D scenes from monocular videos is a fundamental yet challenging task. While recent 3D foundation models provide strong geometric priors, their performance significantly degrades in dynamic environments. This degradation stems from a fundamental tension: the inherent coupling of camera ego-motion and object motion within global attention mechanisms. In this paper, we propose a novel, training-free progressive decoupling framework that disentangles dynamics from statics in a principled, coarse-to-fine manner. Our core insight is to resolve the tension by first stabilizing the camera pose, followed by geometric refinement. Specifically, our approach consists of three synergistic components: (1) a Dynamic-Mask-Guided Pose Decoupling module that isolates pose estimation from dynamic interference, yielding a stable motion-free reference frame; (2) a Topological Subspace Surgery mechanism that orthogonally decomposes the depth manifold, safely preserving dynamic objects while injecting refined, mask-aware geometry into static regions; and (3) an Information-Theoretic Confidence-Aware Fusion strategy that formulates depth integration as a heteroscedastic Bayesian inference problem, adaptively blending multi-pass predictions via inverse-variance weighting. Extensive experiments on standard 4D reconstruction benchmarks demonstrate that our method achieves consistent and substantial improvements across principal point-cloud metrics. Notably, our approach shows competitive performance in robust 4D scene reconstruction without requiring fine-tuning, suggesting the potential of mathematically grounded dynamic-static disentanglement.

[98] OmniHumanoid: Streaming Cross-Embodiment Video Generation with Paired-Free Adaptation cs.CVPDF

Yiren Song, Xiyao Deng, Pei Yang, Yihan Wang, Mike Zheng Shou

TL;DR: OmniHumanoid是一个用于跨具身视频生成的框架，旨在将运动在不同人形具身（如人-机器人、机器人-机器人）之间进行迁移，以支持具身智能的可扩展数据生成。该方法通过解耦可迁移的运动学习和具身特定的适应，利用轻量级适配器仅需非配对视频即可适应新具身，并引入分支隔离注意力设计以减少干扰。

Details

Motivation: 解决跨具身视频生成中运动动态部分可迁移而外观形态具身特定的挑战，避免现有方法因因素纠缠或依赖配对数据而限制可扩展性的问题。

Result: 在合成和真实世界基准测试中，OmniHumanoid实现了强大的运动保真度和具身一致性，并能在不重新训练共享运动模型的情况下可扩展地适应未见的人形具身。

Insight: 创新点包括将运动迁移与具身适应因子化、使用轻量级适配器进行非配对适应、分支隔离注意力设计以减少干扰，以及构建合成跨具身数据集以促进模型学习。

Abstract: Cross-embodiment video generation aims to transfer motions across different humanoid embodiments, such as human-to-robot and robot-to-robot, enabling scalable data generation for embodied intelligence. A major challenge in this setting is that motion dynamics are partly transferable across embodiments, whereas appearance and morphology remain embodiment-specific. Existing approaches often entangle these factors, and many require paired data for every target embodiment, which limits scalability to new robots. We present OmniHumanoid, a framework that factorizes transferable motion learning and embodiment-specific adaptation. Our method learns a shared motion transfer model from motion-aligned paired videos spanning multiple embodiments, while adapting to a new embodiment using only unpaired videos through lightweight embodiment-specific adapters. To reduce interference between motion transfer and embodiment adaptation, we further introduce a branch-isolated attention design that separates motion conditioning from embodiment-specific modulation. In addition, we construct a synthetic cross-embodiment dataset with motion-aligned paired videos rendered across diverse humanoid assets, scenes, and viewpoints. Experiments on both synthetic and real-world benchmarks show that OmniHumanoid achieves strong motion fidelity and embodiment consistency, while enabling scalable adaptation to unseen humanoid embodiments without retraining the shared motion model.

[99] Anomaly-Aware Vision-Language Adapters for Zero-Shot Anomaly Detection cs.CV | cs.AI | cs.LGPDF

Muhammad Aqeel, Maham Nazir, Uzair Khan, Marco Cristani, Francesco Setti

TL;DR: 本文提出AVA-DINO，一种异常感知的视觉-语言适配框架，用于零样本异常检测。该方法利用正常与异常数据分布的不对称性，设计两个专门分支分别处理正常和异常模式，并通过文本引导的路由机制动态组合分支，实现不对称激活，在无需目标类别训练的情况下识别未见类别的缺陷。

Details

Motivation: 现有零样本异常检测方法通常对所有样本应用相同的特征变换，忽略了正常数据（紧凑）与异常数据（多样）之间本质上的不对称分布，导致性能受限。本文旨在利用这种自然不对称性，设计专门化的特征处理机制。

Result: 在九个工业和医学基准测试上实现了最先进的性能，其中在MVTec-AD上达到93.5%的图像AUROC，并在无需领域特定微调的情况下展现出强大的跨领域泛化能力（如迁移到医学影像）。

Insight: 创新点在于提出双分支异常感知适配器，结合文本引导的路由机制和显式路由正则化，鼓励分支专门化，从而实现对正常与异常模式的不对称特征变换。从客观角度看，该方法将分布不对称性显式建模到网络架构中，避免了退化的均匀路由，提升了零样本场景下的判别能力。

Abstract: Zero-shot anomaly detection aims to identify defects in unseen categories without target-specific training. Existing methods usually apply the same feature transformation to all samples, treating normal and anomalous data uniformly despite their fundamentally asymmetric distributions, compact normals versus diverse anomalies. We instead exploit this natural asymmetry by proposing AVA-DINO, an anomaly-aware vision-language adaptation framework with dual specialized branches for normal and anomalous patterns that adapt frozen DINOv3 visual features. During training on auxiliary data, the two branches are learned jointly with a text-guided routing mechanism and explicit routing regularization that encourages branch specialization. At test time, only the input image and fixed, predefined language descriptions are used to dynamically combine the two branches, enabling an asymmetric activation. This design prevents degenerate uniform routing and allows context-specific feature transformations. Experiments across nine industrial and medical benchmarks demonstrate state-of-the-art performance, achieving 93.5% image-AUROC on MVTec-AD and strong cross-domain generalization to medical imaging without domain-specific fine-tuning. https://github.com/aqeeelmirza/AVA-DINO

[100] Design Your Ad: Personalized Advertising Image and Text Generation with Unified Autoregressive Models cs.CV | cs.CL | cs.IRPDF

Yexing Xu, Wei Feng, Shen Zhang, Haohan Wang, Yuxin Qin

TL;DR: 本文提出了一种名为Uni-AdGen的统一自回归模型，用于联合生成个性化的图像-文本广告。该模型通过前景感知模块和指令微调提升生成内容的真实性，并采用从粗到细的偏好理解模块从用户历史点击行为中捕捉兴趣以驱动个性化生成。此外，研究构建了首个大规模个性化广告图文数据集PAd1M，并引入了产品背景相似度（PBS）指标用于训练和评估。实验表明，该方法在通用和个性化广告生成任务上均优于基线模型。

Details

Motivation: 现有广告生成方法通常使用多个独立模型，并依赖点击率（CTR）进行可控生成，但缺乏跨模态感知且CTR仅反映平均偏好，因此需要一种能够从用户历史行为中联合生成个性化图文广告的解决方案。

Result: 在构建的PAd1M数据集上进行的大量实验表明，该方法在通用和个性化广告生成任务上均超越了基线模型，取得了更好的性能。

Insight: 创新点包括：1) 提出了一个统一的单自回归框架（Uni-AdGen）来联合生成广告图像和文本；2) 设计了前景感知模块和指令微调以增强内容真实性；3) 引入了从粗到细的偏好理解模块，从嘈杂的多模态历史行为中有效提取用户兴趣；4) 构建了首个大规模个性化广告图文数据集PAd1M并提出了PBS评估指标。

Abstract: Generating realistic and user-preferred advertisements is a key challenge in e-commerce. Existing approaches utilize multiple independent models driven by click-through-rate (CTR) to controllably create attractive image or text advertisements. However, their pipelines lack cross-modal perception and rely on CTR that only reflects average preferences. Therefore, we explore jointly generating personalized image-text advertisements from historical click behaviors. We first design a Unified Advertisement Generative model (Uni-AdGen) that employs a single autoregressive framework to produce both advertising images and texts. By incorporating a foreground perception module and instruction tuning, Uni-AdGen enhances the realism of the generated content. To further personalize advertisements, we equip Uni-AdGen with a coarse-to-fine preference understanding module that effectively captures user interests from noisy multimodal historical behaviors to drive personalized generation. Additionally, we construct the first large-scale Personalized Advertising image-text dataset (PAd1M) and introduce a Product Background Similarity (PBS) metric to facilitate training and evaluation. Extensive experiments show that our method outperforms baselines in general and personalized advertisement generation. Our project is available at https://github.com/JD-GenX/Uni-AdGen.

[101] BARISTA: A Multi-Task Egocentric Benchmark for Compositional Visual Understanding cs.CVPDF

Patrick Knab, Orgest Xhelili, Inis Buzi, Drago Andres Guggiana Nilo, Mohd Saquib Khan

TL;DR: BARISTA是一个用于组合式视觉理解的多任务第一人称视角基准数据集，包含185个真实世界咖啡制作视频，提供密集的逐帧场景图注释，并从中衍生出多个零样本语言任务，旨在全面评估模型在程序性视频理解中的综合能力。

Details

Motivation: 现有基准通常单独评估物体定位、手物交互、关系解析、时间推理和步骤级程序推断等能力，难以诊断模型在程序性任务上失败的原因，因此需要一个新的综合性基准来评估和诊断模型对物理过程的理解。

Result: 实验表明，不同任务族（如短语定位、手物交互识别、指代、活动识别、关系提取和时间视觉问答）上的模型表现存在显著差异，且没有一致的占主导地位的模型家族，这使BARISTA成为一个具有挑战性的诊断性基准。

Insight: 论文的创新点在于构建了一个密集注释、多任务、基于真实世界程序性视频（咖啡制作）的第一人称视角基准，通过统一的场景图结构链接多种视觉元素，并从中系统地衍生出零样本语言任务，为全面评估和诊断视频理解模型提供了新工具。

Abstract: Scene understanding is central to general physical intelligence, and video is a primary modality for capturing both state and temporal dynamics of a scene. Yet understanding physical processes remains difficult, as models must combine object localization, hand-object interactions, relational parsing, temporal reasoning, and step-level procedural inference. Existing benchmarks usually evaluate these capabilities separately, limiting diagnosis of why models fail on procedural tasks. We introduce BARISTA, a densely annotated egocentric dataset and benchmark of 185 real-world coffee-preparation videos covering fully automatic, portafilter-based, and capsule-based workflows. BARISTA provides verified per-frame scene graphs linking persistent object identities to masks, tracks, boxes, attributes, typed relations, hand-object interactions, activities, and process steps. From these graphs, we derive zero-shot language-based tasks spanning phrase grounding, hand-object interaction recognition, referring, activity recognition, relation extraction, and temporal visual question answering. Experiments reveal strong variation across task families and no consistently dominant model family, positioning BARISTA as a challenging diagnostic benchmark for procedural video understanding. Code and dataset available at https://huggingface.co/datasets/ramblr/BARISTA.

[102] The Missing GAP: From Solving Square Jigsaw Puzzles to Handling Real World Archaeological Fragments cs.CV | cs.AIPDF

Ofir Itzhak Shahar, Gur Elkin, Ohad Ben-Shahar

TL;DR: 本文提出了GAP数据集和PuzzleFlow框架，用于解决现实世界中形状不规则的考古碎片拼图问题。GAP数据集包含合成且严重侵蚀的非方形碎片，而PuzzleFlow基于ViT和流匹配技术，在GAP上超越了现有方法。

Details

Motivation: 现有拼图求解方法大多局限于严格方形碎片，无法处理真实考古碎片的不规则形状和侵蚀问题，本文旨在填补这一空白。

Result: PuzzleFlow在GAP数据集上表现出色，优于该领域的经典和近期突出工作，实现了更优性能。

Insight: 创新点包括引入基于真实考古碎片分布学习的非方形合成数据集GAP，以及结合ViT和流匹配的PuzzleFlow框架，可推广到复杂形状的拼图求解任务。

Abstract: Jigsaw puzzle solving has been an increasingly popular task in the computer vision research community. Recent works have utilized cutting-edge architectures and computational approaches to reassemble groups of pieces into a coherent image, while achieving increasingly good results on well established datasets. However, most of these approaches share a common, restricting setting: operating solely on strictly square puzzle pieces. In this work, we introduce GAP, a set of novel jigsaw puzzles datasets containing synthetic, heavily eroded pieces of unrestricted shapes, generated by a learned distribution of real-world archaeological fragments. We also introduce PuzzleFlow, a novel ViT and Flow-Matching based framework for jigsaw puzzle solving, capable of handling complex puzzle pieces and demonstrating superior performance on GAP when compared to both classic and recent prominent works in this domain.

[103] UniCustom: Unified Visual Conditioning for Multi-Reference Image Generation cs.CVPDF

Yiyan Xu, Qiulin Wang, Wenjie Wang, Yunyao Mao, Xintao Wang

TL;DR: 本文提出UniCustom，一个统一视觉条件化框架，用于多参考图像生成任务。该框架通过早期融合ViT语义特征和VAE外观特征，使视觉语言模型能够联合编码参考对象的语义和视觉细节，从而解决现有方法中存在的属性泄漏和跨参考混淆问题。

Details

Motivation: 现有基于VLM增强的扩散模型采用解耦的视觉条件化方式，将语义特征和外观特征分开处理，导致模型在多参考图像生成中难以将语义主体与正确的视觉细节关联，出现身份保持失败和细粒度外观丢失的问题。

Result: 在两个多参考图像生成基准测试上的实验表明，UniCustom在主体一致性、指令跟随和组合保真度方面均优于强基线模型。

Insight: 创新点在于提出早期特征融合策略，将语义和外观特征在VLM编码前统一，配合两阶段训练（重建预训练和监督微调）和槽绑定正则化，有效减少跨参考纠缠，提升多参考条件下的生成质量。

Abstract: Multi-reference image generation aims to synthesize images from textual instructions while faithfully preserving subject identities from multiple reference images. Existing VLM-enhanced diffusion models commonly rely on decoupled visual conditioning: semantic ViT features are processed by the VLM for instruction understanding, whereas appearance-rich VAE features are injected later into the diffusion backbone. Despite its intuitive design, this separation makes it difficult for the model to associate each semantically grounded subject with visual details from the correct reference image. As a result, the model may recognize which subject is being referred to, but fail to preserve its identity and fine-grained appearance, leading to attribute leakage and cross-reference confusion in complex multi-reference settings. To address this issue, we propose UniCustom, a unified visual conditioning framework that fuses ViT and VAE features before VLM encoding. This early fusion exposes the VLM to both semantic cues and appearance-rich details, enabling its hidden states to jointly encode the referred subject and corresponding visual appearance with only a lightweight linear fusion layer. To learn such unified representations, we adopt a two-stage training strategy: reconstruction-oriented pretraining that preserves reference-specific appearance details in the fused hidden states, followed by supervised finetuning on single- and multi-reference generation tasks. We further introduce a slot-wise binding regularization that encourages each image slot to preserve low-level details of its corresponding reference, thereby reducing cross-reference entanglement. Experiments on two multi-reference generation benchmarks demonstrate that UniCustom consistently improves subject consistency, instruction following, and compositional fidelity over strong baselines.

[104] EchoTracker2: Enhancing Myocardial Point Tracking by Modeling Local Motion cs.CVPDF

Md Abulkalam Azad, Vegard Holmstrøm, John Nyberg, Lasse Lovstakken, Håvard Dalen

TL;DR: 本文提出了一种名为EchoTracker2的新型心肌点跟踪架构，专门用于超声心动图视频中的心肌运动估计。该模型摒弃了传统两阶段（由粗到精）架构中的粗初始化阶段，专注于精细阶段，通过融合局部时空上下文信息和长程联合时序推理来实现鲁棒跟踪。

Details

Motivation: 心肌运动与自然视频中的运动存在本质差异，它源于生理约束下的形变，在空间和时间上具有连续性，且运动轨迹通常局部受限。现有通用点跟踪方法的两阶段架构中的粗初始化步骤在该领域可能是不必要的，因此需要重新审视架构设计。

Result: 在分布内、分布外和公共合成数据集上的实验表明，与领域内最先进的模型相比，EchoTracker2将位置精度提高了6.5%，并将轨迹中值误差降低了12.2%；与最佳通用点跟踪方法相比，分别提高了2.0%和5.3%。此外，该模型与专家得出的整体纵向应变（GLS）具有更好的一致性，并提高了测试-重测的再现性。

Insight: 论文的核心创新在于针对心肌运动的特定物理约束（局部性、时空连续性），提出了一个仅包含精细阶段的简化架构。其借鉴之处在于将局部时空上下文增强与长程时序推理相结合，以替代传统的由粗到精范式，这为特定领域（如医学图像分析）的运动估计任务提供了新的架构设计思路。

Abstract: Myocardial point tracking (MPT) has recently emerged as a promising direction for motion estimation in echocardiography, driven by advances in general-purpose point tracking methods. However, myocardial motion fundamentally differs from motion encountered in natural videos, as it arises from physiologically constrained deformation that is spatially and temporally continuous throughout the cardiac cycle. Consequently, motion trajectories typically remain locally confined despite substantial tissue deformation. Motivated by these properties, we revisit the architectural design for MPT and find that coarse initialization in commonly used two-stage coarse-to-fine architectures may be unnecessary in this domain. In this work, we propose a fine-stage-only architecture, \textbf{EchoTracker2}, which enriches pixel-precise features with local spatiotemporal context and integrates them with long-range joint temporal reasoning for robust tracking. Experimental results across in-distribution, out-of-distribution (OOD), and public synthetic datasets show that our model improves position accuracy by $6.5%$ and reduces median trajectory error by $12.2%$ relative to a domain-specific state-of-the-art (SOTA) model. Compared to the best general-purpose point tracking method, the improvements are $2.0%$ and $5.3%$, respectively. Moreover, EchoTracker2 shows better agreement with expert-derived global longitudinal strain (GLS) and enhances test-rest reproducibility. Source code will be available at: https://github.com/riponazad/ptecho.

Souptik Sen, Raneen Younis, Zahra Ahmadi

TL;DR: 本文提出了一种名为CoDAAR的新型多模态学习框架，旨在解决现有方法在跨模态泛化与模态特定结构之间难以平衡的问题。该框架通过索引级对齐建立跨模态特定码本的语义共识，从而在统一离散空间中保留模态独特结构的同时实现可泛化的跨模态表示。

Details

Motivation: 当前多模态学习方法在连续（隐式）方法保留细粒度先验但泛化困难，与离散（显式）方法强制共享原型但牺牲模态特异性之间存在长期权衡。本文动机是设计一种能同时兼顾跨模态泛化能力和模态特定结构保留的新方法。

Result: 在跨模态泛化基准测试中，包括事件分类、定位、视频分割和跨数据集迁移任务，CoDAAR均取得了最先进的性能（SOTA）。

Insight: 创新点在于通过索引级对齐实现跨模态语义共识，结合离散时间对齐（DTA）进行细粒度时间量化与级联语义对齐（CSA）促进渐进式跨模态语义一致，从而构建无竞争的统一直观表示空间。这为离散且可泛化的多模态表示学习提供了新范式。

Abstract: Multimodal learning seeks to integrate information across diverse sensory sources, yet current approaches struggle to balance cross-modal generalizability with modality-specific structure. Continuous (implicit) methods preserve fine-grained priors but render generalization challenging, while discrete (explicit) approaches enforce shared prototypes at the expense of modality specificity. We introduce CoDAAR (Cross-modal Discrete Alignment And Reconstruction), a novel framework that resolves this long-standing trade-off by establishing semantic consensus across modality-specific codebooks through index-level alignment. This design uniquely allows CoDAAR to preserve modality-unique structures while achieving generalizable cross-modal representations within a unified discrete space. CoDAAR combines two complementary mechanisms: Discrete Temporal Alignment (DTA), which enables fine-grained temporal quantization, and Cascading Semantic Alignment (CSA), which promotes progressive cross-modal semantic agreement. Together, they establish a competition-free unified representation space. Trained with self-supervised reconstruction objectives on paired multimodal sequences, CoDAAR demonstrates robust cross-modal and cross-domain generalization. Across Cross-Modal Generalization benchmarks, including event classification, localization, video segmentation, and cross-dataset transfer, CoDAAR achieves state-of-the-art performance, establishing a new paradigm for discrete and generalizable multimodal representation learning.

[106] Self-Consistent Latent Reasoning: Long Latent Sequence Reasoning for Vision-Language Model cs.CVPDF

Chenfeng Wang, Wei He, Xuhan Zhu, Chunpeng Zhou, Qizhen Li

TL;DR: 本文提出了一种名为SCOLAR（Self-COnsistent LAtent Reasoning）的新方法，用于解决视觉语言模型中长潜在序列推理性能下降的问题。该方法通过引入轻量级detransformer，利用LLM的全序列隐藏状态一次性生成辅助视觉token，并结合三阶段监督微调（SFT）和ALPO强化学习，显著扩展了可接受的潜在思维链长度，在真实世界推理基准上取得了SOTA性能，并展现出强大的分布外泛化能力。

Details

Motivation: 在语言推理中，更长的思维链通常带来更好的性能，但现有视觉潜在推理方法在潜在序列变长时性能却会系统性下降。作者发现其根本原因是信息增益崩溃（Information Gain Collapse）以及过度池化的图像嵌入作为监督目标时信号不足的问题。

Result: SCOLAR将可接受的潜在CoT长度扩展了30倍以上，在真实世界推理基准上比骨干模型提升了14.12%，达到了开源模型中的最先进水平（SOTA），并展示了强大的分布外泛化能力。

Insight: 创新点在于揭示了视觉潜在推理中长序列性能下降的根源（信息增益崩溃和无效监督），并提出了通过detransformer一次性生成独立锚定于原始视觉空间的辅助token的解决方案，结合分阶段训练策略，有效实现了长序列稳健推理。

Abstract: In language reasoning, longer chains of thought consistently yield better performance, which naturally suggests that visual latent reasoning may likewise benefit from longer latent sequences. However, we discover a counterintuitive phenomenon: the performance of existing latent visual reasoning methods systematically degrades as the latent sequence grows longer. We reveal the root cause: Information Gain Collapse – autoregressive generation makes each step highly dependent on prior outputs, so subsequent tokens can barely introduce new information. We further identify that heavily pooled ($\geq 128\times$) image embeddings used as supervision targets provide no more signal than meaningless placeholders. Motivated by these insights, we propose SCOLAR (Self-COnsistent LAtent Reasoning), which introduces a lightweight detransformer that leverages the LLM’s full-sequence hidden states to generate auxiliary visual tokens in a single shot, with each token independently anchored to the original visual space. Combined with three-stage SFT and ALPO reinforcement learning, SCOLAR extends acceptable latent CoT length by over $30\times$, achieves state-of-the-art among open-source models on real-world reasoning benchmarks (+14.12% over backbone), and demonstrates strong out-of-distribution generalization.

[107] SyncDPO: Enhancing Temporal Synchronization in Video-Audio Joint Generation via Preference Learning cs.CVPDF

Xin Cheng, Xihua Wang, Ying Ba, Yuyue Wang, Kaisi Guan

TL;DR: 本文提出了一种名为SyncDPO的后训练框架，旨在通过偏好学习增强视频-音频联合生成中的时间同步性。该方法利用直接偏好优化（DPO）来提高模型对音频事件与其视觉触发之间细粒度对齐的敏感性，并通过一套基于规则的在线负样本构建策略来避免昂贵的采样-排序过程，从而高效地提供显式的负监督。

Details

Motivation: 当前视频-音频联合生成在语义对应上取得了显著成功，但实现精确的时间同步（即音频事件与其视觉触发之间的细粒度对齐）仍然是一个挑战。传统的监督微调方法中常用的均方误差损失对细微的时间错位惩罚不足，而直接偏好优化（DPO）虽然能通过引入显式错位样本来提高时间敏感性，但其依赖的采样-排序过程计算成本高昂。

Result: 在从环境声音视频到人类语音视频的四个不同基准测试上进行的大量主客观实验表明，SyncDPO在提升模型时间对齐能力方面显著优于其他方法。同时，该方法在分布外基准测试上也表现出优异的泛化能力，能够捕捉内在的运动-声音动态。

Insight: 论文的核心创新点在于提出了一种高效的基于规则的在线负样本构建策略，用于生成时间结构扭曲的视频-音频对作为负样本，从而避免了DPO中昂贵的采样-排序开销。此外，引入课程学习策略，逐步增加负样本的难度（从粗粒度错位到细微不一致），以有效强化时间对齐能力。这为后训练阶段增强多模态生成模型的时间同步性提供了一种可扩展且高效的解决方案。

Abstract: Recent advancements in video-audio joint generation have achieved remarkable success in semantic correspondence. However, achieving precise temporal synchronization, which requires fine-grained alignment between audio events and their visual triggers, remains a challenging problem. The post-training method for joint generation is largely dominated by Supervised Fine-Tuning, but the commonly used Mean Squared Error loss provides insufficient penalties for subtle temporal misalignments. Direct Preference Optimization offers an alternative by introducing explicit misaligned counterparts to better improve temporal sensitivity. In this paper we propose a post-training framework SyncDPO, leveraging DPO to improve the temporal sensitivity of V-A joint generation. Conventional DPO pipelines typically depend on costly sampling-and-ranking procedures to construct preference pairs, resulting in substantial computational cost. To improve efficiency, we introduce a suite of on-the-fly rule-based negative construction strategies that distort temporal structures without incurring additional annotation or sampling. We demonstrate that the temporal alignment capability can be effectively reinforced by providing explicit negative supervision through temporally distorted V-A pairs. Accordingly, we implement a curriculum learning strategy that progressively increases the difficulty of negative samples, transitioning from coarse misalignment to subtle inconsistencies. Extensive objective and subjective experiments across four diverse benchmarks, ranging from ambient sound videos to human speech videos, demonstrate that SyncDPO significantly outperforms other methods in improving model’s temporal alignment capability. It also demonstrates superior generalization on out-of-distribution benchmark by capturing intrinsic motion-sound dynamics. Demo and code is available in https://syncdpo.github.io/syncdpo/.

[108] Enhancing Domain Generalization in 3D Human Pose Estimation through Controllable Generative Augmentation cs.CVPDF

Xinhao Hu, Yiyi Zhang, Liqing Zhang, Jianfu Zhang

TL;DR: 本文提出了一种可控的人体姿态生成框架，通过系统性地改变姿态、背景和相机视角来合成多样化的视频数据，以增强3D人体姿态估计模型的领域泛化能力。该方法通过融合室内/真实世界和室外/虚拟数据集，构建了针对真实部署场景的增强训练数据，从而缓解了训练与测试数据分布差异导致的领域差距问题。

Details

Motivation: 解决3D人体姿态估计中因训练与测试数据分布差异（领域差距）导致的模型泛化能力不足的问题，特别是行人运动因其因果性质受领域差距影响显著。

Result: 大量实验表明，使用增强数据集显著提升了模型在未见场景和数据集上的性能，验证了所提方法的有效性。

Insight: 创新点在于提出了一个可控的生成式数据增强框架，通过系统性地操纵姿态、背景和视角来合成多样化数据，以进行跨领域数据融合，从而专门针对领域泛化问题进行数据增强，而非简单的随机增强。

Abstract: Pedestrian motion, due to its causal nature, is strongly influenced by domain gaps arising from discrepancies between training and testing data distributions. Focusing on 3D human pose estimation, this work presents a controllable human pose generation framework that synthesizes diverse video data by systematically varying poses, backgrounds, and camera viewpoints. This generative augmentation enriches training datasets, enhances model generalization, and alleviates the limitations of existing methods in handling domain discrepancies. By leveraging both indoor/real-world and outdoor/virtual datasets, we perform cross-domain data fusion and controllable video generation to construct enriched training data, tailored to realistic deployment settings. Extensive experiments show that the augmented datasets significantly improve model performance on unseen scenarios and datasets, validating the effectiveness of the proposed approach.

[109] Learning Ego-Centric BEV Representations from a Perspective-Privileged View: Cross-View Supervision for Online HD Map Construction cs.CVPDF

Daniel Lengerer, Mathias Pechinger, Klaus Bogenberger, Carsten Markgraf

TL;DR: 本文提出了一种名为跨视角监督（CVS）的表示学习范式，旨在解决基于多摄像头输入的鸟瞰图（BEV）表示在在线高精地图构建中因视角效应和空间稀疏性导致的结构推理不一致问题。该方法通过将几何和拓扑先验从与自车对齐的俯视视角（即视角特权视图）迁移到基于摄像头的BEV编码器中，在不改变推理架构或测试时引入额外输入的情况下，提升了BEV表示的结构一致性。

Details

Motivation: 现有基于多摄像头BEV表示的高精地图构建方法主要依赖自车中心（ego-centric）的监督，这需要从不完整观测、遮挡以及远距离信息密度衰减中推断大规模场景结构，导致因视角效应和空间稀疏性而难以进行一致的结构推理。

Result: 在nuScenes数据集上，使用AID4AD跨视角扩展提供的与自车对齐的航空图像进行实验。CVS方法在保持纯摄像头推理架构不变的情况下，相比StreamMapNet基线模型，在标准的60x30米区域上平均精度（mAP）提升了3.9，在扩展的100x50米区域上mAP提升了9.9，后者对应于远距离区域44%的相对性能增益。

Insight: 论文宣称的核心创新点是提出了跨视角监督（CVS）范式，利用一个视角特权（perspective-privileged）的教师模型（基于对齐的俯视图像）在共享的BEV特征空间中对齐表示并蒸馏全局一致的结构知识到自车中心的学生骨干网络中。从客观角度看，这是一种新颖的、不改变推理架构的知识蒸馏训练策略，通过引入额外的、结构信息更完整的监督信号来增强BEV表示学习，尤其对提升远距离场景的结构推理能力有显著效果。

Abstract: Bird’s-eye-view (BEV) representations derived from multi-camera input have become a central interface for online high-definition (HD) map construction. However, most approaches rely solely on ego-centric supervision, requiring large-scale scene structure to be inferred from incomplete observations, occlusions, and diminishing information density at long range, where perspective effects and spatial sparsity hinder consistent structural reasoning. We introduce Cross-View Supervision (CVS), a representation learning paradigm that transfers geometric and topological priors from an ego-aligned overhead perspective into camera-based BEV encoders. Rather than adding auxiliary semantic losses, CVS aligns representations in a shared BEV feature space and distills globally consistent structural knowledge from a perspective-privileged teacher into the ego-centric backbone. This supervision enhances structural coherence without modifying the inference architecture or requiring overhead input at test time. Experiments on nuScenes using ego-aligned aerial imagery from the AID4AD cross-view extension demonstrate consistent improvements over StreamMapNet while maintaining identical camera-only inference. CVS yields +3.9,mAP in the standard $60\times30,\mathrm{m}$ region and +9.9,mAP in the extended $100\times50,\mathrm{m}$ setting, corresponding to a 44% relative gain at long range. These results highlight perspective-privileged structural supervision as a promising training principle for improving BEV representation learning in HD map construction.

[110] UHR-Micro: Diagnosing and Mitigating the Resolution Illusion in Earth Observation VLMs cs.CVPDF

Shuo Ni, Tong Wang, Jing Zhang, He Chen, Haonan Guo

TL;DR: 本文针对超高分辨率（UHR）地球观测图像中视觉语言模型（VLMs）存在的‘分辨率幻觉’问题，即高分辨率输入未能可靠感知微尺度目标，提出了UHR-Micro基准测试和Micro-evidence Active Perception（MAP）方法。UHR-Micro包含11,253条基于1,212张UHR图像的指令，用于评估VLMs在微尺度目标上的性能；MAP-Agent通过将查询分解为证据搜索步骤、主动检查候选区域并基于局部观察进行回答，以证据为中心提升微尺度感知能力。

Details

Motivation: 解决地球观测VLMs中大规模场景上下文与微尺度目标之间的严重尺度不匹配问题，即‘分辨率幻觉’——高分辨率输入虽提供更丰富的视觉细节，但未必能可靠感知空间上小且任务相关的证据。

Result: 在UHR-Micro基准上对代表性高分辨率VLMs的实验显示，尽管有高分辨率输入，模型在空间定位和证据解析方面仍存在显著失败；进一步分析表明，仅增加模型容量无法完全解决此问题。提出的MAP-Agent通过证据中心的推理改善了微尺度感知性能。

Insight: 创新点在于诊断并缓解了地球观测VLMs中的‘分辨率幻觉’，通过引入UHR-Micro基准进行细粒度错误归因，并设计MAP-Agent以证据为中心的主动感知框架，将高分辨率推理从图像中心转向证据中心，为评估和推进高分辨率推理提供了诊断平台。

Abstract: Vision-Language Models (VLMs) increasingly operate on ultra-high-resolution (UHR) Earth observation imagery, yet they remain vulnerable to a severe scale mismatch between large-scale scene context and micro-scale targets. We refer to this empirical gap as a “resolution illusion”: higher input resolution provides the appearance of richer visual detail, but does not necessarily yield reliable perception of spatially small, task-relevant evidence. To benchmark this challenge, we introduce UHR-Micro, a benchmark comprising 11,253 instructions grounded in 1,212 UHR images, designed to evaluate VLMs at the spatial limits of native Earth observation imagery. UHR-Micro spans diverse micro-target scales, context requirements, task families, and visual conditions, and provides diagnostic annotations that support controlled evaluation and fine-grained error attribution. Experiments with representative high-resolution VLMs show substantial failures in spatial grounding and evidence parsing, despite access to high-resolution inputs. Further analysis suggests that these failures are not fully resolved by increasing model capacity, but are closely tied to insufficient guidance in locating and using task-relevant micro-evidence. Motivated by this finding, we propose Micro-evidence Active Perception (MAP), a reference agent that decomposes queries into evidence-seeking steps, actively inspects candidate regions, and grounds its answers in localized observations. MAP-Agent improves micro-level perception by making high-resolution reasoning evidence-centered rather than image-centered. Together, UHR-Micro and MAP-Agent provide a diagnostic platform for evaluating, understanding, and advancing high-resolution reasoning in Earth observation VLMs. Datasets and source code were released at https://github.com/MiliLab/UHR-Micro.

[111] H3D-MarNet: Wavelet-Guided Dual-Path Learning for Metal Artifact Suppression and CT Modality Transformation for Radiotherapy Workflows cs.CVPDF

Mubashara Rehman, Niki Martinel, Michele Avanzo, Riccardo Spizzo, Christian Micheloni

TL;DR: H3D-MarNet是一个两阶段框架，用于抑制CT图像中的金属伪影并实现从千伏CT到兆伏CT的模态转换。第一阶段通过基于小波的预处理模块进行频率感知去噪以抑制伪影，第二阶段通过结合CNN和Transformer的混合体素学习架构进行域转换，最终重建出伪影抑制的MVCT体积图像。

Details

Motivation: 解决CT图像中金属植入物引起的高密度伪影问题，该问题会严重降低图像质量，影响诊断准确性和放射治疗计划，特别是在癌症患者中。

Result: 在完整数据集的受伪影影响切片上，H3D-MarNet取得了28.14 dB的PSNR和0.717的SSIM，表明其能有效抑制金属伪影并保持解剖结构，在临床放疗工作流中具有可靠CT模态转换的潜力。

Insight: 创新点包括：1) 两阶段设计，先进行小波引导的频域伪影抑制预处理，再进行模态转换；2) 混合架构Domain-TransNet，结合CNN编码器捕捉局部细节和Transformer编码器建模长程体素依赖，并通过注意力机制融合特征；3) 采用深度监督的多阶段注意力引导解码器进行渐进式重建。这为医学图像去伪影和跨模态转换提供了可借鉴的频域处理与多模态特征融合思路。

Abstract: Metal artifacts in computed tomography (CT) severely degrade image quality, compromising diagnostic accuracy and radiotherapy planning, especially in cancer patients with high-density implants. We propose H3D-MarNet, a two-stage framework for artifact-aware CT domain transformation from kilo-voltage CT (kVCT) to mega-voltage CT (MVCT). In the first stage, a wavelet-based preprocessing module suppresses metal-induced artifacts through frequency-aware denoising while preserving anatomical structures. In second stage, Domain-TransNet performs kVCT-to-MVCT domain transformation using a hybrid volumetric learning architecture. Domain-TransNet integrates a CNN-based encoder to capture fine-grained local anatomical details and a transformer-based encoder to model long-range volumetric dependencies. The complementary representations are fused through an attention-based feature fusion mechanism to ensure spatial and contextual coherence across slices. A multi-stage, attention-guided decoder, supported by deep supervision, progressively reconstructs artifact-suppressed MVCT volumes. Extensive experiments demonstrate that H3D-MarNet achieves 28.14 dB PSNR and 0.717 SSIM on artifact-affected slices from full dataset, indicating effective metal artifact suppression and anatomical preservation, highlighting its potential for reliable CT modality transformation in clinical radiotherapy workflows.

[112] Beyond Text Prompts: Visual-to-Visual Generation as A Unified Paradigm cs.CVPDF

Yaofang Liu, Kangning Cui, Meng Chu, Zhaoqing Li, Suiyun Zhang

TL;DR: 该论文提出了一种名为视觉到视觉（V2V）生成的新范式，旨在让用户通过视觉规格页面（如草图、参考图像）而非文本提示来指导生成模型。作者提出了一个无需训练的框架V2V-Zero，它利用现有视觉语言模型的隐藏状态，将视觉页面映射到生成器的条件空间中，从而在多种视觉条件任务上实现了有效的生成控制。

Details

Motivation: 当前视觉生成模型主要依赖文本提示，这限制了用户表达复杂视觉意图（如空间结构、精确外观、字形）的能力。论文的动机是绕过文本瓶颈，允许用户直接使用视觉文档作为生成条件，以更自然、更丰富的方式指定生成内容。

Result: 在GenEval基准上，V2V-Zero使用冻结的Qwen-Image骨干网络达到了0.85分，与其优化的文本到图像性能相当。在作者新提出的Simple-V2V Bench（涵盖7个视觉条件任务和7个模型）上，V2V-Zero得分为32.7/100，超过了评估的开源图像基线模型，并揭示了能力层次：属性绑定较强，内容生成不可靠，结构控制即使对商业系统也较难。视频扩展（HunyuanVideo-1.5）得分为20.2/100。

Insight: 论文的核心创新点在于提出了V2V生成范式，并实现了一个无需训练、即插即用的框架V2V-Zero，它巧妙地利用了现有视觉语言模型的多模态对齐能力，将视觉页面的隐藏状态直接用作生成条件。从客观角度看，其创新之处在于将视觉条件视为一种统一的接口，并证明了该范式在图像和视频生成中的可行性和通用性，同时通过机制分析揭示了其工作主要依赖于视觉路由（95%的注意力在视觉页面隐藏状态上）。

Abstract: Humans often specify and create through visual artifacts: typography sheets, sketches, reference images, and annotated scenes. Yet modern visual generators still ask users to serialize this intent into text, a bottleneck that compresses signals like spatial structure, exact appearance, and glyph shape. We propose \textbf{\emph{visual-to-visual} (V2V)} generation, in which the user conditions a generative model with a visual specification page rather than a text prompt. The page is not an edit target, but a visual document that specifies the desired output. We introduce \textbf{V2V-Zero}, a training-free framework that exposes this interface in existing vision-language model (VLM) conditioned generators by replacing text-only conditioning with final-layer hidden states extracted from visual pages, exploiting the fact that the frozen VLM already maps both text and images into the generator’s conditioning space. On GenEval, V2V-Zero reaches 0.85 with a frozen Qwen-Image backbone, closely matching its optimized text-to-image performance without fine-tuning. To evaluate the broader V2V space, we introduce \textbf{Simple-V2V Bench}, spanning seven visual-conditioning tasks and seven models, including GPT Image 2, Nano Banana 2, Seedream 5.0 Lite, open-weight baselines, and a video extension. V2V-Zero scores 32.7/100, outperforming evaluated open-weight image baselines and revealing a clear capability hierarchy: attribute binding is strong, content generation is unreliable, and structural control remains hard even for commercial systems. A HunyuanVideo-1.5 extension scores 20.2/100, showing the interface transfers beyond images. Mechanistic analysis shows the default reasoning path is primarily visually routed, with 95.0% of conditioning-token attention mass on visual-page hidden states.

[113] Large-Small Model Collaboration for Farmland Semantic Change Detection cs.CVPDF

Xinjia Li, Rui Wang, Qiurong Peng, Lingfei Ye, Dengrong Zhang

TL;DR: 本文针对农田语义变化检测（SCD）中存在的细粒度标注不足和物候伪变化干扰问题，构建了HZNU-FCD大规模细粒度农田SCD基准数据集，并提出了一种大-小模型协作的SCD框架。该框架结合了用于学习密集变化表示的小型视觉模型FD-Mamba，以及利用CLIP先验进行语义仲裁和伪变化抑制的大型视觉-语言模型通路CMLA，并通过硬区域协同训练策略实现有效协作。

Details

Motivation: 现有农田SCD基准缺乏专门的“从-到”细粒度标注，且视觉变化检测模型易受作物轮作、季节变化和光照差异引起的物候伪变化干扰，难以进行精细的耕地转换监测。

Result: 在自建的HZNU-FCD基准上，该方法取得了97.63% F1、96.32% IoU和96.35% SCD_IoU_mean的SOTA结果，仅需6.65M可训练参数。相比利用视觉-语言信息的ChangeCLIP-ViT，F1提升了10.19个百分点。在LEVIR-CD和WHU-CD基准上也表现出强大的鲁棒性和泛化能力。

Insight: 创新点在于构建了具有统一五类农田-非农田标注协议的大规模细粒度农田SCD数据集，并提出了一个新颖的大-小模型协作框架。该框架通过任务驱动的小模型处理密集视觉细节，利用冻结的大模型提供跨模态语义先验进行逻辑仲裁，并通过仅对低置信度像素进行监督的硬区域协同训练策略实现高效协作，有效抑制了伪变化并提升了细粒度检测性能。

Abstract: Farmland Semantic Change Detection (SCD) is essential for cultivated land protection, yet existing benchmarks and models remain insufficient for fine-grained farmland conversion monitoring. Current datasets often lack dedicated “from-to” annotations, while visual change detection models are easily disturbed by phenology-induced pseudo-changes caused by crop rotation, seasonal variation, and illumination differences. To address these challenges, we construct HZNU-FCD, a large-scale fine-grained farmland SCD benchmark with a unified five-class farmland-to-non-farmland annotation protocol. It contains 4,588 bitemporal image pairs with pixel-level labels for practical farmland protection. Based on this benchmark, we propose a large-small collaborative SCD framework that integrates a task-driven small visual model with a frozen large vision-language model. The small model, Fine-grained Difference-aware Mamba (FD-Mamba), learns dense change representations for boundary preservation and small-region localization. The large-model pathway, Cross-modal Logical Arbitration (CMLA), introduces CLIP-based textual priors for prompt-guided semantic arbitration and pseudo-change suppression. To enable effective collaboration, we design a hard-region co-training strategy that supervises the CMLA semantic score map only on low-confidence pixels. Experiments show that our method achieves 97.63% F1, 96.32% IoU, and 96.35% SCD_IoU_mean on HZNU-FCD with only 6.65M trainable parameters. Compared with the multimodal ChangeCLIP-ViT, which leverages vision-language information for change detection, our method improves F1 by 10.19 percentage points on HZNU-FCD. It also achieves 91.43% F1 and 84.21% IoU on LEVIR-CD, and 93.85% F1 and 88.41% IoU on WHU-CD, demonstrating strong robustness and generalization. The code is available at https://github.com/Lovelymili/FD-Mamba.

[114] Images in Sentences: Scaling Interleaved Instructions for Unified Visual Generation cs.CVPDF

Yabo Zhang, Kunchang Li, Dewei Zhou, Xinyu Huang, Xun Wang

TL;DR: 本文提出INSET模型，将图像作为原生词汇嵌入文本指令中，实现统一视觉生成。通过将视觉特征直接置于语义槽位，利用Transformer的上下文局部性进行精确对象绑定，并构建了包含1500万高质量交错样本的数据引擎。在InterleaveBench评估中，INSET在多图像一致性和文本对齐方面显著优于现有方法，且能自然扩展到多模态图像编辑。

Details

Motivation: 现有多模态语言模型在处理复杂交错指令时性能受限，源于图像与文本的结构分离导致长距离依赖匹配困难。

Result: 在InterleaveBench基准测试中，INSET显著超越SOTA方法，尤其在输入复杂度增加时性能差距扩大，在多图像一致性和文本对齐方面表现优异。

Insight: 创新点在于将图像视为密集表达性语言标记嵌入指令，利用Transformer的上下文局部性实现精确视觉-语言对齐；数据引擎通过合成大规模高质量交错样本支持模型训练，可扩展至创造性视觉编辑任务。

Abstract: While recent advancements in multimodal language models have enabled image generation from expressive multi-image instructions, existing methods struggle to maintain performance under complex interleaved instructions. This limitation stems from the structural separation of images and text in current paradigms, which forces models to bridge difficult long-range dependencies to match descriptions with visual targets. To address these challenges, we propose \texttt{I}mages i\texttt{N} \texttt{SE}n\texttt{T}ences (\textit{a.k.a}, INSET), a unified generation model that seamlessly embeds images as native vocabulary within textual instructions. By positioning visual features directly at their corresponding semantic slots, INSET leverages the contextual locality of transformers for precise object binding, effectively treating images as dense, expressive language tokens. Furthermore, we introduce a scalable data engine that synthesizes 15M high-quality interleaved samples from standard image and video datasets, utilizing VLMs and LLMs to construct rich, long-horizon sequences. Evaluation results on InterleaveBench demonstrate that INSET significantly outperforms state-of-the-art methods in multi-image consistency and text alignment, with performance gaps widening as input complexity increases. Beyond standard generation, our approach inherently extends to multimodal image editing, integrating visual content as part of the instruction to facilitate highly expressive and creative visual manipulations.

[115] G$^2$TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models cs.CVPDF

Junxian Li, Kai Liu, Zizhong Ding, Zhixin Wang, Zhikai Chen

TL;DR: 本文提出G²TR框架，用于解决分离编码器统一多模态模型（UMMs）中视觉令牌处理计算成本高的问题。该方法通过生成分支引导的视觉令牌缩减，在理解阶段筛选并合并冗余视觉令牌，显著降低计算量同时保持模型的理解和编辑能力。

Details

Motivation: 现有视觉令牌缩减方法主要面向判别式推理任务，假设最终目标是判别性推理，但UMMs需要同时保持图像编辑能力，因此需要新的令牌缩减方法。

Result: 在图像理解和编辑基准测试中，G²TR将视觉令牌和预填充计算量减少1.94倍，同时保持推理准确性和编辑质量，在几乎所有基准测试中优于基线方法。

Insight: 创新点在于利用生成分支提供任务无关的信号来识别对潜在空间图像重建重要的视觉令牌，通过VAE潜在一致性估计令牌重要性，并进行平衡令牌选择和合并。该方法无需训练、即插即用，且兼容现有UMM推理流程。

Abstract: The development of separate-encoder Unified multimodal models (UMMs) comes with a rapidly growing inference cost due to dense visual token processing. In this paper, we focus on understanding-side visual token reduction for improving the efficiency of separate-encoder UMMs. While this topic has been widely studied for MLLMs, existing methods typically rely on attention scores, text-image similarity and so on, implicitly assuming that the final objective is discriminative reasoning. This assumption does not hold for UMMs, where understanding-side visual tokens must also preserve the model’s capabilities for editing images. We propose G$^2$TR, a generation-guided visual token reduction framework for separate-encoder UMMs. Our key insight is that the generation branch provides a task-agnostic signal for identifying understanding-side visual tokens that are not only semantically relevant but also important for latent-space image reconstruction and generation. G$^2$TR estimates token importance from consistency with VAE latent, performs balanced token selection, and merges redundant tokens into retained representatives to reduce information loss. The method is training-free, plug-and-play, and applied only after the understanding encoding stage, making it compatible with existing UMM inference pipelines. Experiments on image understanding and editing benchmarks show that G$^2$TR substantially reduces visual tokens and prefill computation by 1.94x while maintaining both reasoning accuracy and editing quality, outperforming baselines on almost all benchmarks.

[116] Contrastive Learning under Noisy Temporal Self-Supervision for Colonoscopy Videos cs.CVPDF

Luca Parolari, Pietro Gori, Lamberto Ballan, Carlo Biffi, Loic Le Folgoc

TL;DR: 本文提出了一种针对结肠镜视频的噪声感知对比学习方法，通过利用结肠镜检查的时序结构生成自监督关联，并设计噪声感知对比损失来处理这些关联中的噪声，从而学习稳健的息肉轨迹表示。该方法在息肉检索与重识别、大小估计和组织学分类等多个下游任务中表现出色。

Details

Motivation: 学习稳健的息肉轨迹表示对于多种AI辅助结肠镜应用至关重要，但监督对比学习依赖正确的正负样本定义，而收集这些标签成本高且需要临床专业知识。因此，本文旨在利用结肠镜检查的时序结构进行自监督学习，并处理由此产生的噪声关联。

Result: 该方法在多个下游任务（息肉检索与重识别、大小估计和组织学分类）上超越了先前的自监督和监督基线，并与近期的基础模型相当或更优，且仅使用27个视频训练轻量编码器。

Insight: 创新点在于利用结肠镜检查的时序工作流程生成自监督关联，并引入噪声感知对比损失来鲁棒地处理这些关联中的噪声，从而在数据有限且标注成本高的医疗视频领域实现有效的表示学习。

Abstract: Learning robust representations of polyp tracklets is key to enabling multiple AI-assisted colonoscopy applications, from polyp characterization to automated reporting and retrieval. Supervised contrastive learning is an effective approach for learning such representations, but it typically relies on correct positive and negative definitions. Collecting these labels requires linking tracklets that depict the same underlying polyp entity throughout the video, which is costly and demands specialized clinical expertise. In this work, we leverage the sequential workflow of colonoscopy procedures to derive self-supervised associations from temporal structure. Since temporally derived associations are not guaranteed to be correct, we introduce a noise-aware contrastive loss to account for noisy associations. We demonstrate the effectiveness of the learned representations across multiple downstream tasks, including polyp retrieval and re-identification, size estimation, and histology classification. Our method outperforms prior self-supervised and supervised baselines, and matches or exceeds recent foundation models across all tasks, using a lightweight encoder trained on only 27 videos. Code is available at https://github.com/lparolari/ntssl.

[117] VIP: Visual-guided Prompt Evolution for Efficient Dense Vision-Language Inference cs.CVPDF

Hao Zhu, Shuo Jin, Wenbin Liao, Jiayu Xiao, Yan Zhu

TL;DR: 本文提出了一种名为VIP（Visual-guided Prompt Evolution）的方法，旨在解决CLIP模型固有的空间偏差问题，以实现无需训练、高效且泛化性强的开放词汇语义分割。该方法基于空间感知的dino.txt框架，通过视觉引导的提示词演化来纠正文本查询的语义模糊性，从而提升细粒度物体感知能力。

Details

Motivation: 现有基于CLIP的开放词汇语义分割方法存在空间偏差和效率问题，而dino.txt框架虽具有空间感知能力，但其文本查询的语义模糊性导致密集跨模态交互中的严重不匹配。本文旨在克服这些限制，开发一种更高效、高质量的密集预测方法。

Result: 在广泛的评估中，VIP方法在平均mIoU上超越了当前领先方法1.4%至8.4%，在多个挑战性领域表现出良好的泛化能力，并且仅需边际推理时间和内存开销。

Insight: 创新点在于提出视觉引导的提示词演化机制，结合别名扩展和视觉引导蒸馏来挖掘鲁棒的语义线索，并以显著性感知方式聚合，从而提升预测的保真度。这为无需训练的密集视觉-语言推理提供了一种高效且通用的解决方案。

Abstract: Pursuing training-free open-vocabulary semantic segmentation in an efficient and generalizable manner remains challenging due to the deep-seated spatial bias in CLIP. To overcome the limitations of existing solutions, this work moves beyond the CLIP-based paradigm and harnesses the recent spatially-aware dino.txt framework to facilitate more efficient and high-quality dense prediction. While dino.txt exhibits robust spatial awareness, we find that the semantic ambiguity of text queries gives rise to severe mismatch within its dense cross-modal interactions. To address this, we introduce \textcolor{oursblue}{\textbf{VI}}sual-guided \textcolor{oursblue}{\textbf{P}}rompt evolution (\textcolor{oursblue}{\textbf{\textit{VIP}}}) to rectify the semantic expressiveness of text queries in dino.txt, unleashing its potential for fine-grained object perception. Towards this end, \VIP integrates alias expansion with a visual-guided distillation mechanism to mine valuable semantic cues, which are robustly aggregated in a saliency-aware manner to yield a high-fidelity prediction. Extensive evaluations demonstrate that \VIP: \ding{182} surpasses the top-leading methods by $1.4% \sim 8.4%$ average mIoU, \ding{183} generalizes well to diverse challenging domains, and \ding{184} requires marginal inference time and memory overhead. \href{https://github.com/MiSsU-HH/VIP}{Our code is publicly available at GitHub \faGithub}.

[118] Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models cs.CV | cs.AI | cs.LGPDF

Yanting Miao, Yutao Sun, Dexin Wang, Mengyu Zhou, Pascal Poupart

TL;DR: 本文提出了一种名为GAP（Granular Alignment Paradigm）的细粒度对齐范式，用于解决多模态大语言模型（MLLM）在视觉潜在推理中因特征空间不匹配而导致性能不稳定的问题。该方法通过在特征、上下文和容量三个层面进行对齐，稳定并提升了模型生成中间视觉证据的能力。

Details

Motivation: 现有视觉潜在推理方法通常遵循输出即输入的潜在范式，并重用解码器隐藏状态作为预测的潜在输入。然而，这些状态与模型训练时所消费的输入嵌入处于不同的范数体系，导致特征空间不匹配，使得直接的潜在反馈不可靠，从而造成性能不稳定。

Result: 在Qwen2.5-VL 7B模型上，应用GAP的模型在所有监督变体中取得了最佳的感知与推理平均聚合性能。推理时干预探测进一步表明，生成的潜在特征提供了超越单纯增加令牌槽的任务相关视觉信号。

Insight: 论文的核心创新点是提出了一个三层次的细粒度对齐范式（GAP）：特征级对齐通过轻量级PCA对齐潜在头将解码器输出映射为与输入兼容的视觉潜在特征；上下文级对齐通过可检查的辅助视觉监督来锚定潜在目标；容量引导对齐则选择性地将潜在监督分配给基础MLLM难以处理的样本。这为解决MLLM内部特征不匹配问题提供了一个系统性的框架。

Abstract: Visual latent reasoning lets a multimodal large language model (MLLM) create intermediate visual evidence as continuous tokens, avoiding external tools or image generators. However, existing methods usually follow an output-as-input latent paradigm and yield unstable gains. We identify evidence for a feature-space mismatch that can contribute to this instability: dominant visual-latent models build on pre-norm MLLMs and reuse decoder hidden states as predicted latent inputs, even though these states occupy a substantially different norm regime from the input embeddings the model was trained to consume~\citep{xie2025mhc,li2026siamesenorm,team2026attention}. This mismatch can make direct latent feedback unreliable. Motivated by this diagnosis, we propose \textbf{GAP}, a \textbf{G}ranular \textbf{A}lignment \textbf{P}aradigm for visual latent modeling. GAP aligns visual latent reasoning at three levels: feature-level alignment maps decoder outputs into input-compatible visual latents through a lightweight PCA-aligned latent head; context-level alignment grounds latent targets with inspectable auxiliary visual supervision; and capacity-guided alignment assigns latent supervision selectively to examples where the base MLLM struggles. On Qwen2.5-VL 7B, the resulting model achieves the best mean aggregate perception and reasoning performance among our supervised variants. Inference-time intervention probing further suggests that generated latents provide task-relevant visual signal beyond merely adding token slots.

[119] Beyond Localization: A Comprehensive Diagnosis of Perspective-Conditioned Spatial Reasoning in MLLMs from Omnidirectional Images cs.CVPDF

Yuangong Chen, Wai Keung Wong, Jiaxing Li, Ioannis Patras, Xu Zheng

TL;DR: 本文研究了多模态大语言模型（MLLMs）在360度全景图像中进行视角条件空间推理（PCSR）的能力，发现其存在显著的感知-推理鸿沟。作者提出了诊断性基准PCSR-Bench，并评估了14个代表性MLLM，结果显示其在基础感知任务上表现尚可，但在高级PCSR任务上性能急剧下降。通过基于强化学习的诊断研究，作者发现PCSR能力具有部分可塑性，可通过针对性优化得到有限但有效的提升。

Details

Motivation: 尽管MLLMs展现出强大的视觉感知能力，但在视角变化下的空间推理能力仍然有限。本文旨在系统诊断MLLMs在全景图像中处理视角条件空间推理这一挑战性任务的能力。

Result: 在提出的PCSR-Bench基准上评估了14个代表性MLLM。基础感知任务（如相对方向）准确率可达57.59%，但高级PCSR任务表现很差：以自我为中心的旋转任务准确率降至13.49%，以自我为中心的畸变任务为7.13%，开放式组合推理任务仅为0.64%。通过基于强化学习的奖励塑形，一个7B规模的模型在受控设置下准确率从31.10%提升至60.06%。

Insight: 论文的创新点在于：1）明确定义并系统研究了视角条件空间推理（PCSR）这一关键瓶颈问题；2）构建了大规模、多任务的诊断性基准PCSR-Bench，涵盖从基础感知到高级推理的八个任务；3）通过基于强化学习的诊断研究，首次揭示了MLLMs的PCSR能力具有部分可塑性，而非完全固化，为针对性优化提供了依据。从客观角度看，该研究为理解和提升MLLMs的空间推理能力提供了重要的评估框架和初步的改进路径。

Abstract: Multimodal Large Language Models (MLLMs) show strong visual perception, yet remain limited in reasoning about space under changing viewpoints. We study this challenge as Perspective-Conditioned Spatial Reasoning (PCSR) in 360-degree omnidirectional images, where broad scene coverage reduces ambiguity from partial observations without eliminating the need for viewpoint-dependent inference. To assess this capability, we introduce PCSR-Bench, a diagnostic benchmark of 84,373 question-answer pairs from 2,600 omnidirectional images across 26 indoor environments. PCSR-Bench contains eight tasks spanning foundational perception (e.g., object counting, relative distance, and relative direction) and advanced PCSR, including compositional chains, egocentric rotation, perspective re-anchoring, ego-distortion, and limited-FOV visibility. We evaluate 14 representative MLLMs and observe a substantial perception-reasoning gap: accuracy reaches 57.59% on foundational relative direction, but drops to 13.49% on egocentric rotation, 7.13% on egocentric distortion, and 0.64% on open-ended compositional reasoning. To probe the plasticity of this gap, we conduct an RL-based diagnostic study on a 7B-scale model. Reward shaping improves a matched 7B baseline from 31.10% to 60.06% under a controlled setting, suggesting that PCSR is partial plasticity rather than being fully immutable. Still, the gains are task-selective, sensitive to reward design including both weight allocation and reward formulation, and partially dependent on the evaluation protocol. These results position PCSR as a key bottleneck in current MLLMs and highlight limited but meaningful room for recovery under targeted optimization.

[120] AOI-SSL: Self-Supervised Framework for Efficient Segmentation of Wire-bonded Semiconductors In Optical Inspection cs.CVPDF

Joaquín Figueira, Rob Van Gastel, Giacomo D’Amicantonio, Zhuoran Liu, Ioan Gabriel Bucur

TL;DR: 本文提出了AOI-SSL框架，用于焊线半导体光学检测中的高效语义分割。该框架结合了针对小领域数据的自监督预训练和上下文推理，旨在减少对标注数据的依赖，并实现对新设备或数据分布变化的快速适应。

Details

Motivation: 解决焊线半导体自动光学检测中分割模型通常针对特定设备、在新设备或数据分布变化时需要重新训练的问题，旨在开发一个训练高效、标注需求少的框架。

Result: 在小型工业检测数据集上，自监督预训练（特别是掩码自编码器）显著提升了下游分割质量，优于从头训练和ImageNet预训练主干网络。基于检索的分割在针对单一设备图像时表现优于微调，可实现近即时适应。

Insight: 创新点在于将小领域自监督预训练与上下文推理（特别是基于简单相似性的补丁级检索）结合，为工业视觉任务提供了一种高效、低标注依赖的解决方案。该方法表明，在此设定下，简单的基于相似性的检索与更复杂的基于注意力的聚合方法性能相当。

Abstract: Segmentation models in automated optical inspection of wire-bonded semiconductors are typically device-specific and must be re-trained when new devices or distribution shifts appear. We introduce AOI-SSL, a training-efficient framework for semantic segmentation of wire-bonded semiconductors by combining small-domain self-supervised pre-training of vision transformers with in-context inference that minimizes the need of labeled examples. We pre-train SOTA self-supervised algorithms in a small industrial inspection dataset and find that Masked Autoencoders are the most effective in this small-data setting, improving downstream segmentation while reducing the labeled fine-tuning effort. We further introduce in-context, patch-level retrieval methods that predict masks directly from dense encoder embeddings with negligible additional training. We show that, in this setting, simple similarity-based retrieval performs on par with more complex attention-based aggregation used currently in the literature. Furthermore, our experiments demonstrate that self-supervised pre-training significantly improves segmentation quality compared to training from scratch and to ImageNet pre-trained backbones under a fixed fine-tuning computational budget. Finally, the results reveal that retrieval based segmentation outperforms fine-tuning when targeting single device images, allowing for near-instant adaptation to difficult samples.

[121] GaitProtector: Impersonation-Driven Gait De-Identification via Training-Free Diffusion Latent Optimization cs.CVPDF

Huiran Duan, Qian Zhou, Zhongliang Guo, Junhao Dong, Yuqi Li

TL;DR: 本文提出了GaitProtector，一种基于模仿驱动的步态去身份识别框架，旨在解决传统方法在身份抑制和时空保真度之间的权衡问题。该方法将隐私保护表述为一个统一目标，包含混淆（使受保护步态远离源身份）和模仿（使其接近选定的目标身份）两个紧密耦合的组件。通过利用预训练的3D视频扩散模型先验，以无需训练的方式优化潜在代码，生成在视觉和时间质量上均保持良好、同时有效保护隐私的步态序列。

Details

Motivation: 传统步态去身份识别方法存在固有权衡：要么身份抑制不足，要么引入时空失真，影响对结构敏感的下游应用。本文旨在通过一个统一框架同时实现强身份抑制和保持步态结构合理性。

Result: 在CASIA-B数据集上的实验表明，该方法在黑盒步态识别下实现了56.7%的模仿成功率，并将Rank-1识别准确率从89.6%降至15.0%，同时保持了良好的视觉和时间质量。在下游Scoliosis1K数据集上，诊断准确率仅从91.4%降至74.2%。

Insight: 核心创新点在于将去身份识别形式化为一个包含混淆和模仿的联合优化目标，并利用目标身份作为语义锚点，在预训练扩散先验的约束下引导优化，从而生成结构合理的步态。技术实现上，首次以无需训练的方式，利用预训练的3D扩散模型先验进行基于轮廓的步态去身份识别，避免了为每个数据集重新训练生成器，提高了通用性和效率。

Abstract: Conventional gait de-identification methods often encounter an inherent trade-off: they either provide insufficient identity suppression or introduce spatiotemporal distortions that impede structure-sensitive downstream applications. We propose GaitProtector, an impersonation-driven gait de-identification framework that formulates privacy protection as a unified objective with two tightly coupled components: (i) obfuscation, which repels the protected gait from the source identity, and (ii) impersonation, which attracts it toward a selected target identity. The target identity serves as a semantic anchor that biases optimization toward structurally plausible gait patterns under the pretrained diffusion prior, helping preserve dominant body shape and motion dynamics. We instantiate this idea through a training-free diffusion latent optimization pipeline. Instead of retraining a generator for each dataset, we invert each input silhouette sequence into the latent trajectory of a pretrained 3D video diffusion model and iteratively optimize latent codes with a differentiable adversarial objective to synthesize protected gaits. Experiments on the CASIA-B dataset show that GaitProtector achieves a 56.7% impersonation success rate under black-box gait recognition and reduces Rank-1 identification accuracy from 89.6% to 15.0%, while maintaining favorable visual and temporal quality. We further evaluate downstream utility on the Scoliosis1K dataset, where diagnostic accuracy decreases only from 91.4% to 74.2%. To the best of our knowledge, this work is the first to leverage pretrained 3D diffusion priors in a training-free manner for silhouette-based gait de-identification.

[122] LychSim: A Controllable and Interactive Simulation Framework for Vision Research cs.CVPDF

Wufei Ma, Chloe Wang, Siyi Chen, Jiawei Peng, Patrick Li

TL;DR: 本文介绍了LychSim，一个基于Unreal Engine 5构建的高度可控、交互式的仿真框架，旨在降低计算机视觉研究中仿真工具的使用门槛。它通过简化的Python API、程序化数据流水线以及原生集成Model Context Protocol（MCP），支持生成多样化的高保真环境与丰富的2D/3D真值数据，并可作为闭环优化和OOD评估的平台，应用于合成数据生成、强化学习对抗性测试及语言驱动的场景布局生成等多个下游任务。

Details

Motivation: 当前仿真平台技术门槛高，需要大量计算机图形学和游戏开发专业知识，这限制了其在视觉系统闭环优化和严格分布外（OOD）评估中的应用。LychSim旨在弥合这一差距，提供一个易于使用且功能强大的仿真工具。

Result: 论文通过多个下游应用展示了LychSim的能力，包括作为合成数据引擎、支持基于强化学习的对抗性审查器以及实现交互式语言驱动的场景布局生成。框架将公开源代码和各种数据标注，以惠及更广泛的视觉社区。

Insight: 主要创新点包括：1) 通过简化的Python API抽象底层引擎复杂性，降低使用门槛；2) 程序化数据流水线能生成具有多样化OOD视觉挑战的高保真环境及丰富真值；3) 原生集成MCP协议，将仿真器转变为支持推理型智能体大语言模型（LLMs）的动态闭环实验场。此外，通过标注场景级程序规则和对象级姿态对齐，实现了语义对齐的3D真值和自动化场景修改，增强了仿真的可控性和实用性。

Abstract: While self-supervised pretraining has reduced vision systems’ reliance on synthetic data, simulation remains an indispensable tool for closed-loop optimization and rigorous out-of-distribution (OOD) evaluation. However, modern simulation platforms often present steep technical barriers, requiring extensive expertise in computer graphics and game development. In this work, we present LychSim, a highly controllable and interactive simulation framework built upon Unreal Engine 5 to bridge this gap. LychSim is built around three key designs: (1) a streamlined Python API that abstracts away underlying engine complexities; (2) a procedural data pipeline capable of generating diverse, high-fidelity environments with varying out-of-distribution (OOD) visual challenges, paired with rich 2D and 3D ground truths; and (3) a native integration of the Model Context Protocol (MCP) that transforms the simulator into a dynamic, closed-loop playground for reasoning agentic LLMs. We further annotate scene-level procedural rules and object-level pose alignments to enable semantically aligned 3D ground truths and automated scene modification. We demonstrate LychSim’s capability across multiple downstream applications, including serving as a synthetic data engine, powering reinforcement learning-based adversarial examiners, and facilitating interactive, language-driven scene layout generation. To benefit the broader vision community, LychSim will be made publicly available, including full source code and various data annotations.

[123] OmniNFT: Modality-wise Omni Diffusion Reinforcement for Joint Audio-Video Generation cs.CV | cs.AIPDF

Guohui Zhang, XiaoXiao Ma, Jie Huang, Hang Xu, Hu Yu

TL;DR: 本文提出OmniNFT，一种新颖的模态感知在线扩散强化学习框架，用于联合音视频生成。该框架通过模态优势路由、分层梯度手术和区域损失重加权三项关键创新，解决了多目标优势不一致、多模态梯度失衡和均匀信用分配等挑战，从而在音视频感知质量、跨模态对齐和音视频同步方面实现了全面改进。

Details

Motivation: 解决现有联合音视频生成方法在单模态保真度、跨模态对齐和细粒度同步方面的不足，并探索将强化学习应用于多目标、多模态联合生成这一尚未充分研究的领域。

Result: 在JavisBench和VBench基准测试上，基于LTX-2骨干网络的大量实验表明，OmniNFT在音频和视频感知质量、跨模态对齐以及音视频同步方面均取得了全面的提升。

Insight: 创新点在于提出了模态感知的强化学习框架，具体包括：1）模态优势路由，将独立奖励优势路由到各自的模态生成分支；2）分层梯度手术，选择性分离浅层音频层上的视频分支梯度，同时保留跨模态交互层的梯度；3）区域损失重加权，将策略优化调制到与音视频同步和细粒度对齐相关的关键区域。从客观角度看，该研究系统性地识别并解决了将RL应用于多模态生成时的核心障碍，具有方法论上的创新性。

Abstract: Recent advances in joint audio-video generation have been remarkable, yet real-world applications demand strong per-modality fidelity, cross-modal alignment, and fine-grained synchronization. Reinforcement Learning (RL) offers a promising paradigm, but its extension to multi-objective and multi-modal joint audio-video generation remains unexplored. Notably, our in-depth analysis first reveals that the primary obstacles to applying RL in this stem from: (i) multi-objective advantages inconsistency, where the advantages of multimodal outputs are not always consistent within a group; (ii) multi-modal gradients imbalance, where video-branch gradients leak into shallow audio layers responsible for intra-modal generation; (iii) uniform credit assignment, where fine-grained cross-modal alignment regions fail to get efficient exploration. These shortcomings suggest that vanilla RL fine-tuning strategy with a single global advantage often leads to suboptimal results. To address these challenges, we propose OmniNFT, a novel modality-aware online diffusion RL framework with three key innovations: (1) Modality-wise advantage routing, which routes independent per-reward advantages to their respective modality generation branches. (2) Layer-wise gradient surgery, which selectively detaches video-branch gradients on shallow audio layers while retaining those for cross-modal interaction layers. (3) Region-wise loss reweighting, which modulates policy optimization toward critical regions related to audio-video synchronization and fine-grained alignment. Extensive experiments on JavisBench and VBench with the LTX-2 backbone demonstrate that OmniNFT achieves comprehensive improvements in audio and video perceptual quality, cross-modal alignment, and audio-video synchronization.

[124] Elastic Attention Cores for Scalable Vision Transformers cs.CV | cs.LGPDF

Alan Z. Song, Yinjie Chen, Mu Nan, Rui Zhang, Jiahang Cao

TL;DR: 本文提出了一种名为VECA（Visual Elastic Core Attention）的新型视觉Transformer架构，旨在解决传统ViT中全连接自注意力计算复杂度随图像分辨率呈二次方增长的问题。VECA通过引入一组可学习的核心（core）嵌入作为通信接口，使图像块（patch）令牌仅通过核心令牌间接交互，从而将计算复杂度降低到线性O(N)。该模型支持在推理时弹性权衡计算与精度，并在分类和密集预测任务上取得了与当前先进视觉基础模型相当的性能。

Details

Motivation: 传统视觉Transformer（ViT）依赖全连接自注意力，其计算成本随图像分辨率二次方增长，限制了其在高分辨率领域的应用。本文挑战了“图像块间直接交互对学习丰富视觉语义表示是必要的”这一假设，旨在设计一种更高效的注意力机制。

Result: 在分类和密集预测任务上，VECA取得了与最新的视觉基础模型（如Swin Transformer等）具有竞争力的性能，同时显著降低了计算成本。

Insight: 核心创新在于提出了“弹性核心-外围”注意力结构，用一组可学习的核心令牌作为所有图像块令牌的间接通信中介，避免了直接的块间交互，实现了线性复杂度。模型支持通过沿核心轴的嵌套训练，在推理时进行弹性计算-精度权衡，为构建可扩展的视觉Transformer提供了新的基础模块。

Abstract: Vision Transformers (ViTs) achieve strong data-driven scaling by leveraging all-to-all self-attention. However, this flexibility incurs a computational cost that scales quadratically with image resolution, limiting ViTs in high-resolution domains. Underlying this approach is the assumption that pairwise token interactions are necessary for learning rich visual-semantic representations. In this work, we challenge this assumption, demonstrating that effective visual representations can be learned without any direct patch-to-patch interaction. We propose VECA (Visual Elastic Core Attention), a vision transformer architecture that uses efficient linear-time core-periphery structured attention enabled by a small set of learned cores. In VECA, these cores act as a communication interface: patch tokens exchange information exclusively through the core tokens, which are initialized from scratch and propagated across layers. Because the $N$ image patches only directly interact with a resolution invariant set of $C$ learned “core” embeddings, this yields linear complexity $O(N)$ for predetermined $C$, which bypasses quadratic scaling. Compared to prior cross-attention architectures, VECA maintains and iteratively updates the full set of $N$ input tokens, avoiding a small $C$-way bottleneck. Combined with nested training along the core axis, our model can elastically trade off compute and accuracy during inference. Across classification and dense tasks, VECA achieves performance competitive with the latest vision foundation models while reducing computational cost. Our results establish elastic core-periphery attention as a scalable alternative building block for Vision Transformers.

[125] AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in UMMs via Decompositional Verifiable Reward cs.CV | cs.AI | cs.LGPDF

Runhui Huang, Jie Wu, Rui Yang, Zhe Liu, Hengshuang Zhao

TL;DR: 本文提出了AlphaGRPO框架，通过将组相对策略优化（GRPO）应用于AR-Diffusion统一多模态模型（UMMs），无需额外冷启动阶段即可增强多模态生成能力。该方法解锁了模型执行高级推理任务的内在潜力，包括推理式文本到图像生成和自我反思式精炼。为解决现实世界多模态生成中提供稳定监督的挑战，引入了可分解可验证奖励（DVReward），利用LLM将复杂用户请求分解为原子化的、可验证的语义和质量问题，并由通用MLLM评估以提供可靠且可解释的反馈。

Details

Motivation: 动机在于增强统一多模态模型（UMMs）的多模态生成能力，特别是解决高级推理任务（如推断用户隐含意图和自主诊断纠正生成输出中的错位）以及为复杂现实生成任务提供稳定、可解释的监督信号。

Result: 在多个多模态生成基准测试（包括GenEval、TIIF-Bench、DPG-Bench和WISE）上取得了稳健的改进，并且在未针对编辑任务进行训练的情况下，在GEdit编辑任务上也实现了显著提升，验证了该自反思强化学习方法能有效利用内在理解来指导高保真生成。

Insight: 创新点在于将GRPO策略优化与UMMs结合以解锁自反思生成能力，并提出了DVReward这一新颖的奖励机制，通过LLM驱动的可分解、可验证问题来提供稳定且可解释的反馈，从而避免了传统标量奖励的局限性，并促进了模型内在推理潜力的发挥。

Abstract: In this paper, we propose AlphaGRPO, a novel framework that applies Group Relative Policy Optimization (GRPO) to AR-Diffusion Unified Multimodal Models (UMMs) to enhance multimodal generation capabilities without an additional cold-start stage. Our approach unlocks the model’s intrinsic potential to perform advanced reasoning tasks: Reasoning Text-to-Image Generation, where the model actively infers implicit user intents, and Self-Reflective Refinement, where it autonomously diagnoses and corrects misalignments in generated outputs. To address the challenge of providing stable supervision for real-world multimodal generation, we introduce the Decompositional Verifiable Reward (DVReward). Unlike holistic scalar rewards, DVReward utilizes an LLM to decompose complex user requests into atomic, verifiable semantic and quality questions, which are then evaluated by a general MLLM to provide reliable and interpretable feedback. Extensive experiments demonstrate that AlphaGRPO yields robust improvements across multimodal generation benchmarks, including GenEval, TIIF-Bench, DPG-Bench and WISE, while also achieving significant gains in editing tasks on GEdit without training on editing tasks. These results validate that our self-reflective reinforcement approach effectively leverages inherent understanding to guide high-fidelity generation. Project page: https://huangrh99.github.io/AlphaGRPO/

[126] CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narratives cs.CVPDF

Yihao Meng, Zichen Liu, Hao Ouyang, Qiuyu Wang, Ka Leong Cheng

TL;DR: CausalCine是一个用于多镜头视频叙事的实时自回归生成框架，它将多镜头视频生成转化为在线导演过程，通过因果基础模型学习复杂镜头转换，并引入内容感知记忆路由（CAMR）动态检索历史信息以保持跨镜头连贯性，最终通过蒸馏实现实时交互生成。

Details

Motivation: 现有自回归视频生成模型主要针对短时延续训练，将长序列视为单一镜头的延伸，导致在长序列生成中出现运动停滞和语义漂移，无法满足需要事件演进、视角转换和离散镜头边界的电影叙事需求。

Result: 大量实验表明，CausalCine在自回归基线模型上表现显著更优，其能力接近双向模型，同时解锁了因果生成的流式交互性。

Insight: 创新点在于将多镜头视频生成视为在线导演的交互过程，提出了在原生多镜头序列上训练的因果基础模型以学习镜头转换先验，并设计了CAMR机制，基于注意力相关性而非时间邻近性动态检索历史KV条目，在有限活动内存下保持跨镜头连贯性，最终通过蒸馏实现实时交互生成。

Abstract: Autoregressive video generation aims at real-time, open-ended synthesis. Yet, cinematic storytelling is not merely the endless extension of a single scene; it requires progressing through evolving events, viewpoint shifts, and discrete shot boundaries. Existing autoregressive models often struggle in this setting. Trained primarily for short-horizon continuation, they treat long sequences as extended single shots, inevitably suffering from motion stagnation and semantic drift during long rollouts. To bridge this gap, we introduce CausalCine, an interactive autoregressive framework that transforms multi-shot video generation into an online directing process. CausalCine generates causally across shot changes, accepts dynamic prompts on the fly, and reuses context without regenerating previous shots. To achieve this, we first train a causal base model on native multi-shot sequences to learn complex shot transitions prior to acceleration. We then propose Content-Aware Memory Routing (CAMR), which dynamically retrieves historical KV entries according to attention-based relevance scores rather than temporal proximity, preserving cross-shot coherence under bounded active memory. Finally, we distill the causal base model into a few-step generator for real-time interactive generation. Extensive experiments demonstrate that CausalCine significantly outperforms autoregressive baselines and approaches the capability of bidirectional models while unlocking the streaming interactivity of causal generation. Demo available at https://yihao-meng.github.io/CausalCine/

[127] SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture cs.CVPDF

Haiwen Diao, Penghao Wu, Hanming Deng, Jiahao Wang, Shihao Bai

TL;DR: SenseNova-U1是一种基于NEO-unify架构的原生统一多模态范式，旨在打破传统视觉语言模型中理解与生成任务分离的局限，将两者视为单一底层过程的协同视图。论文提出了两个变体模型（8B密集型和30B稀疏激活的MoE型），在多种理解任务和生成任务上均表现出色，并展示了其在视觉语言动作和世界模型场景中的潜力。

Details

Motivation: 当前大型视觉语言模型普遍存在理解与生成任务被割裂处理的问题，导致架构碎片化、流程级联和表示空间不对齐，这被视为阻碍原生多模态智能出现的结构性限制。

Result: 模型在文本理解、视觉语言感知、知识推理、智能体决策和空间智能等理解任务上媲美顶级纯理解模型；在任意到图像合成、复杂图文生成和交错视觉语言生成等任务上展现出强大的语义一致性和视觉保真度；初步证据表明其在视觉语言动作和世界模型场景中也表现强劲。

Insight: 核心创新在于提出了一个将多模态理解与生成统一为单一底层过程协同视图的原生架构（NEO-unify），挑战了传统任务分离范式，并展示了统一模型内部可自然涌现出感知、生成乃至行动等多种必要能力，为构建真正统一的多模态智能系统提供了新路径。

Abstract: Recent large vision-language models (VLMs) remain fundamentally constrained by a persistent dichotomy: understanding and generation are treated as distinct problems, leading to fragmented architectures, cascaded pipelines, and misaligned representation spaces. We argue that this divide is not merely an engineering artifact, but a structural limitation that hinders the emergence of native multimodal intelligence. Hence, we introduce SenseNova-U1, a native unified multimodal paradigm built upon NEO-unify, in which understanding and generation evolve as synergistic views of a single underlying process. We launch two native unified variants, SenseNova-U1-8B-MoT and SenseNova-U1-A3B-MoT, built on dense (8B) and mixture-of-experts (30B-A3B) understanding baselines, respectively. Designed from first principles, they rival top-tier understanding-only VLMs across text understanding, vision-language perception, knowledge reasoning, agentic decision-making, and spatial intelligence. Meanwhile, they deliver strong semantic consistency and visual fidelity, excelling in conventional or knowledge-intensive any-to-image (X2I) synthesis, complex text-rich infographic generation, and interleaved vision-language generation, with or without think patterns. Beyond performance, we show detailed model design, data preprocessing, pre-/post-training, and inference strategies to support community research. Last but not least, preliminary evidence demonstrates that our models extend beyond perception and generation, performing strongly in vision-language-action (VLA) and world model (WM) scenarios. This points toward a broader roadmap where models do not translate between modalities, but think and act across them in a native manner. Multimodal AI is no longer about connecting separate systems, but about building a unified one and trusting the necessary capabilities to emerge from within.

cs.AI [Back]

[128] Unlocking LLM Creativity in Science through Analogical Reasoning cs.AI | cs.CL | q-bio.QMPDF

Andrew Shen, Shaul Druckmann, James Zou

TL;DR: 本文提出了一种基于类比推理（AR）的方法来增强大语言模型在开放性问题解决中的创造性和多样性，通过跨领域类比生成新颖解决方案，并在多个生物医学任务上验证了其有效性。

Details

Motivation: 解决大语言模型在开放性问题生成中容易陷入模式崩溃、生成方案多样性不足的问题，以推动自主科学发现，特别是在生物医学等复杂领域。

Result: AR方法显著提升了生成方案的多样性（多样性指标提升90-173%），超过50%的生成方案是新颖的（基线最低仅1.6%）。在四个生物医学任务中，AR在扰动效应预测的分布指标上实现了近13倍的提升，在细胞间通讯预测的AUPRC上优于所有基线，推断脑区交互的Spearman相关系数达到0.729，并在寡核苷酸性质预测的两个数据集上取得了SOTA性能。

Insight: 通过引入基于共享关系结构的跨领域类比推理来激发LLM的创造性，有效缓解模式崩溃，生成多样且新颖的解决方案，并可集成到现有方法中以扩展搜索空间。

Abstract: Autonomous science promises to augment scientific discovery, particularly in complex fields like biomedicine. However, this requires AI systems that can consistently generate novel and diverse solutions to open-ended problems. We evaluate LLMs on the task of open-ended solution generation and quantify their tendency to mode collapse into low-diversity generations. To mitigate this mode collapse, we introduce analogical reasoning (AR) as a new approach to solution generation. AR generates analogies to cross-domain problems based on shared relational structure, then uses those analogies to search for novel solutions. Compared to baselines, AR discovers significantly more diverse generations (improving solution diversity metrics by 90-173%), generates novel solutions over 50% of the time (compared to as little as 1.6% for baselines), and produces high-quality analogies. To validate the real-world feasibility of AR, we implement AR-generated solutions across four biomedical problems, yielding consistent quantitative gains. AR-generated approaches achieve a nearly 13-fold improvement on distributional metrics for perturbation effect prediction, outperform all baselines on AUPRC when predicting cell-cell communication, infer brain region interactions with a high Spearman correlation ($ρ$=0.729) to published methods, and establish state-of-the-art performance on 2 datasets for oligonucleotide property prediction. The novel and diverse solutions produced by AR can be used to augment the search space of existing solution generation methods.

[129] LatentRouter: Can We Choose the Right Multimodal Model Before Seeing Its Answer? cs.AI | cs.CL | cs.CVPDF

Xueqi Cheng, Yushun Dong

TL;DR: 本文提出了LatentRouter，一种用于多模态大语言模型（MLLM）的路由器，它将路由问题建模为反事实多模态效用预测。该方法通过学习多模态路由胶囊、模型能力令牌以及它们之间的潜在通信，来预测每个候选模型对给定图像-问题输入的潜在性能，从而在模型池中选择最合适的模型。

Details

Motivation: 现有的多模态大语言模型在OCR、图表理解、空间推理、视觉问答、成本和延迟等方面各有优劣，因此有效的路由不仅需要评估查询难度，更需要将输入的多模态需求与每个候选模型的能力进行匹配。

Result: 在MMR-Bench和VL-RouterBench基准测试上的实验表明，LatentRouter在性能导向和性能-成本权衡的路由任务上，均优于固定模型、特征级和现有学习型路由器基线，尤其在那些模型选择依赖于视觉、布局敏感或推理导向需求的多模态任务组上提升最显著。

Insight: 核心创新在于将MLLM路由形式化为反事实效用预测，并引入了潜在通信机制（latent communication）来估计模型性能；此外，其分布化结果头和有界胶囊校正机制有助于提升预测精度，而共享的按模型评分与可用性掩码则支持动态变化的候选模型池。

Abstract: Multimodal large language models (MLLMs) have heterogeneous strengths across OCR, chart understanding, spatial reasoning, visual question answering, cost, and latency. Effective MLLM routing therefore requires more than estimating query difficulty: a router must match the multimodal requirements of the current image-question input with the capabilities of each candidate model. We propose LatentRouter, a router that formulates MLLM routing as counterfactual multimodal utility prediction. Given an image-question query, LatentRouter extracts learned multimodal routing capsules, represents each candidate MLLM with a model capability token, and performs latent communication between these states to estimate how each model would perform if selected. A distributional outcome head predicts model-specific counterfactual quality, while a bounded capsule correction refines close decisions without allowing residual signals to dominate the prediction. The resulting utility-based policy supports performance-oriented and performance-cost routing, and handles changing candidate pools through shared per-model scoring with availability masking. Experiments on MMR-Bench and VL-RouterBench show that LatentRouter outperforms fixed-model, feature-level, and learned-router baselines. Additional analyses show that the gains are strongest on multimodal task groups where model choice depends on visual, layout-sensitive, or reasoning-oriented requirements, and that latent communication is the main contributor to the improvement. The code is available at: https://github.com/LabRAI/LatentRouter.

[130] Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning cs.AI | cs.CL | cs.LOPDF

Zihao Han, Tiangang Zhang, Huaibin Wang, Yilun Sun

TL;DR: 本文提出了一种名为自适应教师暴露的自蒸馏方法（ATESD），用于改进大语言模型的推理能力。该方法通过动态调整教师模型在训练过程中看到的参考推理内容的比例，来解决传统自蒸馏方法中教师-学生能力不匹配的问题。实验表明，该方法在多个推理基准测试上显著优于现有的自蒸馏和强化学习方法。

Details

Motivation: 当前主流的大语言模型推理自蒸馏方法默认让教师模型看到完整的参考推理过程，这可能导致教师给出的监督信号过于强大，超出了学生模型当前的学习能力，造成师生能力不匹配，从而影响学习效率。

Result: 在AIME 24、AIME 25和HMMT 25三个推理基准测试上，使用Qwen3-{1.7B, 4B, 8B}模型进行实验，ATESD方法相比基线方法OPSD，在Average@12指标上分别提升了+0.95、+2.05和+2.33个点，取得了显著的性能提升，并超越了其他竞争性的自蒸馏和强化学习基线。

Insight: 核心创新点在于将教师模型的“暴露比例”（即能看到多少参考推理）从一个固定的超参数，转变为一个基于训练状态动态学习的控制变量。这通过一个轻量级的Beta策略控制器来实现，并使用一种基于“未来学习进步”的延迟奖励信号来优化控制器，有效解决了策略蒸馏中的信用分配延迟问题。这为推理自蒸馏提供了一个新的、有效的优化维度。

Abstract: On-policy self-distillation has become a strong recipe for LLM reasoning, where a privileged teacher supervises the student’s own rollouts while conditioning on the reference solution. A design choice shared by nearly all such methods, however, has gone unquestioned: the teacher always sees the full reference reasoning. We argue that this default itself is part of the problem and identify a teacher-side exposure mismatch: when the teacher conditions on reasoning far beyond the student’s current competence, the resulting token targets become too strong to absorb. A controlled fixed-exposure sweep makes this concrete on two fronts: 1) full exposure is not reliably the best choice, and 2) student-teacher mismatch grows monotonically as the teacher sees more privileged reasoning. This motivates treating teacher exposure not as a fixed hyperparameter but as a learnable training-time control variable. We therefore propose Adaptive Teacher Exposure for Self-Distillation (ATESD). ATESD models the reveal ratio with a lightweight Beta-policy controller conditioned on compact training-state statistics, and uses one sampled exposure for a short hold window of student updates. To make this exposure controller learnable, we optimize it with a discounted learning-progress reward that scores each held decision by its effect on the student’s future improvement rather than its immediate loss change, addressing the delayed credit assignment induced by on-policy distillation. Experiments on AIME 24, AIME 25, and HMMT 25 across Qwen3-{1.7B, 4B, 8B} show that ATESD consistently outperforms competitive self-distillation and RL baselines, improving over OPSD by +0.95, +2.05, and +2.33 Average@12 points respectively, and establishing adaptive teacher exposure as an effective new axis for reasoning self-distillation.

[131] AutoLLMResearch: Training Research Agents for Automating LLM Experiment Configuration – Learning from Cheap, Optimizing Expensive cs.AI | cs.CL | cs.LGPDF

Taicheng Guo, Nitesh V. Chawla, Olaf Wiest, Xiangliang Zhang

TL;DR: 本文提出了AutoLLMResearch框架，旨在自动化高成本的大语言模型实验配置。该框架通过智能体从低成本实验中学习通用原则，并外推至昂贵实验设置，以高效识别有前景的配置。

Details

Motivation: 现有自动化方法适用于低成本、可重复试错的场景，而可扩展的LLM实验成本过高，无法进行大量迭代，导致配置过程依赖专家直觉且劳动密集。本文旨在填补这一空白，实现高成本LLM实验配置的自动化。

Result: 在包含超过一百万GPU小时可验证实验结果的LLMConfig-Gym环境中，与多种强基线方法在保留实验上进行广泛评估，证明了该框架的有效性、泛化性和可解释性。

Insight: 创新点在于提出了一个多保真度环境（LLMConfig-Gym）和一个结构化的训练流程，将配置研究建模为长视野马尔可夫决策过程，并激励跨保真度的外推推理，从而模拟人类研究者的学习过程，为可扩展的LLM实验自动化提供了通用解决方案。

Abstract: Effectively configuring scalable large language model (LLM) experiments, spanning architecture design, hyperparameter tuning, and beyond, is crucial for advancing LLM research, as poor configuration choices can waste substantial computational resources and prevent models from realizing their full potential. Prior automated methods are designed for low-cost settings where repeated trial and error is feasible, but scalable LLM experiments are too expensive for such extensive iteration. To our knowledge, no work has addressed the automation of high-cost LLM experiment configurations, leaving this problem labor-intensive and dependent on expert intuition. Motivated by this gap, we propose AutoLLMResearch, an agentic framework that mimics how human researchers learn generalizable principles from low-fidelity experiments and extrapolate to efficiently identify promising configurations in expensive LLM settings. The core challenge is how to enable an agent to learn, through interaction with a multi-fidelity experimental environment that captures the structure of the LLM configuration landscape. To achieve this, we propose a systematic framework with two key components: 1) LLMConfig-Gym, a multi-fidelity environment encompassing four critical LLM experiment tasks, supported by over one million GPU hours of verifiable experiment outcomes; 2) A structured training pipeline that formulates configuration research as a long-horizon Markov Decision Process and accordingly incentivizes cross-fidelity extrapolation reasoning. Extensive evaluation against diverse strong baselines on held-out experiments demonstrates the effectiveness, generalization, and interpretability of our framework, supporting its potential as a practical and general solution for scalable real-world LLM experiment automation.

[132] Allegory of the Cave: Measurement-Grounded Vision-Language Learning cs.AI | cs.CL | cs.CVPDF

Kepeng Xu, Li Xu, Gang He, Wenxin Yu

TL;DR: 该论文提出了测量基础视觉语言学习框架，通过将视觉接口从后处理的RGB图像前移至更接近原始相机测量的RAW数据域，以解决RGB渲染过程中信息丢失导致的视觉语言模型（VLM）基础错误问题。论文实例化了PRISM-VL模型，采用RAW衍生的Meas.-XYZ输入、相机条件基础化和曝光包围监督聚合等技术，在低光、HDR等挑战性场景下显著提升了模型性能。

Details

Motivation: 当前视觉语言模型通常基于后处理RGB图像进行推理，但RGB渲染过程可能对传感器原始测量数据进行裁剪、抑制或量化，导致信息丢失，从而影响模型的基础能力。论文旨在探究将视觉接口移至更接近相机原始测量域是否能改善模型的基础性能。

Result: 在针对低光、高动态范围（HDR）、对可见性敏感和幻觉敏感场景的保留基准测试中，PRISM-VL-8B模型达到了0.6120 BLEU、0.4571 ROUGE-L和82.66%的LLM-Judge准确率，相比基于RGB的Qwen3-VL-8B基线模型，分别提升了+0.1074 BLEU、+0.1071 ROUGE-L和+4.46个百分点，展现了显著的性能改进。

Insight: 论文的创新点在于提出了测量基础视觉语言学习范式，并设计了PRISM-VL模型，其核心包括使用RAW衍生的Meas.-XYZ输入保留更丰富的传感器证据、引入相机条件基础化以适配不同相机特性，以及通过曝光包围监督聚合将RGB域的监督信号有效迁移至测量域。从客观角度看，这项工作揭示了RGB渲染信息损失是VLM基础错误的一个重要来源，并证明了在测量域保留证据能有效提升多模态推理能力，为未来VLM设计提供了新的数据表示方向。

Abstract: Vision-language models typically reason over post-ISP RGB images, although RGB rendering can clip, suppress, or quantize sensor evidence before inference. We study whether grounding improves when the visual interface is moved closer to the underlying camera measurement. We formulate measurement-grounded vision-language learning and instantiate it as PRISM-VL, which combines RAW-derived Meas.-XYZ inputs, camera-conditioned grounding, and Exposure-Bracketed Supervision Aggregation for transferring supervision from RGB proxies to measurement-domain observations. Using a quality-controlled 150K instruction-tuning set and a held-out benchmark targeting low-light, HDR, visibility-sensitive, and hallucination-sensitive cases, PRISM-VL-8B reaches 0.6120 BLEU, 0.4571 ROUGE-L, and 82.66% LLM-Judge accuracy, improving over the RGB Qwen3-VL-8B baseline by +0.1074 BLEU, +0.1071 ROUGE-L, and +4.46 percentage points. These results suggest that part of VLM grounding error arises from information lost during RGB rendering, and that preserving measurement-domain evidence can improve multimodal reasoning.

[133] Do Enterprise Systems Need Learned World Models? The Importance of Context to Infer Dynamics cs.AI | cs.CL | cs.LGPDF

Jishnu Sethumadhavan Nair, Patrice Bechard, Rishabh Maheshwary, Surajit Dasgupta, Sravan Ramachandran

TL;DR: 本文探讨了在企业系统中，当环境动态可由配置定义且可读时，智能体是否仍需学习世界模型。作者提出企业发现智能体，通过在运行时读取系统配置来发现相关转移动态，而非仅依赖离线训练的内部表示。研究引入CascadeBench基准，用于评估企业级级联预测，并证明基于发现的智能体在部署变化下比离线训练的世界模型更鲁棒。

Details

Motivation: 企业系统的动态常由租户特定的业务逻辑定义，这些逻辑随部署不同且随时间演变，导致基于历史转移训练的模型在部署变化时脆弱。本文旨在研究当规则在推理时可读时，智能体是否仍需学习这些动态。

Result: 在CascadeBench基准（基于World of Workflows评估方法的合成环境）上进行实验，结果显示离线训练的世界模型在分布内表现良好，但随着动态变化性能下降；而基于发现的智能体通过将预测基于当前实例，在变化下更鲁棒。

Insight: 创新点在于提出企业发现智能体，强调在可配置企业环境中，智能体不应仅依赖固定的内部化动态，而应结合运行时发现相关转移逻辑的机制。从客观角度看，这为动态变化环境中的智能体设计提供了新思路，即利用可读配置来增强鲁棒性。

Abstract: World models enable agents to anticipate the effects of their actions by internalizing environment dynamics. In enterprise systems, however, these dynamics are often defined by tenant-specific business logic that varies across deployments and evolves over time, making models trained on historical transitions brittle under deployment shift. We ask a question the world-models literature has not addressed: when the rules can be read at inference time, does an agent still need to learn them? We argue, and demonstrate empirically, that in settings where transition dynamics are configurable and readable, runtime discovery complements offline training by grounding predictions in the active system instance. We propose enterprise discovery agents, which recover relevant transition dynamics at runtime by reading the system’s configuration rather than relying solely on internalized representations. We introduce CascadeBench, a reasoning-focused benchmark for enterprise cascade prediction that adopts the evaluation methodology of World of Workflows on diverse synthetic environments, and use it together with deployment-shift evaluation to show that offline-trained world models can perform well in-distribution but degrade as dynamics change, whereas discovery-based agents are more robust under shift by grounding their predictions in the current instance. Our findings suggest that, in configurable enterprise environments, agents should not rely solely on fixed internalized dynamics, but should incorporate mechanisms for discovering relevant transition logic at runtime.

physics.geo-ph [Back]

[134] Parameter-Efficient Adaptation of Pre-Trained Vision Foundation Models for Active and Passive Seismic Data Denoising physics.geo-ph | cs.CV | cs.LGPDF

Jiahua Zhao, Umair bin Waheed, Jing Sun, Yang Cui, Nikos Savva

TL;DR: 本文提出了一种参数高效的微调框架，将通用的视觉基础模型（VFMs）重新用于地震数据去噪任务。该方法采用预训练的DINOv3编码器，并通过低秩适应（LoRA）进行少量参数调整。此外，引入了一个基于峰度的无监督测试时适应模块，在推理过程中仅更新LoRA参数，以增强模型在未见野外条件下的鲁棒性。

Details

Motivation: 解决密集地震数据（如DAS阵列和大规模勘测）中复杂噪声抑制的挑战，同时避免传统监督方法需要大量配对数据、任务特定且易受域偏移影响的缺点，以及从头预训练地震基础模型所需的海量数据和计算成本。

Result: 在犹他州FORGE站点的公开勘探地震图像和DAS垂直地震剖面数据上的实验表明，该框架达到或超越了特定领域模型的性能。在中国陆地勘测和德国Groß Schönebeck地热站点的未见跨站点数据测试中，进一步展示了强大的泛化能力和有效的信噪分离效果。

Insight: 创新点在于将通用视觉基础模型通过参数高效的LoRA微调迁移到地球物理任务，并引入基于峰度的无监督测试时适应模块，实现了在无地面真值情况下的模型自校准。这为数据密集型勘探地震学问题提供了一种高效利用预训练模型的新途径。

Abstract: The demand for high-resolution subsurface imaging and continuous Earth monitoring has driven rapid growth in active and passive seismic data from dense geophone deployments, distributed acoustic sensing (DAS) arrays, and large-scale 2D and 3D surveys. This expansion makes complex noise suppression increasingly challenging, especially when signal fidelity must be preserved. Conventional supervised deep learning methods are often task-specific, require large paired datasets, and can suffer from domain shift under new acquisition conditions. Foundation models offer a promising alternative, but pre-training seismic foundation models from scratch requires massive domain-specific data and substantial computation. We propose an efficient framework that repurposes general-purpose Vision Foundation Models (VFMs) for geophysical tasks through Parameter-Efficient Fine-Tuning. The architecture uses a pre-trained VFM, a DINOv3 encoder, adapted with Low-Rank Adaptation (LoRA) to enable effective feature adaptation with few additional parameters. To improve robustness under unseen field conditions without ground truth, we introduce a kurtosis-guided unsupervised test-time adaptation module that updates only LoRA parameters during inference. This module self-calibrates the model to site-specific noise by identifying information-rich regions via kurtosis and performing self-training without labeled data. Experiments on public exploration seismic images and DAS vertical seismic profiling data from the Utah FORGE site show that the framework matches or outperforms domain-specific models. Tests on unseen cross-site data from a land survey in China and the Groß Schönebeck geothermal site in Germany further demonstrate strong generalization and effective signal-noise separation. These results highlight the potential of adapting pre-trained VFMs to data-intensive problems in exploration seismology.

cs.SE [Back]

[135] StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning cs.SE | cs.CLPDF

Hao Wang, Rui Li, Lei Sha, Jie M. Zhang

TL;DR: 本文提出了StepCodeReasoner框架，通过强化学习将代码推理与逐步执行轨迹对齐。该方法通过在代码中自动插入基于打印的结构化执行轨迹锚点，使模型能够预测每一步的运行时状态，从而将代码推理转化为可验证的逐步执行建模问题。

Details

Motivation: 现有代码推理方法主要监督最终代码输出，忽略了中间状态，常导致奖励黑客问题，即通过不一致的推理获得正确答案。本文旨在通过引入显式的中间执行状态监督来解决此问题。

Result: 在CRUXEval和LiveCodeBench基准测试中，7B模型分别达到91.1%和86.5%的准确率，超越了CodeReasoner-7B基线（86.0%和77.7%）和GPT-4o（85.6%和75.1%），实现了SOTA性能。在REval执行轨迹基准上，模型得分为82.9%，也优于基线模型和GPT-4o。

Insight: 创新点在于将代码推理转化为可验证的逐步执行建模，并提出了双层GRPO强化学习算法，在轨迹间比较替代执行路径，在轨迹内根据中间准确性对下游正确性的影响进行奖励分配，从而实现了结构化的信用分配。该方法不仅提升了代码推理性能，也改善了代码生成能力。

Abstract: Existing code reasoning methods primarily supervise final code outputs, ignoring intermediate states, often leading to reward hacking where correct answers are obtained through inconsistent reasoning. We propose StepCodeReasoner, a framework that introduces explicit intermediate execution-state supervision. By automatically inserting structured print-based execution-trace anchors into code, the model is trained to predict runtime states at each step, transforming code reasoning into a verifiable, stepwise execution modeling problem. Building on this execution-aware method, we introduce Bi-Level GRPO, a reinforcement learning algorithm for structured credit assignment at two levels: inter-trajectory, comparing alternative execution paths, and intra-trajectory, rewarding intermediate accuracy based on its impact on downstream correctness. Extensive experiments demonstrate that StepCodeReasoner achieves SOTA performance in code reasoning. In particular, our 7B model achieves 91.1% on CRUXEval and 86.5% on LiveCodeBench, outperforming the CodeReasoner-7B baseline (86.0% and 77.7%) and GPT-4o (85.6% and 75.1%). Furthermore, on the execution-trace benchmark REval, our model scores 82.9%, outperforming baseline CodeReasoner-7B (72.3%), its 14B counterpart (81.1%), and GPT-4o (77.3%). Additionally, our approach also improves code generation performance, demonstrating that explicit execution modeling enhances both code reasoning and code generation.

cs.MM [Back]

Che Liu, Lichao Ma, Xiangyu Tony Zhang, Yuxin Zhang, Haoyang Zhang

TL;DR: 本文研究了全模态语言模型的评估偏差问题，并提出了一种视觉去偏的评估方法。作者通过视觉探测审计了九个全模态基准，移除了仅凭视觉线索即可解答的查询，构建了名为OmniClean的清洗后评估集。在此基础上，他们提出了一个基于Qwen2.5-Omni-3B模型的三阶段后训练方案OmniBoost，包括混合双模态SFT、混合模态RLVR和基于自蒸馏数据的SFT。实验表明，在视觉去偏的评估下，小模型通过分阶段后训练可以达到与更大模型相当的性能。

Details

Motivation: 当前全模态语言模型的基准测试存在视觉捷径偏差，即仅凭视觉信息就能回答问题，这夸大了模型在整合音频、视觉和语言多模态证据方面的真实能力。本文旨在研究如何通过视觉去偏的评估来更准确地衡量模型性能，并探索有效的后训练方法。

Result: 在构建的视觉去偏评估集OmniClean上，提出的三阶段后训练方案OmniBoost使3B参数模型达到了与Qwen3-Omni-30B-A3B-Instruct模型相当甚至略优的聚合性能，而未使用更强的全模态教师模型。

Insight: 论文的创新点在于：1) 揭示了全模态基准中的视觉泄漏问题，并提出了系统性的视觉探测和清洗方法，构建了更可靠的评估集OmniClean；2) 提出了一种分阶段的后训练方案，特别是结合了强化学习与视觉推理(RLVR)和自蒸馏监督，有效提升了小模型的全模态理解能力，证明了精心设计的后训练可以弥补模型规模的不足。

Abstract: Omni-modal language models are intended to jointly understand audio, visual inputs, and language, but benchmark gains can be inflated when visual evidence alone is enough to answer a query. We study whether current omni-modal benchmarks separate visual shortcuts from genuine audio-visual-language evidence integration, and how post-training behaves under a visually debiased evaluation setting. We audit nine omni-modal benchmarks with visual-only probing, remove visually solvable queries, and retain full subsets when filtering is undefined or would make comparisons unstable. This yields OmniClean, a cleaned evaluation view with 8,551 retained queries from 16,968 audited queries. On OmniClean, we evaluate OmniBoost, a three-stage post-training recipe based on Qwen2.5-Omni-3B: mixed bi-modal SFT, mixed-modality RLVR, and SFT on self-distilled data. Balanced bi-modal SFT gives limited and uneven gains, RLVR provides the first broad improvement, and self-distillation reshapes the benchmark profile. After SFT on self-distilled data, the 3B model reaches performance comparable to, and in aggregate slightly above, Qwen3-Omni-30B-A3B-Instruct without using a stronger omni-modal teacher. These results show that omni-modal progress is easier to interpret when evaluation controls visual leakage, and that small omni-modal models can benefit from staged post-training with self-distilled omni-query supervision.

cs.CR [Back]

[137] TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection cs.CR | cs.CL | cs.LGPDF

Tom Sander, Hongyan Chang, Tomáš Souček, Tuan Tran, Valeriu Lacatusu

TL;DR: 本文提出了TextSeal，一种用于大语言模型（LLM）的先进水印技术。该方法基于Gumbel-max采样，通过双密钥生成恢复输出多样性，并结合熵加权评分和多区域定位以提升检测能力。它支持推测解码和多令牌预测等推理优化，且不增加推理开销。TextSeal在检测强度上严格优于SynthID-text等基线方法，对稀释具有鲁棒性，即使在大量混合人机生成的文档中也能保持可靠的局部化检测。该方案理论上无失真，在多个推理基准测试中证实其能保持下游任务性能；同时，一项多语言人工评估（6000次A/B比较，5种语言）显示其没有可感知的质量差异。除了用于来源检测，TextSeal还具有‘放射性’：其水印信号可通过模型蒸馏传递，从而能够检测未经授权的使用。

Details

Motivation: 解决大语言模型生成文本的来源追溯和未经授权使用（如通过模型蒸馏）的检测与保护问题。

Result: 在检测强度上严格优于SynthID-text等基线方法；在多个推理基准测试中保持下游性能；多语言人工评估显示无感知质量差异；对稀释具有鲁棒性，能在混合文档中实现可靠的局部化检测。

Insight: 创新点包括：基于Gumbel-max采样的双密钥生成机制以恢复输出多样性；熵加权评分和多区域定位技术提升检测精度与鲁棒性；支持推测解码等推理优化且无推理开销；水印信号具有‘放射性’，可通过模型蒸馏传递，为模型版权保护提供了新思路。

Abstract: We introduce TextSeal, a state-of-the-art watermark for large language models. Building on Gumbel-max sampling, TextSeal introduces dual-key generation to restore output diversity, along with entropy-weighted scoring and multi-region localization for improved detection. It supports serving optimizations such as speculative decoding and multi-token prediction, and does not add any inference overhead. TextSeal strictly dominates baselines like SynthID-text in detection strength and is robust to dilution, maintaining confident localized detection even in heavily mixed human/AI documents. The scheme is theoretically distortion-free, and evaluation across reasoning benchmarks confirms that it preserves downstream performance; while a multilingual human evaluation (6000 A/B comparisons, 5 languages) shows no perceptible quality difference. Beyond its use for provenance detection, TextSeal is also ``radioactive’’: its watermark signal transfers through model distillation, enabling detection of unauthorized use.

cs.LG [Back]

[138] VERDI: Single-Call Confidence Estimation for Verification-Based LLM Judges via Decomposed Inference cs.LG | cs.CL | cs.IRPDF

Jasmine Qi, Danylo Dantsev, Muyang Sun

TL;DR: 本文提出了一种名为VERDI（VERification-Decomposed Inference）的方法，用于为基于验证的LLM评判系统（LLM-as-Judge）提供单次调用的置信度估计。该方法通过分解评判过程产生的推理轨迹，提取三种结构化信号，并结合逻辑回归进行校准，无需额外调用LLM。

Details

Motivation: 当前广泛部署的LLM-as-Judge系统缺乏可靠的方法来判断评判结果的置信度。标准的后验置信度信号（如词元对数概率）对许多商业LLM不可用，且在处理结构化JSON输出时容易饱和至接近1.0，无法有效区分不确定性。

Result: 在三个公开基准测试上，VERDI在GPT-4.1-mini上取得了0.72-0.91的AUROC，在GPT-5.4-mini上为0.66-0.80。对于Qwen3.5系列模型（其词元对数概率存在反校准问题，AUROC仅0.32-0.49），VERDI将AUROC提升至0.56-0.70。此外，在一个生产系统的八个评估准则上，事实性准则的AUROC达到0.73-0.88，并展示了跨模型迁移能力（AUROC 0.66-0.69）。

Insight: 论文的创新点在于从现有结构化评判的推理轨迹中提取置信度信号，无需额外推理成本。具体包括：1）将验证式评估分解为子检查步骤；2）定义了三种结构化信号：步骤-裁决对齐、声明级边际和证据基础分数；3）使用Platt缩放的逻辑回归进行信号组合。客观来看，该方法为解决商业LLM置信度估计的可用性和校准问题提供了一种实用且可扩展的替代方案，特别是证明了小型NLI模型可作为正则表达式提取的可扩展替代。

Abstract: LLM-as-Judge systems are widely deployed for automated evaluation, yet practitioners lack reliable methods to know when a judge’s verdict should be trusted. Token log-probabilities, the standard post-hoc confidence signal, are unavailable for many commercial LLMs and, even when accessible, saturate above 0.999 with structured JSON output. We introduce VERDI (VERification-Decomposed Inference), a method that extracts confidence from the reasoning trace a structured judge already produces, with no additional inference calls. VERDI decomposes each verification-style evaluation into sub-checks and derives three structural signals: Step-Verdict Alignment, Claim-Level Margin, and Evidence Grounding Score. We combine them with Platt-scaled logistic regression. On three public benchmarks, VERDI achieves AUROC 0.72-0.91 on GPT-4.1-mini and 0.66-0.80 on GPT-5.4-mini. On Qwen3.5-4B/9B/27B, where answer-token logprobs are anti-calibrated (higher confidence on errors, AUROC 0.32-0.49), VERDI achieves 0.56-0.70. We additionally validate on a production system with eight rubrics (AUROC 0.73-0.88 on factual rubrics), demonstrate cross-model transfer (AUROC 0.66-0.69), and show that a 33M-parameter NLI (Natural Language Inference) model provides a scalable alternative to regex extraction.

[139] Test-Time Compute for Dense Retrieval: Agentic Program Generation with Frozen Embedding Models cs.LG | cs.CL | cs.IRPDF

Han Xiao

TL;DR: 本文挑战了’仅大型推理模型受益于测试时计算’的普遍观点，证明了小型嵌入模型同样能从中获益。通过一个基于代理的程序搜索循环，在冻结的嵌入模型API上探索了大量候选推理程序，发现帕累托前沿收敛于一个简单的代数公式：查询向量与局部前K个文档向量的软最大值加权质心的插值。

Details

Motivation: 动机在于探索如何在不重新训练的情况下，利用冻结嵌入模型继承自大型LLM骨干网络的表示空间，通过额外的推理计算（测试时计算）来提升其检索性能。

Result: 该方法在七个参数规模相差十倍的嵌入模型家族上，均显著提升了nDCG@10指标，并在完整的BEIR基准验证集上确认了每个测试模型的性能提升。

Insight: 核心创新点在于揭示了冻结嵌入模型通过一个参数无关的、基于软最大值加权质心插值的简单代数运算，即可有效利用测试时计算提升检索效果，这为优化嵌入模型推理提供了新的、轻量化的思路。

Abstract: Test-time compute is widely believed to benefit only large reasoning models. We show it also helps small embedding models. Most modern embedding checkpoints are distilled from large LLM backbones and inherit their representation space; a frozen embedding model should therefore benefit from extra inference compute without retraining. Using an agentic program-search loop, we explore 259 candidate inference programs over a frozen embedding API across ninety generations. The entire Pareto frontier collapses onto a single algebra: a softmax-weighted centroid of the local top-K documents interpolated with the query. This parameter-free default lifts nDCG@10 statistically significantly across seven embedding-model families spanning a tenfold parameter range, with held-out full-BEIR validation confirming the lift on every model tested.

[140] fg-expo: Frontier-guided exploration-prioritized policy optimization via adaptive kl and gaussian curriculum cs.LG | cs.AI | cs.CLPDF

Mingxiong Lin, Zhangquan Gong, Maowen Tang, Qian Li, Chuangchuang Wang

TL;DR: 本文提出FG-ExPO，一种针对LLM数学推理中强化学习（RLVR）的改进算法。它通过自适应KL惩罚和基于高斯分布的课程采样，解决了现有主流算法GRPO在探索限制和训练数据利用上的两个低效问题，从而在多个数学推理基准上取得了更好的性能。

Details

Motivation: 针对GRPO算法中存在的两个被忽视的低效问题：固定的KL系数在模型需要大幅偏离参考策略时过度限制了探索；以及均匀问题采样忽略了中等难度问题能提供最具信息量的梯度信号。

Result: 在DeepSeek-R1-Distill-Qwen-1.5B和Qwen3-8B-Base模型上，于六个主流数学推理基准进行评估。FG-ExPO一致优于原始GRPO，在AIME 2025的pass@32指标上绝对提升13.34%（从63.33%到76.67%），在8B模型上平均pass@32增益为2.66。pass@32相比pass@1的更大提升验证了其在固定推理预算下扩大了模型的有效探索空间。

Insight: 创新点在于引入了两个轻量级组件：基于批次平均准确率的平滑非线性函数自适应调整KL惩罚强度的AKL，以及以中等准确率（约0.5）为中心的高斯分布为问题分配采样权重的GCS。核心思想是将训练聚焦于模型的学习前沿，并动态平衡探索与利用。

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become the standard paradigm for LLM mathematical reasoning, with Group Relative Policy Optimization (GRPO) serving as the dominant algorithm. We identify two overlooked inefficiencies inherent in GRPO. First, a fixed KL coefficient overly restricts policy exploration at moments when the model needs to diverge significantly from the reference policy. Second, uniform question sampling overlooks that moderately difficult problems produce the most informative gradient signals. We propose FG-ExPO, short for Frontier-Guided Exploration-Prioritized Policy Optimization, which integrates two lightweight components. Accuracy-Conditioned KL Scaling (AKL) adjusts the KL penalty strength through a smooth nonlinear function of batch average accuracy, loosening the constraint when the model performs poorly and strengthening it when the model achieves satisfactory results. Gaussian Curriculum Sampling (GCS) assigns sampling weights to questions following a Gaussian distribution centered at a moderate accuracy level around 0.5, focusing model training on its learning frontier. We conduct evaluations on DeepSeek-R1-Distill-Qwen-1.5B and Qwen3-8B-Base across six mainstream mathematical reasoning benchmarks. Experimental results demonstrate that FG-ExPO consistently outperforms vanilla GRPO. It delivers an absolute improvement of 13.34 on the AIME 2025 pass@32 metric, rising from 63.33 percent to 76.67 percent, and obtains an average pass@32 gain of 2.66 on the 8B model. The substantially larger performance gains observed on pass@32 compared to pass@1 verify that FG-ExPO enlarges the model’s effective exploration space under a fixed inference budget.

[141] MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification cs.LG | cs.AI | cs.CLPDF

Bo Zheng, Yudong Chen, Zihua Xiong, Shuai Fang, Peidong He

TL;DR: 本文提出了MaskTab，一个专为工业级表格数据设计的统一预训练框架。它通过专用的可学习标记编码缺失值，并采用双路径架构联合优化掩码重建与任务特定监督，以及通过MoE增强的损失自适应路由特征。该框架旨在解决表格数据高维、缺失值多、大规模标注稀缺的挑战，并证明表格数据在尊重其结构特性的前提下，同样适用于基础模型范式。

Details

Motivation: 解决工业表格数据固有的高维度、大量缺失值、大规模标注稀缺的难题，并为表格学习提供一个通用的自监督预训练框架，以弥补其与视觉和语言领域在基础模型应用上的差距。

Result: 在工业级基准测试中，相较于先前技术，在严格扩展下实现了AUC提升5.04%和KS提升8.28%。其表征能有效蒸馏到轻量级模型中，在严格的延迟和可解释性约束下，仍能实现AUC提升2.55%和KS提升4.85%，并增强了对分布偏移的鲁棒性。

Insight: 创新点包括：使用专用可学习标记编码缺失值以区分结构性缺失与随机丢弃；采用双路径架构联合优化掩码重建与监督任务；引入MoE增强的损失进行自适应特征路由。客观来看，该工作为表格数据提供了一种尊重其结构特性的基础模型处理方法，并展示了通过蒸馏实现高效部署的潜力。

Abstract: Tabular data forms the backbone of high-stakes decision systems in finance, healthcare, and beyond. Yet industrial tabular datasets are inherently difficult: high-dimensional, riddled with missing entries, and rarely labeled at scale. While foundation models have revolutionized vision and language, tabular learning still leans on handcrafted features and lacks a general self-supervised framework. We present MaskTab, a unified pre-training framework designed specifically for industrial-scale tabular data. MaskTab encodes missing values via dedicated learnable tokens, enabling the model to distinguish structural absence from random dropout. It jointly optimizes a hybrid supervised pre-training scheme–utilizing a twin-path architecture to reconcile masked reconstruction with task-specific supervision–and an MoE-augmented loss that adaptively routes features through specialized subnetworks. On industrial-scale benchmarks, it achieves +5.04% AUC and +8.28% KS over prior art under rigorous scaling. Moreover, its representations distill effectively into lightweight models, yielding +2.55% AUC and +4.85% KS under strict latency and interpretability constraints, while improving robustness to distribution shifts. Our work demonstrates that tabular data admits a foundation-model treatment–when its structural idiosyncrasies are respected.

[142] Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information cs.LG | cs.AI | cs.CLPDF

Guobin Shen, Xiang Cheng, Chenxiao Zhao, Lei Huang, Jindong Li

TL;DR: 本文提出了一种名为反自蒸馏（Anti-Self-Distillation, AntiSD）的新方法，用于提升语言模型在数学推理任务中的性能。该方法通过最大化学生模型与教师模型（即模型自身的特权上下文版本）之间的散度，而非传统的缩小散度，来特别关注对多步搜索至关重要的‘思考过程’token，从而更有效地利用自蒸馏进行推理强化学习。

Details

Motivation: 在数学推理任务中，传统的基于策略的自蒸馏方法（即学生模型向一个拥有特权上下文（如已验证的解决方案）的自身副本学习）效果不稳定。研究发现，问题根源在于特权上下文会扭曲教师模型对不同类型的token（如结构性连接词与思考性词汇）的置信度，从而抑制了模型的多步搜索能力。

Result: 在涵盖4B到30B参数规模的五个模型上，于数学推理基准测试中，AntiSD方法仅用基线方法（GRPO）2到10分之一的训练步数就达到了其准确率，并且最终准确率最高提升了11.5个百分点。

Insight: 核心创新点在于颠覆了传统自蒸馏的优化方向，通过最大化学生与教师模型的散度来反向增强对推理过程关键token的学习。该方法引入了一个基于教师模型熵的门控机制，使其成为一个即插即用的自蒸馏替代方案，为语言模型通过自身训练信号实现可扩展的自我提升开辟了新路径。

Abstract: On-policy self-distillation, where a student is pulled toward a copy of itself conditioned on privileged context (e.g., a verified solution or feedback), offers a promising direction for advancing reasoning capability without a stronger external teacher. Yet in math reasoning the gains are inconsistent, even when the same approach succeeds elsewhere. A pointwise mutual information analysis traces the failure to the privileged context itself: it inflates the teacher’s confidence on tokens already implied by the solution (structural connectives, verifiable claims) and deflates it on deliberation tokens (“Wait”, “Let”, “Maybe”) that drive multi-step search. We propose Anti-Self-Distillation (AntiSD), which ascends a divergence between student and teacher rather than descending it: this reverses the per-token sign and yields a naturally bounded advantage in one step. An entropy-triggered gate disables the term once the teacher entropy collapses, completing a drop-in replacement for default self-distillation. Across five models from 4B to 30B parameters on math reasoning benchmarks, AntiSD reaches the GRPO baseline’s accuracy in 2 to 10x fewer training steps and improves final accuracy by up to 11.5 points. AntiSD opens a path to scalable self-improvement, where a language model bootstraps its own reasoning through its training signal.

[143] Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control cs.LG | cs.CLPDF

Jiazheng Zhang, Ziche Fu, Junrui Shen, Yunbin Zhao, Yunke Zhang

TL;DR: 本文针对大语言模型（LLM）的强化学习微调（RLVR）中的策略熵控制问题，提出了一个理论框架来分析熵变化的令牌级机制。该框架引入了熵极性（entropy polarity）这一有符号的令牌级量，用于预测采样更新是扩大还是收缩策略熵，并揭示了强化高频高概率令牌倾向于收缩熵、而扩大熵通常需要低概率样本的结构性不对称。基于此，作者提出了极性感知策略优化（PAPO）方法，通过优势重加权自适应地在熵扩张和熵收缩更新间重新分配优化压力。

Details

Motivation: 现有基于熵的方法主要通过全局目标调节熵，而采样策略更新在令牌级别如何重塑策略熵的机制尚未得到充分探索。本文旨在从理论上理解RLVR中熵变化的微观机制，并实现对熵的精细控制。

Result: 在数学推理和智能体基准测试上的实验表明，PAPO方法在训练效率和奖励提升方面均持续优于竞争基线，取得了优越的性能。

Insight: 主要创新点在于：1）提出了熵极性的理论框架，首次对令牌级熵变化方向进行了一阶近似和预测；2）揭示了熵变化的结构性不对称性；3）设计了PAPO方法，利用熵轨迹作为在线相位信号，自适应地平衡探索与利用。这为RL微调中的熵控制提供了新的理论视角和更精细的调控工具。

Abstract: Policy entropy has emerged as a fundamental measure for understanding and controlling exploration in reinforcement learning with verifiable rewards (RLVR) for LLMs. However, existing entropy-aware methods mainly regulate entropy through global objectives, while the token-level mechanism by which sampled policy updates reshape policy entropy remains underexplored. In this work, we develop a theoretical framework of entropy mechanics in RLVR. Our analysis yields a first-order approximation of the entropy change, giving rise to entropy polarity, a signed token-level quantity that predicts how much a sampled update expands or contracts entropy. This analysis further reveals a structural asymmetry: reinforcing frequent high-probability tokens triggers contraction tendencies, whereas expansive tendencies typically require lower-probability samples or stronger distributional correction. Empirically, we show that entropy polarity reliably predicts entropy changes, and that positive and negative polarity branches play complementary roles in preserving exploration while strengthening exploitation. Building on these insights, we propose Polarity-Aware Policy Optimization (PAPO), which preserves both polarity branches and implements entropy control through advantage reweighting. With the empirical entropy trajectory as an online phase signal, PAPO adaptively reallocates optimization pressure between entropy-expanding and entropy-contracting updates. Experiments on mathematical reasoning and agentic benchmarks show that PAPO consistently outperforms competitive baselines, while delivering superior training efficiency and substantial reward improvements.

[144] GEAR: Granularity-Adaptive Advantage Reweighting for LLM Agents via Self-Distillation cs.LG | cs.AI | cs.CLPDF

Sijia Li, Yuchen Huang, Zifan Liu, Yanping Li, Jingjing Fu

TL;DR: 本文提出了一种名为GEAR（Granularity-adaptivE Advantage Reweighting）的粒度自适应优势重加权框架，用于改进LLM智能体的强化学习训练。该方法通过自蒸馏获取细粒度信号，自适应地识别轨迹中的语义偏差段，并调整其优势权重，从而在长视野任务中实现更有效的信用分配。

Details

Motivation: 动机在于解决LLM智能体强化学习中，仅依赖结果级奖励进行粗粒度监督所面临的挑战，即如何为长轨迹中的正确部分分配可靠的细粒度信用，以实现更有效的策略更新。

Result: 在八个数学推理和智能体工具使用基准测试中，使用Qwen3 4B和8B模型进行的实验表明，GEAR一致性地优于标准的GRPO、仅使用自蒸馏的基线方法以及token级或turn级的信用分配方法。在GRPO基线准确率较低的基准上，提升尤为显著，最高可达约20%。

Insight: 核心创新点在于通过自蒸馏获取教师-学生模型的参考引导差异信号，自适应地识别语义偏差起始点作为锚点，并据此划分信用区域：在保持一致的区域保留token级分辨率，在发生偏差的区域将后续内容分组为自适应段，并用偏差点的差异来调制该段的优势权重。这提供了一种动态、自适应的粒度信用分配机制，特别适用于具有挑战性的长视野任务。

Abstract: Reinforcement learning has become a widely used post-training approach for LLM agents, where training commonly relies on outcome-level rewards that provide only coarse supervision. While finer-grained credit assignment is promising for effective policy updates, obtaining reliable local credit and assigning it to the right parts of the long-horizon trajectory remains an open challenge. In this paper, we propose Granularity-adaptivE Advantage Reweighting (GEAR), an adaptive-granularity credit assignment framework that reshapes the trajectory-level GRPO advantage using token- and segment-level signals derived from self-distillation. GEAR compares an on-policy student with a ground-truth-conditioned teacher to obtain a reference-guided divergence signal for identifying adaptive segment boundaries and modulating local advantage weights. This divergence often spikes at the onset of a semantic deviation, while later tokens in the same autoregressive continuation may return to low divergence. GEAR therefore treats such spikes as anchors for adaptive credit regions: where the student remains aligned with the teacher, token-level resolution is preserved; where it departs, GEAR groups the corresponding continuation into an adaptive segment and uses the divergence at the departure point to modulate the segment’ s advantage. Experiments across eight mathematical reasoning and agentic tool-use benchmarks with Qwen3 4B and 8B models show that GEAR consistently outperforms standard GRPO, self-distillation-only baselines, and token- or turn-level credit-assignment methods. The gains are especially strong on benchmarks with lower GRPO baseline accuracy, reaching up to around 20% over GRPO, suggesting that the proposed adaptive reweighting scheme is especially useful in more challenging long-horizon settings.

[145] ORCE: Order-Aware Alignment of Verbalized Confidence in Large Language Models cs.LG | cs.CLPDF

Chen Li, Xiaoling Hu, Songzhu Zheng, Jiawei Zhou, Chao Chen

TL;DR: 本文提出了一种名为ORCE的解耦且顺序感知的框架，用于校准大语言模型中的语言化置信度。该方法首先生成答案，然后基于固定的问题-答案对估计置信度，从而在不干扰答案生成过程的情况下优化置信度校准。通过构建基于采样的代理并使用基于排序的强化学习目标，鼓励正确性估计更高的回答获得更高的语言化置信度。

Details

Motivation: 大语言模型经常在答案错误时仍表现出高度确定性，因此可靠的置信度估计对于实际部署至关重要。现有的语言化置信度方法通常联合优化答案生成和置信度生成，这可能导致置信度对齐目标干扰答案准确性。

Result: 在推理和知识密集型基准测试上的实验表明，该方法在基本保持答案准确性的同时，提高了校准和失败预测性能。

Insight: 核心创新点在于将置信度估计与答案生成解耦，并优化不同回答间置信度的相对排序。这提供了一种更可靠的语言化置信度对齐方法，避免了联合优化可能带来的干扰。

Abstract: Large language models (LLMs) often produce answers with high certainty even when they are incorrect, making reliable confidence estimation essential for deployment in real-world scenarios. Verbalized confidence, where models explicitly state their confidence in natural language, provides a flexible and user-facing uncertainty signal that can be applied even when token logits are unavailable. However, existing verbalized-confidence methods often optimize answer generation and confidence generation jointly, which can cause confidence-alignment objectives to interfere with answer accuracy. In this work, we propose a decoupled and order-aware framework for verbalized confidence calibration. Our method first generates an answer and then estimates confidence conditioned on the fixed question–answer pair, allowing confidence optimization without directly perturbing the answer-generation process. To align confidence with correctness likelihood, we construct a sampling-based surrogate from multiple model completions and optimize rank-based reinforcement learning objectives that encourage responses with higher estimated correctness likelihood to receive higher verbalized confidence. Experiments on reasoning and knowledge-intensive benchmarks show that our method improves calibration and failure prediction performance while largely preserving answer accuracy. These results demonstrate that verbalized confidence can be more reliably aligned by decoupling confidence estimation from answer generation and optimizing the relative ordering of confidence across responses.

[146] Solve the Loop: Attractor Models for Language and Reasoning cs.LG | cs.AI | cs.CL | cs.NEPDF

Jacob Fein-Ashley, Paria Rashidinejad

TL;DR: 本文提出了Attractor Models，一种新型循环Transformer架构，通过隐式微分求解固定点来迭代优化潜在表示，解决了传统循环模型训练不稳定、计算成本高和深度固定的问题。该方法在语言建模和推理任务上均取得显著提升，并展现出独特的平衡内化现象。

Details

Motivation: 传统循环Transformer虽能通过迭代优化提升语言建模和推理能力，但存在训练不稳定、优化部署成本高且循环深度固定的局限性，Attractor Models旨在通过固定点求解和隐式微分实现稳定、高效且自适应的迭代优化。

Result: 在语言建模中，Attractor Models相比标准Transformer和稳定循环模型实现了帕累托改进，困惑度最高降低46.6%，下游任务准确率最高提升19.7%，训练成本更低；在推理任务上，仅2700万参数的模型在Sudoku-Extreme和Maze-Hard上分别达到91.4%和93.1%的准确率，优于Claude和GPT o3等前沿模型。

Insight: 创新点在于将循环优化转化为可学习的固定点求解问题，通过隐式微分实现恒定训练内存和自适应迭代深度，并发现平衡内化现象，使模型在推理时可移除求解器而性能几乎无损，为迭代优化提供了可扩展的新范式。

Abstract: Looped Transformers offer a promising alternative to purely feed-forward computation by iteratively refining latent representations, improving language modeling and reasoning. Yet recurrent architectures remain unstable to train, costly to optimize and deploy, and constrained to small, fixed recurrence depths. We introduce Attractor Models, in which a backbone module first proposes output embeddings, then an attractor module refines them by solving for the fixed point, with gradients obtained through implicit differentiation. Thus, training memory remains constant in effective depth, and iterations are chosen adaptively by convergence. Empirically, Attractor Models outperform existing models across two regimes, large-scale language-model pretraining and reasoning with tiny models. In language modeling, Attractor Models deliver a Pareto improvement over standard Transformers and stable looped models across sizes, improving perplexity by up to 46.6% and downstream accuracy by up to 19.7% while reducing training cost. Notably, a 770M Attractor Model outperforms a 1.3B Transformer trained on twice as many tokens. On challenging reasoning tasks, we show that our model with only 27M parameters and approximately 1000 examples achieves 91.4% accuracy on Sudoku-Extreme and 93.1% on Maze-Hard, scaling favorably where frontier models like Claude and GPT o3, fail completely, and specialized recursive reasoners collapse at larger sizes. Lastly, we show that Attractor Models exhibit a novel phenomenon, which we call equilibrium internalization: fixed-point training places the model’s initial output embedding near equilibrium, allowing the solver to be removed at inference time with little degradation. Together, these results suggest that Attractor Models make iterative refinement scalable by turning recurrence into a computation the model can learn to internalize.

[147] Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts cs.LG | cs.CLPDF

Sagi Ahrac, Noya Hochwald, Mor Geva

TL;DR: 本文研究了稀疏混合专家（SMoE）模型中路由器与专家之间的几何耦合机制，揭示了路由器权重与专家权重在梯度更新方向上的一致性，并分析了辅助负载均衡损失如何破坏这种耦合结构。通过引入无参数的在线K-Means路由器，验证了几何耦合对有效路由的重要性。

Details

Motivation: SMoE模型在扩展语言模型时面临路由崩溃到少数专家以及辅助负载均衡损失可能降低专家专业化的问题，因此需要从机制上理解路由决策的形成过程。

Result: 在从头训练的1B参数SMoE模型中，更高的路由器分数预测了更强的专家神经元激活；在线K-Means路由器在仅带来适度困惑度增加的情况下实现了最低的负载不平衡，优于辅助损失和无损失平衡方法。

Insight: 创新点在于揭示了路由器与专家之间的几何耦合机制，即它们沿相同输入方向接收梯度，这支持了有效的分工；通过在线K-Means路由器验证了几何耦合是路由学习的关键部分，为改进SMoE训练提供了新视角。

Abstract: Sparse Mixture-of-Experts (SMoE) models enable scaling language models efficiently, but training them remains challenging, as routing can collapse onto few experts and auxiliary load-balancing losses can reduce specialization. Motivated by these hurdles, we study how routing decisions in SMoEs are formed mechanistically. First, we reveal a geometric coupling between routers and their corresponding experts. For a given token, the router weights for the selected expert and the expert weights processing it receive gradients along the same input direction, differing only in scalar coefficients. Thus, matched router–expert directions accumulate the same routed token history. This theoretical coupling also appears empirically in routing dynamics. In a $1$B SMoE trained from scratch, higher router scores predict stronger expert neuron activations, showing that routing decisions are mirrored inside the selected expert. Next, we analyze the effects of auxiliary load balancing on the router–expert geometric coupling, showing that such losses break this structure by spreading input-directed gradients across router weights, making distinct router directions nearly three times more similar to each other. Last, we demonstrate the centrality of geometric coupling for effective routing with a parameter-free online K-Means router, in which each expert maintains a running average of the hidden states routed to it and tokens are assigned based on cosine similarity. Compared with auxiliary-loss and loss-free balancing, this router achieves the lowest load imbalance with only a modest perplexity increase, indicating that geometric coupling captures a substantial part of what the router learns. Overall, our results explain how routers form assignment geometry that supports an effective division of labor.

[148] MEME: Multi-entity & Evolving Memory Evaluation cs.LG | cs.CLPDF

Seokwon Jung, Alexander Rubinstein, Arnas Uselis, Sangdoo Yun, Seong Joon Oh

TL;DR: MEME是一个用于评估LLM智能体在持久环境中多实体和动态演化记忆能力的基准测试，定义了六个任务，包括三个新任务：级联、缺失和删除。评估发现现有记忆系统在依赖推理任务上表现极差，仅Claude Opus 4.7结合文件代理能部分提升性能，但成本极高。

Details

Motivation: 现有基准仅评估单实体更新，无法满足LLM智能体在持久环境中跨会话存储、更新和推理多实体动态信息的需求，因此提出MEME以全面评估多实体和演化记忆能力。

Result: 在100个受控情节上评估六种记忆系统，默认配置下依赖推理任务平均准确率极低（级联3%、缺失1%），提示优化、深度检索、减少噪声和更强LLM均无法显著改善，仅Claude Opus 4.7结合文件代理能部分提升但成本增加约70倍。

Insight: 创新点在于定义了多实体和演化记忆的完整评估空间，引入三个新任务以评估依赖推理和删除后状态；客观分析表明当前记忆系统在依赖推理上存在根本性缺陷，且性能提升严重依赖高成本配置，揭示了实际部署的挑战。

Abstract: LLM-based agents increasingly operate in persistent environments where they must store, update, and reason over information across many sessions. While prior benchmarks evaluate only single-entity updates, MEME defines six tasks spanning the full space defined by the multi-entity and evolving axes, including three not scored by prior work: Cascade and Absence (dependency reasoning) and Deletion (post-removal state). Evaluating six memory systems spanning three memory paradigms on 100 controlled episodes, we find that all systems collapse on dependency reasoning under the default configuration (Cascade: 3%, Absence: 1% in average accuracy) despite adequate static retrieval performance. Prompt optimization, deeper retrieval, reduced filler noise, and most stronger LLMs fail to close this gap. Only a file-based agent paired with Claude Opus 4.7 as its internal LLM partially closes the gap, but at ~70x the baseline cost, indicating closure currently depends on configurations that are not practical at scale. Code and data are available on the project page: https://seokwonjung-jay.github.io/meme-eval/.

[149] TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment cs.LG | cs.AI | cs.CVPDF

Jiaming Li, Chenyu Zhu, Zhiyuan Ma, Nanxi Yi, Youjun Bao

TL;DR: 本文提出了一种名为轨迹匹配策略优化（TMPO）的新方法，用于解决扩散模型与下游任务对齐时常见的奖励黑客问题，该方法通过匹配轨迹级别的奖励分布，而非最大化标量奖励，来保持生成多样性和质量。

Details

Motivation: 现有基于强化学习的扩散模型对齐方法存在严重的奖励黑客问题，导致视觉模式崩溃和生成多样性下降，其根源在于这些方法具有模式寻求特性，会过度集中于少数高奖励路径。

Result: 在人类偏好、组合生成和文本渲染等多种对齐任务上的广泛实验表明，TMPO在生成多样性上比最先进方法提升了9.1%，在所有下游任务和效率指标上均取得了有竞争力的性能，实现了奖励与多样性的最佳权衡。

Insight: 核心创新点在于将标量奖励最大化问题转化为轨迹级奖励分布匹配问题，并提出了Softmax轨迹平衡目标，该目标继承了前向KL散度的模式覆盖特性；同时，通过动态随机树采样技术，有效减少了大规模流匹配模型的多轨迹训练时间。

Abstract: Reinforcement learning (RL) has shown extraordinary potential in aligning diffusion models to downstream tasks, yet most of them still suffer from significant reward hacking, which degrades generative diversity and quality by inducing visual mode collapse and amplifying unreliable rewards. We identify the root cause as the mode-seeking nature of these methods, which maximize expected reward without effectively constraining probability distribution over acceptable trajectories, causing concentration on a few high-reward paths. In contrast, we propose Trajectory Matching Policy Optimization (TMPO), which replaces scalar reward maximization with trajectory-level reward distribution matching. Specifically, TMPO introduces a Softmax Trajectory Balance (Softmax-TB) objective to match the policy probabilities of K trajectories to a reward-induced Boltzmann distribution. We prove that this objective inherits the mode-covering property of forward KL divergence, preserving coverage over all acceptable trajectories while optimizing reward. To further reduce multi-trajectory training time on large-scale flow-matching models, TMPO incorporates Dynamic Stochastic Tree Sampling, where trajectories share denoising prefixes and branch at dynamically scheduled steps, reducing redundant computation while improving training effectiveness. Extensive results across diverse alignment tasks such as human preference, compositional generation and text rendering show that TMPO improves generative diversity over state-of-the-art methods by 9.1%, and achieves competitive performance in all downstream and efficiency metrics, attaining the optimal trade-off between reward and diversity.

[150] Unlocking Compositional Generalization in Continual Few-Shot Learning cs.LG | cs.CVPDF

Phu-Quy Nguyen-Lam, Phu-Hoa Pham, Dao Sy Duy Minh, Chi-Nguyen Tran, Huynh Trung Kiet

TL;DR: 本文提出了一种新的持续少样本学习范式，通过解耦表示学习和组合推理来解决组合泛化问题。该方法利用自监督视觉Transformer的补丁级语义几何，在训练阶段优化槽表示以保持整体类别身份和可泛化的对象级几何，在推理阶段动态组合槽以匹配新场景。

Details

Motivation: 现有持续学习模型要么将场景坍缩为全局嵌入，要么使用部件级匹配目标导致表示与已见模式绑定过紧，无法泛化到真正的新概念，因此需要解决表示学习与组合推理之间的结构冲突。

Result: 在标准持续学习基准测试中，该方法实现了最先进的未见概念泛化性能，并保持了最小的遗忘。

Insight: 核心创新在于严格解耦表示学习和组合推理，利用自监督ViT的固有几何特性，通过双阶段策略（训练时优化整体类别身份，推理时动态组合）同时防止表示漂移并保持特征对新概念的迁移能力。

Abstract: Object-centric representations promise a key property for few-shot learning: Rather than treating a scene as a single unit, a model can decompose it into individual object-level parts that can be matched and compared across different concepts. In practice, this potential is rarely realized. Continual learners either collapse scenes into global embeddings, or train with part-level matching objectives that tie representations too closely to seen patterns, leaving them unable to generalize to truly novel concepts. In this paper, we identify this fundamental structural conflict and pioneer a new paradigm that strictly decouples representation learning from compositional inference. Leveraging the inherent patch-level semantic geometry of self-supervised Vision Transformers (ViTs), our framework employs a dual-phase strategy. During training, slot representations are optimized entirely toward holistic class identity, preserving highly generalizable, object-level geometries. At inference, preserved slots are dynamically composed to match novel scenes. We demonstrate that this paradigm offers dual structural benefits: The frozen backbone naturally prevents representation drift, while our lightweight, holistic optimization preserves the features’ capacity for novel-concept transfer. Extensive experiments validate this approach, achieving state-of-the-art unseen-concept generalization and minimal forgetting across standard continual learning benchmarks.

[151] Resilient Vision-Tabular Multimodal Learning under Modality Missingness cs.LG | cs.CVPDF

Camillo Maria Caruso, Valerio Guarrasi, Paolo Soda

TL;DR: 本文提出了一种针对医学应用中视觉（如X光图像）与表格（如临床变量）多模态学习的鲁棒性Transformer框架，旨在解决现实场景中模态缺失的普遍问题。该方法通过可学习的模态令牌加权单模态表示，并利用掩码自注意力进行中间融合，在训练中引入模态丢弃正则化以增强模型在部分数据可用时的互补信息利用能力。

Details

Motivation: 现有多模态学习方法通常假设模态完整可用，而现实临床环境中常出现整个模态或部分特征缺失的情况，因此需要一种无需依赖数据填补或启发式模型切换、能直接处理普遍模态缺失的鲁棒性多模态学习框架。

Result: 在MIMIC-CXR数据集（配对MIMIC-IV结构化临床数据）的14种诊断发现的多标签分类任务上，通过系统性的压力测试（逐步增加训练和推理中各模态的缺失比例），该方法在所有缺失情况下均优于代表性基线，表现出更平缓的性能下降和更强的鲁棒性。

Insight: 创新点包括：1）通过掩码自注意力在注意力层级排除缺失模态，防止其参与信息聚合和梯度传播；2）引入模态丢弃正则化策略，鼓励模型在部分模态可用时利用互补信息；3）中间融合与联合微调被证明是鲁棒多模态推理的关键。从客观角度看，该方法为处理现实世界不完整多模态数据提供了一种端到端的通用解决方案。

Abstract: Multimodal deep learning has shown strong potential in medical applications by integrating heterogeneous data sources such as medical images and structured clinical variables. However, most existing approaches implicitly assume complete modality availability, an assumption that rarely holds in real-world clinical settings where entire modalities and individual features are frequently missing. In this work, we propose a multimodal transformer framework for joint vision-tabular learning explicitly designed to operate under pervasive modality missingness, without relying on imputation or heuristic model switching. The architecture integrates three components: a vision, a tabular, and a multimodal fusion encoder. Unimodal representations are weighted through learnable modality tokens and fused via intermediate fusion with masked self-attention, which excludes missing tokens and modalities from information aggregation and gradient propagation. To further enhance resilience, we introduce a modality-dropout regularization strategy that stochastically removes available modalities during training, encouraging the model to exploit complementary information under partial data availability. We evaluate our approach on the MIMIC-CXR dataset paired with structured clinical data from MIMIC-IV for multilabel classification of 14 diagnostic findings with incomplete annotations. Two parallel systematic stress-test protocols progressively increase training and inference missingness in each modality separately, spanning fully multimodal to fully unimodal scenarios. Across all missingness regimes, the proposed method consistently outperforms representative baselines, showing smoother performance degradation and improved robustness. Ablation studies further demonstrate that attention-level masking and intermediate fusion with joint fine-tuning are key to resilient multimodal inference.

eess.IV [Back]

[152] Streaming of rendered content with adaptive frame rate and resolution eess.IV | cs.CV | cs.GR | cs.MMPDF

Yaru Liu, Joseph G. March, Rafal K. Mantiuk

TL;DR: 该论文提出了一种自适应调整帧率和分辨率的渲染内容流传输系统，旨在在带宽受限条件下提升感知质量并降低渲染成本。通过利用人类视觉系统的时空限制，系统根据场景内容和运动速度动态调整帧率与分辨率，并使用轻量级神经网络预测最优组合。

Details

Motivation: 现有解决方案在服务器上以固定帧率（如30或60 fps）渲染内容，并在带宽受限时降低分辨率，导致在带宽约束下渲染质量不佳。本文旨在通过自适应调整帧率和分辨率来解决这一问题，以提升感知质量。

Result: 论文提出了一种基于神经网络的预测系统，在给定传输带宽、内容和运动速度下预测最优帧率与分辨率组合，显著提升了感知质量并最小化了计算成本。系统在大型渲染内容数据集上训练，并使用感知视频质量指标进行标注。

Insight: 创新点在于结合人类视觉系统的时空限制，自适应调整帧率和分辨率，而非仅固定帧率或降低分辨率。该方法与编解码器无关，仅需对现有渲染基础设施进行最小修改，可借鉴于实时流媒体和图形渲染优化领域。

Abstract: Streaming rendered content is an attractive way to bring high-quality graphics to billions of mobile devices that do not have sufficient rendering power. Existing solutions render content on a server at a fixed frame rate, typically 30 or 60 frames per second, and reduce resolution when bandwidth is restricted. However, this strategy leads to suboptimal rendering quality under the bandwidth constraints. In this work, we exploit the spatio-temporal limits of the human visual system to improve perceived quality while reducing rendering costs by adaptively adjusting both frame rate and resolution based on scene content and motion. Our approach is codec-agnostic and requires only minimal modifications to existing rendering infrastructure. We propose a system in which a lightweight neural network predicts the optimal combination of frame rate and resolution for a given transmission bandwidth, content, and motion velocity. This prediction significantly enhances perceptual quality while minimizing computational cost under bandwidth constraints. The network is trained on a large dataset of rendered content labeled with a perceptual video quality metric. The dataset and further information can be found at the project web page: https://www.cl.cam.ac.uk/research/rainbow/projects/adaptive_streaming/.

cs.IR [Back]

[153] Much of Geospatial Web Search Is Beyond Traditional GIS cs.IR | cs.AI | cs.CL | cs.HCPDF

Ilya Ilyankou, Stefano Cavazzi, James Haworth

TL;DR: 本研究通过分析MS MARCO语料库中的101万条真实Bing搜索查询，发现地理空间网络搜索的占比远高于传统标注方案所显示的6.17%，实际达到18.0%。研究揭示了地理空间查询主要由交易性和实用性查询主导，如成本、营业时间等，这些查询大多超出了传统地理信息系统（GIS）和知识图谱的服务范围。

Details

Motivation: 现有标注方案低估了网络搜索中与地点相关的查询频率，且大规模地理空间网络查询的分布特征尚未得到充分刻画。

Result: 识别出181,827条地理空间查询（占比18.0%），构建了包含88个类别的分类体系，其中成本与价格类查询占比15.3%，远超自然地理主题的总和。

Insight: 地理空间搜索的实际需求广泛存在于传统GIS覆盖范围之外，这对混合检索架构和大型语言模型的地理推理基准测试提出了新要求；研究公开了标注数据集、分类器和分类体系。

Abstract: Web search queries concern place far more often than existing labelling schemes suggest, yet the landscape of geospatial web search queries - what people ask of place, and how often - remains poorly characterised at scale. We apply dense sentence embeddings, a lightweight SetFit classifier, and density-based clustering to the full MS MARCO corpus of 1.01 million real Bing queries without prior filtering for toponyms or spatial keywords, identifying 181,827 geospatial queries (18.0%), nearly threefold the 6.17% labelled as Location in the original annotations. The resulting taxonomy of 88 query categories reveals that geospatial web search is dominated by transactional and practical lookups: costs and prices alone account for 15.3% of geospatial queries, nearly twice the size of the entire physical geography theme. Much of this activity - costs, opening hours, contact details, weather, travel recommendations - falls outside the scope traditional GIS systems and knowledge graphs are built to serve. The categories vary substantially in the kind of answer they admit, from deterministic lookups answerable from spatial databases or knowledge graphs to evaluative or temporally volatile queries that require generative or real-time systems. We discuss implications for hybrid retrieval architectures and for benchmarks of geographic reasoning in large language models. We openly release the labelled dataset, classifier, and taxonomy.

[154] Very Efficient Listwise Multimodal Reranking for Long Documents cs.IR | cs.AI | cs.CV | cs.MMPDF

Yiqun Sun, Pengfei Wei, Lawrence B. Hsieh

TL;DR: 本文提出了一种名为ZipRerank的高效列表式多模态重排序器，用于处理长文档的视觉中心检索和多模态检索增强生成任务。它通过轻量级查询-图像早期交互机制减少输入长度，并通过单次前向传播为所有候选打分来消除自回归解码，从而显著降低了计算开销。

Details

Motivation: 解决现有基于视觉语言模型的多模态重排序器因长视觉令牌序列和多步自回归解码导致的计算成本高、延迟大的问题，以提升其在延迟敏感的实际系统中的实用性。

Result: 在MMDocIR基准测试上的大量实验表明，ZipRerank在性能上匹配或超越了最先进的多模态重排序器，同时将LLM推理延迟降低了一个数量级。

Insight: 创新点包括：1) 轻量级查询-图像早期交互以减少输入序列长度；2) 单次前向传播的非自回归打分机制；3) 结合大规模文本数据列表式预训练和VLM教师蒸馏软排序监督的两阶段训练策略。这些设计在保持高精度的同时大幅提升了效率。

Abstract: Listwise reranking is a key yet computationally expensive component in vision-centric retrieval and multimodal retrieval-augmented generation (M-RAG) over long documents. While recent VLM-based rerankers achieve strong accuracy, their practicality is often limited by long visual-token sequences and multi-step autoregressive decoding. We propose ZipRerank, a highly efficient listwise multimodal reranker that directly addresses both bottlenecks. It reduces input length via a lightweight query-image early interaction mechanism and eliminates autoregressive decoding by scoring all candidates in a single forward pass. To enable effective learning, ZipRerank adopts a two-stage training strategy: (i) listwise pretraining on large-scale text data rendered as images, and (ii) multimodal finetuning with VLM-teacher-distilled soft-ranking supervision. Extensive experiments on the MMDocIR benchmark show that ZipRerank matches or surpasses state-of-the-art multimodal rerankers while reducing LLM inference latency by up to an order of magnitude, making it well-suited for latency-sensitive real-world systems. The code is available at https://github.com/dukesun99/ZipRerank.

cs.RO [Back]

[155] DreamAvoid: Critical-Phase Test-Time Dreaming to Avoid Failures in VLA Policies cs.RO | cs.AI | cs.CL | cs.CVPDF

Xianzhe Fan, Yuxiang Lu, Shenyuan Gao, Xiaoyang Wu, Ruihua Han

TL;DR: 本文提出DreamAvoid框架，通过关键阶段测试时‘做梦’机制，使视觉-语言-动作模型能够预见并避免失败，从而提高精细操作任务的鲁棒性。

Details

Motivation: 现有VLA模型主要依赖成功演示进行训练，缺乏对关键阶段失败风险的显式认知，导致在精细操作中微小的动作误差可能迅速演变为不可恢复的失败。

Result: 在真实世界操作任务和仿真基准测试中，DreamAvoid能有效避免失败，提升了整体任务成功率。

Insight: 创新点包括：关键阶段触发机制、动作块采样与评估的‘做梦’框架，以及基于混合数据（成功、失败和边界案例）训练的自适应边界学习范式，增强了模型对成功与失败微妙边界的理解。

Abstract: Vision-Language-Action (VLA) models are often brittle in fine-grained manipulation, where minor action errors during the critical phases can rapidly escalate into irrecoverable failures. Since existing VLA models rely predominantly on successful demonstrations for training, they lack an explicit awareness of failure during these critical phases. To address this, we propose DreamAvoid, a critical-phase test-time dreaming framework that enables VLA models to anticipate and avoid failures. We also introduce an autonomous boundary learning paradigm to refine the system’s understanding of the subtle boundary between success and failure. Specifically, we (1) utilize a Dream Trigger to determine whether the execution has entered a critical phase, (2) sample multiple candidate action chunks from the VLA via an Action Proposer, and (3) employ a Dream Evaluator, jointly trained on mixed data (success, failure, and boundary cases), to “dream” the short-horizon futures corresponding to the candidate actions, evaluate their values, and select the optimal action. We conduct extensive evaluations on real-world manipulation tasks and simulation benchmarks. The results demonstrate that DreamAvoid can effectively avoid failures, thereby improving the overall task success rate. Our code is available at https://github.com/XianzheFan/DreamAvoid.

[156] World Action Models: The Next Frontier in Embodied AI cs.RO | cs.CL | cs.CVPDF

Siyin Wang, Junhao Shi, Zhaoyang Fu, Xinzhe He, Feihong Liu

TL;DR: 这篇论文对世界行动模型（WAMs）这一新兴的具身AI范式进行了首次系统性综述。WAMs将预测环境动态的世界模型与视觉-语言-行动（VLA）模型的行动生成能力相结合，旨在联合建模未来状态和行动。论文明确了WAMs的定义，梳理了其与相关概念的区别，并追溯了VLA与世界模型研究融合的起源。作者构建了一个涵盖级联与联合WAMs的分类法，系统分析了支撑其发展的数据生态系统和新兴评估协议，旨在为该快速发展的领域提供一个统一的概念框架。

Details

Motivation: 现有VLA模型在具身策略学习中实现了良好的语义泛化，但它们学习的是反应式的观测到行动的映射，没有显式地建模物理世界在干预下的演化过程。为了弥补这一局限，研究开始将预测环境动态的世界模型整合到行动生成流程中，从而催生了WAMs这一新范式。然而，相关文献在架构、学习目标和应用场景上仍较为分散，缺乏统一的概念框架。

Result: 本文是一篇综述性论文，未提出具体的新模型，因此没有报告定量的性能结果。其主要成果是首次对WAMs领域进行了系统性的梳理和总结，构建了结构化的分类法，并综合了围绕视觉保真度、物理常识和行动合理性组织的新兴评估协议。

Insight: 论文的核心创新在于提出了“世界行动模型（WAMs）”这一统一概念，并为其建立了清晰的定义和分类体系。它将预测建模与行动生成统一起来，强调对未来状态和行动的联合分布进行建模，这超越了仅预测行动的VLA模型。从客观角度看，该综述为这个新兴且分散的研究领域提供了一个宝贵的“地图”，明确了级联与联合两种核心架构范式及其权衡，并系统性地分析了数据来源和评估维度，有助于引导未来的研究方向。

Abstract: Vision-Language-Action (VLA) models have achieved strong semantic generalization for embodied policy learning, yet they learn reactive observation-to-action mappings without explicitly modeling how the physical world evolves under intervention. A growing body of work addresses this limitation by integrating world models, predictive models of environment dynamics, into the action generation pipeline. We term this emerging paradigm World Action Models (WAMs): embodied foundation models that unify predictive state modeling with action generation, targeting a joint distribution over future states and actions rather than actions alone. However, the literature remains fragmented across architectures, learning objectives, and application scenarios, lacking a unified conceptual framework. We formally define WAMs and disambiguate them from related concepts, and trace the foundations and early integration of VLA and world model research that gave rise to this paradigm. We organize existing methods into a structured taxonomy of Cascaded and Joint WAMs, with further subdivision by generation modality, conditioning mechanism, and action decoding strategy. We systematically analyze the data ecosystem fueling WAMs development, spanning robot teleoperation, portable human demonstrations, simulation, and internet-scale egocentric video, and synthesize emerging evaluation protocols organized around visual fidelity, physical commonsense, and action plausibility. Overall, this survey provides the first systematic account of the WAMs landscape, clarifies key architectural paradigms and their trade-offs, and identifies open challenges and future opportunities for this rapidly evolving field.

[157] Overcoming Dynamics-Blindness: Training-Free Pace-and-Path Correction for VLA Models cs.RO | cs.AI | cs.CV | cs.LGPDF

Yanyan Zhang, Chaoda Song, Vikash Singh, Xinpeng Li, Kai Ye

TL;DR: 该论文提出了一种名为Pace-and-Path Correction的训练免费、闭式推理时操作符，用于解决视觉-语言-动作模型在动态场景中因单帧观察范式导致的时序动态盲区问题。该方法通过单一二次成本函数的联合最小化，分解出两个正交通道来修正动作块的执行步调和空间路径，从而吸收感知到的动态变化。

Details

Motivation: 现有VLA模型大多在单帧观察范式下训练，导致其结构上对时序动态不敏感，在非平稳场景中性能严重下降。现有方法要么需要昂贵的重新训练，要么存在延迟瓶颈和动作块间时间一致性问题。

Result: 在专门设计的诊断基准MoveBench上评估，该方法在纯动态和静态-动态混合环境中，分别比基础VLA模型的绝对成功率提升了28.8%和25.9%，并持续优于最先进的训练免费包装器和动态自适应方法。

Insight: 创新点在于提出了一种无需训练、闭式求解的推理时包装器，通过将动态补偿分解为步调（时间）和路径（空间）两个正交通道进行联合优化，统一解决了动作块执行中的时序一致性和空间偏移问题，为VLA模型提供了高效且通用的动态适应能力。

Abstract: Vision-Language-Action (VLA) models achieve remarkable flexibility and generalization beyond classical control paradigms. However, most prevailing VLAs are trained under a single-frame observation paradigm, which leaves them structurally blind to temporal dynamics. Consequently, these models degrade severely in non-stationary scenarios, even when trained or finetuned on dynamic datasets. Existing approaches either require expensive retraining or suffer from latency bottlenecks and poor temporal consistency across action chunks. We propose Pace-and-Path Correction, a training-free, closed-form inference-time operator that wraps any chunked-action VLA. From a single quadratic cost, joint minimization yields a unified solution that decomposes orthogonally into two distinct channels. The pace channel compresses execution along the planned direction, while the path channel applies an orthogonal spatial offset, jointly absorbing the perceived dynamics within the chunk window. We evaluate our approach on a comprehensive diagnostic benchmark MoveBench designed to isolate motion as the sole controlled variable. Empirical results demonstrate that our framework consistently outperforms state-of-the-art training-free wrappers and dynamic-adaptive methods and improves success rates by up to 28.8% and 25.9% in absolute terms over foundational VLA models in dynamic-only and static-dynamic mixed environments, respectively.

[158] See What Matters: Differentiable Grid Sample Pruning for Generalizable Vision-Language-Action Model cs.RO | cs.CVPDF

Yixu Feng, Zinan Zhao, Yanxiang Ma, Chenghao Xia, Chengbin Du

TL;DR: 本文提出了一种名为可微分网格采样器（GridS）的即插即用模块，用于解决视觉-语言-动作（VLA）模型在机器人操作中计算成本高的问题。该方法通过任务感知的连续视觉令牌重采样，在保持关键空间几何信息（如接触点）的同时，实现了大幅压缩（使用少于10%的原始视觉令牌），从而在显著减少计算量（FLOPs减少76%）的情况下不损失任务成功率。

Details

Motivation: 现有VLA模型的令牌剪枝方法存在一个根本性权衡：激进的压缩会丢弃关键的几何细节（如接触点），导致性能严重下降，从而限制了可实现的压缩率和加速潜力。本文旨在打破这一权衡，将压缩重新构想为视觉编码器中一种几何感知的连续令牌重采样。

Result: 在LIBERO基准测试和真实机器人平台上的实验表明，GridS在保持成功率不下降的前提下，实现了76%的FLOPs减少，并验证了迄今为止报告的最低可行视觉令牌数量。

Insight: 核心创新点在于将压缩问题重新定义为几何感知的连续重采样，而非离散的剪枝。具体而言，GridS通过自适应预测一组最小化的显著坐标，并利用可微分插值提取特征，从而在实现高压缩率的同时，保留了任务关键的空间信息。这为VLA模型的高效部署提供了一种新的、性能无损的压缩思路。

Abstract: Vision-Language-Action (VLA) models have shown remarkable promise in robotics manipulation, yet their high computational cost hinders real-time deployment. Existing token pruning methods suffer from a fundamental trade-off: aggressive compression using pruning inevitably discards critical geometric details like contact points, leading to severe performance degradation. This forces a compromise, limiting the achievable compression rate and thus the potential speedup. We argue that breaking this trade-off requires rethinking compression as a geometry-aware, continuous token resampling in the vision encoder. To this end, we propose the Differentiable Grid Sampler (GridS), a plug-and-play module that performs task-aware, continuous resampling of visual tokens in VLA. By adaptively predicting a minimal set of salient coordinates and extracting features via differentiable interpolation, GridS preserves essential spatial information while achieving drastic compression (with fewer than 10% original visual tokens). Experiments on both LIBERO benchmark and a real robotic platform demonstrate that validating the lowest feasible visual token count reported to date, GridS achieves a 76% reduction in FLOPs with no degradation in the success rate. The code is available at https://github.com/Fediory/Grid-Sampler.

[159] From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation cs.RO | cs.CVPDF

Yajie Li, Bozhou Zhang, Chun Gu, Zipei Ma, Jiahui Zhang

TL;DR: 本文提出MoLA（Mixture of Latent Actions），一种面向控制的接口，旨在解决视频生成模型在机器人操作中预测未来观测时，视觉真实性与控制相关性不匹配的问题。MoLA通过预训练的逆动力学模型混合，从生成的视觉转换中推断隐含的潜在动作，从而将想象的未来视频转化为可执行的表示，以提升任务成功率、时间一致性和泛化能力。

Details

Motivation: 视频生成模型为机器人操作提供了预测长期未来观测的想象机制，但现有方法（如基于预测帧的策略或直接解码视频为动作）存在视觉真实性与控制相关性不匹配的问题，导致预测观测强调感知保真度而非状态转换的动作中心原因，从而产生间接和不稳定的控制。

Result: 在模拟基准（LIBERO、CALVIN和LIBERO-Plus）和真实世界机器人操作任务上评估，MoLA在任务成功率、时间一致性和泛化方面均取得了持续提升，表明其有效性。

Insight: 创新点在于提出了一种控制导向的接口，利用模态感知的逆动力学模型混合（捕捉语义、深度和流线索）来推断潜在动作，提供结构化且基于物理的动作表示，从而弥合视频想象与策略执行之间的差距。从客观角度看，该方法通过多模态线索整合，增强了动作表示的物理基础，可能为机器人控制中的视频生成应用提供更直接和稳定的解决方案。

Abstract: Video generation models offer a promising imagination mechanism for robot manipulation by predicting long-horizon future observations, but effectively exploiting these imagined futures for action execution remains challenging. Existing approaches either condition policies on predicted frames or directly decode generated videos into actions, both suffering from a mismatch between visual realism and control relevance. As a result, predicted observations emphasize perceptual fidelity rather than action-centric causes of state transitions, leading to indirect and unstable control. To address this gap, we propose MoLA (Mixture of Latent Actions), a control-oriented interface that transforms imagined future videos into executable representations. Instead of passing predicted frames directly to the policy, MoLA leverages a mixture of pretrained inverse dynamics models to infer a mixture of latent actions implied by generated visual transitions. These modality-aware inverse dynamics models capture complementary semantic, depth, and flow cues, providing a structured and physically grounded action representation that bridges video imagination and policy execution. We evaluate our approach on simulated benchmarks (LIBERO, CALVIN, and LIBERO-Plus) and real-world robot manipulation tasks, achieving consistent gains in task success, temporal consistency, and generalization.

Table of Contents

cs.CL [Back]

[1] ClinicalBench: Stress-Testing Assertion-Aware Retrieval for Cross-Admission Clinical QA on MIMIC-IV cs.CL | cs.AI | cs.IRPDF

[2] The Bicameral Model: Bidirectional Hidden-State Coupling Between Parallel Language Models cs.CL | cs.AI | cs.LGPDF

[3] ReVision: Scaling Computer-Use Agents via Temporal Visual Redundancy Reduction cs.CLPDF

[4] HEBATRON: A Hebrew-Specialized Open-Weight Mixture-of-Experts Language Model cs.CLPDF

[5] ReAD: Reinforcement-Guided Capability Distillation for Large Language Models cs.CL | cs.AIPDF

[6] An Empirical Study of Automating Agent Evaluation cs.CLPDF

[7] Deep Reasoning in General Purpose Agents via Structured Meta-Cognition cs.CL | cs.AIPDF

[8] Agent-BRACE: Decoupling Beliefs from Actions in Long-Horizon Tasks via Verbalized State Uncertainty cs.CL | cs.AIPDF

[9] Checkup2Action: A Multimodal Clinical Check-up Report Dataset for Patient-Oriented Action Card Generation cs.CL | cs.CVPDF

[10] Taming Extreme Tokens: Covariance-Aware GRPO with Gaussian-Kernel Advantage Reweighting cs.CL | cs.AI | cs.LGPDF

[11] OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models cs.CLPDF

[12] Human-Grounded Multimodal Benchmark with 900K-Scale Aggregated Student Response Distributions from Japan’s National Assessment of Academic Ability cs.CLPDF

[13] Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation cs.CLPDF

[14] Probabilistic Calibration Is a Trainable Capability in Language Models cs.CLPDF

[15] Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Models cs.CL | cs.LGPDF

[16] YFPO: A Preliminary Study of Yoked Feature Preference Optimization with Neuron-Guided Rewards for Mathematical Reasoning cs.CLPDF

[17] On Predicting the Post-training Potential of Pre-trained LLMs cs.CLPDF

[18] Towards Visually-Guided Movie Subtitle Translation for Indic Languages cs.CLPDF

[19] Learning Agentic Policy from Action Guidance cs.CLPDF

[20] SAGE: Scalable Automated Robustness Augmentation for LLM Knowledge Evaluation cs.CLPDF

[21] SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphs cs.CLPDF

[22] Latent Causal Void: Explicit Missing-Context Reconstruction for Misinformation Detection cs.CL | cs.SIPDF

[23] Combining On-Policy Optimization and Distillation for Long-Context Reasoning in Large Language Models cs.CLPDF

[24] Overview of the MedHopQA track at BioCreative IX: track description, participation and evaluation of systems for multi-hop medical question answering cs.CL | cs.IRPDF

[25] MedHopQA: A Disease-Centered Multi-Hop Reasoning Benchmark and Evaluation Framework for LLM-Based Biomedical Question Answering cs.CL | cs.AI | cs.IRPDF

[26] Context Convergence Improves Answering Inferential Questions cs.CL | cs.IRPDF

[27] Scalable Token-Level Hallucination Detection in Large Language Models cs.CL | cs.AI | cs.LGPDF

[28] Question Difficulty Estimation for Large Language Models via Answer Plausibility Scoring cs.CL | cs.IRPDF

[29] ORBIT: Preserving Foundational Language Capabilities in GenRetrieval via Origin-Regulated Merging cs.CL | cs.IR | cs.LGPDF

[30] A Causal Language Modeling Detour Improves Encoder Continued Pretraining cs.CL | cs.AIPDF

cs.CV [Back]

[31] Principle-Guided Supervision for Interpretable Uncertainty in Medical Image Segmentation cs.CVPDF

[32] HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer cs.CV | cs.MMPDF

[33] Birds of a Feather Flock Together: Background-Invariant Representations via Linear Structure in VLMs cs.CV | cs.AIPDF

[34] USEMA: a Scalable Efficient Mamba Like Attention for Medical Image Segmentation cs.CVPDF

[35] Unpacking the Eye of the Beholder: Social Location, Identity, and the Moving Target of Political Perspectives cs.CVPDF

[36] Hi-GaTA: Hierarchical Gated Temporal Aggregation Adapter for Surgical Video Report Generation cs.CVPDF

[37] ABRA: Agent Benchmark for Radiology Applications cs.CV | cs.AIPDF

[38] DenseTRF: Texture-Aware Unsupervised Representation Adaptation for Surgical Scene Dense Prediction cs.CV | cs.AI | cs.LGPDF

[39] PG-3DGS: Optimizing 3D Gaussian Splatting to Satisfy Physics Objectives cs.CV | cs.GR | cs.LGPDF

[40] Real-Scale Island Area and Coastline Estimation using Only its Place Name or Coordinates cs.CVPDF

[41] Can Graphs Help Vision SSMs See Better? cs.CVPDF

[42] CheXTemporal: A Dataset for Temporally-Grounded Reasoning in Chest Radiography cs.CVPDF

[43] Vision2Code: A Multi-Domain Benchmark for Evaluating Image-to-Code Generation cs.CV | cs.LGPDF

[44] Quantifying Rodda and Graham Gait Classification from 3D Makerless Kinematics derived from a Single-view Video in a Heterogeneous Pediatric Clinical Cohort cs.CV | cs.AIPDF

[45] PresentAgent-2: Towards Generalist Multimodal Presentation Agents cs.CV | cs.CLPDF

[46] VidSplat: Gaussian Splatting Reconstruction with Geometry-Guided Video Diffusion Priors cs.CVPDF

[47] ZeroIDIR: Zero-Reference Illumination Degradation Image Restoration with Perturbed Consistency Diffusion Models cs.CVPDF

[48] Instruct-ICL: Instruction-Guided In-Context Learning for Post-Disaster Damage Assessment cs.CV | cs.LGPDF

[49] Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts cs.CVPDF

[50] SpatialForge: Bootstrapping 3D-Aware Spatial Reasoning from Open-World 2D Images cs.CV | cs.AIPDF

[51] LDDR: Linear-DPP-Based Dynamic-Resolution Frame Sampling for Video MLLMs cs.CVPDF

[52] PoseBridge: Bridging the Skeletonization Gap for Zero-Shot Skeleton-Based Action Recognition cs.CVPDF

[53] LiBrA-Net: Lie-Algebraic Bilateral Affine Fields for Real-Time 4K Video Dehazing cs.CVPDF

[54] GeoR-Bench: Evaluating Geoscience Visual Reasoning cs.CVPDF

[55] When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs cs.CV | cs.AIPDF

[56] TCP-SSM: Efficient Vision State Space Models with Token-Conditioned Poles cs.CV | cs.AIPDF

[57] Dynamic Execution Commitment of Vision-Language-Action Models cs.CVPDF

[58] Logit-Attention Divergence: Mitigating Position Bias in Multi-Image Retrieval via Attention-Guided Calibration cs.CVPDF

[59] HorizonDrive: Self-Corrective Autoregressive World Model for Long-horizon Driving Simulation cs.CVPDF

[60] Keep What Audio Cannot Say: Context-Preserving Token Pruning for Omni-LLMs cs.CV | cs.AIPDF

[61] Grounding by Remembering: Cross-Scene and In-Scene Memory for 3D Functional Affordances cs.CVPDF

[62] Single-Shot HDR Recovery via a Video Diffusion Prior cs.CVPDF

[63] Unlocking UML Class Diagram Understanding in Vision Language Models cs.CV | cs.AIPDF

[64] Hide to See: Reasoning-prefix Masking for Visual-anchored Thinking in VLM Distillation cs.CV | cs.AI | cs.CLPDF

[65] Weather-Robust Cross-View Geo-Localization via Prototype-Based Semantic Part Discovery cs.CV | cs.AI | cs.ROPDF

[66] Reviving In-domain Fine-tuning Methods for Source-Free Cross-domain Few-shot Learning cs.CV | cs.AIPDF

[67] ShapeCodeBench: A Renewable Benchmark for Perception-to-Program Reconstruction of Synthetic Shape Scenes cs.CVPDF

[68] DORA: Dynamic Online Reinforcement Agent for Token Merging in Vision Transformers cs.CVPDF

[69] CAST: Collapse-Aware multi-Scale Topology Fusion for Multimodal Coreset Selection cs.CVPDF

[70] EPIC: Efficient Predicate-Guided Inference-Time Control for Compositional Text-to-Image Generation cs.CV | cs.LGPDF

[71] CaC: Advancing Video Reward Models via Hierarchical Spatiotemporal Concentrating cs.CV | cs.AIPDF

[72] WorldComp2D: Spatio-semantic Representations of Object Identity and Location from Local Views cs.CV | cs.LGPDF

[73] BronchoLumen: Analysis of recent YOLO-based architectures for real-time bronchial orifice detection in video bronchoscopy cs.CVPDF

[74] M$^4$-SAM: Multi-Modal Mixture-of-Experts with Memory-Augmented SAM for RGB-D Video Salient Object Detection cs.CVPDF

[75] Revisiting Shadow Detection from a Vision-Language Perspective cs.CVPDF

[76] Urban Risk-Aware Navigation via VQA-Based Event Maps for People with Low Vision cs.CVPDF

[77] SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions cs.CVPDF