cs.CL [Total: 38]
cs.CV [Total: 80]
cs.AI [Total: 7]
cs.CR [Total: 1]
cs.LO [Total: 1]
cs.SE [Total: 2]
cs.RO [Total: 3]
cs.LG [Total: 7]
cs.HC [Total: 1]

cs.CL [Back]

[1] RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents cs.CLPDF

Mingchen Li, Hansi Zeng, Zhuo Qian, Jiatan Huang, Hamed Zamani

TL;DR: RICE-PO是一个无需评论家的策略优化框架，旨在解决交互式检索智能体中信用分配的挑战。它将检索交互转化为局部学习信号，通过选择高不确定性的可执行动作作为锚点，评估局部反事实分支，并在推理对动作影响强且未来残差效应稳定时，将信用传播给潜在推理步骤。

Details

Motivation: 动机在于训练交互式推理检索智能体时面临的信用分配难题：可执行动作（如查询）可由检索器直接评估，而潜在推理步骤不可直接观察且仅影响未来动作，这种不对称性使得结果级奖励分配不可靠。

Result: 在BRIGHT和BEIR基准测试中，RICE-PO在相同检索器设置下，持续优于基于提示的智能体和基于群体的强化学习基线方法。

Insight: 创新点在于利用智能体-环境交互结构本身为基于推理的检索智能体训练提供监督，通过局部反事实评估和条件信用传播，将检索交互直接转化为学习信号，避免了传统结果奖励的不可靠性。

Abstract: Retrieval is increasingly moving from one-shot matching toward interactive reasoning, where language agents iteratively inspect evidence, reformulate queries, and search again. Training such agents raises a credit-assignment challenge: executable actions such as queries or summaries can be directly evaluated by the retriever, while latent reasoning steps are not directly observable and only affect future executable actions. This asymmetry makes outcome-level reward assignment unreliable, as the same final reward may credit reasoning steps that did not actually shape retrieval success. We propose RICE-PO, a critic-free policy optimization framework that converts retrieval interactions into localized learning signals. RICE-PO selects high-uncertainty executable actions as anchors, evaluates local counterfactual branches using retrieval metrics, and propagates credit to latent reasoning steps only when reasoning-to-action influence is strong and future residual effects are stable. On BRIGHT and BEIR, RICE-PO consistently outperforms prompt-based agents and group-based RL baselines under the same retriever setting. These results show that the structure of agent-environment interaction itself can provide useful supervision for training reasoning-based retrieval agents.

[2] Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline cs.CL | cs.LGPDF

Tony Lee, Percy Liang

TL;DR: 本文提出了一种名为Self-Verified Distillation（自我验证蒸馏）的后训练优化算法，旨在让大型语言模型仅使用未标注的提示进行自我提升。该方法通过模型自身生成候选答案，并利用基于提示的自我验证机制进行筛选，从而构建高质量的自监督数据集，最终在数学、科学和编程三个推理领域显著提升了模型性能。

Details

Motivation: 研究动机是探索大型语言模型能否在没有外部教师或工具反馈的情况下，仅利用未标注的提示（即只有问题没有标准答案）进行自我改进，以解决模型在推理任务上的后训练优化问题。

Result: 在多个基准测试中，该方法显著提升了模型性能：例如，对于Qwen3-4B模型，在数学（AIME26和HMMT）、科学（GPQA Diamond和HLE）和编程（LCBv5和LCBv6）领域的综合pass@1指标分别提升了16.7、11.1和8.3个百分点，且效果扩展到0.6B和8B模型；相比仅依赖测试时计算的方法（UQ-TTC），本方法在多数场景下性能更优且测试时只需单次推理。

Insight: 创新点在于将基于验证的过滤思想（受UQ基准启发）引入自训练过程，设计了一个包含循环一致性、事实性和正确性检查的三阶段级联自我验证流程，仅当候选答案通过所有阶段且获得一致投票时才被接受，从而实现了高质量自监督数据的自动构建；客观来看，该方法通过增加训练时生成候选和验证的预算来提升数据质量，是一种高效且无需外部监督的模型自我精炼范式。

Abstract: Can post-trained large language models (LLMs) further improve themselves using only unlabeled prompts, without external teachers or feedback from tools? We study this setting starting only from unlabeled seed questions with no ground-truth solutions, across three reasoning domains: math, science, and coding. We propose Self-Verified Distillation, a simple post-training refinement algorithm in which the model generates candidate solutions to these seed questions, filters them using prompt-based self-verification, and trains on the resulting self-curated dataset. Inspired by the UQ benchmark’s use of multiple validators to screen candidate answers to hard unsolved questions, we adapt this validation-based filtering idea to self-training: the model filters its own generated solutions through a three-stage cascade of cycle-consistency, factuality, and correctness checks, accepting a solution only if it passes all stages with unanimous judge votes. We find that sampling more candidate generations and using a larger verification budget during training data construction produces higher-quality self-curated data and, in turn, better reasoning models. We then train Qwen3 models at multiple scales with Self-Verified Distillation and obtain gains across all three domains. For Qwen3-4B, our method improves aggregate held-out pass@1 by +16.7 points in math (AIME26 and HMMT), +11.1 points in science (GPQA Diamond and HLE), and +8.3 points in coding (LCBv5 and LCBv6), with gains also extending to 0.6B and 8B models. Compared to our test-time-only baseline (UQ-TTC), which improves performance by spending extra compute at inference time, Self-Verified Distillation achieves better performance in most settings while requiring only a single inference call at test time.

[3] Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations cs.CL | cs.AIPDF

Shanghao Li, Jinda Han, Yibo Wang, Yuanjie Zhu, Zihe Song

TL;DR: 该论文研究了大型语言模型在处理线性化结构化知识（如图表和表格）时产生幻觉的机制。研究发现，幻觉源于系统性的内部动态，而非随机噪声，具体表现为注意力过度集中于类似捷径的结构线索，以及前馈层未能有效锚定所提供知识，导致模型回退到参数记忆。

Details

Motivation: 尽管有充足的结构化外部知识，LLM在处理线性化表示时仍会产生幻觉输出，其背后的失败机制尚不明确。论文旨在深入理解这些幻觉产生的根本原因。

Result: 研究结果表明，幻觉与注意力分配和前馈层语义锚定失败的系统性模式相关，这些模式在单跳图、多跳图和表格设置中具有普遍性，可用于跨结构化知识格式的有效幻觉检测。

Insight: 创新点在于从机制层面（注意力分配和前馈层表示）揭示了LLM在结构化知识推理中产生幻觉的系统性根源，而非归因于随机错误，这为理解和检测幻觉提供了新的视角和可推广的模式。

Abstract: In many reasoning tasks, large language models (LLMs) rely on structured external knowledge, such as graphs and tables, which is typically linearized into sequential token representations. However, even when sufficient knowledge is available, LLMs can still produce hallucinated outputs, and the underlying mechanisms behind such failures remain poorly understood. We investigate these mechanisms and find that hallucinations arise from systematic internal dynamics rather than random noise. First, attention disproportionately concentrates toward shortcut-like structural cues rather than distributing across the full context. Second, feed-forward representations fail to ground the provided knowledge, causing the model to revert to parametric memory. Moreover, our results indicate that hallucination is consistently associated with failures in semantic grounding within feed-forward layers, while attention allocation exhibits greater task-dependent variability. Finally, we show that these mechanistic patterns generalize beyond single-hop graphs to multi-hop and tabular settings, enabling effective hallucination detection across structured knowledge formats.

[4] Memory Architectures for Multi-Turn Text-to-SQL: A Benchmark and Empirical Study cs.CLPDF

Ravi Kumar Tummalapenta, Suman Addanki

TL;DR: 本文提出了EnterpriseMem-Bench，一个包含300个会话和1400轮对话的多轮Text-to-SQL基准测试，并系统评估了五种前沿大语言模型在五种不同记忆架构下的性能。研究发现，无状态模型在第三轮对话后执行准确率降至零，工作记忆窗口是影响性能的主导因素，而更复杂的记忆架构并不总能带来提升。

Details

Motivation: 多轮Text-to-SQL对企业分析至关重要，但现有研究主要评估单轮场景，缺乏对记忆架构影响的系统性基准测试和实证研究。

Result: 在提出的EnterpriseMem-Bench上评估了GPT-5 mini、GPT-5.2、Claude Sonnet 4.5/4.6和Opus 4.6等模型。结果显示，无状态模型在第三轮后准确率崩溃至零；工作记忆是主要影响因素；Claude Sonnet 4.6在SEC EDGAR数据集上比4.5版本表现差17-33个百分点。

Insight: 创新点在于提出了一个程序化构建的、带有确定性真值和逐轮记忆关键性标注的多轮Text-to-SQL基准测试，并引入了Memory Benefit Score作为诊断指标。研究揭示了记忆架构复杂性并非总是有益，工作记忆窗口大小是关键，这为设计高效的对话式SQL系统提供了重要见解。

Abstract: Multi-turn Text-to-SQL is central to enterprise analytics yet remains predominantly evaluated in single-turn settings. We introduce EnterpriseMem-Bench, a multi-turn Text-to-SQL benchmark of 300 sessions and 1,400 turns built programmatically from three enterprise domains (BIRD financial, SEC EDGAR, Northwind), with deterministic ground truth and per-turn memory-critical annotation. We evaluate five frontier models – GPT-5 mini, GPT-5.2, Claude Sonnet 4.5, Sonnet 4.6, and Opus 4.6 – across five memory conditions enabling a three-way ablation isolating working-memory window size, episodic retrieval, and semantic augmentation as independent effects. All Claude models are evaluated with extended thinking enabled to maintain parity with GPT reasoning models. We introduce the Memory Benefit Score (MBS) as a per-turn diagnostic metric. Four findings emerge: (1) stateless multi-turn Text-to-SQL collapses to zero execution accuracy by Turn 3 across all five models, even under reasoning; (2) memory-architecture complexity does not monotonically improve accuracy – working memory dominates, and additional components produce model- and dataset-dependent effects from +14 to -16 percentage points; (3) Claude Sonnet 4.6 underperforms Sonnet 4.5 by 17-33pp on SEC EDGAR across conditions, a generational regression persisting under reasoning; (4) under reasoning, Claude error distributions become mono-modal – every non-correct turn is a wrong-result error. We release the benchmark, agent, and evaluation code.

[5] Towards Just-in-Time Adaptive Feedback: Enhancing Student Learning via Knowledge-Grounded LLM cs.CLPDF

Younghun Lee, Amir Bralin, Nobel Sanjay Rebello, Dan Goldwasser

TL;DR: 本文提出了一种基于领域专家知识的大语言模型框架，用于在教育场景中提供及时自适应反馈。该框架通过分析学生的书面推理逻辑，识别潜在错误类型，并生成非侵入式反馈以澄清缺失或错误概念。在一门大规模大学课程中部署该框架，使学生的表现相比以往学期提升了80%以上。

Details

Motivation: 当前基于大语言模型的自适应反馈研究缺乏在真实教学环境中提供及时反馈的明确方法，因此需要一种能够结合领域专家知识、分析学生推理并提供非侵入式反馈的框架。

Result: 在一门超过1000名学生的大规模大学课程中部署该框架，学生表现相比以往学期提升了80%以上，并通过分析学习轨迹验证了其教学效用。

Insight: 创新点在于将大语言模型与领域专家知识结合，通过分析学生书面推理逻辑来识别错误并提供及时自适应反馈，从而促进从错误概念到正确理解的转变。

Abstract: Educational interventions are effective tools for enhancing student learning. While Large Language Models (LLMs) allow for generating adaptive feedback at scale, current studies lack clear methodologies for providing Just-in-Time (JiT) feedback in authentic instructional settings. In this paper, we present a framework that provides adaptive feedback by grounding LLMs with domain-specific expert knowledge. Our approach collects written reasoning logic (strategy essays) from students, analyzes potential error types based on the content of that reasoning, and delivers non-intrusive feedback designed to clarify missing or incorrect concepts. We deploy this framework in a large-scale university course (N > 1000), where it improved student performance by over 80% compared to previous semesters. Lastly, we validate the framework’s pedagogical utility by analyzing the learning trajectories; we demonstrate how iterative conversations with LLM facilitate shifting one’s misconception to correct understanding.

Lin Yao

TL;DR: 本文针对LLaDA2.1等离散掩码扩散语言模型中Token-to-Token（T2T）编辑机制存在的根本缺陷，提出了无需训练的Token-to-Mask（T2M）重掩码方法。该方法将疑似错误的已生成令牌重置回掩码状态，在更干净的上下文中重新预测，从而提升生成质量。

Details

Motivation: 解决T2T编辑机制存在的三个核心问题：错误检测与替换的耦合、潜在错误令牌污染生成上下文，以及训练-推理阶段的噪声不匹配（模型系统性错误与训练时的随机扰动不同）。

Result: 在涵盖知识、推理、数学、编程和指令遵循的12个基准测试中，T2M普遍提升了需要精确令牌级输出的任务性能，其中在数学任务（CMATH）上提升最大（+5.92%）。错误分析表明，T2M修复了59.4%的“最后一英里令牌损坏”错误。

Insight: 核心创新点在于将“令牌编辑”范式转变为“令牌到掩码”的细化范式，通过重置错误令牌而非直接替换，实现了生成上下文的净化、将系统性推理错误转换回模型原生的掩码噪声类型，并支持延迟承诺以实现多位置联合优化。该方法无需重新训练，是即插即用的改进方案。

Abstract: Discrete masked diffusion language models such as LLaDA generate text through iterative denoising, where mask tokens are progressively replaced with predicted tokens. LLaDA2.1 introduced a Token-to-Token (T2T) editing mechanism that accelerates generation by directly replacing committed tokens suspected of being incorrect. However, we identify fundamental limitations of T2T editing: it couples error detection with replacement, pollutes the generation context with potentially incorrect tokens, and introduces a train-inference noise mismatch where systematic model-generated errors differ from the random perturbations seen during training. We propose Token-to-Mask (T2M) remasking, a training-free, drop-in replacement for T2T editing that resets suspected erroneous tokens back to the mask state, allowing the diffusion process to re-predict them under cleaner context. We design and empirically validate three complementary error detection strategies – probability-based, trigger-mirrored, and temporal-difference-based – and provide a unified theoretical analysis showing that T2M remasking purifies the generation context, converts systematic inference errors back to the model’s native mask noise type, and enables delayed commitment for joint multi-position optimization. Comprehensive experiments across 12 benchmarks spanning knowledge, reasoning, mathematics, coding, and instruction following show that T2M generally improves performance on tasks requiring precise token-level output, with the largest gain on mathematics (+5.92% on CMATH). Error analysis on CMATH reveals that the dominant failure mode is last-mile token corruption – where correct reasoning produces a corrupted final answer – and that T2M repairs 59.4% of such cases.

[7] Towards Error-Free EHRs: Reasoning-Intensive Consistency Verification Between Clinical Notes and Structured Tables in Electronic Health Records cs.CL | cs.AIPDF

Yeonsu Kwon, Jiho Kim, Junseong Choi, Paloma Rabaey, Minseo Kim

TL;DR: 本文提出了EHR-ReasonCon基准和EHR-Inspector框架，用于验证电子健康记录中非结构化临床笔记与结构化表格之间的一致性。EHR-ReasonCon是一个基于MIMIC-III构建、包含专家标注的推理密集型基准，而EHR-Inspector是一个基于大语言模型的框架，通过分割笔记、提取实体和时间信息，并利用表格探索工具来验证一致性。

Details

Motivation: 现有方法主要依赖数值或简单事件的表层匹配，无法捕捉真实世界电子健康记录文档中的临床解释、事件关系和时序变化等深层推理，因此需要开发一个推理密集型的验证基准和方法来解决这一差距。

Result: 在严格和宽松标准下，使用专家验证的LLM-as-a-judge指标进行评估，EHR-Inspector在多个模型骨干上均达到了最先进的性能。

Insight: 创新点在于引入了推理密集型的基准EHR-ReasonCon，它强调临床解释和时序推理；以及提出了EHR-Inspector框架，它结合了笔记分割、实体提取和专门的表格探索工具，系统化地进行一致性验证，这为医疗数据一致性检查提供了新的方法论。

Abstract: Data consistency between unstructured clinical notes and structured tables in Electronic Health Records (EHRs) is essential for patient safety and clinical decision-making. However, existing work on note-table consistency verification mainly relies on surface-level matching of numeric values or simple events. Such approaches fail to capture the reasoning underlying real-world EHR documentation, including clinical interpretation, event relations, and temporal changes. To address this gap, we introduce EHR-ReasonCon, a reasoning-intensive benchmark for note-table consistency verification. Built on MIMIC-III with expert-guided annotations, it comprises 8,048 entities derived from clinical notes and provides high-quality ground-truth labels. The annotation protocol is supported by specialized table-exploration tools to ensure systematic evidence retrieval and reliable consistency assessment. We also propose EHR-Inspector, an LLM-based framework that segments notes, extracts anchor entities and temporal references, and uses table-exploration tools to verify consistency against structured tables. Evaluated using expert-validated LLM-as-a-judge metrics under harsh and lenient criteria, EHR-Inspector achieves state-of-the-art performance across multiple model backbones. Analyses further demonstrate the effectiveness of its components and highlight differences from human verification.

[8] FAB-Bench: A Framework for Adaptive RAG Benchmarking in Semiconductor Manufacturing cs.CL | cs.IRPDF

Jingbin Qian, Congwen Yi, Min Xia, Wen Wu, Jun Zhu

TL;DR: 本文提出了FAB-Bench，一个用于半导体制造领域自适应RAG系统评估的端到端框架。该框架定义了六项诊断指标，并构建了一个包含200个高质量查询-答案对的基准测试集。通过对多个LLM和RAG框架的系统性评估，揭示了三种不同的上下文扩展行为及其背后的性能衰减机制。

Details

Motivation: 在垂直领域（如半导体制造）评估RAG系统性能非常困难，主要挑战在于领域复杂性、上下文规模多样以及对昂贵、不一致且难以扩展的专家评估的严重依赖。

Result: 在超过1300个生成的候选中，构建了一个包含200个查询-答案对的高质量基准测试集。对四个LLM和四个RAG框架的系统性评估揭示了三种上下文扩展行为（对数增长、早期饱和和冷启动动态），并确定注意力稀释是极端上下文长度下性能下降的主要机制。在三个额外生产RAG系统上的跨框架验证证实了评估的可移植性。

Insight: 创新点在于提出了一个针对垂直领域（半导体制造）的自适应RAG评估框架，其核心是耦合检索器诊断与生成器级推理分析，并量化了检索精度和生成保真度如何随上下文范围扩展而共同演化。客观来看，其定义的六项诊断指标和构建的多策略基准测试集，为领域特定RAG系统的精细化评估提供了可借鉴的方法论。

Abstract: Retrieval-Augmented Generation (RAG) has become critical for knowledge-intensive applications, yet evaluating its performance in vertical domains remains difficult due to domain complexity, diverse context scales, and heavy reliance on expert assessments that are costly, inconsistent, and non-scalable. We introduce FAB-Bench, an end-to-end framework for adaptive benchmarking of RAG systems in semiconductor manufacturing. FAB-Bench defines six diagnostic metrics measuring factual accuracy, contextual utilization, completeness, retrieval relevance, technical depth, and reasoning consistency. The framework couples retriever diagnostics with generator-level reasoning analysis across context windows of 4K-32K tokens, quantifying how retrieval precision and generative fidelity co-evolve as contextual scope expands. From over 1,300 generated candidates, we curated a high-quality benchmark of 200 query-answer pairs spanning three synthesis strategies: needle-in-haystack, intra-document multi-topic, and cross-document multi-hop. Systematic evaluation across four LLMs and four RAG frameworks reveals three distinct context-scaling behaviors: logarithmic growth, early saturation, and cold-start dynamics, and identifies attention dilution as the primary mechanism behind performance degradation at extreme context lengths. Cross-framework validation on three additional production RAG systems confirms evaluation portability.

[9] Conceptual Steganography cs.CLPDF

Zhejian Zhou, Jonathan May

TL;DR: 本文提出了概念隐写术，这是一种新的语言模型隐写方法，它通过思维链中高层推理行为的模式而非词汇选择来传递隐蔽信息。研究表明，该方法在四个模型系列和两个推理领域中，比标准的基于关键词的隐写方法对强重述防御更具鲁棒性，且不影响思维链的推理效用。文章最后指出，一种策略感知的重述器可以很大程度上阻断该信道，强调了确保LLM忠实推理的新挑战和防御建议。

Details

Motivation: 动机在于揭示语言模型（LM）思维链（CoT）可能被滥用的新风险：一个未对齐的模型可能在其CoT中嵌入隐蔽信息以逃避人类监督，即所谓的编码推理。现有LM隐写方案在词元或词汇层面操作，而内容保留的重述器是近期工作的标准有效防御手段，本文旨在探索一种更隐蔽、更鲁棒的隐写形式。

Result: 在四个模型系列和两个推理领域的实验中，概念隐写术被证明比标准的关键词方法对强重述防御具有一致的更高鲁棒性，并且将信息编码到CoT中不会影响其在推理过程中的效用。

Insight: 创新点在于将隐写从词汇空间提升到概念/推理行为模式层面，从而规避了基于词汇修改的防御。客观来看，这揭示了LLM推理过程本身可能成为隐蔽通信信道的新安全风险，并提出了策略感知防御的新研究方向。

Abstract: Language Models (LMs) emit Chains-of-Thought (CoTs) that drive much of their capability. However, the same sequence that carries useful reasoning can also covertly convey messages: a misaligned model may embed covert information in its CoT that slips through human supervision, a form of steganography known as encoded reasoning. Prior LM steganography schemes operate in the token or lexical space, and a content-preserving paraphraser is the canonical and effective defense in recent work. We introduce conceptual steganography, in which each step of a CoT carries information through patterns of high-level reasoning behavior, rather than through lexical choice. Across four model families and two reasoning domains, this backdoor communication channel is shown to be consistently more robust to a strong paraphrase defense than standard keyword approaches, and the encoding of information into CoTs does not affect their utility in the reasoning process. Having raised awareness of this new risk, we then demonstrate that a strategy-aware paraphraser can close much of the channel, highlighting new challenges and recommended defenses for ensuring faithful LLM reasoning in the wild.

[10] Why Prompt Optimization Works, and Why It Sometimes Doesn’t: A Causal-Inspired Edit-Level Analysis cs.CL | cs.LG | cs.NEPDF

Shuzhi Gong, Hechuan Wen

TL;DR: 本文通过因果推断启发的观察性分析，探究了自动化提示优化方法（如DSpy、TextGrad）在不同任务间泛化能力不足的原因。研究发现，提示性能的异质性源于编辑类型（如复杂度增加、元指令编辑）与任务特性（如数学推理、多跳推理）之间的系统性交互作用，而非随机优化伪影。

Details

Motivation: 自动化提示优化方法虽能提升大语言模型性能，但其优化后的提示在不同基准任务间的泛化能力有限，且该问题在不同模型骨干间依然存在。本文旨在探究提示性能异质性的未充分探索来源。

Result: 分析表明，复杂度增加和元指令编辑与数学及多跳推理性能呈负相关，而逐步分解和元认知编辑则能提升逻辑与顺序推理任务。这些效应在不同认知负荷标注、表层文本特征和编辑模式分析中均稳健，且能跨优化框架泛化。

Insight: 创新点在于采用因果推断启发的倾向性调整关联分析，结合多种互补的提示编辑表示，识别出任务条件化编辑的一致模式。这为优化器行为提供了特征级表征，并启发了未来任务条件化优化器的设计方向。

Abstract: Automated prompt optimization methods (e.g., DSpy, TextGrad) can substantially improve the performance of large language model (LLM), however, their generalization ability across different tasks remains underperformed. In practice, the superiority of the optimized prompt on one benchmark often fails to transfer to another, and this limitation persists even when switching across different LLM backbones. To investigate the underexplored sources of heterogeneity in prompt performance, we conduct a causal inference-inspired observational analysis of optimized prompts across a diverse set of optimization frameworks, LLM backbones, and NLP benchmarks. To achieve the goal, we build upon the propensity-adjusted associational analysis together with multiple complementary representations of prompt edits, where the consistent task-conditioned edits patterns are identified. We find that complexity-increasing and meta-instructional edits are negatively associated with mathematical and multi-hop reasoning performance, whereas step-by-step and meta-cognitive edits improve logical and sequential reasoning tasks. These effects are robust across cognitive-load annotations, surface-level text features, and edit-motif analyses, and can generalize across optimization frameworks. Overall, these results indicate that prompt optimization failures arise from systematic interactions between edit families and task characteristics rather than random optimization artifacts, providing feature-level characterization of optimizer behavior and motivating future task-conditioned optimizer design.

[11] Rethinking the Multilingual Reasoning Gap with Layer Swap cs.CLPDF

Maxence Lasbordes, Amélie Chatelain, Djamé Seddah

TL;DR: 本文重新审视了多语言推理模型中的‘母语推理差距’问题。通过在大规模、可比较的监督下，对六种语言（英、法、德、西、中、斯瓦希里语）进行微调实验，发现该差距远小于先前报告。权重空间分析揭示了模型存在一个语言无关的‘推理核心’和语言特定的外层。基于此，作者提出‘层交换’方法，将英语专家的中间层移植到母语专家中，有效缩小了推理差距，同时保持了目标语言的思维链。

Details

Motivation: 解决现有大型语言模型在非英语语言上进行‘母语推理’（即用输入语言生成思维链）时，性能显著低于‘英语枢纽推理’（用英语推理后回答）的问题，并探究其根本原因。

Result: 在数学、科学、常识和代码等任务上，在可比监督下，五个非英语语言的母语推理平均差距缩小至1.9%-3.5%，远小于先前研究。通过层交换方法，可以基本消除这一差距，同时保持目标语言的思维链输出。

Insight: 创新点在于通过大规模实验和权重分析，揭示了多语言模型中存在一个语言无关的‘推理核心’，并据此提出了‘层交换’这一轻量级干预方法。这为提升多语言模型的母语推理能力提供了新的、高效的思路，即分离并强化核心推理模块，而非完全依赖语言特定的训练。

Abstract: Recent reasoning Large Language Models produce a chain-of-thought (CoT) predominantly in English, even when prompted in non-English languages. Prior work suggests that forcing the CoT to remain in the input language (\emph{native reasoning}) substantially degrades performance relative to allowing the model to reason in English before answering in the input language (\emph{English-pivoted reasoning}). However, most studies of this native reasoning gap rely on inference-time interventions or limited native-language training data. We revisit this comparison at a larger scale and under comparable supervision. We construct long multilingual reasoning datasets across six languages (English, French, German, Spanish, Chinese and Swahili); fine-tune specialists in both native and English-pivoted regimes on top of \texttt{Qwen/Qwen3-8B-Base}, and evaluate across mathematics, science, general knowledge, and code. In this setting, the average native reasoning gap shrinks to 1.9–3.5% across the five non-English languages, considerably smaller than previously reported. Weight-space analysis of the native specialists reveals aligned fine-tuning updates in the middle layers and divergence in the outer layers. This points to a largely language-agnostic reasoning core surrounded by language-specific layers. Exploiting this structure, we introduce a Layer Swap: transferring the English specialist’s stronger reasoning mid-layers into each native specialist, closing most of the native reasoning gap across the five non-English languages while preserving CoT in the target language. We release all models and datasets.

[12] KARMA: Karma-Aligned Reward Model Adaptation cs.CLPDF

Jared Scott, Jesse Roberts

TL;DR: 本文提出KARMA框架，通过从Reddit大规模社交互动数据中学习上下文敏感的对话行为。该框架训练一个奖励模型来预测基于上下文的回复价值，并利用该信号通过强化学习微调语言模型，以提升语用相关任务的性能。研究发现，性能最高的奖励模型并不一定带来更好的下游模型对齐，而仅依赖对话上下文的奖励模型虽然对Reddit karma的预测较差，却产生了更好的下游性能。

Details

Motivation: 人类交流依赖于隐含的社交信号，其有效性由语调、上下文和对话规范而不仅仅是语义内容决定。本文旨在让LLM从大规模社交互动数据中学习上下文敏感的对话行为。

Result: 应用KARMA的下游模型在语用相关行为上表现出改进，且不良副作用大幅减轻。但在所有条件下，包括下游模型未直接接触Reddit数据时，事实性均一致下降，表明这种权衡内嵌于奖励信号本身。

Insight: 创新点在于提出了一个从社交互动数据中学习语用对齐的奖励模型适应框架。客观分析认为，其关键发现是奖励模型的下游对齐效果与其对原始数据指标（如karma）的预测性能可能解耦，这为奖励模型设计提供了新视角；同时揭示了事实性与语用行为改善之间可能存在的内在权衡。

Abstract: Human communication depends on implicit social signals where effectiveness is shaped by tone, context, and conversational norms rather than semantic content alone. We introduce KARMA (Karma-Aligned Reward Model Adaptation), a framework for LLM learning of context-sensitive conversational behavior from large-scale social interaction data. KARMA trains a reward model on Reddit conversations to predict response valuation conditioned on context, and uses this signal to fine-tune language models via reinforcement learning to improve performance on pragmatics-mediated tasks. Critically, we find that the highest performing reward model does not lead to better downstream model alignment: a reward model relying exclusively on conversational context was a worse predictor of Reddit karma but yielded substantially better downstream performance. We evaluate the effects of KARMA applied to a downstream model with and without direct exposure to the social media data. The resulting models show improved pragmatics-mediated behaviors with largely mitigated undesirable side effects. Factuality is consistently diminished by KARMA across all conditions, including when the downstream model has no direct exposure to Reddit data, suggesting that this tension is embedded in the reward signal itself rather than introduced by noisy training data.

[13] SeDT: Sentence-Transformer Decision-Transformer Conditioning for Multi-Turn Conversation Reliability cs.CL | cs.AIPDF

Ramakrishna Vamsi Setti, Jagadeesh Rachapudi, Sachin Chaudhary, Praful Hambarde, Amit Shukla

TL;DR: SeDT是一种无需训练、在推理时使用的方法，旨在解决大语言模型在多轮对话中性能显著下降的问题（即’Lost in Conversation’现象）。该方法通过从离线强化学习引入’return-to-go’条件，为对话片段计算累积相关性分数，并在最终轮次将完整注释历史呈现给模型，从而显著提升模型在多轮对话中的可靠性和性能。

Details

Motivation: 大语言模型在单轮任务中表现出色，但在多轮增量揭示相同任务时性能会大幅下降（高达39%），这主要源于可靠性故障。根本原因是扁平化的对话历史对所有先前轮次赋予相同的隐式权重，导致模型无法区分关键约束与无关对话。

Result: 在Lost-in-Conversation基准测试中，针对三种LLM和三种生成任务的所有九种模型-任务组合，SeDT均优于分片基线，平均性能提升最高达+37.7%，并在九种组合中的七种同时降低了不可靠性。

Insight: 创新点在于将离线强化学习中的’return-to-go’条件引入LLM推理，通过结合语义、词汇和位置信号计算对话片段的相关性分数，以无训练、不改权重、不丢弃上下文的方式，明确告知模型哪些历史轮次重要，从而有效缓解多轮对话中的性能衰退问题。

Abstract: Large language models (LLMs) achieve impressive performance when a task is fully specified in a single turn, yet the same models lose up to 39% of that performance when the identical task is revealed incrementally across multiple turns, a phenomenon documented at scale as Lost in Conversation. Crucially, this collapse is almost entirely a reliability failure; the best case, the aptitude only falls 16%, while the unreliability more than doubles (+112%). We argue that the root cause is structural, a flat conversation history assigns equal implicit weight to every prior turn, giving the model no signal to distinguish a critical constraint from incidental dialog. We present SeDT Sentence-transformer Decision-Transformer, a training-free inference-time method that resolves this by importing return-to-go conditioning from offline reinforcement learning. SeDT annotates each conversation shard with a cumulative relevance score derived from three complementary semantic, lexical, and positional signals and presents the full annotated history to the model at the final turn, without weight changes, without training data, and without discarding context. Evaluated on the Lost-in-Conversation benchmark in three LLMs and three generation tasks, SeDT outperforms the sharded baseline in all nine model-task combinations, with gains up to +37.7% in mean performance P and simultaneous reductions in unreliability in seven of the nine combinations. In short, telling the model which past turns matter is sufficient to substantially recover the performance lost in conversation.

[14] Generating Logically Consistent Synthetic Supply Chain Data with LLM-Driven Knowledge Graph Reasoning cs.CLPDF

Yunbo Long, Ge Zheng, Liming Xu, Alexandra Brintrup

TL;DR: 本文提出TabKG框架，利用知识图谱引导生成逻辑一致的供应链合成表格数据。该方法首先通过多LLM集成构建列关系知识图谱（CR-KG）来表示数据间的操作依赖关系，然后使用潜在扩散模型生成独立列，并依据CR-KG中的验证关系确定性地重建依赖列，从而在生成过程中强制保持操作逻辑一致性。

Details

Motivation: 解决供应链分析中合成数据在保持统计分布之外，还需保留供应链流程操作逻辑（如时间顺序、数学依赖、层次分类和条件规则）的问题，现有表格生成模型常生成违反基本操作约束的数据。

Result: 论文未在摘要中提及具体定量结果或基准测试，但强调所提方法能通过构建的CR-KG确保生成数据在操作规则上的逻辑一致性。

Insight: 创新点在于将供应链数据操作逻辑视为“物理定律”，并引入知识图谱来显式建模列间依赖关系，结合LLM推理与扩散模型，以构造性方式强制逻辑一致性，而非仅依赖统计拟合。

Abstract: Synthetic data offers a promising solution to two persistent barriers in supply chain analytics: data scarcity and data privacy. However, for synthetic data to support operational simulation and decision-making, it must do more than reproduce the statistical distributions of real records, and also preserve the \emph{operational logic} that governs supply chain processes, including the temporal orderings, mathematical dependencies, hierarchical taxonomies, and conditional rules that make a record operationally plausible. We consider this logic as the ``physics’’ of supply chain data. Existing tabular generative models are primarily optimized for distributional fidelity and downstream predictive utility, and therefore often generate records that appear statistically realistic but violate fundamental operational constraints. This paper introduces \textbf{\textit{TabKG}}, a knowledge-graph-guided framework for logically consistent synthetic supply chain tabular data generation. TabKG constructs a \textbf{\textit{Column Relationship Knowledge Graph (CR-KG)}} to represent data operational dependencies. It uses a multi-LLM ensemble with majority voting to propose candidate relationships from column metadata, validates these relationships against real data to remove hallucinated or unsupported edges, and then uses the validated CR-KG to guide generation. Specifically, TabKG compresses the original table into independent columns, generates these columns using a latent diffusion model, and deterministically reconstructs dependent columns according to the validated relationships, enforcing logical consistency by construction with respect to the discovered operational rules.

[15] ContextGuard: Structured Self-Auditing for Context Learning in Language Models cs.CL | cs.AIPDF

Hongbo Jin, Chi Wang, Haoran Tang, Zhongjing Du, Xu Jiang

TL;DR: 这篇论文提出了ContextGuard框架，旨在解决大语言模型在应用复杂上下文知识时存在的忠实性问题。它通过结构化自审计机制，帮助模型在遵循核心推理路径的同时，识别并满足外围、持久或格式敏感的上下文要求。

Details

Motivation: 尽管大语言模型具备强大的推理能力，但现有基准测试表明，它们在忠实应用复杂上下文知识方面仍存在困难。这些失败通常不是整体推理崩溃，而是在上下文丰富的任务中，模型可能遵循核心推理路径，却忽略了外围、持久或格式敏感的细节要求。

Result: 摘要中未提及具体的定量结果、基准测试或SOTA比较。

Insight: 论文的核心创新点是提出了一个结构化自审计框架（ContextGuard），用于系统性地检查和确保模型对上下文所有方面的忠实遵循，而不仅仅是核心逻辑。这为解决模型在复杂任务中“抓大放小”的常见问题提供了一种新思路。

Abstract: Recent benchmarks reveal that despite strong reasoning capabilities, large language models (LLMs) still struggle to faithfully apply complex contextual knowledge. These failures are often not wholesale reasoning collapses: in context-rich tasks, models may follow the central reasoning path while missing peripheral, persistent, or format-sensitive requirements.

[16] Optimising Factual Consistency in Summarisation via Preference Learning from Multiple Imperfect Metrics cs.CLPDF

Yuxuan Ye, Raul Santos-Rodriguez, Edwin Simpson

TL;DR: 本文提出了一种通过偏好学习优化摘要事实一致性的自动化训练流程。该方法利用多个不完善的事实性度量指标，通过聚合它们的评分来构建高质量偏好数据集，从而指导模型学习生成事实更准确的摘要。

Details

Motivation: 现有的事实一致性度量指标发展不充分，单独使用不可靠，限制了其作为强化学习奖励信号来塑造模型行为的效果。

Result: 实验表明，该方法在不同模型（从早期的编码器-解码器架构到现代大语言模型）上均能带来一致的事实性提升，较小的模型能达到与较大模型相当的事实一致性水平。

Insight: 核心创新在于利用多个弱指标的聚合来更有效地捕捉多样的事实错误，并通过将评分映射为偏好、过滤指标间高分歧案例来避免复杂的奖励塑造。此外，通过为每个源文档生成词汇相似但事实性不同的摘要对，使模型能从细微的词汇差异中学习事实差异，仅使用源文档即可构建高质量偏好数据集。

Abstract: Reinforcement learning with evaluation metrics as rewards is widely used to enhance specific capabilities of language models. However, for tasks such as factually consistent summarisation, existing metrics remain underdeveloped, limiting their effectiveness as signals for shaping model behaviour.While individual factuality metrics are unreliable, their combination can more effectively capture diverse factual errors. We leverage this insight to introduce an automated training pipeline that improves factual consistency in summaries by aggregating scores from different weak metrics. Our approach avoids the need for complex reward shaping by mapping scores to preferences and filtering out cases with high disagreement between metrics. For each source document, we generate lexically similar summary pairs by varying decoding strategies, enabling the model to learn from factual differences caused by subtle lexical differences. This approach constructs a high-quality preference dataset using only source documents.Experiments demonstrate consistent factuality gains across models, ranging from early encoder-decoder architectures to modern large language models, with smaller models reaching comparable factuality to larger ones.

[17] Uncertainty-Aware Budget Allocation for Adaptive Test-Time Reasoning cs.CLPDF

Manh Nguyen, Sunil Gupta, Hung Le

TL;DR: 本文提出了一种不确定性感知的预算分配方法（UAB），用于自适应地优化测试时推理过程中的采样预算分配。该方法通过两阶段策略：第一阶段为每个问题生成一个响应并计算平均负对数似然作为难度信号；第二阶段根据不确定性重新分配剩余采样预算，优先探索困难问题。

Details

Motivation: 现有方法在语言模型推理中均匀分配采样预算效率低下，容易问题被过度采样而困难问题探索不足。本文旨在通过基于不确定性的预算重分配，提高固定采样预算下的推理性能。

Result: 在六个参数规模从1.5B到27B的开源和黑盒模型上，以及涵盖数学、逻辑和偏好任务的五个推理基准测试中，UAB相比基线方法在平均准确率上提升最高达+3%，在个别基准上提升最高达+5%，且在低资源设置下收益最大。

Insight: 创新点包括：1）利用输出对数概率直接提取平均负对数似然作为无额外推理成本的难度信号；2）通过凹整数优化框架和边际贪婪算法实现预算的精确重分配；3）无需辅助模型或额外LLM调用即可提升性能。

Abstract: Sampling multiple responses improves language model reasoning, but uniform compute allocation is inefficient: easy questions are over-sampled while hard questions remain under-explored. We propose Uncertainty-Aware Budget Allocation (UAB), a concave integer optimization framework that reallocates a fixed sampling budget based on per-question uncertainty estimated at no additional inference cost. In Phase 1, every question receives one generation; its average negative log-likelihood (ANLL), extracted directly from output log-probabilities, serves as a difficulty signal while the generation contributes to the final vote. In Phase 2, the remaining budget is allocated by a marginal-greedy algorithm that solves a concave coverage-maximization surrogate exactly: uncertain questions receive more sampling budget while confident questions receive fewer additional samples. Evaluated on six open-weight and black-box models spanning 1.5B to 27B parameters and five reasoning benchmarks covering math, logic, and preference tasks, UAB outperforms baselines by up to +3% in average accuracy and up to +5% on individual benchmarks, with the largest gains in low-resource settings, requiring no auxiliary model or additional LLM call. Code is publicly available at https://github.com/manhitv/UAB.

[18] GeoFaith: A Spatio-Temporal Dual View of Faithful Chain-of-Thought cs.CL | cs.AIPDF

Weijiang Lv, Wentong Zhao, Jiayu Wang, Yuhao Wu, Jiaheng Wei

TL;DR: 本文提出GeoFaith框架，通过时空双视角（几何结构和熵动态）来诊断和增强思维链推理的忠实性。该方法构建了可扩展的标注流程，训练出超越GPT-5的忠实性检测器，并设计了联合优化正确性、忠实性和一致性的强化学习框架，在保持准确性的同时生成更简洁、可解释的推理链。

Details

Motivation: 针对现有思维链推理中普遍存在的后合理化问题（即生成看似合理但不忠实的推理过程），以及现有忠实性评估方法难以扩展、昂贵或不可靠的局限性，旨在开发可扩展且可靠的忠实性诊断与增强框架。

Result: 在标准基准测试中，训练的8B参数忠实性检测器性能优于GPT-5；所提方法在忠实性检测和下游推理任务上均取得优异性能，生成的推理链更短、更可解释且不牺牲准确性。

Insight: 创新点在于从时空双视角（潜在几何结构和熵动态）建模推理过程的忠实性，并构建了可扩展的标注流程与联合优化多目标（结果正确性、过程忠实性、轨迹一致性）的强化学习框架，为可解释AI提供了新思路。

Abstract: Chain-of-Thought (CoT) reasoning has advanced large language models (LLMs), but outcome-based supervision leads to pervasive post-hoc rationalization, producing plausible yet unfaithful reasoning chains. Most prior faithfulness assessment methods are either unscalable, expensive, or unreliable. We propose GeoFaith, a spatio-temporal framework that leverages latent geometric structure and entropy dynamics to diagnose and enforce faithful reasoning. We develop a scalable bootstrapping pipeline expanding step-level annotations from 1k to 20k samples across four domains, train an 8B faithfulness detector outperforming GPT-5 on standard benchmarks, and design a faithfulness-aware reinforcement learning framework jointly optimizing outcome correctness, process faithfulness, and trajectory consistency. Experiments show the proposed method achieves superior performance on both faithfulness detection and downstream reasoning, producing shorter, more interpretable chains without sacrificing accuracy. Our code will be made available publicly.

[19] Are Video Models Zero-Shot Learners and Reasoners in Education? EduVideoBench, A Knowledge-Skills-Attitude Benchmark for Educational Video Generation cs.CLPDF

Unggi Lee, Hoyoung Ahn, Yoon Choi, Seonmin Eun, Jahyun Jeong

TL;DR: 本文提出了首个教育领域的视频生成模型基准EduVideoBench，该基准基于知识-技能-态度（KSA）框架，旨在评估视频生成模型在教育场景下的教学有效性和安全性，而不仅仅是感知质量或通用安全性。通过对五个前沿视频生成模型的评估，发现它们在知识、技能和态度方面均存在显著不足，尚未达到课堂就绪水平。

Details

Motivation: 现有视频生成模型基准主要评估感知质量、内在忠实度、通用安全性或作为推理媒介的能力，缺乏对教育有效性的评估，而视频生成模型正快速进入课堂，因此需要建立一个专门评估其教学有效性和安全性的基准。

Result: 在五个前沿视频生成模型上的评估结果表明，它们在知识、技能和态度维度均有很大改进空间，尚未达到课堂就绪水平。

Insight: 创新点在于提出了首个基于KSA框架的教育视频生成基准，将教学充分性和教育安全性作为核心评估维度，而非临时性的质量指标；客观分析认为，其核心贡献是强调了教育有效性是多组分的，单个元素（如节奏、清晰度或符号）的错位就可能导致整个视频无效，这为开发教育适用的视频生成模型提供了明确指导。

Abstract: Video generation models (VGMs) are rapidly entering classrooms, yet existing benchmarks evaluate only perceptual quality, intrinsic faithfulness, generic safety, or video as a reasoning medium, and none assesses whether the outputs are educationally valid. In this work, we present EduVideoBench, the first balanced benchmark in the education domain, grounded in the Knowledge-Skills-Attitude (KSA) framework so that pedagogical adequacy and educational safety are evaluated jointly rather than as ad-hoc quality dimensions. Across five frontier VGMs, our results show substantial room for improvement across knowledge, skills, and attitude before they are classroom-ready. We complement this with a qualitative analysis of expert comments, finding that educational validity is multi-component, where a single misaligned element such as pacing, legibility, or notation can invalidate an otherwise correct video. We hope EduVideoBench will guide the development of VGMs that are pedagogically grounded and safe for the classroom.

[20] Learning to Adapt SFT Data for Better Reasoning Generalization cs.CLPDF

Lisong Sun, Li Wang, Chen Zhang, Jinyang Wu, Kui Zhang

TL;DR: 本文提出了一种名为DART（Data Adaptation for Reasoning Tuning）的方法，旨在解决在监督微调（SFT）中，当外部专家数据分布与目标模型自身分布不匹配时，会损害模型泛化能力的问题。DART通过强化学习训练一个映射器模型，将原始的SFT数据转换为更匹配目标模型分布和学习偏好的监督信号，从而提升模型在推理任务上的泛化性能。

Details

Motivation: 动机在于直接使用分布不匹配的专家数据进行监督微调会损害大型语言模型（LLMs）的泛化能力，尤其是在提升推理能力时。因此，需要一种方法来适配SFT数据，使其更好地与目标模型对齐。

Result: 在多个模型和数据集上的实验表明，DART方法提升了泛化能力，获得了比直接使用强化学习（RL）更高的训练效率，并帮助模型超越了标准的SFT方法。

Insight: 创新点在于将固定、可能分布不匹配的SFT数据集的使用形式化为一个关于演示转换的优化问题，并引入一个通过强化学习训练的映射器模型来动态适配数据，这是一种数据层面的适配方法，而非直接修改模型架构或训练算法。

Abstract: Large language models (LLMs) have achieved remarkable progress, with post-training playing a crucial role in enhancing their reasoning capabilities. Among post-training paradigms, supervised fine-tuning (SFT) is widely used: it leverages external data to provide dense supervision and enables efficient training. However, directly fine-tuning on expert data can hurt generalization when the data distribution is mismatched with the target model’s own distribution. In this work, we propose Data Adaptation for Reasoning Tuning (DART), which formulates the use of a fixed, potentially distributionally misaligned SFT dataset as an optimization problem over demonstration transformations. DART trains a mapper model with reinforcement learning to convert original SFT data into model-adapted supervision that better matches the target model’s distribution and learning preferences. The transformed data are then used for SFT, allowing the target model to better exploit external supervision. Experiments across multiple models and datasets show that DART improves generalization, achieves higher training efficiency than direct RL, and helps models surpass standard SFT. Our code is available at https://anonymous.4open.science/r/DART525E50D.

[21] Beyond Questions: Evaluating What Large Language Models (Actually) Know cs.CL | cs.AIPDF

Luca Giordano, Simon Razniewski

TL;DR: 本文提出了一种名为’开放知识评估’的新范式，用于评估大语言模型（LLMs）的参量知识。该范式通过开放式提示（如’告诉我关于马丁·路德·金的一切’）来评估模型自然表达的知识，而非依赖预定义的狭窄问题。研究者基于此范式构建了名为BeQu的基准，包含10,000个实体及其参考语料库，用于验证模型生成的陈述。

Details

Motivation: 现有知识基准通常依赖预定义的问题进行评估，这存在可用性偏差，只能评估基准设计者明确选择查询的知识，而LLMs的参量知识作为其成功的基石，目前仍缺乏深入理解。

Result: 研究使用BeQu基准评估了广泛的LLM，并分析了推理努力、模型规模、提示格式和知识领域等因素的影响。相关数据和排行榜已在GitHub仓库和基准网站上公开。

Insight: 创新点在于从’预定义答案检索’转向’表征模型自然表达的知识’的评估范式。这有助于更全面地理解LLMs实际掌握的知识结构，而不仅仅是其回答特定问题的能力，为知识评估提供了新视角。

Abstract: Parametric knowledge in large language models (LLMs) is a cornerstone of their success, yet remains poorly understood. Existing knowledge benchmarks typically rely on predefined questions (e.g., “What is the birth date of M.L. King?”), evaluating only knowledge that benchmark designers explicitly choose to query, a problematic availability bias. In this paper, we introduce open knowledge evaluation, a new paradigm for LLM knowledge benchmarking. Instead of asking narrow questions, it evaluates models on the knowledge they choose to surface in response to open-ended elicitation prompts (e.g., “Tell me everything you know about M.L. King”). This shifts the focus from predefined answer retrieval toward characterizing the knowledge models naturally express. We instantiate this paradigm with BeQu (Beyond Questions), a benchmark of 10,000 entities paired with reference corpora for statement verification. Using BeQu, we evaluate a broad range of language models and analyze the effects of reasoning effort, model scale, prompt format, and knowledge domain. Data and leaderboard are available on this work’s GitHub repository and at the benchmark’s website.

[22] Reasoning Depth and Environment Complexity: A Controlled Study of RLVR Data Allocation across Logical Reasoning Tasks cs.CL | cs.AIPDF

Yihua Zhu, Qianying Liu, Fei Cheng, Jiaxin Wang, Akiko Aizawa

TL;DR: 本文研究了强化学习可验证奖励（RLVR）在逻辑推理任务中的数据分配问题，通过构建一个合成知识图谱环境，系统分析了推理深度和环境复杂性两个维度对模型性能的影响。研究发现，联合考虑深度和复杂性的数据覆盖策略优于单一维度方法，不同推理家族（演绎、溯因、归纳、类比）对数据分布的响应存在差异，且均匀混合数据优于分阶段课程学习。

Details

Motivation: 现有RLVR研究对推理空间的刻画过于狭窄，仅将难度视为推理深度，且奖励集中于正向演绎状态追踪。本文旨在通过引入环境复杂性维度（如干扰项和交互结构）以及扩展奖励推理形式（包括演绎、溯因、归纳和类比），更全面地刻画真实世界的推理任务，以探究数据分配如何影响不同推理能力。

Result: 在可控的合成知识图谱环境中进行实验，结果表明：1）联合覆盖深度和复杂性的数据分配方案性能最佳；2）不同推理家族响应不均，溯因推理在RL覆盖区域外性能下降明显，任务相关性聚类为演绎-溯因和归纳-类比对；3）在固定预算下，均匀数据混合优于分阶段课程学习。此外，现成的近期模型也表现出相同的演绎优于溯因的不对称性。

Insight: 创新点在于将推理难度从单一深度维度扩展为深度与复杂性的二维空间，并系统评估了四种核心推理形式（演绎、溯因、归纳、类比）对数据分布的敏感性。客观来看，该研究为RLVR数据分配提供了更细致的分析框架，揭示了不同推理能力依赖不同的训练数据模式，对设计更均衡的推理模型训练策略具有借鉴意义。

Abstract: Reinforcement learning with verifiable rewards (RLVR) has become central to post-training reasoning models, yet a key limitation of existing studies is their narrow view of the reasoning space: difficulty is treated as reasoning depth alone, and reward is concentrated on forward deductive state tracking. We instead characterize the reasoning space along two dimensions. Difficulty. Beyond reasoning depth, we study environment complexity, where models must identify the correct path amid distractors and interacting structures. Rewarded reasoning form. We consider four abilities core to real-world reasoning: deductive state tracking, abductive recovery of hidden events or facts, inductive rule induction, and analogical transfer. To disentangle these factors, we construct a synthetic knowledge-graph environment with controlled pre- and post-training distributions, where each instance varies along depth, complexity, and task family. Three findings emerge: joint depth-complexity coverage outperforms single-axis recipes; reasoning families respond non-uniformly, with abductive reasoning degrading outside the RL-covered region and task correlations clustering into deductive-abductive and inductive-analogy pairs; and uniform mixing outperforms staged curricula under a fixed budget. We also find that recent off-the-shelf models exhibit the same deductive-over-abductive asymmetry, suggesting that this gap is not merely an artifact of our controlled setup.

[23] Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancement cs.CLPDF

Dingwei Chen, Zefang Zong, Zhipeng Ma, Leo Luo, Yang Li

TL;DR: 本文提出AKBE方法，通过动态探测模型的内在知识边界来优化基于LLM的智能体强化学习训练。该方法在训练过程中并行执行带工具和不带工具的推理路径，通过对比正确性来构建针对性的监督信号，从而指导智能体形成高效的工具使用模式。

Details

Motivation: 现有智能体强化学习训练会导致冗余工具调用增加并模糊模型的内在知识边界，即模型无法区分何时需要工具调用、何时仅凭参数知识即可解决问题；而基于奖励塑形的现有方案会产生粗粒度的优化目标，容易引发奖励破解问题。

Result: 在七个QA基准测试上的实验表明，AKBE相比标准智能体强化学习方法平均任务准确率提升+1.85，工具调用减少18%，工具生产率提高25%，且没有准确率-效率的权衡。

Insight: 创新点在于定义了基于每个实例的知识边界（判断是否需要工具及所需最少工具调用），并通过双路径推演动态探测该边界，从而构建细粒度的监督信号；该方法具有即插即用的兼容性，可无缝集成到现有智能体RL训练循环中。

Abstract: Agentic reinforcement learning (RL) has proven effective for training LLM-based agents with external tool-use capabilities. However, we identify that agentic RL training induces increasing redundant tool calls and blurs the model’s intrinsic knowledge boundary, where the model fails to distinguish when tools are needed versus when parametric knowledge suffices. Existing solutions based on reward shaping create coarse-grained optimization targets that tend to incentivize indiscriminate tool-call suppression, leading to reward hacking. In this paper, we propose AKBE (Agentic Knowledge Boundary Enhancement), an on-policy method that dynamically probes the model’s intrinsic knowledge boundary through dual-path (with-tool and no-tool) rollouts during training. We define the knowledge boundary as the per-instance determination of whether tools are required and the minimum tool calls necessary. By comparing correctness across paths, AKBE categorizes trajectories and constructs targeted supervisory signals that guide efficient tool-use patterns for each question. These signals are integrated seamlessly into the agentic RL training loop. Experiments on seven QA benchmarks demonstrate that AKBE improves task accuracy by +1.85 on average and reduces tool calls by 18% over standard agentic RL, yielding 25% higher tool productivity without any accuracy-efficiency trade-off. Further analysis suggests its plug-and-play compatibility across different RL algorithms and the mechanism of each signal category. Our code is available at https://github.com/CuSO4-Chen/AKBE.

[24] Tournament-GRPO: Group-Wise Tournament Rewards for Reinforcement Learning in Open-Ended Long-Form Generation cs.CL | cs.AIPDF

Zixuan Yang, Yiqun Chen, Wei Yang, Erhan Zhang, Zihan Shen

TL;DR: 本文提出Tournament-GRPO，一种用于开放式长文本生成的强化学习方法，通过将基于量规的LLM评分转换为组内多轮锦标赛的相对奖励，以解决现有方法中绝对评分难以校准、区分度弱和优化易饱和的问题。

Details

Motivation: 开放式长文本生成中，可靠的参考答案和自动评估指标往往缺失，现有基于量规的方法依赖点对点的LLM评分，但绝对评分在复杂响应间难以校准、区分度不足且优化时易饱和。

Result: 在Deep Research Bench上的实验表明，Tournament-GRPO持续优于现有奖励设计基线，整体得分比最强基线提高了4.52分，展现了更好的效果-效率权衡。

Insight: 创新点在于将量规引导的LLM判断通过同查询生成结果间的多轮锦标赛转换为相对奖励，利用组内比较和累积结果归一化来提供更有效的强化学习信号，避免了绝对评分的局限性。

Abstract: Reinforcement learning in open-ended long-form generation is challenging because reliable reference answers and automatic metrics are often unavailable. Existing rubric-based methods typically rely on pointwise LLM-as-a-judge scoring, but absolute scores are difficult to calibrate across complex responses, may provide weak discrimination among same-query rollouts, and can become saturated during optimization. We propose Tournament-GRPO, a group-wise reward framework that converts rubric-guided LLM judgments into relative rewards through repeated multi-round tournaments among same-query rollouts. Tournament-GRPO compares candidates within groups, accumulates tournament outcomes, and normalizes them into group-wise rewards for GRPO training. Experiments on Deep Research Bench show that Tournament-GRPO consistently outperforms existing reward-design baselines, achieving a 4.52-point overall-score improvement over the strongest baseline. Further analyses show that tournament rewards provide a favorable effectiveness–efficiency trade-off and that tournament design affects training dynamics. These results suggest that rubric-guided tournament comparison provides an effective reward signal for reinforcement learning in open-ended long-form generation.

[25] Recon: Reconstruction-Guided Reasoning Synthesis for User Modeling cs.CL | cs.AIPDF

Alan Zhu, Mihran Miroyan, Carolyn Wang, Andrew Zhou, Lisa Dunlap

TL;DR: 本文提出了一种名为Recon的新方法，用于用户建模中的推理合成，该方法通过动作重建来评估推理轨迹的预测能力，从而避免后验合理化，并生成更真实、可解释的推理。

Details

Motivation: 现有方法在用户建模中通过合成推理轨迹来模拟用户行为，但通常基于上下文和动作的条件生成，这导致后验合理化而非真实推理，无法捕捉潜在的因果决策路径。

Result: 在四个领域上，Recon相比后验合理化基线Backward Synthesis取得了54.7%的胜率；使用Recon奖励训练推理合成模型后，下游用户建模性能进一步提升，胜率最高达70.0%。

Insight: 创新点在于引入动作重建作为评分机制来评估推理质量，强调有用且可解释的推理应自然地从上下文中引出动作，这有助于生成更准确的用户模拟并提升模型间的可迁移性。

Abstract: User modeling aims to use language models (LMs) to mimic an individual’s behavior from a corpus of past context-action pairs (e.g., conversation turns), enabling the simulation of users in settings like behavioral science, human-AI collaboration, and market research. Recent approaches augment these corpora with synthesized reasoning traces, typically generated by conditioning on both context and action. However, such conditioning constitutes post-hoc rationalization rather than reasoning: the trace is guaranteed to justify the action, but may not encode the underlying latent causal decision paths. We propose Recon, which uses action reconstruction to score reasoning traces by their predictive power: given a context and candidate reasoning, a reconstruction model predicts the action, and reconstruction fidelity determines reasoning quality. Across four domains, Recon achieves a 54.7% win rate over Backward Synthesis, a standard post-hoc rationalization baseline. Further, we find that training a reasoning synthesis model with rewards derived from Recon improves downstream user modeling performance, achieving a win rate of up to 70.0% over baselines. We further show that Recon-synthesized reasoning transfers across models, and improves user modeling beyond the reconstruction model. Our work demonstrates that post-hoc rationalization is insufficient for reasoning synthesis, and that useful and interpretable reasoning should naturally elicit the action from the context.

[26] Cast a Wider Net: Coordinated Pass@K Policy Optimization for Code Reasoning cs.CL | cs.AIPDF

Yilong Li, Suman Banerjee, Tong Che

TL;DR: 本文提出了一种名为CPPO（Coordinated Pass@K Policy Optimization）的方法，用于优化代码生成任务中的pass@K策略。该方法通过一个规划器生成K个不同的高级方法策略，然后由共享求解器为每个策略尝试生成解决方案，从而避免标准独立采样方法中常见的重复推理路径问题，更有效地利用计算预算。

Details

Motivation: 标准代码生成方法在验证器下重复采样以分配测试时计算资源，但通常从单一答案分布中独立抽取K个样本，导致尝试经常坍缩到近似的推理路径上，浪费预算在冗余的rollouts上。在竞争性编程等场景中，许多问题存在多种不同的算法策略，而pass@K只需要一个正确尝试即可，因此需要一种能协调探索不同策略的方法。

Result: 在APPS、CodeContests和LiveCodeBench-v6三个基准测试上，CPPO在相同的K=4求解器尝试预算下，相比直接采样、规划基线、仅规划器SFT以及面向pass@K的RL方法，均提升了pass@4性能，在九个模型-基准单元中有六个取得了统计显著的增益。最大单次增益是在Qwen3.5-9B模型上的LiveCodeBench-v6基准，相比最强基线PKPO提升了0.16（从0.588提升至0.748，配对bootstrap检验p<0.05）。

Insight: 论文的创新点在于将pass@K生成转化为对策略的联合探索，通过规划器生成多样化的高级方法元组，并设计了一个乘法规划器奖励（R_plan = J_ψ·R_out）来训练联合策略，该奖励仅对能导致验证器确认的pass@K成功的有效策略元组分配信用。这鼓励了策略多样性，避免了冗余采样，从而更高效地利用有限的求解尝试预算。

Abstract: Repeated sampling with a verifier is the standard way to allocate test-time compute for code generation, with pass@$K$ as the canonical metric. Yet the standard policy class draws $K$ independent samples from a single answer distribution, so attempts often collapse onto near-duplicate reasoning paths and waste the budget on redundant rollouts. This failure is costly in competitive programming, where many problems admit multiple distinct algorithmic strategies and pass@$K$ requires only one correct attempt. We propose Coordinated Pass@$K$ Policy Optimization (CPPO), which turns pass@$K$ generation into joint exploration over strategies: a planner emits a tuple of $K{=}4$ alternative high-level methods, and a shared solver attempts one solution per method. CPPO trains this joint policy with a multiplicative planner reward, $R_{\mathrm{plan}} = J_ψ\cdot R_{\mathrm{out}}$, assigning credit only to valid strategy tuples that lead to verifier-confirmed pass@$K$ success. Across APPS, CodeContests, and LiveCodeBench-v6, CPPO improves pass@$4$ over direct sampling, planning baselines, planner-only SFT, and pass@$K$-oriented RL under the same $K{=}4$ solver-attempt budget, with statistically significant gains on six of nine model–benchmark cells. The largest single gain is $+0.16$ on Qwen3.5-9B LiveCodeBench-v6 over the strongest baseline, PKPO ($0.588 \rightarrow 0.748$; paired bootstrap, $p < 0.05$).

Xinglin Wang, Hao Lin, Shaoxiong Feng, Peiwen Yuan, Yiwei Li

TL;DR: 本文提出了一种名为协作并行思考（CPT）的无训练推理框架，旨在解决现有并行测试时扩展（TTS）方法中因分支信息隔离导致的冗余探索问题。CPT通过在并行分支间共享搜索过程中的中间发现，构建去重的信息池并广播，使各分支能复用已有发现而非重复探索，从而在相同计算预算下提升推理效率与准确性。

Details

Motivation: 现有并行TTS方法在搜索过程中各分支相互隔离，中间发现无法及时共享，导致大量冗余探索和更多搜索步骤，降低了推理效率。本文旨在通过促进搜索时的跨分支信息共享来解决这一问题。

Result: 在HMMT和AIME基准测试上的实验表明，在不同计算预算和模型规模下，CPT在准确率-延迟帕累托前沿上均优于强基线方法，证明了搜索时协作对于高效并行TTS的有效性。

Insight: 核心创新在于提出了一个无需训练、在推理时实现跨分支信息共享的框架，通过提取、去重和广播中间信息来减少冗余计算。这为提升并行推理效率提供了一个新的有效方向，即从信息协作而非单纯增加分支数量的角度进行优化。

Abstract: Test-Time Scaling (TTS) enhances the reasoning capabilities of large language models by allocating additional inference compute to explore the solution space. However, existing parallel TTS methods typically keep branches isolated during search: intermediate discoveries remain branch-private and cannot guide other branches in time. This information isolation causes substantial redundant exploration, as branches repeatedly rediscover information already found elsewhere and require more search steps to collect complete decision information needed to reach correct answers. To bridge this gap, we propose \textbf{Collaborative Parallel Thinking (CPT)}, a training-free inference framework that enables search-time information sharing across parallel branches. CPT extracts compact intermediate information from ongoing branches, maintains a deduplicated query-level information pool, and broadcasts pool entries through the input context, allowing each branch in subsequent search steps to reuse discoveries made by other branches rather than rediscover the same information. Empirically, experiments on HMMT and AIME benchmarks show that CPT establishes a stronger accuracy–latency Pareto frontier than strong baselines across rollout budgets and model scales, highlighting search-time collaboration as an effective direction for efficient parallel TTS.

Ye Yuan, Rui Song, Weien Li, Zeyu Li, Haochen Liu

TL;DR: 本文介绍了QUACK，一个用于评估多模态社交推理中智能体语言是否基于真实感知和行为的开源环境与评估框架。该框架通过游戏结果、行为轨迹和话语一致性三个层面进行审计，并利用声明验证管道自动检测空间幻觉、无根据指控、欺骗崩溃和语言-行为不一致等问题。

Details

Motivation: 现有社交推理游戏大多仅通过胜率等游戏结果评估智能体，且多为纯文本交互，难以判断智能体语言是否真正基于其感知和行为，也无法识别其行为背后的失败模式。

Result: 在评估三个前沿视觉语言模型（VLMs）时，发现即使最强的智能体也有15.1%的可验证空间声明存在幻觉，且超过一半的指控缺乏基于证据的支撑。

Insight: 创新点在于提出了一个多层次的评估框架，通过自动化的声明验证管道来审计智能体语言的真实性，这有助于更细致地诊断多模态社交推理中的失败模式，并为智能体行为分析提供了新的工具和方法。

Abstract: Social deduction games have become a popular testbed for probing reasoning, deception, coordination, and belief modeling in Large Language Model (LLM) agents. However, most environments are scored only by game outcomes such as win rates and largely remain to text-only interaction, making it difficult to tell whether an agent’s language is actually grounded in what it perceived and did, or to identify the failure modes underlying its behavior. To address this gap, we introduce QUACK, an open-source environment and evaluation framework for auditing the grounding of agent language in multimodal social reasoning. QUACK evaluates agents at three levels: game outcomes, behavioral trajectories, and utterance-level consistency. Its core Statement Verification Pipeline reconstructs each agent’s ground-truth trajectory from engine logs and checks every discussion claim against it, automatically flagging spatial hallucination, unsupported accusation, deception collapse, and language-action inconsistency. Evaluating three frontier VLMs in both homogeneous and cross-model adversarial settings, we find that even the strongest agent hallucinates 15.1% of its verifiable spatial claims and makes over half of its accusations without grounded evidence. We release the full engine, evaluation framework, toolkit, and logs at https://github.com/AAAAA-Academia-Attractions/QUACK.

[29] Large Language Model-Powered Query-Driven Event Timeline Summarization in Industrial Search cs.CL | cs.IRPDF

Mingyue Wang, Xingyu Xie, Hang Yang, Li Gao, Lixin Su

TL;DR: 本文介绍了QDET，一个部署在百度搜索引擎中的查询驱动事件时间线摘要生产系统。该系统从每日检索的数百万文档中识别并组织与查询密切相关的子事件，生成聚焦的事件时间线。其核心创新在于通过多任务监督微调和基于强化学习的摘要生成，使紧凑模型在特定领域达到与超大通用模型相当的性能，同时大幅降低计算成本。

Details

Motivation: 为了解决搜索引擎在处理趋势新闻查询时，需要理解事件随时间演变过程的问题，传统以主题为中心的方法追求全面覆盖，但QDET旨在从海量噪声候选文档中，构建与特定查询高度相关的、聚焦的事件时间线摘要。

Result: 在线A/B测试显示，QDET在百度搜索中带来了5.5%的点击率提升、4.6%的更长的停留时间和4.4%的更深度的探索。其微调后的7B参数模型在时间线摘要任务上取得了76.2%的F1分数，略超过DeepSeek-R1-671B的零样本性能（76.1% F1），同时参数量仅为后者的1%。在约束满足度上，其强化学习方法比671B规模模型高出7.7个百分点。

Insight: 论文宣称的创新点包括：1）包含时序排序、因果判断和时间线补全三个辅助任务的多任务监督微调，使紧凑模型在专业领域能匹配超大通用模型的性能；2）基于强化学习的事件简洁摘要生成，在严格长度约束下保持语义质量。从客观角度看，其将领域特定优化与强化学习约束控制相结合，为生产环境部署高效、高质量的专用模型提供了可行路径，并展示了时间线理解能力向下游任务（如热度预测）的知识迁移潜力。

Abstract: Understanding how events evolve over time is essential for search engines handling queries about trending news. We present QDET (Query-Driven Event Timeline Summarization), a production system deployed on Baidu Search that constructs focused event timelines to explain specific query events. Unlike traditional topic-centric approaches that aim for comprehensive coverage, QDET identifies and organizes sub-events closely relevant to the query from noisy candidate sets formed by millions of documents retrieved daily. QDET incorporates two key innovations: (1) multi-task supervised fine-tuning with three auxiliary tasks-temporal ordering, causal judgment, and timeline completion-that enable compact models to match the performance of much larger general-purpose models in specialized domains; (2) reinforcement learning-based event concise summarization that enforces strict length constraints while maintaining semantic quality, achieving 88.2% length compliance and outperforming 671B-scale models by 7.7 points in constraint satisfaction. Our fine-tuned 7B parameter model achieves 76.2% F1 score on timeline summarization, slightly surpassing the zero-shot performance of DeepSeek-R1-671B (76.1% F1) while using only 1% of its parameters-demonstrating that domain-specific optimization enables production-ready models with comparable quality at drastically reduced computational costs. Online A/B tests on Baidu Search validate real-world effectiveness, showing 5.5% CTR improvement, 4.6% longer dwell time, and 4.4% deeper exploration compared to single-task baselines. We further demonstrate that timeline understanding transfers to heat prediction, confirming effective knowledge transfer to downstream tasks.

[30] LLMs Are Already Good Tutors: Training-Free Prompt Optimization for Pedagogical Math Tutoring cs.CL | cs.LGPDF

Unggi Lee, Minchul Shin, Yeil Jeong, Sookbun Lee, Jeongsu Moon

TL;DR: 本文探讨了无需训练、仅通过API调用优化系统提示的方法，作为替代传统基于强化学习的数学辅导大语言模型对齐方案。研究评估了12种提示优化方法，发现所有方法均超越了最强的RL训练基线，其中ParetoGrad在多项教学指标上取得了最佳平衡。行为分析表明，无训练方法更依赖教学知识模式，并揭示了任务依赖的推理模式效应。

Details

Motivation: 传统基于强化学习的LLM数学辅导对齐方法需要多GPU基础设施和训练成本高，本文旨在研究是否可以通过无需训练的提示优化（仅通过API调用演化系统提示）作为一种实用替代方案。

Result: 在2个OOD基准测试套件上的5种条件下评估了12种方法，所有方法的最佳配置均超越了最强的RL训练基线（R_total = 0.633）。提出的ParetoGrad方法在后测解决率、泄漏控制和帮助性方面取得了最佳的帕累托平衡，而非在单一组件上占优。

Insight: 创新点在于系统性地将提示优化方法应用于数学教学对齐任务，并提出了专门针对教育的优化方法。研究发现无训练方法显著增加了教学知识模式的使用率（是RL模型的2-3倍），同时意图层面的脚手架减少了约10个百分点，且任务依赖的推理模式效应在两种范式中一致存在，这为仅用提示高效开发教学对齐的LLM辅导提供了新途径。

Abstract: Aligning LLMs for math tutoring typically requires RL-based training with multi-GPU infrastructure. We investigate whether training-free prompt optimization-evolving only the system prompt via API calls-can serve as a practical alternative. We adapt 7 published methods and propose 5 education-specialized methods, evaluating these 12 methods under 5 conditions on 2 OOD benchmark suites. All 12 best-per-method configurations surpass the strongest RL-trained baseline (R_total = 0.633), and our ParetoGrad achieves the best Pareto balance across post-test solve rate, leak control, and helpfulness, rather than dominating any single component. Behavioral analysis with an 82-code educational codebook reveals that training-free methods rely on teaching-knowledge patterns at 2-3x the rate of RL-trained models, with a compensating ~10 percentage-point reduction in intent-level scaffolding. We also find a task-dependent reasoning mode effect consistent across training-free and RL-based paradigms. Our approach enables efficient development of pedagogically aligned LLM tutors with prompts alone and minimal compute.

[31] Learning When to Think While Listening in Large Audio-Language Models cs.CL | cs.AI | cs.LG | cs.SDPDF

Zhiyuan Song, Weici Zhao, Yang Xiao, Suhao Yu, Cheng Zhu

TL;DR: 本文针对大型音频语言模型在流式语音交互中的推理时机问题，提出了一种可学习的‘等待-思考-回答’控制框架。该框架允许模型在接收到部分音频证据时，动态决定是继续等待、输出紧凑的推理更新还是给出最终答案，以平衡推理质量和响应速度。

Details

Motivation: 动机在于解决流式语音交互中推理质量与响应速度之间的固有矛盾：等待语音结束再推理能提升答案质量但会增加用户可见的延迟，而过早回答则可能在决定性证据到达前做出错误承诺。

Result: 在一个六任务的合成语音推理问答基准上，使用六项奖励的DAPO控制器将行加权准确率从67.6%提升至70.3%，并在相同部署条件下将端点后的最终思考长度减少了14%。在包含186条真人录音的Real Audio Bench上，SFT方法获得了最高的准确率，而六奖励DAPO控制器是唯一能将最终思考长度降至基线以下的已学习变体。

Insight: 核心创新点是提出了一个可学习的控制器来动态管理流式音频处理中的推理时机，将决策从单纯的‘何时回答’扩展为‘等待、思考更新、回答’的序列。其奖励函数设计全面，不仅优化最终答案正确性，还优化了整个决策轨迹的多个维度（如动作有效性、更新时机、延迟同步、推理质量和链一致性），这为构建更自然、高效的流式人机对话系统提供了新思路。

Abstract: Recent advances in Large Audio-Language Models (LALMs) have made real-time, streaming spoken interaction increasingly practical. In this setting, reasoning quality and responsiveness are tightly coupled: delaying reasoning until the speech endpoint can improve answer quality but moves deliberation into user-visible response delay, while answering too early risks committing before decisive evidence arrives. We introduce a learnable wait-think-answer control formulation for LALMs. Motivated by the incremental nature of human conversation, the controller decides under partial audio evidence when to wait, when to externalize a compact reasoning update, and when to answer. Using Qwen2.5-Omni-7B as the base model, we construct aligned wait-think-answer traces from spoken reasoning data, train the controller with supervised fine-tuning (SFT), and then apply Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO). The reward combines answer correctness, action validity, update timing, latency synchronization, reasoning quality, and chain consistency, optimizing the complete wait-think-answer trajectory and not the final answer alone. On a six-task synthetic spoken reasoning question answering (SRQA) benchmark, the six-reward DAPO controller improves the row-weighted accuracy from 67.6% to 70.3% while reducing post-endpoint final-think length by 14% under the same Qwen deployment harness. On a 186-item human-recorded Real Audio Bench, a transfer check beyond text-to-speech (TTS)-rendered speech, the controller family remains functional: SFT achieves the strongest accuracy, while the six-reward DAPO controller is the only learned variant whose final-think length falls below the base. These results suggest that a streaming model should learn when to make intermediate reasoning explicit during the audio stream.

[32] Not All Tokens Matter Equally: Dynamic In-context Vector Distillation with Decisive-Token Supervision for Long-form Medical Report Generation cs.CL | cs.CV | cs.LGPDF

Ning Wu, Rui Liu, Xinkun Lin, Weixing Chen, Jinxi Xiang

TL;DR: 本文提出了一种名为DIVE的轻量级蒸馏框架，用于解决长文本医学报告生成任务中传统方法因对所有输出词元平等对待而导致的监督不足问题。该框架通过关键词元监督和状态条件动态引导两种机制，重点增强病理相关词元和序列结束事件的监督，并动态调整注入信号，从而在冻结骨干网络的情况下提升生成质量。

Details

Motivation: 现有多模态蒸馏方法主要针对短文本任务评估，在长文本生成（如医学报告生成）中，由于输出中高频模板和语法词元占主导，而真正决定质量的关键词元（如病理相关词元和序列结束事件）分布稀疏，均匀的交叉熵监督不足，且自回归解码会偏离教师轨迹，导致生成效果受限。

Result: 在MIMIC-CXR和CheXpert Plus数据集上，使用两种医学视觉语言模型骨干进行实验，DIVE在词汇和临床代理指标上均表现优异，在所有数据集-骨干设置中取得了最佳的BLEU-4、ROUGE-L和RadGraph F1分数，同时在粗粒度标签级CheXbert F1上保持竞争力。

Insight: 创新点在于识别并针对长文本生成中关键监督信号稀疏的问题，提出了关键词元监督以重新平衡损失权重，以及状态条件动态引导以自适应调整蒸馏信号，从而在不微调骨干网络的情况下有效提升长文本生成的准确性和终止控制。

Abstract: Distilling demonstration effects into hidden-space interventions offers a lightweight alternative to full finetuning. However, existing multimodal variants are mostly evaluated on short-form tasks, where outputs end after a few tokens. Extending these methods to long-form generation exposes a fundamental yet underexamined limitation: token-level distillation implicitly treats all output tokens as equally informative, but long-form outputs are dominated by high-frequency template and grammatical tokens, while the tokens that actually determine output quality are sparsely distributed. In medical report generation (MRG), two such decisive tokens stand out: pathology-related tokens that determine diagnostic content, and the end-of-sequence (EOS) event that determines termination. Both receive insufficient supervision under uniform cross-entropy, and autoregressive decoding further compounds the problem by drifting away from teacher-forced trajectories. We propose DIVE, a frozen-backbone distillation framework that addresses long-form report generation through two complementary mechanisms matched to these failures. Decisive-token supervision restores supervision balance by upweighting the cross-entropy contribution of pathology-related tokens and the EOS event, ensuring that content fidelity and termination are learned during training rather than imposed at decoding time. State-conditioned dynamic steering replaces fixed open-loop residuals with hidden-state-dependent adapters, allowing the injected signal to adapt as decoding drifts. Experiments on MIMIC-CXR and CheXpert Plus with two medical VLM backbones show that DIVE consistently ranks among the strongest methods across lexical and clinical-proxy metrics. Our method achieves the best BLEU-4, ROUGE-L, and RadGraph F1 in all dataset–backbone settings, while remaining competitive on coarse label-level CheXbert F1.

[33] EpiCurveBench: Evaluating VLMs on Epidemic Curve Digitization cs.CLPDF

Thomas Berkane, Maimuna S. Majumder

TL;DR: 本文提出了EpiCurveBench基准测试和EpiCurveSimilarity (ECS)评估指标，用于评估视觉语言模型在流行病曲线图数字化任务上的性能。该基准包含1000张真实世界的流行病曲线图，而ECS指标通过动态规划对齐预测和真实时间序列，容忍局部时间偏移和间隙。实验发现，现有最强模型仅达到52.3%的ECS得分，且ECS比传统键值对指标能更好地区分不同模型的性能。

Details

Motivation: 现有图表数据提取基准测试存在两个问题：一是性能提升空间有限（前沿VLMs在ChartQA上已超过89%），二是评估指标将提取点视为无序键值对，忽略了时间序列的时间结构，并将小的对齐偏移视为灾难性错误。本文旨在解决这两个缺陷。

Result: 在EpiCurveBench上评估了六种方法（包括三个前沿闭源VLM、一个开源VLM和两个专用图表提取系统），最强模型仅达到52.3%的ECS得分。ECS指标将四个通用VLMs的性能分布在25分的范围内，而传统键值对指标（如RMS、SCRM）则将其压缩在5分的窄带内。ECS与四个下游流行病学汇总统计量的误差相关性比动态时间规整（DTW）强1.5到3.6倍。

Insight: 创新点在于提出了一个专注于时间序列结构（特别是流行病曲线）的真实世界基准测试，以及一个能容忍局部时间偏移和间隙、并对其进行比例惩罚的评估指标ECS。这为评估图表数据提取，尤其是时间序列图表，提供了更符合实际应用需求的框架，并揭示了现有模型在该任务上的显著不足。

Abstract: Chart-to-data extraction with vision-language models (VLMs) is increasingly evaluated on benchmarks that show diminishing headroom (frontier VLMs exceed 89% on ChartQA) and with metrics that treat extracted points as unordered key-value pairs, ignoring the temporal structure of time series and penalizing small alignment shifts as catastrophic failures. We address both gaps with EpiCurveBench, a benchmark of 1,000 real-world epidemic curve images curated from diverse public-health sources, and EpiCurveSimilarity (ECS), an evaluation metric that aligns predicted and ground-truth series via dynamic programming, tolerating local temporal shifts and gaps while penalizing them proportionally. Evaluating six methods–three frontier closed VLMs, one open VLM, and two specialized chart-extraction systems–we find the strongest model reaches only 52.3% ECS, and that ECS spreads the four general-purpose VLMs over a 25-point range where key-value metrics (RMS, SCRM) compress them into a 5-point band. We further validate ECS against four downstream epidemiological summary statistics, finding that higher ECS predicts smaller errors in total counts, peak timing, and peak magnitude, and higher growth-rate fidelity; across all four, ECS correlates 1.5–3.6 times more strongly than Dynamic Time Warping, which lacks a gap penalty and therefore cannot distinguish a truncated prediction from a temporally faithful one. EpiCurveBench targets a high-impact public-health application–unlocking decades of outbreak data trapped in published figures–but the benchmark and metric apply directly to any structured time-series chart-extraction setting.

[34] Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs cs.CL | cs.AIPDF

Wenhui Tan, Minghao Li, Xiaoqian Ma, Siqi Fan, Xiusheng Huang

TL;DR: 本文提出了一种名为Pair-In, Pair-Out (PIPO)的新方法，旨在统一输入端的潜在压缩和输出端的多令牌预测，以高效降低大语言模型自回归推理的成本。该方法通过将两个输入令牌折叠为一个潜在表示，同时从一个隐藏状态展开生成一个额外输出令牌，并引入一个轻量级置信度头来验证草稿令牌，从而无需昂贵的验证器。实验表明，PIPO在多个基准测试上显著提升了推理速度和答案生成质量。

Details

Motivation: 现有方法分别针对输入端（潜在压缩）和输出端（推测解码和多令牌预测）进行优化，但两者独立且输出端方法需要昂贵的验证器来确认不可靠的草稿令牌，这增加了推理成本。PIPO旨在统一这两方面，并消除验证器开销，同时保持可靠性。

Result: 在AIME 2025、GPQA-Diamond、LiveCodeBench v6和LongBench v2等基准测试上，使用Qwen3.5-4B和9B骨干网络，PIPO将pass@4指标提升了高达+7.15点，同时实现了高达2.64倍的首令牌延迟加速和2.07倍的每令牌延迟加速，显示出显著的性能提升。

Insight: 创新点在于将潜在压缩器和多令牌预测头视为镜像操作进行统一，并引入基于策略蒸馏训练的轻量级置信度头来替代昂贵的验证器，这既降低了推理成本，又保持了生成可靠性，为高效LLM推理提供了新思路。

Abstract: Long chain-of-thought reasoning has made autoregressive decoding the dominant inference cost of modern large language models. Existing methods target either the input side (latent compression) or the output side (speculative decoding and multi-token prediction, MTP), but the two lines of work have been pursued independently. Moreover, output-side methods must incur an expensive verifier pass to validate the unreliable draft tokens predicted by MTP. To address these issues, we propose \textbf{Pair-In, Pair-Out (PIPO)}, which unifies both sides by viewing a latent compressor and an MTP head as mirror-image operations: the compressor folds two input tokens into one latent representation, while the MTP head unfolds one hidden state into one additional output token. To remove the verifier cost without sacrificing reliability, PIPO trains a lightweight confidence head that decides whether draft tokens should be accepted. We observe that On-Policy Distillation (OPD) naturally matches the rejection-sampling criterion of speculative decoding, so the confidence head can be trained alongside OPD with negligible extra cost. Experiments on AIME 2025, GPQA-Diamond, LiveCodeBench v6, and LongBench v2 with Qwen3.5-4B and 9B backbones show that PIPO improves pass@4 over regular decoding by up to $+7.15$ points, while delivering up to $2.64\times$ first-token-latency and $2.07\times$ per-token-latency speedups.

[35] Self-Ensembling Vision-Language Models for Chart Data Extraction cs.CLPDF

Thomas Berkane, Qianyi Wang, Maimuna S. Majumder

TL;DR: 本文提出了一种用于图表数据提取的自集成视觉语言模型方法。该方法通过对同一图表图像从VLM中重复采样多个表格输出，并在单元格级别进行聚合，通过数值中位数生成更准确的共识表格。该方法还包括收敛检测和基于样本分散度的不确定性估计，以提升提取可靠性。

Details

Motivation: 图表数据常以图像形式存在，难以复用和分析，手动数字化耗时且易错。现有VLM方法在处理数据点多或风格变化大的图表时性能不足，需要更鲁棒的自动提取方法。

Result: 在ChartQA和新建的WB-ChartExtract基准测试中，该方法相比单次VLM输出提升了提取准确率，在WB-ChartExtract上实现了最高23%的相对改进。WB-ChartExtract包含平均数据点数量是ChartQA 7倍的更复杂图表。

Insight: 创新点包括VLM自集成策略、基于单元格中位数聚合的共识机制、收敛检测和不确定性估计。客观来看，该方法通过采样多样性提升鲁棒性，新建的复杂基准推动了领域评估标准的发展。

Abstract: Charts effectively convey quantitative information, but the underlying data are often locked in image form, hindering reuse and analysis. Manually digitizing charts is time-consuming and error-prone, motivating automatic chart-to-table extraction. Recent approaches use specialized vision-language models (VLMs), yet performance still lags on charts with many datapoints or substantial stylistic variation. We propose a VLM self-ensembling method that repeatedly samples multiple tabular outputs from the same VLM for a fixed chart image and aggregates them at the level of individual table cells. We align candidate tables and take per-cell medians over numerical values to produce a more accurate consensus table. Our method also includes convergence detection to stop sampling once the aggregated table stabilizes, and uncertainty estimation based on dispersion across samples to help users assess extraction reliability. Because existing chart extraction benchmarks contain relatively simple plots with limited room for improvement, we introduce WB-ChartExtract, a new benchmark built from World Bank data with more complex and stylistically diverse charts; on average, its charts contain 7 times more datapoints than those in the ChartQA benchmark. Across both ChartQA and WB-ChartExtract, our approach improves extraction accuracy over single-pass VLM outputs, yielding up to 23% relative improvement on WB-ChartExtract after ensembling. More broadly, our method helps unlock tabular data previously siloed in chart images, enabling downstream analysis and reuse.

[36] It’s Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertainty cs.CL | cs.AI | cs.LGPDF

Kevin H. Guo, Chao Yan, Avinash Baidya, Katherine Brown, Xiang Gao

TL;DR: 本文提出了一种名为MUSE的两阶段评估框架，旨在区分驱动大型语言模型（LLM）顺从行为的机制。研究发现，LLM的顺从行为不仅源于从人类反馈强化学习中学到的奉承，还与其推理时的认知不确定性有关。具体而言，顺从行为由奉承性顺从和不确定性驱动顺从两个因素共同驱动，并受用户感知专业度和建议合理性的影响。

Details

Motivation: 先前研究主要将LLM放弃初始立场以顺从用户反驳的行为归因于奉承，但本文假设模型的认知不确定性也是驱动顺从行为的关键因素。

Result: 研究通过MUSE框架证明，顺从行为确实由奉承性顺从和不确定性驱动顺从共同驱动，且两者都随用户感知专业度和建议合理性的增加而增强。

Insight: 创新点在于提出了MUSE框架来解耦顺从行为的驱动机制，区分了由对齐诱导的奉承和由训练语料驱动的不确定性，这有助于制定更有针对性的干预策略。

Abstract: Large language models (LLMs) are known to abandon their initial stance to conform to user pushback. While prior research largely attributes this behavior to sycophancy learned during reinforcement learning from human feedback, we hypothesize that conformity is also driven by a model’s epistemic uncertainty at inference time. In this paper, we introduce MUSE, a two-stage evaluation framework to disentangle the mechanisms driving LLM conformity. Specifically, MUSE maps a model’s epistemic uncertainty in responding to a query against its likelihood to yield to user pushback in a subsequent turn. We demonstrate that the mechanisms driving conformity extend beyond sycophancy alone. Specifically, we characterize two distinct factors that jointly drive conformity: sycophantic conformity, where a model aligns with user pushback even with absolute certainty in its initial response, and uncertainty-driven conformity, where a model’s likelihood for conformity increases alongside its uncertainty. Furthermore, we conduct ablation studies to demonstrate that both sycophantic conformity and uncertainty-driven conformity grow with 1) the LLM’s perceived expertise of the user and 2) the plausibility of the user’s suggestions. More broadly, MUSE informs more targeted intervention strategies by distinguishing alignment-induced sycophancy and training-corpora-driven uncertainty.

[37] Real Images, Worse Judgments: Evaluating Vision-Language Models on Concreteness and Imagery cs.CLPDF

Yifan Jiang, Ruoxi Ning, Sheng Yao, Freda Shi

TL;DR: 本文研究了视觉-语言模型在词汇判断任务中视觉输入的作用，发现真实图像上下文不仅未能一致提升模型性能，反而在视觉证据最不相关时显著降低与人类评分的一致性。通过分析发现，真实图像会导致表征偏移和对虚假视觉线索的过度敏感，从而削弱目标词汇属性的可恢复性。研究进一步表明，在推理时指示模型仅关注文本内容可以减轻这种性能下降。

Details

Motivation: 动机是检验视觉输入是否总是能提升多模态模型的语言理解能力，特别是探究视觉-语言模型能否在词汇判断中区分有用的视觉证据与无关的图像上下文。

Result: 实验基于人类具体性和意象性评分数据集进行，结果显示真实图像上下文并未带来一致增益，反而经常损害模型与人类评分的一致性，尤其是在视觉证据最不相关时（如抽象、低意象性词汇）性能下降最明显。通过指令调整让模型在推理时仅关注文本，可以在这些脆弱子集上获得最清晰的性能提升。

Insight: 论文的创新点在于揭示了当前指令调优的视觉-语言模型在词汇判断任务中对视觉上下文的校准不足，容易受到无关图像信息的干扰。客观分析认为，其核心洞察是指出模型需要更好地判断何时应该利用视觉信息，并提出了通过推理时指令来缓解问题的简单有效方法。

Abstract: Visual inputs are often assumed to improve language understanding in multimodal models. We examine this assumption by asking whether vision-language models (VLMs) can distinguish useful visual evidence from incidental image context in lexical judgments. We use human concreteness and imagery ratings because they span words with varying expected visual relevance, from abstract and low-imagery words to concrete and high-imagery words. We find that real-image contexts do not yield consistent gains and often hurt alignment with human ratings, most sharply when visual evidence is least relevant. Through probing and canonical correlation analysis, complemented by an attribution case study, we find that real-image contexts are associated with representational shifts and greater sensitivity to spurious visual cues, coinciding with weaker recoverability of the targeted lexical properties. We further show that instructing models to focus solely on textual content at inference time can reduce this degradation, with the clearest gains on these vulnerable subsets. Our findings suggest that current instruction-tuned VLMs need better calibration of when visual context should inform lexical judgments.

[38] Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models cs.CL | cs.CVPDF

Yifan Jiang, Dae Yon Hwang, Jesse C. Cresswell, Freda Shi

TL;DR: 本文提出了Chartographer框架，通过生成反事实图表来严格评估视觉语言模型（VLMs）的视觉推理能力。该框架将图表逆向工程为可执行代码，生成可控的反事实变体，并基于可执行的问答逻辑推导新答案。实验表明，反事实图表能揭示模型在单一图表评估中隐藏的泛化失败问题。

Details

Motivation: 现有图表问答（QA）基准中，模型可能通过捷径或先验知识而非视觉推理来回答问题，因此需要更严格的评估方法。本文旨在通过反事实图表（保持任务不变但改变图表和答案）来强制模型进行视觉推理。

Result: 在现有图表QA数据集上评估了专有和开源VLMs，测量了模型对变化的敏感性和泛化能力。反事实图表显示，模型在正确回答原始图表后常无法泛化到变体，尤其是在需要新视觉推理路径时失败率最高。

Insight: 创新点在于提出反事实图表生成框架，通过可执行代码实现图表重构和可控变体生成，从而系统评估VLMs的视觉推理泛化能力。客观来看，该方法为评估模型对视觉变化的鲁棒性提供了新视角。

Abstract: Chart question-answering (QA) benchmarks aim to pose questions that require visual reasoning to correctly answer, but models can often reach solutions through shortcuts or prior familiarity with a chart based on their own background knowledge. To strictly evaluate visual reasoning, we propose counterfactual charts where the chart-question task remains fixed, but underlying chart and the corresponding answer are varied. We introduce Chartographer, a framework to reverse engineer charts into executable code, validate reconstruction fidelity, generate seed-controlled counterfactual variants, and derive new answers from executable QA logic. We apply this framework to existing chart QA datasets and evaluate proprietary and open-source vision-language models (VLMs), measuring variation sensitivity and generalizability. Counterfactual charts reveal failures hidden by single-chart performance: VLMs often fail to generalize after answering the original chart correctly. We find failures are most prevalent when updated charts require novel visual reasoning pathways.

cs.CV [Back]

[39] Not All Modalities Are Equal: Instruction-Aware Gating for Multimodal Videos cs.CVPDF

Bonan Ding, Umair Nawaz, Ufaq Khan, Abdelrahman M. Shaker, Muhammad Haris Khan

TL;DR: 本文提出了一种名为UniMVU的统一多模态视频理解框架，旨在解决预训练视频大语言模型在处理包含音频、深度图等辅助流视频时面临的模态干扰问题。该框架通过指令感知的动态门控机制（包括模态内门控和模态级门控）自适应地平衡不同模态的重要性，并结合跨模态自注意力和快慢融合策略提升性能。

Details

Motivation: 现有视频大语言模型在视频包含音频、深度图等辅助流时表现不佳，因为均匀融合会导致模态干扰，让不相关通道分散模型注意力。

Result: 在AVQA、AVSD、Music-AVQA、ScanQA、SQA3D和MVBench六个基准测试上，UniMVU相比静态融合基线取得了一致提升，在CIDEr指标上最高提升13.5分。

Insight: 创新点在于提出了指令感知的双层动态门控机制（模态内和模态级），能够根据文本指令自适应调整模态重要性；同时采用快慢融合减少时间对齐流的冗余，为多模态视频理解提供了无需手工设计融合规则的统一方案。

Abstract: Pre-trained video large language models excel at visual reasoning. However, they struggle when videos arrive with auxiliary streams, such as audio, depth map, or dense temporal evidence. In such a scenario, uniform fusion induces modality interference, allowing irrelevant channels to distract the model. To address this issue, we present a unified multimodal video understanding framework, named UniMVU, that performs instruction-aware fusion across video, audio, depth map, or any other modality inputs via two levels of dynamic gating: inner-modality gates emphasize salient regions within each modality, whereas modality-level gates re-weight whole streams; both are conditioned on the text instruction to adaptively balance modality importance. Our UniMVU combines cross-modal self-attention with instruction-driven inner-modality gating module and a modality-level gating module with control token; for time-aligned streams we further adopt a fast-to-slow fusion scheme that reduces redundancy. Across six benchmarks (AVQA, AVSD, Music-AVQA, ScanQA, SQA3D and MVBench), our UniMVU achieves consistent gains over static-fusion baselines achieving gains as high as 13.5 in terms of CIDEr metric. Further, our analysis shows that the gating mechanism aligns with the human-interpretable modality relevance, and ablations show the contributions of inner-modality and modality-level gating. Our UniMVU provides a simple, unified recipe for instruction-aware multimodal video understanding that scales to diverse modalities without hand-crafted fusion rules.

[40] Sentinel: Embodied Cooperative Spatial Reasoning and Planning cs.CV | cs.MAPDF

Xiangye Lin, Hongxin Zhang, Ruxi Deng, Qinhong Zhou, Chuang Gan

TL;DR: 论文提出Sentinel Challenge基准测试，研究去中心化具身智能体在城市规模户外环境中通过自然语言协作寻找安全会合点的能力。作者提出CoSaR框架，结合基础模型的高层通信规划能力与经典空间导航算法的精确性，使智能体能够交换情境更新、推理空间约束并协作重规划轨迹。在14个城市级场景中评估，CoSaR实现了更快的聚集速度、更短路径和更高安全性。

Details

Motivation: 研究协作空间智能，即去中心化具身智能体在城市规模户外动态约束下有效协调的能力，解决多智能体在复杂环境中通过自然语言协商安全会合点并规避动态巡逻哨兵的挑战。

Result: 在14个城市级场景（3-5个智能体）的评估中，CoSaR框架在Sentinel Challenge基准上实现了更快的聚集时间、更短路径长度和更高的安全性，证明了动态通信与空间推理结合对多智能体协作的重要性。

Insight: 创新点在于提出结合基础模型（高层通信规划）与经典空间导航算法（精确控制）的混合框架CoSaR，以及建立城市规模多智能体协作基准Sentinel Challenge，为具身多智能体系统的协作空间智能研究奠定基础。

Abstract: In this work, we study Cooperative Spatial Intelligence, the ability of decentralized embodied agents to coordinate effectively under dynamic environmental constraints across city-scale outdoor domains. We introduce Sentinel Challenge, a benchmark where multiple decentralized embodied agents must communicate in natural language to agree on a mutually safe and convenient meeting point within large, city-scale outdoor environments. Each agent must then navigate safely while avoiding dynamic sentinels patrolling the area, using a tool that provides coarse spatial information. To address this, we propose CoSaR (Cooperative Spatial Reasoning and Planning), a framework that bridges the high-level communication and planning abilities of foundation models with the precision of classical spatial navigation algorithms. CoSaR enables agents to exchange situational updates, reason over evolving spatial constraints, and collaboratively replan trajectories. Evaluated across 14 city-level scenes with 3-5 agents, CoSaR consistently leads to faster gathering, shorter path lengths, and improved safety. Our results demonstrate that integrating dynamic communication with spatial reasoning is essential for robust multi-agent cooperation. By formalizing this new setting and providing a scalable benchmark, we aim to build a foundation for advancing cooperative spatial intelligence in embodied multi-agent systems. Code and challenge are available at https://github.com/UMass-Embodied-AGI/Sentinel.

[41] RoMo: A Large-Scale, Richly Organized Dataset and Semantic Taxonomy for Human Motion Generation cs.CVPDF

Jiahao Zhang, Joseph Liu, Young-Yoon Lee, Seonghyeon Moon, Victor Zordan

TL;DR: 本文介绍了RoMo数据集，这是一个大规模、高质量、经过精心筛选的野外人体运动数据集，旨在解决现有数据集在规模与质量之间的权衡问题。数据集包含详细的文本描述，并采用新颖的三级语义分类法进行组织，支持细粒度的评估。基于该数据集训练的模型在保真度、多样性和对复杂文本提示的理解方面达到了最先进的水平。

Details

Motivation: 现有3D人体运动生成研究受限于数据集选择：要么是小规模、高保真的动作捕捉数据，要么是大规模但包含大量静态或低质量序列的野外数据。本文旨在通过构建一个大规模、高质量、结构化的野外运动数据集来打破这一局限。

Result: 在RoMo数据集上训练的模型在保真度和多样性方面达到了最先进的水平（SOTA），并且对复杂、微妙的文本提示展现出更优越的理解能力。

Insight: 主要创新点在于提出了一个结合分类法感知的过滤流程来保证数据质量，并引入了一个三级语义分类法来组织数据，这支持了细粒度的、按类别的模型评估，能揭示被全局指标掩盖的模型优缺点。同时，发布的Motion Toolbox为运动生成研究提供了标准化工具。

Abstract: Success in generative modeling across language, image, and video demonstrates that large, well-curated datasets are the key driver for building capable models. 3D Human motion, however, has lagged behind, constrained by an unsatisfying choice between small, high-fidelity motion capture datasets and large-scale in-the-wild collections dominated by static or low-quality sequences. We introduce RoMo, a rich, large-scale, carefully curated dataset of in-the-wild human motions that resolves these tradeoffs. To ensure quality, we introduce a taxonomy-aware filtering pipeline that aggressively removes static and artifact-prone sequences. Every sequence is annotated with detailed captions and organized by a novel three-level semantic taxonomy. This hierarchical structure enables fine-grained, per-category evaluation, that reveals model strengths and weaknesses obscured by global metrics. We demonstrate that models trained on RoMo achieve state-of-the-art fidelity and diversity while gaining a superior understanding of complex, subtle text prompts. Finally, we release the Motion Toolbox to standardize metrics, data conversion, and visualization, establishing a foundation for reproducible and interpretable motion generation research.

[42] LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV cs.CV | cs.MM | cs.SDPDF

Tengfei Liu, Yang Shi, Xuanyu Zhu, Jiafu Tang, Liu Yang

TL;DR: 本文提出了LongAV-Compass，一个用于评估分钟级长音频-视频生成任务的系统性基准测试。该基准包含284个测试用例，覆盖文本到音视频（T2AV）、图像到音视频（I2AV）和视频到音视频（V2AV）三种条件生成模态，并集成了多模态大语言模型辅助评估与多种感知及多模态指标的统一评估框架。

Details

Motivation: 现有评估协议主要局限于短片段（5-10秒）的文本条件生成，缺乏对分钟级长内容生成、多种条件模态的统一评估，以及对身份一致性、叙事连贯性和音视频对齐在长时间跨度上如何退化的深入洞察。

Result: 通过在11个代表性模型上进行实验并结合人工对齐验证，LongAV-Compass为分析当前系统在维持跨模态、长时间、连贯且语义对齐的音视频生成方面的局限性提供了一个诊断性测试平台。

Insight: 创新点在于构建了一个覆盖多种条件模态和复杂度的分钟级音视频生成基准，并设计了一个结合分类学指导的基准构建与统一评估框架，该框架集成了MLLM辅助评估及DINO-v2、ArcFace等细粒度多模态指标，可评估超过20个维度的生成质量。

Abstract: Audio-visual generation is rapidly advancing from short clips to minute-long content, while existing evaluation protocols remain largely confined to short-form settings. Existing benchmarks primarily focus on 5–10 second text-conditioned generation and rarely support unified evaluation across text, image, and video conditioning modalities. Moreover, they provide limited insight into how identity consistency, narrative coherence, and audio-visual alignment degrade over extended temporal horizons. To bridge this gap, we introduce LongAV-Compass, a systematic benchmark for minute-long audio-visual generation. LongAV-Compass contains 284 curated test cases spanning text-to-audio-video (T2AV), image-to-audio-video (I2AV), and video-to-audio-video (V2AV), organized by application scenario and generation complexity. The benchmark combines taxonomy-guided benchmark construction with a unified evaluation framework that integrates MLLM-assisted assessment with complementary perceptual and multimodal metrics, including DINO-v2, ArcFace, CLIP, and ImageBind. The framework evaluates more than 20 fine-grained dimensions covering within-segment quality, cross-segment consistency, global narrative coherence, semantic alignment, and audio-visual synchronization. Through experiments on 11 representative models together with human-alignment validation, LongAV-Compass provides a diagnostic testbed for analyzing the limitations of current systems in sustaining coherent, semantically aligned, and temporally consistent minute-scale audio-visual generation across diverse input modalities.

[43] Frequency-Guided Fusion For RGB-Thermal Semantic Segmentation cs.CVPDF

İsmail Emre Canıtez, Özgür Erkent

TL;DR: 本文提出了一种用于RGB-热红外语义分割的频率引导融合方法，通过双ConvNeXt V2骨干网络，在不同特征抽象阶段采用模态自适应的融合策略。早期特征使用基于频率的融合模块分解热红外特征，并利用空间注意力与置信门控残差机制与RGB特征融合；晚期特征使用带有跨模态注意力和多尺度深度卷积的语义融合模块。该方法在MFNet和PST900数据集上以较少的参数量和计算成本实现了SOTA性能。

Details

Motivation: 解决在复杂环境（如城市驾驶场景）中，RGB图像在恶劣光照条件下信息不足的问题，通过RGB-热红外融合来提升场景理解，并应对在不同特征抽象层次上有效整合异构模态的挑战。

Result: 在MFNet和PST900数据集上，最轻量变体分别达到61.73%和86.24%的mIoU，仅使用35.43M参数，超越了近期方法，同时显著减少了参数量和计算成本。

Insight: 创新点包括：阶段式模态自适应融合策略，早期特征中基于频率分解（高斯滤波）和双分支空间注意力来选择性强调热模式和细粒度边界，以及晚期特征中结合跨模态注意力和多尺度深度卷积的语义融合模块。从客观角度看，该方法通过频率域引导和置信门控机制，实现了更精细的模态互补信息整合，具有较高的效率。

Abstract: Semantic segmentation in complex environments such as urban driving scenes remains challenging under adverse lighting conditions, where RGB images alone provide insufficient information. RGB-Thermal fusion leverages the complementary strengths of visible and infrared imagery to improve scene understanding; however, effectively integrating these heterogeneous modalities at varying levels of feature abstraction remains an open problem. In this paper, we propose a multi-modal fusion architecture built upon dual ConvNeXt V2 backbones that employs stage-wise, modality-adaptive fusion strategies. For early-stage features, we introduce a Frequency-Based Fusion Module that decomposes infrared features into low- and high-frequency components via Gaussian filtering, applies dual-branch spatial attention to selectively emphasize thermal patterns and fine-grained boundaries, and integrates them with RGB features through a confidence-gated residual mechanism. For late-stage features, we design a semantic fusion module with cross-modal attention and multi-scale depthwise convolutions to capture semantic correspondences across modalities. The fused features are decoded via a PANet-style bidirectional decoder with deep supervision. Experiments on MFNet and PST900 demonstrate that our lightest variant achieves 61.73% and 86.24% mIoU, respectively, with only 35.43M parameters, outperforming recent methods while using substantially fewer parameters and lower computational cost. Code is available at https://github.com/ismailemrecntz/VISIBLE-INFRARED-SENSOR-FUSION

[44] Benchmarking Convolutional, Transformer, Hybrid, and Vision Language Models for Multi Disease Retinal Screening cs.CV | cs.LGPDF

Durjoy Dey, Aymane Ajbar, Yuhong Yan

TL;DR: 该研究系统性地评估了四种视觉模型家族（卷积神经网络、视觉Transformer、混合CNN-Transformer骨干网络以及视觉语言模型）在视网膜多疾病筛查任务上的性能。使用标准化协议在RFMiD数据集上评估了12种架构的二元筛查和多标签分类任务，并在Messidor-2数据集上进行了外部验证。结果表明，基于注意力的模型（如SwinTiny）和混合模型（如CoAtNet0）表现最佳，而视觉语言模型与CNN基线相当但未超越最佳Transformer模型。

Details

Motivation: 现代深度学习为自动化视网膜筛查提供了强大工具，但尚不清楚不同视觉模型家族在真实多疾病场景和领域偏移下的表现如何。本研究旨在为多疾病视网膜筛查中的模型选择提供可复现的参考基准。

Result: 在RFMiD数据集上，所有架构在二元筛查任务上AUC均超过84%，其中SwinTiny、CoAtNet0和MaxViTTiny表现最强，并在多标签分类中提升了宏/微F1分数。在Messidor-2外部验证中，AUC范围为66.8%至84.7%，混合和Transformer模型再次显示出强劲性能。

Insight: 研究提供了首个针对多疾病视网膜筛查的跨模型家族系统性基准，表明基于注意力的架构（尤其是混合模型）在该领域具有优势。其标准化的训练、校准和评估协议为未来临床部署的自动化筛查工具提供了重要指导。

Abstract: Modern deep learning offers powerful tools for automated retinal screening, but it remains unclear how different visual model families compare in realistic multi-disease settings and under domain shift. In this work, we benchmark twelve architectures across four model families: convolutional neural networks, vision transformers, hybrid CNN-transformer backbones, and vision-language models, using the Retinal Fundus Multi-disease Image Dataset (RFMiD). We evaluate two tasks: binary screening for any retinal disease and multi-label classification across 28 disease classes. Using standardized training, calibration, and evaluation protocols, we report AUC, F1, precision, recall, and sensitivity at a clinically relevant operating point with specificity near 80%. On RFMiD, all architectures perform well on binary screening, with AUC above 84%, but attention-based models perform best. SwinTiny and the hybrid CoAtNet0 and MaxViTTiny models achieve the strongest binary screening results and improve macro and micro F1 in the multi-label setting. Vision-language models, including CLIP ViT-B/16 and SigLIP-Base384, are competitive with CNN baselines but do not surpass the best transformer and hybrid backbones. In external validation on Messidor-2 for referable diabetic retinopathy, AUC ranges from 66.8% to 84.7%, with hybrid and transformer models again showing strong performance. These results provide a reproducible reference for model selection in multi-disease retinal screening and guide future automated screening tools for clinical deployment.

[45] Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuning cs.CV | cs.CLPDF

Taha Koleilat, Hassan Rivaz, Yiming Xiao

TL;DR: 本文提出了Evi-Steer，一种用于生物医学视觉语言模型BiomedCLIP的证据感知、参数高效微调框架。该方法通过在视觉和文本编码器中执行轻量级的低维令牌更新，并同时估计认知不确定性，实现了仅更新总参数0.11%的高效适应。其核心创新包括基于不确定性更新门控残差，以及利用Dempster-Shafer理论进行跨模态置信度融合，从而在证据不足时保守适应并抑制冲突更新。

Details

Motivation: 现有参数高效的视觉语言基础模型适应方法通常是确定性的，在领域偏移或图像-文本对齐模糊的情况下表现不佳，这在数据稀缺、领域多变的临床环境中尤为关键。因此，需要一种能够感知不确定性、保持鲁棒性的高效微调方法。

Result: 在涵盖8个器官和8种成像模态的15个生物医学影像数据集上，在少样本学习和领域泛化设置下进行了全面评估。Evi-Steer在少样本学习和领域偏移设置下，始终优于最先进的方法，证明了其在真实临床环境中部署的实用性和鲁棒性。

Insight: 主要创新点在于将证据推理（认知不确定性估计）与参数高效微调相结合，通过不确定性动态调节模型更新（门控残差），并引入基于Dempster-Shafer理论的跨模态置信度融合机制，使视觉适应能够以文本置信度为条件，有效处理跨模态的不确定性和冲突，提升了模型在数据稀缺和领域变化下的可靠性与泛化能力。

Abstract: Parameter-efficient adaptation of vision-language foundation models is crucial for precise multimodal understanding of biomedical images, yet existing methods remain deterministic and often struggle under domain shift or ambiguous image-text alignment. This limitation is particularly critical in the clinic, where models should remain robust in low-data regimes and domain shifts. We present Evi-Steer, an evidential cross-modal low-dimensional steering framework for BiomedCLIP that enables uncertainty-aware parameter-efficient fine-tuning while updating only 0.11% of total model parameters. Our approach performs lightweight low-dimensional token updates in both vision and text encoders while simultaneously estimating epistemic uncertainty. These uncertainty estimates update gate residuals, allowing the model to adapt conservatively when evidence is weak. Furthermore, we introduce cross-modal confidence fusion based on Dempster-Shafer theory, enabling visual adaptation to be conditioned on textual confidence and suppressing conflicting or uncertain cross-modal updates. We conduct a comprehensive evaluation on 15 biomedical imaging datasets spanning 8 organs and 8 imaging modalities under few-shot learning and domain generalization settings. Evi-Steer consistently outperforms state-of-the-art methods under few-shot learning and domain shift settings, demonstrating a practical and robust pathway for deploying vision-language models in real-world clinical settings. Code is available at https://github.com/HealthX-Lab/Evi-Steer.

[46] CNNs, Transformers, Hybrid, and Vision Language Models for Skin Cancer Detection cs.CVPDF

Durjoy Dey, Yuhong Yan, Hassan Hajjdiab

TL;DR: 本文对12种深度学习模型在皮肤癌检测任务上进行了统一评估，涵盖了CNN、ViT、混合卷积Transformer骨干网络以及视觉语言模型（VLM）四类架构。实验在PAD-UFES-20数据集上进行，评估指标包括AUC、最大F1分数及其精确率与召回率，以及80%特异性下的灵敏度。结果表明，经过良好调优的CNN提供了强基线，但基于Transformer的模型持续提升了判别性能，其中混合模型（如MaxViT Tiny、CoAtNet0）和基于SigLIP的VLM在排序性能与临床相关操作点之间取得了最佳平衡，而基于CLIP的模型则提供了高精确率。

Details

Motivation: 皮肤癌是全球常见且快速增长的恶性肿瘤，早期检测对改善预后至关重要。尽管基于皮肤镜和临床图像的深度学习模型可支持自动化快速分诊，但现有研究通常仅评估有限架构，且实验设置各异，因此需要统一评估以提供实际部署指导。

Result: 在PAD-UFES-20数据集上的实验显示，混合模型（MaxViT Tiny、CoAtNet0）和基于SigLIP的VLM在整体性能上表现最佳，而基于CLIP的模型实现了高精确率，为皮肤癌筛查的模型选择提供了可复现的参考基准。

Insight: 创新点在于对四类主流模型家族（CNN、ViT、混合、VLM）进行了系统性比较，强调了混合架构和视觉语言模型在临床相关指标上的优势；客观来看，该研究为皮肤癌检测领域建立了统一的评估框架和开源代码库，有助于推动实际应用和未来研究。

Abstract: Skin cancer is a common and fast rising malignancy worldwide. Early detection is critical for improving outcomes. Deep learning models trained on dermoscopic and clinical images can support automated and fast triage. However, many studies evaluate only a limited set of architectures. Experimental setups also vary across studies. In this paper, we present a unified evaluation of twelve deep learning models for binary skin cancer detection on the PAD-UFES-20 dataset. The models span four families: convolutional neural networks (CNN), vision transformers (ViT), hybrid convolution transformer backbones, and vision language models (VLM). Performance is assessed using AUC, the maximum F1 score with its precision and recall, and sensitivity at 80% specificity, reflecting screening oriented requirements. Our results show that well tuned CNNs already provide strong baselines, but transformer based families consistently improve discrimination. Hybrid models (MaxViT Tiny, CoAtNet0) and a SigLIP based VLM achieve the best overall trade off between ranking performance and clinically relevant operating points, while CLIP based model offers high precision. The full codebase for all experiments is publicly released. Together, these findings offer practical guidance on which model families are most suitable for real world deployment in skin cancer screening and establish a reproducible reference point for future work on PAD-UFES-20.

[47] E$^3$C: Video Generation with 3D Environmental Memory and Ego-Exo Human Pose Control cs.CV | cs.AIPDF

Qiao Gu, Lingni Ma, Adam W Harley, Richard Newcombe, Florian Shkurti

TL;DR: 本文提出了E$^3$C，一个用于以自我为中心视角的可控视频生成扩散框架。它通过构建基于半稠密点云的3D环境记忆来解耦持久场景结构和人体驱动动态，并分别通过骨架渲染（第三人称控制）和3D关节与6DoF手腕运动（第一人称控制）来精确控制人物动作。

Details

Motivation: 解决以自我为中心视角视频生成的挑战，包括相机与演员强耦合导致的快速视角变化和自遮挡、动作细微且部分可见，以及需要确保人物和场景状态与指定控制信号一致演化的问题。

Result: 在Nymeria数据集上的实验表明，E$^3$C在视觉保真度、相机运动准确性、物体一致性和对第一人称/第三人称人物控制方面优于强基线模型，并能实现直观的场景编辑。

Insight: 创新点在于构建结构化的3D环境记忆来解耦场景与动态，并引入自我运动编码器以在穿戴者身体部位不可见时保持对第一人称动作的持续控制，这为具身智能体的物理世界推理提供了更可控的生成基础。

Abstract: Controllable and physically grounded egocentric video generation is essential for embodied agents to reason about how their own and others’ actions manifest and change the world. Compared to generic video synthesis, egocentric generation is especially challenging: the camera is tightly coupled to the actor, leading to rapid viewpoint changes and frequent self-occlusions; the underlying actions are subtle, articulated, and often only partially visible; and both the people and the scene state must evolve consistently with the specified controls. We present E$^3$C, a controllable video diffusion framework for egocentric generation that builds structured and compact conditions disentangling persistent scene structure from human-driven dynamics. From context frames, E$^3$C constructs a semi-dense point cloud-based 3D memory and augments each point with appearance descriptors from video-VAE features. Rendering this memory into target viewpoints produces conditioning aligned with the target frames. Human dynamics are modeled separately. The observed people in the scene are controlled by skeleton renderings (exo human control), while the camera wearer is specified by their 3D body joints and 6DoF wrist motion (ego human control). To preserve ego human control when the wearer’s body parts are invisible, we introduce an ego motion encoder that produces persistent cross-attention tokens. Experiments on Nymeria show that E$^3$C improves visual fidelity, camera-motion accuracy, object consistency, and ego & exo human control over strong baselines, while also enabling intuitive scene editing.

[48] RadarSim: Simulating Single-Chip Radar via Multimodal Neural Fields cs.CVPDF

Chuhan Chen, Tianshu Huang, Akarsh Prabhakara, Chaithanya Kumar Mummadi, Zhongxiao Cong

TL;DR: 本文提出了RadarSim，一个基于多模态神经场的可微分渲染器，用于模拟单芯片雷达数据。该方法利用RGB相机的高角度分辨率，从相机初始化的神经场生成多普勒雷达距离图像，以解决雷达数据空间分辨率低的问题。

Details

Motivation: 雷达是相机的理想补充，但雷达数据难以解释且传感器间差异大，需要依赖仿真进行传感器和处理流程的原型设计。现有方法将雷达重建视为新视角合成问题，但受限于雷达固有的低空间分辨率。

Result: 使用自定义手持设备采集的校准雷达-相机数据集进行实验，RadarSim相比仅使用雷达的重建方法，能生成更清晰的几何和多普勒距离帧。

Insight: 创新点在于通过相机初始化的神经场统一融合雷达和相机模态，利用相机的高角度分辨率提升雷达模拟的几何细节，实现了跨模态的差异化渲染以增强仿真质量。

Abstract: Radars are an ideal complement to cameras: both are inexpensive, solid-state sensors, with cameras offering fine angular resolution, while radars provide metric depth and robustness under adverse weather. However, radar data is more difficult to interpret than camera images and varies significantly between sensors, necessitating increased reliance on simulation for prototyping sensors and processing pipelines. Recent work treating radar reconstruction as a novel view synthesis problem has shown great promise in reconstructing radar-relevant geometry and simulating low-level radar data. However, such methods are constrained by the low spatial resolution of the underlying radar. To address this, we propose a unified differentiable renderer, RadarSim, which leverages the high angular resolution of RGB cameras to generate Doppler radar range images from a camera-initialized neural field. Using a novel data set of calibrated radar camera recordings from a custom hand-held rig, we demonstrate that RadarSim produces sharper geometry and Doppler range frames than radar-only reconstructions.

[49] Personalized Generative Models for Contextual Debiasing cs.CV | cs.AI | cs.LGPDF

Xinran Liang, Esin Tureci, Prachi Sinha, Ye Zhu, Vikram V. Ramaswamy

TL;DR: 本文提出DecoupleGen方法，通过个性化文本到图像扩散模型生成具有罕见上下文的图像，以缓解视觉模型对常见场景的过拟合问题，并作为训练数据增强提升模型在物体分类和识别任务上的性能。

Details

Motivation: 针对现实世界中不同视觉模式出现频率差异导致训练模型更易识别常见场景物体，而识别罕见上下文（如沙滩球在道路上）可能更为重要的问题，研究如何通过生成罕见上下文图像来弥补数据偏差。

Result: 在复杂场景数据集上的物体分类和识别任务评估中，该方法相比先前方法取得了持续改进，生成图像在语义上具有意义且视觉上与原始数据集对齐。

Insight: 创新点在于通过个性化扩散模型解耦上下文模式，在保持原始视觉细节的同时生成多样化的罕见上下文图像，并应用验证约束确保增强数据的相关性，为缓解数据集偏差提供了可扩展的数据增强方案。

Abstract: Different visual patterns appear with different frequencies in the world: e.g., beach balls appear on sand more often than they do on a road. These statistics are reflected in vision datasets, and as a result trained models more easily recognize objects in common scenarios. However, recognizing a beach ball on a road may arguably be even more important than recognizing it on sand. We study how to mitigate this discrepancy. Since collecting uncommon images in the real world may be difficult, we explore whether generating images with less frequent contexts can serve as effective training augmentation. A key challenge is guiding generations to remain close to the original dataset distribution while creating diverse images with uncommon contexts. We introduce Decoupling Contextual Patterns with Generations (DecoupleGen), a method that personalizes text-to-image diffusion models to facilitate coherent synthesis of images with rare contexts while preserving original visual details. The generated images contain semantically meaningful content and remain visually aligned with the original datasets. We further apply verification constraints to ensure relevance of the augmented data. We evaluate our approach on object classification and recognition tasks on complex scene datasets. Our experiments demonstrate consistent improvements over previous approaches, and our analyses identify factors underlying these improvements.

[50] Joint Instance Segmentation and Geometric Attribute Regression for Roof Structures in Aerial Imagery cs.CVPDF

Luuk Versteeg, Rob G. J. Wijnhoven, Martin R. Oswald

TL;DR: 本文提出了一种从单张航空正射影像中联合预测实例级屋顶分割掩码及三个连续几何属性（建筑高度、屋顶坡度、屋顶方位角）的方法。该方法扩展了Mask R-CNN，引入了条件方位角损失和对数归一化高度表示等创新，并在大规模荷兰航空影像数据集上进行了训练与评估。

Details

Motivation: 旨在从单张航拍图像中同时完成屋顶实例分割和几何属性回归，以支持从二维图像重建简化的三维建筑模型（LoD2），从而减少对昂贵三维参考数据的依赖。

Result: 在基于3DBAG LiDAR数据构建的大规模数据集上，使用DINOv3 ConvNeXt-Base骨干网络，方法在实例分割上AP$_{50}$达到0.566，几何属性回归的平均绝对误差分别为：屋顶坡度约4度、方位角约7度、建筑高度约1米。

Insight: 主要创新点包括：1）在Mask R-CNN框架中集成了专用的属性回归分支；2）设计了条件方位角损失，以抑制平坦屋顶（方位角标签噪声大）的监督信号；3）采用对数归一化高度表示来处理建筑高度分布严重偏斜的问题。这些设计提升了多任务学习的鲁棒性和精度。

Abstract: We present a method for jointly predicting instance-level roof segment masks together with three continuous geometric attributes – building height, roof slope, and roof azimuth – from a single aerial orthophoto. Our approach extends Mask R-CNN with a dedicated attribute regression branch and introduces two key innovations: a conditional azimuth loss that suppresses supervision for flat roof segments where azimuth labels are inherently noisy, and a log-normalized height representation that addresses the heavily skewed distribution of building heights. We train and evaluate on a large-scale dataset of Dutch aerial images paired with automatically derived ground truth from 3DBAG, a nationwide LiDAR-based 3D building dataset. Using a DINOv3 ConvNeXt-Base backbone, our method achieves a mean absolute error of approximately 4 degrees for roof slope, 7 degrees for azimuth, and 1 meter for building height, with an instance segmentation AP$_{50}$ of 0.566. The predicted per-segment masks and attributes are sufficient to reconstruct simplified 3D building models (LoD2) from a single overhead image, requiring expensive 3D reference data only for training.

[51] BioFact-MoE: Biologically Factorized Mixture of Experts for Vision-Language Prognostic Modeling in Hepatocellular Carcinoma cs.CV | cs.AI | cs.LGPDF

Junlin Yang, Tian Yu, Nicha C. Dvornek, Yuexi Du, Peiyu Duan

TL;DR: 本文提出BioFact-MoE，一种用于肝细胞癌（HCC）预后建模的生物因子化混合专家框架。该模型通过多参数MRI和放射学报告等多模态信息，在残差MoE生存架构中，利用生物监督的专家显式分解肝脏和肿瘤相关因子，以提升预测准确性和生物学可解释性。

Details

Motivation: HCC具有生物学异质性，现有视觉语言模型学习单一纠缠的潜在表示，混合了肝脏和肿瘤因子，限制了预测准确性和生物学可解释性。

Result: 在N=588患者的HCC队列上（预训练使用4,582个3D MRI图像-报告对），BioFact-MoE在多个时间点的生存预测上均优于所有基线，12、18、24个月的AUC分别达到75.33%、75.85%和73.96%。肝脏和肿瘤嵌入在无监督情况下分别与肝功能及肿瘤负荷标志物选择性相关（p<0.05）。

Insight: 创新点在于提出生物因子化的MoE框架，显式分解不同生物因子，结合门控专家权重实现表型感知的风险分层和通路信息门控，揭示了临床有意义的治疗相关生存异质性，增强了模型的可解释性。

Abstract: Hepatocellular carcinoma (HCC) is biologically heterogeneous, shaped by the interplay between hepatic functional reserve and tumor-related oncologic factors; thus, similar survival outcomes may reflect fundamentally different underlying biological processes. Prognostic modeling in HCC is informed by rich multimodal information from multiparametric MRI and radiology reports from routine clinical practice. Existing prognostic vision-language models (VLMs) learn a single entangled latent representation that blends hepatic and tumor-related factors, limiting both accuracy and biological interpretability. We present BioFact-MoE, a biologically factorized Mixture of Experts (MoE) framework that explicitly decomposes liver and tumor factors via biologically supervised experts within a residual MoE survival architecture. On a HCC cohort of N=588 patients (pretrained on 4,582 3D MRI image-report pairs), BioFact-MoE consistently improves survival prediction over all baselines across time horizons, achieving 12-, 18-, and 24-month AUCs of 75.33%, 75.85%, and 73.96%. Beyond scalar risk prediction, gated expert weights enable phenotype-aware risk stratification. Pathway-informed gating uncovers clinically meaningful treatment-associated survival heterogeneity. In held-out validation, hepatic and tumor embeddings show selective associations with liver function and tumor burden markers, respectively (p<0.05), without supervision. The code is available at https://github.com/jy-639/BioFact-MoE.

[52] VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenes cs.CV | cs.AIPDF

Jingru Chen, Yiming Liu, Mingtao Chen, Sijie Chen, Richeng Xuan

TL;DR: 本文提出了VisualNeedle基准，用于评估多模态大语言模型在信息密集场景中的主动视觉搜索能力。该基准通过设计关键证据仅存在于微小区域且无法一眼识别的挑战性场景，并引入反事实的crop-black设置来测试模型是否真正依赖中间视觉证据。实验评估了9个主流MLLM，发现无工具时准确率低于20%，最佳工具辅助模型仅达56.01%，仍低于人类多数投票准确率63.00%，揭示了模型在细粒度视觉搜索上的持续局限。

Details

Motivation: 现有MLLM在细粒度感知基准上报告的高准确率可能源于语言先验、全局语义捷径或中间图像损坏不影响最终答案等虚假提升，而非真正利用视觉证据。因此需要构建一个能激发真实主动视觉搜索的挑战性基准。

Result: 在VisualNeedle基准上，无工具设置下所有模型准确率低于20%；标准工具辅助下最佳模型（GPT-4o）达到56.01%，但仍显著低于人类多数投票准确率63.00%。通过crop-black消融实验证实，成功依赖于真实的中间视觉证据。

Insight: 创新点在于构建了信息密集且细粒度的VisualNeedle基准，其关键证据局限于微小区域；并设计了crop-black反事实设置来检验模型对工具返回视觉证据的真实依赖性，为评估MLLM的细粒度视觉推理提供了更可靠的测试框架。

Abstract: Frontier multimodal large language models (MLLMs) have been reported to achieve over 90% accuracy on fine-grained perception benchmarks. However, such scores do not necessarily imply faithful use of visual evidence. Prior studies have identified three shortcuts that inflate benchmark performance. First, linguistic priors and lexical cues in questions often enable models to infer plausible answers without seeing the image. Second, coarse global semantics from the visual encoder can bypass fine-grained local details. Third, in some ``think-with-images’’ benchmarks, corrupting the intermediate images returned by visual tools barely affects the final answer. These findings suggest that higher input resolution or larger question pools alone do not elicit genuine active visual search. To address this, we introduce VisualNeedle, a challenging, information-dense, and fine-grained benchmark for scenes where critical evidence is spatially constrained to minute regions and not discernible at a glance. We further propose a counterfactual crop-black setting, which replaces crops returned by tools with black images of the same size, to test whether tool-enabled performance truly relies on intermediate visual evidence. We evaluate 9 promninent MLLMs across three settings: no-tool, standard tool-enabled, and crop-black. No-tool accuracy stays below 20%, and the best tool-enabled model reaches only 56.01%, still trailing the 63.00% human majority-vote accuracy. These results reveal persistent limitations in fine-grained visual search, while the crop-black ablation confirms that success on VisualNeedle hinges on genuine intermediate visual evidence.

Niels Sombekke, Rob G. J. Wijnhoven, Martin R. Oswald

TL;DR: 本文提出了一种基于Perceiver IO架构的多模态建筑检查框架，通过共享的DINOv2骨干网络融合卫星图像和街景图像，以预测屋顶元素和材料类别。该方法无需填充或固定尺寸池化即可处理每个建筑的可变数量街景视图，并在一个包含32,135个建筑的大规模数据集上评估了四种掩码策略，其中提出的RGB-M掩码策略（将建筑足迹掩码作为第四输入通道）表现最佳。

Details

Motivation: 解决建筑检查中多模态图像（卫星和街景）融合的挑战，特别是处理每个建筑可变数量的街景视图，并联合预测多标签屋顶元素和材料类别。

Result: 在包含十个国家的大规模数据集上，Perceiver IO融合模型优于其他融合策略，对于街景可见的属性（如石板屋顶、天窗）取得了显著的每类AP提升（例如石板+11.3 AP，天窗+1.3 AP），但仅使用卫星图像的基线在宏观平均mAP上对主要从上可见的类别仍略有优势。

Insight: 创新点包括使用Perceiver IO架构灵活处理可变数量输入，以及提出RGB-M掩码策略（将建筑足迹掩码作为软空间先验加入输入通道），这为多模态建筑检查提供了一个可扩展且灵活的架构，可适应异构输入和多个输出任务。

Abstract: We present a multi-modal classification framework that fuses satellite and street-level imagery through a Perceiver IO architecture operating on spatial patch tokens from a shared DINOv2 backbone. The design naturally handles a variable number of street-level views per building without padding or fixed-size pooling, and jointly predicts multi-label roof element and roof material classes. We construct a large-scale dataset of 32,135 buildings (61,672 segments) spanning ten countries, pairing satellite images with up to eight street-level views per segment and evaluating four masking strategies for isolating the target building. We propose an RGB-M masking strategy that appends the building footprint mask as a fourth input channel, providing a soft spatial prior that outperforms hard cropping across both modalities. The Perceiver IO fusion model improves over all other fusion strategies and yields substantial per-class gains for attributes visible from street level (e.g., +11.3 AP for slate, +1.3 AP for dormers), though the satellite-only baseline retains a slight advantage in macro-averaged mAP for classes that are predominantly visible from above. These results establish a scalable, flexible architecture for multi-modal building inspection that can accommodate heterogeneous inputs and multiple output tasks.

[54] Detail Consistent Stage-Wise Distillation for Efficient 3D MRI Segmentation cs.CVPDF

Mengchen Fan, Baocheng Geng, Xi Xiao, Tianyang Wang, Siyuan Mei

TL;DR: 该论文提出了一种名为细节一致性蒸馏（DCD）的阶段式蒸馏框架，旨在压缩高性能3D医学图像分割模型（如nnU-Net），以解决其部署时面临的内存占用高和推理延迟大的问题。该方法通过在训练期间，在多个分辨率阶段对齐师生模型在小波分解表示中的特征，特别是蒸馏方向性细节成分，以保留跨尺度的精细结构信息（如小病灶和锐利边界），且不增加推理开销。

Details

Motivation: 部署高性能3D医学图像分割器（如nnU-Net）常受限于内存占用和推理延迟，因此需要模型压缩。但紧凑的3D编码器在跨多分辨率阶段重复下采样时，容易丢失精细的结构线索。

Result: 在BraTS 2024和ISLES 2022基准测试上的实验表明，该方法在使用3D多模态数据进行MRI分割时取得了优越的性能。

Insight: 创新点在于提出阶段式蒸馏，并在小波域中对齐特征以保留方向性细节，避免对全局语义的过度正则化。从客观角度看，将小波变换的多尺度分析特性与知识蒸馏结合，针对性地处理医学图像中的细节信息，是一种有借鉴价值的压缩策略。

Abstract: Deploying high-performing 3D medical image segmenters (e.g., nnU-Net) is often limited by memory footprint and inference latency. Compression is therefore necessary, but compact 3D encoders tend to lose fine structural cues (small lesions and sharp boundaries) as downsampling repeats across multi-resolution stages. We propose Detail Consistent Distillation (DCD), a stage-wise distillation framework that preserves structural detail across scales by aligning teacher-student features in a wavelet-decomposed representation. At each encoder stage, DCD distills directional detail components in the wavelet domain while leaving the coarse approximation comparatively unconstrained, avoiding over-regularization of global semantics. DCD is used only during training and introduces no inference-time overhead. Experiments on the BraTS 2024 and ISLES 2022 benchmarks demonstrate that our approach achieves superior performance in MRI segmentation using 3D multi-modal data. Code and implementation details for DCD are publicly available at https://github.com/ClinicaAlpha/DCD-3D-MedSeg.

[55] OmniGF: A Dual-Branch Vision-Language Framework for Unified Gaze Following cs.CVPDF

Qiaomu Miao, Haoyu Wu, Jingyi Xu, Minh Hoai, Dimitris Samaras

TL;DR: 本文提出OmniGF，一个双分支视觉语言框架，用于统一解决视线跟随任务。该框架通过结合结构化语言分支和连续空间分支，克服了传统模型仅能进行空间定位或仅依赖文本生成坐标的局限性，实现了对多人场景中精确空间目标估计、语义视线预测和复杂社交视线推理的统一处理。

Details

Motivation: 传统视线跟随模型局限于纯空间定位，缺乏对语义目标或复杂社交场景的推理能力，且多人推理时计算冗余；而现有视觉语言模型虽具备语义推理能力，但其依赖离散文本生成的方式限制了在连续空间任务（如视线定位）中的精度。

Result: 在多个标准基准测试上的广泛实验表明，该框架取得了新的最先进（SOTA）性能。

Insight: 创新点在于提出了一种双分支解码策略，将离散语义推理与连续空间表示提取相结合，并通过引入头部嵌入来显式地建模多人场景，从而统一了视线跟随的多个子任务，有效利用了基础视觉语言模型的强大语义能力。

Abstract: Understanding human gaze behavior is essential for complex scene comprehension and human-computer interaction. Traditional gaze following models are typically restricted to pure spatial localization, lacking the high-level capacity to reason about semantic targets or complex social contexts. Furthermore, these models often process individuals sequentially, requiring redundant computations over the same scene image for multi-person inference. While recent Vision-Language Models (VLMs) offer the exceptional semantic reasoning needed to address gaze-related semantic tasks, their reliance on discrete text generation inherently limits precision in continuous spatial tasks like gaze localization. To bridge this gap, we propose OmniGF, a unified vision-language framework that adapts foundational VLMs for highly scalable multi-person gaze reasoning. The model adopts a dual-branch decoding strategy: a structured language branch generates discrete reasoning states, while a continuous spatial branch directly taps into the VLM’s dense hidden states. Supervising these extracted representations with high-resolution gaze target heatmaps effectively overcomes the spatial bottleneck of text-only coordinate generation. Furthermore, to explicitly ground the model in multi-person scenes, we augment the input with head embeddings encoded from cropped head images, providing fine-grained appearance and orientation cues for all individuals simultaneously. By modeling all individuals and leveraging the strong semantic capability of VLMs, OmniGF seamlessly integrates precise spatial gaze target estimation, semantic gaze prediction, and complex social gaze reasoning. Extensive experiments demonstrate that our framework establishes new state-of-the-art performance across multiple standard benchmarks. Code is available at https://github.com/cvlab-stonybrook/omnigf.

[56] Zero-Shot Object Re-Identification in Egocentric Kitchen Videos via Multi-Stage SAM3 Feature Fusion cs.CVPDF

Dmytro Klepachevskyi, Alexander Wong, Sirisha Rambhatla, Yuhao Chen

TL;DR: 本文研究了在EPIC-Kitchens基准测试中的零样本物体重识别问题，旨在仅使用预训练视觉特征跨帧匹配食物和厨房工具实例。作者首先评估了五种SOTA特征提取器，发现零样本方法效果不佳，最佳基线仅达到45.3% mAP。为此，他们提出了一种增强的SAM3重识别流程，这是一个围绕SAM3分割构建的多阶段零样本方法，通过背景抑制、特征融合、几何一致性增强和重排序，将性能提升至52.8% mAP。

Details

Motivation: 解决在自我中心厨房视频中，由于视角快速变化、频繁遮挡、场景杂乱以及类内外观差异大，导致物体重识别极具挑战性的问题。同时，标注数据有限使得监督式方法难以扩展，因此需要零样本方法。

Result: 在EPIC-Kitchens基准上，提出的增强SAM3重识别流程将mAP从最佳基线（45.3%）提升了7.5%，达到52.8%。

Insight: 创新点在于以SAM3分割为核心，构建了一个多阶段零样本流程，融合了SAM3、DINOv2和CLIP的嵌入特征，并引入掩模形状IoU来增强几何一致性，最后应用k-互惠重排序。从客观角度看，这种模块化设计有效结合了分割、通用视觉和视觉语言模型的优势，无需微调即可提升性能。

Abstract: Object re-identification (ReID) in egocentric kitchen videos is challenging due to rapid viewpoint changes, frequent occlusions, cluttered scenes, and large intra-class appearance variations. Objects may leave and re-enter the field of view, and the large diversity of instances with limited annotations makes supervised ReID difficult to scale, motivating zero-shot approaches. We study zero-shot object ReID on the EPIC-Kitchens benchmark, where the goal is to match active food and kitchen-tool instances across frames using only pre-trained visual features. We first evaluate five state-of-the-art feature extractors, including Vision-Language Models (VLMs) - CLIP, DINOv2, DreamSim, I-JEPA, and SAM3 - and show that zero-shot methods fail, with the best baseline achieving only 45.3% mAP. We then propose an Enhanced SAM3 ReID Pipeline, a zero-shot multi-stage method built around SAM3 segmentation as the core component. Stage 1 uses SAM3 to suppress background clutter. Stage 2 fuses embeddings from SAM3, DINOv2, and CLIP into a single L2-normalized descriptor. Stage 3 augments cosine similarity with mask-shape IoU for geometric consistency, and Stage 4 applies k-reciprocal re-ranking. The full pipeline improves performance by 7.5% mAP to 52.8%.

[57] The Rescue Effect: Spatio-Semantic Early Exit Bypasses Quantization Collapse in CLIP cs.CV | cs.AIPDF

Kahyeon Nam, Hyesong Choi

TL;DR: 本文针对CLIP等联合嵌入模型在INT8量化部署时出现的‘量化诱导表示崩溃’问题，提出了一种名为LRA-EE的层间表示感知早期退出方法。该方法通过空间语义聚合、学习的多特征门控和层自适应置信度阈值，绕过噪声饱和的深层，在减少计算量的同时显著提升了零样本分类精度。

Details

Motivation: 解决在资源受限硬件上部署CLIP等视觉语言模型时，INT8量化导致的独特失效模式——激活噪声在Transformer块中累积，破坏多模态嵌入向量的方向，从而侵蚀零样本检索所依赖的余弦对齐。

Result: 在ImageNet-1K零样本分类任务上，LRA-EE方法相比INT8基线减少了13.4%的FLOPs，并将Top-1准确率提升了2.44个百分点（从58.72%提升至61.16%）。四象限分解验证了‘救援效应’：9.5%的样本在浅层退出时被正确分类，但在完整深度时因噪声而分类错误。

Insight: 创新点在于将早期退出机制与量化噪声问题结合，提出了针对层间信息噪声比的自适应阈值和空间语义聚合策略。客观来看，其核心洞察是识别并量化了量化噪声在深度网络中的累积效应，并设计了一种动态的、基于表示质量的路径选择机制来规避噪声饱和层，而非简单地压缩所有层。

Abstract: Deploying Vision-Language Models on resource-constrained hardware typically requires INT8 quantization, but in joint-embedding architectures such as CLIP this introduces a failure mode distinct from quantized CNN classifiers: activation noise accumulated across transformer blocks perturbs the direction of the multimodal embedding, eroding the cosine alignment on which zero-shot retrieval depends. We characterize this as Quantization-Induced Representation Collapse (QIRC) and quantify it on INT8 CLIP ViT-B/32, where the layer-wise noise-to-signal ratio grows from below 10% in shallow blocks to 52% at Layer 11. We propose LRA-EE (Layer-wise Representation-Aware Early Exit), which bypasses noise-saturated deep layers via Spatio-Semantic Aggregation (replacing the immature shallow [CLS] with a global patch-token average), a learned multi-feature gate (confidence, top-2 margin, spatial-activation variance), and Layer-adaptive Confidence Thresholding calibrated to each layer’s Information-to-Noise Ratio. On ImageNet-1K zero-shot classification, LRA-EE reduces FLOPs by 13.4% and improves Top-1 accuracy by +2.44%p (58.72% -> 61.16%) over the INT8 baseline. A four-quadrant decomposition isolates the Rescue Effect: 9.5% of samples are correctly classified at shallow exits but lost to noise at full depth, against only 7.1% suffering the inverse.

[58] HydraPrompt: An Adaptive and Asymmetric Framework of Vision-Language Models for Synthetic Image Detection cs.CVPDF

Senyuan Shi, Hao Tan, Zichang Tan, Shuhan Feng, Ajian Liu

TL;DR: 本文提出HydraPrompt，一种用于合成图像检测（SID）的自适应非对称提示框架。该框架利用视觉语言模型（如CLIP），通过非对称提示适配器（APA）动态调整类别中心：对真实图像使用统一的静态提示作为锚点，对伪造图像则构建样本自适应的提示以捕捉多样线索。此外，还引入了条件监督对比（CSC）目标来增强不同合成图像间的区分度。

Details

Motivation: 现有基于可学习文本提示的SID方法使用静态提示作为真假图像的固定边界，无法适应推理过程中出现的多种伪造类型，导致检测性能受限。

Result: 在流行的SID基准测试上进行了广泛实验，结果表明该框架达到了最先进的（SOTA）性能。

Insight: 创新点在于提出了非对称提示框架（APA），对真实和伪造类别采用不同的提示策略（统一锚点 vs. 样本自适应），并结合条件监督对比学习（CSC）来增强细粒度伪造线索的建模能力，从而更灵活地应对多样化的合成图像检测任务。

Abstract: The rapid evolution of generative models has precipitated a proliferation of fabricated content, posing significant challenges to existing Synthetic Image Detection (SID) methods. Capitalizing on advancements in vision-language models (e.g., CLIP), recent attempts have leveraged learnable textual prompts to identify synthetic images. However, they still leverage static prompt as a fixed boundary for real and fake images, failing to adapt to the varying types of forgery that emerge during inference. To overcome this issue, we propose HydraPrompt, an asymmetric prompting framework that dynamically adjusts the category centers by aligning with fine-grained image cues. Specifically, we propose an Asymmetric Prompt Adapter (APA): (1) for authentic category, we introduce a single set of prompts to capture the consistent representative patterns, which serves as a unified anchor for real content. While (2) for fake category, we construct sample-adaptive prompts that specialize in capturing diverse cues from different samples, enabling adaptive modeling of forgery image variations. To increase pronounced discriminability within different synthetic images, we further introduce a Conditional Supervised Contrastive (CSC) objective, which compacts the authentic representations while capturing fine-grained forgery clues. Extensive experiments on popular SID benchmarks demonstrate the state-of-the-art performance of our framework.

[59] Rethinking Weakly-supervised Video Temporal Grounding From a Game Perspective cs.CV | cs.AIPDF

Xiang Fang, Zeyu Xiong, Wanlong Fang, Xiaoye Qu, Chen Chen

TL;DR: 本文提出了一种从博弈论视角重新思考弱监督视频时序定位任务的新方法。该方法将视频帧和查询词建模为博弈参与者，利用多元合作博弈理论学习它们对跨模态相似度得分的贡献，从而捕捉细粒度的跨模态交互，并避免依赖复杂的候选片段提议。

Details

Motivation: 现有基于候选片段选择的弱监督视频时序定位方法存在两个问题：一是仅进行粗粒度的视频-查询全局对齐，未能建模帧与词之间的细粒度一致性以精确定位边界；二是其性能严重依赖耗时且复杂的候选片段提议质量。

Result: 实验表明，该方法在Charades-STA和ActivityNet Caption数据集上取得了优越的性能。

Insight: 创新点在于首次从博弈论视角建模该任务，将帧和词视为博弈参与者，通过博弈论交互量化联盟内合作趋势，从而评估所有不确定但可能的帧-词对应关系，并利用学习到的查询引导的逐帧得分进行定位，避免了传统候选片段框架的局限性。

Abstract: This paper addresses the challenging task of weakly-supervised video temporal grounding. Existing approaches are generally based on the moment proposal selection framework that utilizes contrastive learning and reconstruction paradigm for scoring the pre-defined moment proposals. Although they have achieved significant progress, we argue that their current frameworks have overlooked two indispensable issues: 1) Coarse-grained cross-modal learning: previous methods solely capture the global video-level alignment with the query, failing to model the detailed consistency between video frames and query words for accurately grounding the moment boundaries. 2) Complex moment proposals: their performance severely relies on the quality of proposals, which are also time-consuming and complicated for selection. To this end, in this paper, we make the first attempt to tackle this task from a novel game perspective, which effectively learns the uncertain relationship between each vision-language pair with diverse granularity and flexible combination for multi-level cross-modal interaction.Specifically, we creatively model each video frame and query word as game players with multivariate cooperative game theory to learn their contribution to the cross-modal similarity score. By quantifying the trend of frame-word cooperation within a coalition via the game-theoretic interaction, we are able to value all uncertain but possible correspondence between frames and words. Finally, instead of using moment proposals, we utilize the learned query-guided frame-wise scores for better moment localization.Experiments show that our method achieves superior performance on both Charades-STA and ActivityNet Caption datasets.

[60] Cross-scale Aligned Supervision for Training GANs cs.CV | cs.AIPDF

Sangeek Hyun, MinKyu Lee, Jae-Pil Heo

TL;DR: 本文挑战了传统多尺度生成对抗网络（GAN）中对抗监督的粗到细层次生成解释，指出其存在跨尺度轨迹未对齐问题。为此，作者提出了一种跨尺度对齐Transformer（CAT），通过在生成器侧添加一致性正则化来对齐中间输出与最终输出，从而解决该问题。

Details

Motivation: 论文动机在于指出标准多尺度对抗监督并未构建真正的粗到细层次生成，中间输出与最终输出可能代表不同的生成样本，导致跨尺度轨迹未对齐问题。

Result: 在类别条件ImageNet-256数据集上，CAT-H/2模型仅训练60个epoch后，在一步推理下达到FID-50K为1.56，优于强基线的一步GAN和扩散/流模型。

Insight: 创新点在于提出跨尺度轨迹未对齐问题，并通过生成器侧的一致性正则化实现中间输出与最终输出的对齐，从而提升多尺度生成的一致性和质量。

Abstract: Modern GANs often introduce adversarial supervision on intermediate generator outputs and interpret the resulting multi-stage synthesis as coarse-to-fine hierarchical generation. In this work, we challenge this interpretation. We argue that standard scale-wise adversarial supervision does not construct a proper coarse-to-fine hierarchy: each intermediate image is independently pushed toward the real distribution at its own resolution, but this scale-wise realism does not ensure that outputs across stages represent the identical generated sample. Moreover, the scale-specific image produced at each stage is not used as an explicit refinement target for the subsequent stage. Therefore, its adversarial loss can improve a scale-specific output without constraining later stages to preserve the same sample trajectory, allowing them to move toward a different sample rather than refine the previous output. We refer to this problem as a cross-scale trajectory misalignment problem. To resolve it, we propose CAT, a Cross-scale Aligned Transformer for multi-scale adversarial generation. CAT keeps the discriminator scale-wise, so each intermediate output is evaluated at its own resolution, while adding a simple generator-side consistency regularization that aligns intermediate outputs with the final output. On class-conditional ImageNet-256, CAT-H/2 achieves an FID-50K of 1.56 with one-step inference after only 60 training epochs, outperforming strong one-step GAN and diffusion/flow baselines.

[61] AnchorDiff: Training-Free Concept Grounding for MM-DiTs via Anchor-Based Graph Propagation cs.CV | cs.AIPDF

Jian Zhang, Zhijun Zhang

TL;DR: 本文提出AnchorDiff，一种无需训练的概念定位方法，用于解决多模态扩散变换器（MM-DiTs）中因注意力机制导致的‘概念泄漏’问题。该方法通过基于锚点的图传播，将语义定位与结构细化解耦，并在新构建的多概念混淆数据集上验证了其有效性。

Details

Motivation: 现有基于注意力的方法在处理视觉上易混淆的概念时，常产生重叠激活，导致目标响应溢出到非目标物体（即概念泄漏），因此需要一种更精确的训练免费概念定位方法。

Result: 在ImageNet-Segmentation和PascalVOC基准测试中，AnchorDiff表现出强大的定位性能，同时在多概念混淆数据集上显著减少了概念泄漏现象。

Insight: 创新点在于通过锚点选择和混合图传播（结合输出空间相似性和注意力门控）来解耦语义定位与结构细化，并引入了专门评估概念泄漏的多概念混淆数据集，为概念定位提供了更可靠的评估框架。

Abstract: Multi-Modal Diffusion Transformers (MM-DiTs) encode rich representations for training-free concept grounding, but existing attention-based methods often produce overlapping activations on visually confusable concepts, a failure mode we call concept leakage, where target responses spill over to non-target objects. To address this issue, we propose AnchorDiff, a training-free grounding method that decouples semantic localization from structural refinement. AnchorDiff selects a high-confidence anchor from concept-to-image attention map and propagates it as a one-hot seed over a hybrid graph derived from image-to-image self-attention. The graph uses output-space similarity for dense within-object propagation and a row-wise attention gate to suppress cross-object connections. Additionally, we introduce the Multi-Concept Confusion Dataset, which contains images with multiple visually similar concepts and separate masks, enabling explicit evaluation of concept leakage. Experiments show that AnchorDiff achieves strong grounding performance on ImageNet-Segmentation and PascalVOC, while substantially reducing concept leakage on our Multi-Concept Confusion Dataset.

[62] Triadic Dynamics Aware Diffusion Posterior Sampling for Inverse Problems: Optimizing Guidance and Stochasticity Schedules cs.CVPDF

Junseo Bang, Dong Ju Mun, Hoigi Seo, Seongmin Hong, Se Young Chun

TL;DR: 该论文提出了一种名为TriPS（Triadic Dynamics Aware Posterior Sampling）的新方法，用于优化扩散模型在解决成像逆问题中的后验采样过程。该方法将后验采样视为一个时变控制问题，通过分析数据一致性指导、无分类器指导和随机性三者之间的动态交互，优化了它们的调度策略。

Details

Motivation: 现有方法在利用扩散模型解决成像逆问题时，通常对数据一致性指导、无分类器指导和随机性这三个核心组件的调度采用启发式固定或部分调整的策略，导致性能次优。本文认为，这三者之间的调度交互对于显著提升性能至关重要。

Result: 实验表明，TriPS方法在数据保真度和感知真实性方面均优于最先进的基线方法。

Insight: 论文的核心创新在于揭示了三个组件在采样过程中的动态冲突与协同关系（例如早期激进的CFG会与DC指导冲突，而随机性可将轨迹拉回高概率区域），并据此提出了遵循“DC与随机性尺度递减、CFG尺度递增”三元趋势的优化调度框架。具体通过基于模板的函数先验搜索和基于GRPO的强化学习两种策略来实现，为逆问题求解中的扩散模型调度提供了系统化的优化思路。

Abstract: Generative posterior sampling using diffusion models has emerged as a dominant paradigm for solving inverse problems in imaging, which usually consists of three main components: data consistency (DC) guidance, classifier-free guidance (CFG) and stochasticity. While prior arts have focused on how to develop each or all components, less attention has given to how to schedule them, leading to heuristically fixed or partially adjusted suboptimal schedules. In this work, we argue that the interactions among all three components in terms of scheduling are crucial for significantly improved performance in solving inverse problems in imaging. Our analysis shows that aggressive CFG early in sampling conflict with DC guidance, while stochasticity brings the trajectory back to higher-probability regions. Based on these findings, we propose Triadic Dynamics Aware Posterior Sampling (TriPS), which reformulates posterior sampling as a time-varying control problem and optimizes schedules following a triadic trend of decreasing DC and stochasticity scales alongside increasing CFG scale. TriPS achieves this through two strategies: template-based search over functional priors for reliable baseline schedules, and Group Relative Policy Optimization (GRPO)-based reinforcement learning for more flexible temporal curves. Experiments demonstrate TriPS outperforms state-of-the-art baselines in data fidelity and perceptual realism.

[63] Comparative Study of Vision-Based Metric Measurement for Large-Scale Planar Scenes cs.CV | cs.AIPDF

ZhiXin Sun

TL;DR: 本文针对大规模户外平面场景中的视觉度量测量问题，在真实水库监测场景下使用PTZ相机，比较了三种代表性方法：基于几何的单目测距、图像拼接与鸟瞰变换、以及使用两个联合标定单目相机的立体测距。研究发现，单目测距在俯仰角足够大时可达米级精度，立体测距对俯仰角变化不敏感且可达分米级精度，而图像拼接在小场景有效但随场景扩大其稳定性和可扩展性下降。

Details

Motivation: 解决大规模户外环境中，由于远距离感知、相机变焦和成像条件不稳定导致的视觉度量距离和面积测量的挑战。

Result: 实验在真实水库监测场景中进行，结果表明单目测距在俯仰角足够大时精度为米级，立体测距精度为分米级且对俯仰角变化不敏感，图像拼接对小场景有效但随场景扩大性能下降。

Insight: 创新点在于对三种无需专用硬件（如立体相机）的视觉平面度量方法进行了系统的实证比较，并量化了它们在精度、对相机姿态的敏感性和可扩展性之间的权衡，为实际应用中的方法选择提供了清晰指导。

Abstract: Vision-based metric distance and area measurement remains challenging in large-scale outdoor environments due to long-range sensing, camera zoom, and unstable imaging conditions. This work studies planar metric measurement in a real-world reservoir monitoring scenario using PTZ cameras and compares three representative approaches: geometry-based monocular ranging, image stitching with birds-eye-view transformation, and stereo-based ranging using two jointly calibrated monocular cameras. For monocular ranging, planar localization models are derived from camera geometry and the effect of camera pitch angle is analyzed. Image stitching is investigated for large-area mapping, while a stereo-based scheme is developed for long-range measurement without dedicated stereo hardware. Experiments show clear trade-offs: monocular ranging achieves meter-level accuracy under sufficiently large pitch angles, stereo-based ranging achieves decimeter-level accuracy with reduced sensitivity to pitch variations, and image stitching is effective for small-scale scenes but degrades in stability and scalability as scene size increases.

[64] OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants cs.CV | cs.CLPDF

Xudong Lu, Xueying Li, Annan Wang, Yang Bo, Jinpeng Chen

TL;DR: 本文提出了OmniInteract，一个用于评估实时全模态大语言模型的流式交互基准测试。该基准通过处理原始音视频流进行在线推理，要求模型在无法预知未来内容的情况下，检测多模态触发信号、决定何时响应并在流展开时回答问题。它包含250个视频和1430个时间锚定的响应槽，涵盖实时、主动和嵌套场景以及连续任务监控。

Details

Motivation: 现有基准主要关注离线视频理解或基于文本提示的流式问答，缺乏对真实世界流式交互的评估，即模型需要在线处理嵌入用户查询和环境声音的原始音视频流，并做出实时决策和响应。

Result: 在OmniInteract基准上的实验表明，当前模型在流式交互方面表现较弱，最佳的整体交互感知质量-及时性F1分数（IA-QTF1）仅为0.368，而在连续问答任务上的最佳IA-QTF1仅为0.052。研究还发现，离线推理能力并不能直接迁移到在线交互场景中。

Insight: 创新点在于提出了一个专注于真实世界流式交互的评估框架，强调在线处理、多模态触发检测和实时决策。其设计的评估指标（如IA-QTF1、中断诊断套件和嵌套链完成分数）为衡量模型的交互质量和时机提供了系统方法，推动了面向实时全模态助手的研究。

Abstract: We introduce OmniInteract, a streaming benchmark for real-time omnimodal large language models evaluated through native online inference over audio-visual streams. Unlike offline video understanding or text-prompted streaming QA, OmniInteract preserves the original audio-visual stream and requires models to process it online, without access to future content. User queries and ambient sounds are embedded in the audio track, requiring models to detect multimodal triggers, decide when to respond, and answer while the stream unfolds. OmniInteract contains 250 videos with 1,430 temporally grounded response slots: 1,062 1Q1A slots across real-time, proactive, and nested scenarios, and 368 1QnA slots for continuous task monitoring and step guidance. Each slot includes a trigger, response window, and target answer. We evaluate response correctness, timing, invalid outputs, interruption handling, and context continuity using Interaction-Aware Quality-Timeliness F1, Interruption Diagnostic Suite, and Nested Chain Completion Score. Experiments show that current models remain weak in streaming interaction, with the best overall IA-QTF1 reaching only 0.368 and the best 1QnA IA-QTF1 only 0.052. Further study on mathematical reasoning in full-duplex settings shows that offline capability does not necessarily transfer to online interaction. Code and datasets will be made publicly accessible at https://github.com/Lucky-Lance/OmniInteract.

[65] Clinically-Grounded Counterfactual Reasoning for Medical Video Diagnosis cs.CVPDF

Jianzhe Gao, Churan Wang, Weiyi Zhang, Jianghua Li, Li-An Li

TL;DR: 本文提出了MedVCR，一个用于医学视频诊断的反事实推理框架，旨在模仿临床诊断思维。该框架包含三个核心组件：一个基于扩散模型的反事实生成器，用于合成特定病理状态下的组织演变；一个结合临床规则的反事实表示学习模块；以及一个融合视频级评估与帧级反事实分析的双重诊断预测策略。

Details

Motivation: 现有医学视频诊断方法存在三个主要问题：过度关注外观而非病理本身、缺乏临床先验知识、以及仅从观察结果进行推理而缺乏反事实比较。本研究旨在通过引入反事实推理来模拟临床医生的诊断思维，以解决这些问题。

Result: MedVCR在完全监督（如阴道镜检查）和弱监督（如结肠镜检查）的视频诊断设置下进行了评估，相比领先的基线方法，性能提升了2.6%到10.2%。全面的消融研究进一步验证了每个组件的有效性。

Insight: 论文的核心创新点在于将反事实推理系统性地引入医学视频诊断，通过合成反事实样本进行对比学习，并结合了临床规则（如时间一致性、病理可分离性和反事实对齐）来引导表示学习。从客观角度看，其将生成模型（扩散模型）与诊断任务结合，并设计双重预测策略，是提升模型可解释性和性能的有效途径。

Abstract: Medical video diagnosis involves inferring clinical decisions from dynamic tissue responses throughout examination processes. Existing methods rely on an end-to-end learning paradigm that i) focuses on appearance rather than pathology, ii) lacks clinical priors, and iii) reasons solely from observations without counterfactual comparison. This work introduces MedVCR, a counterfactual reasoning framework that mimics clinical diagnostic thinking. MedVCR comprises three components: a Counterfactual Generator that synthesizes tissue evolution under specified pathological states via a diffusion-based manner; a Counterfactual Representation Learning module that encodes diagnostic knowledge through clinical rules (i.e., temporal consistency, pathological separability, and counterfactual alignment); and a Dual Diagnostic Prediction strategy that integrates video-level assessment with frame-level counterfactual analysis. MedVCR is evaluated under both fully supervised (e.g., colposcopy) and weakly supervised (e.g., colonoscopy) video diagnosis settings, yielding 2.6%-10.2% performance gains compared with leading baselines. Comprehensive ablation studies further validate the effectiveness of each component. The code will be released.

[66] LongCat-Video-Avatar 1.5 Technical Report cs.CVPDF

Meituan LongCat Team, Xunliang Cai, Meng Cheng, Feng Gao, Zhe Kong

TL;DR: 本文介绍了LongCat-Video-Avatar 1.5，这是一个升级版的开源音频驱动视频生成框架，其核心目标是通过系统性的工程优化（而非架构创新）来提升生产就绪度。该版本通过升级音频编码器、扩展训练方案、严格的数据管理和RLHF训练，实现了精准的唇形同步、全身时序稳定性、严格的身份一致性，并能泛化至动漫和动物等风格化领域。此外，通过先进的步数蒸馏技术，将推理速度优化至8个NFE，在服务效率和视觉保真度之间取得了良好平衡。

Details

Motivation: 尽管音频驱动视频生成技术有所进展，但实现商业级的稳定性仍然具有挑战性。本研究旨在通过系统性的工程优化，而非追求架构新颖性，来构建一个生产就绪的开源框架，以缩小学术研究原型与商业级部署之间的差距。

Result: 在包含超过500个多样化测试案例的综合基准测试中，通过广泛的定量指标和严格的人工评估进行验证。结果显示，在人类相似度评分和专家级质量评估方面，v1.5与领先的闭源系统（如HeyGen、OmniHuman 1.5、Kling Avatar 2.0）相比，取得了相当或更优的性能。

Insight: 论文宣称的创新点在于其工程优先的路径，通过升级音频编码器（Whisper Large）、精心扩展训练方案、严格的数据管理和RLHF训练来系统性提升模型性能与鲁棒性。从客观角度看，其将先进的步数蒸馏技术应用于推理加速（优化至8 NFE），在效率与质量间取得平衡，以及模型对复杂现实条件（如多人交互、物体处理）和风格化领域的原生处理能力，是值得借鉴的工程实践创新。

Abstract: Despite advances in audio-driven video generation, achieving commercial-grade stability remains challenging. We present LongCat-Video-Avatar 1.5, an upgraded open-source framework prioritizing systematic engineering and production-readiness over architectural novelty. By upgrading the audio encoder to Whisper Large and meticulously scaling our training recipes, v1.5 achieves accurate lip-synchronization, full-body temporal stability, and robust long-video generation with strict identity consistency. Through rigorous data curation and RLHF Training, the model readily generalizes to stylized domains such as anime and animals, and natively handles complex real-world conditions, such as multi-person interactions and object handling. Furthermore, addressing the practical demands of industrial deployment, we employ advanced step distillation to accelerate inference to an optimal 8 NFE, achieving a favorable trade-off between serving efficiency and visual fidelity. The superiority of our approach is validated through extensive quantitative metrics and a rigorous human evaluation conducted on a comprehensive benchmark of over 500 diverse test cases. Results show that v1.5 achieves competitive or superior performance compared to leading closed-source systems (e.g., HeyGen, OmniHuman 1.5, Kling Avatar 2.0) across human-likeness ratings and expert-level quality assessments on our benchmark. With its open-source release, LongCat-Video-Avatar 1.5 narrows the gap between academic research prototypes and commercial-grade deployment.

Jianzhe Gao, Rui Liu, Yuxuan Xu, Tongtong Cao, Yingxue Zhang

TL;DR: 本文提出了一种用于视觉语言导航的不确定性感知高斯地图方法，通过显式建模几何、语义和外观三种感知不确定性，并将其整合到智能体的观测空间中，以支持更可靠的导航决策。

Details

Motivation: 现有视觉语言导航智能体在导航过程中常遇到感知不确定性（如证据不足或空间线索模糊），但通常忽略这些信息，本文旨在通过建模不确定性来提升决策的可靠性。

Result: 在多个VLN基准测试上的综合评估表明，该方法有效提升了导航性能。

Insight: 创新点在于将三种不确定性统一建模为3D价值地图，作为可供性和约束，为智能体提供更丰富的环境感知信息。

Abstract: Vision-Language Navigation (VLN) requires an agent to navigate 3D environments following natural language instructions. During navigation, existing agents commonly encounter perceptual uncertainty, such as insufficient evidence for reliable grounding or ambiguity in interpreting spatial cues, yet they typically ignore such information when predicting actions. In this work, we explicitly model three forms of perceptual uncertainty (i.e., geometric, semantic, and appearance uncertainty) and integrate them into the agent’s observation space to enable informed decision-making. Concretely, our agent first constructs a Semantic Gaussian Map (SGM), composed of differentiable 3D Gaussian primitives initialized from panoramic observations, that encodes both the geometric structure and semantic content of the environment. On top of SGM, geometric uncertainty is estimated through variational perturbations of Gaussian position and scale to assess structural reliability; semantic uncertainty is captured by perturbing Gaussian semantic attributes to reveal ambiguous interpretations; and appearance uncertainty is characterized by Fisher Information, which measures the sensitivity of rendered observations to Gaussian-level variations. These uncertainties are incorporated into SGM, extending it into a unified 3D Value Map, which grounds them as affordances and constraints that support reliable navigation. Comprehensive evaluations across multiple VLN benchmarks show the effectiveness of our agent.

Xiang Fang, Wanlong Fang, Changshuo Wang

TL;DR: 本文提出了一种名为多模态对抗协同（MMAS）的新型黑盒攻击框架，旨在揭示大型视觉语言模型（LVLMs）的脆弱性。该框架通过联合优化图像和文本扰动，利用基于小波的纹理约束和嵌入空间范数约束，生成难以察觉且语义连贯的通用对抗样本，仅需模型查询即可实现。

Details

Motivation: 大型视觉语言模型在多模态理解任务中表现出色，但其针对同时利用视觉和文本模态的对抗攻击的鲁棒性尚未得到充分探索，这在自动驾驶和内容审核等关键应用中构成风险。现有攻击方法多局限于单模态或需要不切实际的白盒访问权限，限制了其现实意义。

Result: 广泛的实验表明，该攻击方法在流行的LVLMs上展现出强大的通用对抗能力，成功实现了黑盒条件下的多模态攻击。

Insight: 主要创新点在于提出了一个联合优化图像（带纹理约束）和文本（带嵌入空间约束）扰动的统一框架，并引入了一种新颖的跨模态正则化项来对齐梯度方向，从而增强了扰动的协同效应和跨任务、跨模型的可迁移性。从客观角度看，其将通用对抗扰动与黑盒查询优化结合，并专门针对多模态交互进行设计，是提升攻击现实相关性的重要步骤。

Abstract: Large Vision-Language Models (LVLMs) have transformed multi-modal understanding, excelling in tasks like image captioning and visual question answering by integrating visual and textual inputs. However, their robustness against adversarial attacks, particularly those exploiting both modalities, remains underexplored, posing risks to critical applications like autonomous driving and content moderation. Existing attacks focus on single modalities or require impractical white-box access, limiting their real-world relevance. In this paper, we introduce Multi-Modal Adversarial Synergy, a groundbreaking framework that crafts universal, black-box multi-modal attacks against LVLMs. MMAS simultaneously generates a texture scale-constrained universal adversarial perturbation for images and a learnable prompt perturbation for text, optimized jointly using only model queries. The image perturbation leverages wavelet-based texture constraints to ensure imperceptibility and robustness across diverse visual inputs. The text perturbation, constrained by an L-norm in the embedding space, maintains semantic coherence while steering outputs toward a target. A novel cross-modal regularization term aligns the perturbations’ gradient directions, enhancing their synergistic impact and transferability across tasks and models. Extensive experiments show the strong universal adversarial capabilities of our proposed attack with prevalent LVLMs.

[69] Re-M3Dr: Rebalanced MultiModal Mean Deviation Regression cs.CVPDF

Haojie Yin, Chengcheng Feng, Tianyi Liu, Tianqi Zhang, Kaizhu Huang

TL;DR: 本文提出了一种名为Re-M3Dr的新型多模态回归框架，用于从光学相干断层扫描（OCT）和眼底摄影（FP）中预测平均偏差（MD），以评估视野缺损。研究发现，在该医学影像场景中，简单的多模态融合性能反而比单模态模型差，其根本原因是数据分布与模态学习冲突之间的耦合不平衡。Re-M3Dr通过自适应边界的监督对比学习增强单模态表示，并利用锐度感知梯度调制稳定联合优化，从而解决了这一问题。

Details

Motivation: 动机在于直觉上结合OCT和FP两种互补的眼科医学影像应能提升MD预测性能，但实际发现多模态融合效果反而更差，因此需要探究其根本原因并设计有效的解决方案。

Result: 在公开和私有临床数据集上的实验结果表明，与最先进的多模态学习方法相比，Re-M3Dr平均减少了29%的均方误差（MSE），证明了其优越性。

Insight: 创新点在于首次识别出医学影像多模态回归中数据分布与模态学习冲突的耦合不平衡问题，并提出了结合自适应边界监督对比学习和锐度感知梯度调制的框架来稳定优化过程，这为处理类似的不平衡多模态学习问题提供了新思路。

Abstract: Mean Deviation (MD) is a critical metric for assessing visual field loss in ophthalmology. While previous work has focused solely on predicting MD from Optical Coherence Tomography (OCT), it is intuitive to assume that combining OCT with another imaging of fundus photography (FP) could improve performance, as two ophthalmic medical imaging provide complementary information. This is particularly expected when sophisticated multi-objective optimization is applied, as documented in common multimodal classification. Surprisingly, our investigations reveal that multimodal fusion in this medical imaging scenario performs worse than unimodal model. Through detailed analysis, we identify the root cause as a coupled imbalance between data distribution and modality learning conflict. This imbalance distorts the optimization landscape, leading to unstable training. To address this challenge, we propose the method of Rebalanced MultiModal Mean Deviation Regression (Re-M3Dr), a novel multimodal regression framework. We enhance unimodal representation through adaptive margin based supervised contrastive learning. Then, our framework stabilizes the joint optimization with the sharpness-aware gradient modulation. Experimental results on both public and private clinical datasets show average 29% reduction in MSE compared to SOTA multimodal learning methods, demonstrating the superiority of Re-M3Dr. The code is available in the supplementary materials.

Jianzhe Gao, Rui Liu, Wenguan Wang

TL;DR: 本文提出了一种用于视觉语言导航（VLN）的3D高斯地图方法，通过将环境表示为可微分的3D高斯集合，并结合开放集语义分组操作来增强场景理解，进而设计多级动作预测策略来辅助智能体决策。

Details

Motivation: 现有VLN方法在增强智能体空间感知时，往往忽略了复杂3D几何和丰富语义信息，限制了其在多样化和未见环境中的泛化能力。

Result: 在R2R、R4R和REVERIE三个公开基准测试上的广泛实验验证了该方法的有效性。

Insight: 创新点在于构建了在线自中心场景地图，利用稀疏伪激光雷达点云初始化3D高斯以提供几何先验，并通过开放集语义分组将高斯基元与开放世界中的物体实例或场景类别关联，实现了几何与语义的统一表示。

Abstract: Vision-language navigation (VLN) requires an agent to traverse complex 3D environments based on natural language instructions, necessitating a thorough scene understanding. While existing works equip agents with various scene representations to enhance spatial awareness, they often neglect the complex 3D geometry and rich semantics in VLN scenarios, limiting the ability to generalize across diverse and unseen environments. To address these challenges, this work proposes a 3D Gaussian Map that represents the environment as a set of differentiable 3D Gaussians and accordingly develops a navigation strategy for VLN. Specifically, Egocentric Scene Map is constructed online by initializing 3D Gaussians from sparse pseudo-lidar point clouds, providing informative geometric priors for scene understanding. Each Gaussian primitive is further enriched through Open-Set Semantic Grouping operation, which groups 3D Gaussians based on their membership in object instances or stuff categories within the open world, resulting in a unified 3D Gaussian Map. Building on this map, Multi-Level Action Prediction strategy, which combines spatial-semantic cues at multiple granularities, is designed to assist agents in decision-making. Extensive experiments conducted on three public benchmarks (i.e., R2R, R4R, and REVERIE) validate the effectiveness of our method.

[71] CSV-ViT: A Vision Transformer with the Variable-sized Cortical Supervertices for Detection of Alzheimer’s Disease Pathologies cs.CV | cs.AI | cs.LGPDF

Geonwoo Baek, Ikbeom Jang

TL;DR: 本文提出了一种用于阿尔茨海默病（AD）病理检测的视觉Transformer模型CSV-ViT，该模型基于大脑皮层表面数据，通过一种保留感兴趣区域（ROI）的可变大小皮层超顶点（CSV）标记化方法，解决了现有表面模型在边界重复顶点和包含非皮层区域（如内侧壁）的问题。

Details

Motivation: 当前AD确认通常依赖昂贵且有创的PET扫描，因此需要基于结构MRI的预筛查方法；然而，在非欧几里得流形（如大脑皮层表面）上进行深度学习面临球形拓扑数据的挑战，且现有表面模型常因基于面的均匀分块导致边界顶点重复或包含非皮层区域，限制了其对ROI的感知。

Result: 在T1加权MRI数据上，CSV-ViT将AD相关状态分类为AD诊断、淀粉样蛋白阳性和tau阳性三类，实验表明其分类性能优于近期基于表面的模型，支持在PET或脑脊液确认前进行基于MRI的AD状态预测。

Insight: 创新点在于提出了一种ROI保留的、基于顶点的可变大小皮层表面分块方法（CSV），并设计了相应的CSV-ViT模型，该模型通过填充和掩码感知的补丁嵌入，能够容忍可变大小的补丁，从而更有效地处理皮层表面数据，提升AD病理检测的准确性。

Abstract: Confirming Alzheimer’s disease (AD) typically relies on positron emission tomography (PET), which remains costly and invasive, motivating the use of structural MRI-based prescreening. Deep learning on non-Euclidean manifolds, particularly brain cortical surfaces, faces significant challenges due to the data’s spherical topology. Recent surface models have enabled learning from cortical surface data; however, imposing face-based uniform patches often causes duplicate vertices at patch boundaries. In general, many surface-based models are limited in their awareness of the region of interest (ROI), which can result in non-cortical regions, such as the medial wall, being included. We propose a cortical surface tokenization that performs ROI-preserving, vertex-based, variable-sized patch partitioning. We refer to these cortical surface patches as cortical supervertices (CSVs). Building on this representation, we design the CSV Vision Transformer (CSV-ViT), a variable-size patch-tolerant Vision Transformer that uses padding and a mask-aware patch embedding. We used T1-weighted MRI and evaluated our framework by classifying AD-related status into three categories: AD diagnosis, amyloid positivity, and tau positivity. Across the experiments, CSV-ViT achieved higher classification performance than recent surface-based models. The results suggest that the proposed CSV-ViT may support MRI-based prediction of AD-related status prior to PET or CSF confirmation.

[72] InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward cs.CV | cs.AIPDF

Zhiwei Ning, Wenwen Tong, Xiangli Kong, Shengnan Ma, Ziyi Shang

TL;DR: 本文提出了InterSketch模型，一种通过自校正视觉草图和逐步奖励机制来增强视觉-文本交织思维链（VT-CoT）能力的交织推理模型。该模型利用外部工具动态生成中间视觉草图，并与文本推理交织，以解决长视野视觉理解任务。实验表明，其在视觉推理基准测试中表现优异，甚至超越了Gemini-3-Pro等专有模型。

Details

Motivation: 现有视觉语言模型（VLMs）的多轮推理轨迹较浅且以文本为中心，难以应对复杂的视觉挑战。而人类思维通常涉及长视野、视觉与文本交织的推理链，因此需要弥合这一差距。

Result: 在视觉推理基准测试上进行了广泛实验，结果表明InterSketch有效，性能甚至超过了Gemini-3-Pro等专有模型。

Insight: 创新点在于引入了自校正视觉草图和逐步奖励机制来构建和优化交织的VT-CoT。具体包括：在冷启动阶段使用合成的高质量交织VT-CoT数据集和反思机制；在强化学习阶段设计逐步奖励以缓解长视野推理中仅终端监督带来的奖励稀疏问题。

Abstract: While vision-language models (VLMs) have exhibited multi-turn visual reasoning capabilities, their reasoning trajectories remain relatively shallow and are dominated by a text-centric paradigm, limiting their applicability to complex visual challenges. In contrast, human-like thought typically involves long-horizon reasoning with an interleaved visual-textual chain-of-thought (VT-CoT). To bridge this gap, we introduce InterSketch, an interleaved reasoning model to enhance the VT-CoT capability via self-correcting and stepwise reward mechanisms. InterSketch dynamically generates intermediate visual sketches using external tools and interleaves them with textual reasoning, enabling effective perception and logical reasoning over long-horizon visual understanding tasks. Specifically, in the first cold-start stage, we propose a synthesized high-quality interleaved VT-CoT dataset and include a reflection mechanism to enable the model’s capability in multi-turn interleaved reasoning and self-correction. In the subsequent reinforcement learning (RL) stage, we design a stepwise reward mechanism to mitigate the sparsity of reward signals inherent in end-only supervision over long-horizon reasoning. Extensive experiments on visual reasoning benchmarks demonstrate the effectiveness of InterSketch, even outperforming proprietary models such as Gemini-3-Pro.

Yuxu Lu, Dong Yang, Xiaoyu Li, Mengwei Bao, Congcong Zhao

TL;DR: 本文提出了一种基于跨模态交互的船舶轨迹预测框架CmIVTP，通过融合AIS运动数据和CCTV环境数据来提升预测精度。该框架包含目标感知场景编码器、跨模态交互Transformer和船舶群轨迹库，并构建了大规模海事多模态数据集Maritime-MmD+。实验表明，CmIVTP在多模态船舶轨迹预测基准上取得了更好的性能。

Details

Motivation: 解决单一数据源（如AIS数据稀疏或CCTV数据不完整）在船舶轨迹预测中的局限性，通过融合多模态数据来建模船舶动态与环境约束的复杂交互。

Result: 在提出的多模态船舶轨迹预测基准上，CmIVTP实现了更好的性能，具体定量结果未在摘要中给出，但暗示达到了先进水平。

Insight: 创新点包括目标感知场景编码器、跨模态交互Transformer机制，以及通过聚类构建船舶群轨迹库的方法；客观来看，其多模态融合策略和同步AIS与CCTV的大规模数据集构建对海事智能交通研究具有借鉴意义。

Abstract: Maritime intelligent transportation systems (MITS) are essential for ensuring navigation safety and efficiency in busy waterways. However, accurate vessel trajectory prediction remains challenging due to the limitations of single-source data. Automatic identification system (AIS) data is often sparse or unavailable for small vessels, while closed-circuit television (CCTV) data alone cannot fully capture dynamic vessel behavior. To mitigate these challenges, we propose a cross-modal interaction-based vessel trajectory prediction (named CmIVTP) framework to model the intricate interactions between vessel dynamics and environmental constraints. Specifically, we introduce a target-aware scene encoder to extract scene semantic features, effectively capturing vessel-environment interactions and enhancing trajectory prediction accuracy. In addition, we propose a cross-modal interaction transformer, which integrates AIS-derived motion features, CCTV-based environmental features, and scene representations. It leverages cross-modal attention mechanisms to simultaneously capture intra-modal semantics and inter-modal interactions, ensuring dynamically consistent and environmentally feasible predictions. Furthermore, we construct a vessel group trajectory bank by clustering historical AIS trajectories into representative motion patterns, providing an efficient and scalable approach for candidate trajectory generation. Additionally, we introduce the maritime multimodal dataset plus (named Maritime-MmD$^+$), a large-scale dataset that synchronizes AIS data and CCTV video data, providing robust support for multimodal trajectory prediction research. Extensive experiments demonstrate that CmIVTP achieves better performance on multimodal-driven vessel trajectory prediction benchmarks. The code resources for this work can be available at https://github.com/LouisYxLu/CmIVTP.

[74] ReCA: Multi-Shot Long Video Extrapolation via Recursive Context Allocation cs.CV | cs.AIPDF

Akide Liu, Jinbo Xing, Chaojie Mao, Ye Li, Zeyu Zhang

TL;DR: 本文提出了多镜头视频外推（MSVE）任务，旨在将观察到的帧或片段扩展为具有电影结构的多镜头长视频，同时保持锚点状态并推进叙事意图。针对现有方法在长视频生成中面临的全局规划过细、镜头级提示信息稀释以及时间链导致状态衰减等瓶颈，作者提出了递归上下文分配（ReCA）推理框架，通过分层分配上下文来递归分解任务并调用冻结的生成器。

Details

Motivation: 现有视频生成模型在分钟级电影视频生成上存在局限：单镜头外推能保持锚点但缺乏电影结构，而多镜头叙事虽有结构却会自由发明视觉状态而非延续观察到的状态。因此，本文旨在解决如何从有限观察内容外推出结构化的长视频这一核心挑战。

Result: 在提出的MSVE-Bench和NB-Q基准（专为3至5分钟长视频生成设计）上评估，ReCA相比之前最强控制器将平均归一化分数提高了8%至16%，并将多镜头一致性指标提升了28%至43%。

Insight: 创新点在于揭示了长视频生成的失败不仅是上下文长度限制，更是上下文分配问题，并提出了递归上下文分配（ReCA）框架，通过分层递归分解任务和结构化状态传播来有效管理生成过程中的上下文，从而在有限预算下实现高质量的长视频外推。

Abstract: Minute-scale cinematic video generation is a central challenge for generative video models. Existing paradigms address only fragments of this challenge: single-shot extrapolation preserves an anchor but lacks cinematic structure, while multi-shot storytelling imposes structure yet remains free to invent its visual states rather than continue an observed one. We define Multi-Shot Video Extrapolation (MSVE), a task that extends an observed frame or clip into a sequence of cinematically structured shots while preserving anchor state and advancing narrative intent. This setting operates under the finite per-call generation budget of short-video models. We identify three coupled bottlenecks: (1) global planners over-specify unsupported details from full screenplays; (2) shot-level prompts dilute task-relevant state when carrying the complete story; and (3) temporal chaining turns generated frames into a lossy memory in which identity, scene, object, and action state decay. MSVE reveals that long-video failure is not merely a limitation of context length, but a failure of context allocation. We propose Recursive Context Allocation (ReCA), an inference-time framework that allocates context hierarchically across planning and generation. ReCA recursively decomposes MSVE into context-bounded subproblems, invokes frozen generators at leaf nodes, and propagates structured state updates across time. To evaluate this setting, we further propose MSVE-Bench and NB-Q, a source-grounded protocol with prompts purpose-built for 3 to 5 minute long-video generation, a regime not addressed by existing short-clip benchmarks. Compared to previous methods, ReCA improves average normalized score by 8 to 16 percent over the strongest competing controller and improves multi-shot consistency metrics by 28 to 43 percent. View the project page at https://reca.vmv.re.

[75] A Hybrid Vision-Language Architecture for Automated Defect Reasoning and Report Generation in Industrial Inspection cs.CV | cs.AI | cs.CL | cs.LGPDF

Malikussaid, Imad Gohar

TL;DR: 本文提出了一种用于风力涡轮机叶片检测的混合视觉-语言架构，该架构将缺陷定位、空间编码和报告生成三个子任务解耦。系统由基于YOLO的缺陷检测器、确定性的空间编码模块和经过QLoRA微调的小型语言模型组成，并利用检索增强微调（RAFT）来生成结构化的JSON维护报告。实验表明，这种专门构建的解耦架构在报告质量上显著优于通用的端到端视觉语言模型。

Details

Motivation: 当前工业检测中，缺陷定位和报告生成任务通常分离处理，且语言解释依赖人工专家。本文旨在构建一个自动化、可边缘部署的管道，将视觉检测与结构化报告生成统一起来，减少人工干预。

Result: 在BLEU-4、ROUGE-L、幻觉率（HR）和专家评分等指标上，完整系统（BLEU-4 0.41, HR=4%, 专家评分8.6/10）显著优于零样本通用VLM基线（0.07, 65%, 3.3/10）。经过QLoRA微调的1.5B模型在给定相同检测证据下，其报告质量甚至优于671B参数的通用API模型，并在单块T4 GPU上达到每秒47个token的生成速度。

Insight: 核心创新在于将复杂任务解耦为三个专门化组件（检测、编码、生成）的管道式架构，并结合了确定性空间编码、QLoRA高效微调和RAFT检索增强。这表明针对特定领域任务，使用小型专用模型和精心设计的解耦流程，可以超越大型通用端到端模型，在保证性能的同时实现高效部署。

Abstract: Automated industrial inspection requires both precise defect localization and structured maintenance report generation; in current practice these tasks are handled separately, with linguistic interpretation left to human experts. This paper describes a decoupled, edge-deployable pipeline for wind turbine blade inspection built from three components that each handle a distinct sub-task. The Eyes a YOLO26-x-obb oriented bounding-box detector localizes defects at dataset-native resolution. The Bridge a deterministic, parameter-free encoding module maps each detected bounding box to grid-referenced spatial tokens embedded in a structured prompt. The Brain a 4-bit quantized Qwen-2.5-1.5B model adapted with Quantized Low-Rank Adaptation (QLoRA) on 947 synthetically generated maintenance reports generates a structured JSON report from that prompt. Retrieval-Augmented Fine-Tuning (RAFT) further grounds each recommendation in indexed maintenance procedures. Five ablation experiments, scored by BLEU-4, ROUGE-L, Hallucination Rate (HR), and an LLM-as-a-Judge rubric, compare the pipeline against a monolithic vision-language model (VLM) baseline and against partial configurations in which one component is removed. The complete system achieves BLEU-4 0.41, HR=4%, and Expert Score = 8.6/10 compared with 0.07, 65%, and 3.3/10 for the zero-shot VLM baseline. The QLoRA-adapted 1.5B model generates higher-quality reports than a 671B-parameter generalist API model given identical detection evidence, at 47 tokens per second on a single T4-class GPU. The results show that purpose-built decoupled architecture with a small domain-specific training corpus outperforms a generalist end-to-end model on this structured generation task.

[76] O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding cs.CVPDF

Peiran Wu, Yunze Liu, Chi-Hao Wu, Chen Chen, Junxiao Shen

TL;DR: 本文提出了O-MARC框架，旨在解决全模态大语言模型在视频理解中因长序列导致推理成本高的问题。首先，作者构建了一个新的用户生成内容基准UGC-AVQA，用于评估模型对音频和视觉关联的理解。其次，提出了无需训练的压缩方法OMAC来降低推理开销。最后，通过压缩蒸馏框架O-MARC，使紧凑模型能有效处理压缩后的多模态输入，在保持性能的同时显著提升效率。

Details

Motivation: 全模态大语言模型支持统一的音视频理解，但长联合token序列导致推理成本高昂，且现有基准测试无法在嘈杂的用户生成视频中充分隔离音视频关联。

Result: 在Qwen2.5-Omni-3B模型上，O-MARC将四个基准测试的平均得分提升至45.8，优于完整token推理的44.1和OmniZip的41.0。OMAC方法将推理延迟降低了34.6%（1.53倍加速），内存使用减少了34.7%。

Insight: 创新点包括：1）构建了UGC-AVQA基准，通过音频移除测试确保问题需要音视频证据；2）提出了无需训练的压缩方法OMAC，保留关键的视觉记忆和时序锚定的音频信息；3）设计了压缩蒸馏框架O-MARC，使模型能鲁棒地处理压缩后的多模态上下文，实现性能与效率的平衡。

Abstract: Omnimodal large language models enable unified audio video understanding, but long joint token sequences make inference costly, and existing benchmarks do not fully isolate audio visual association in noisy user generated videos. We introduce UGC-AVQA, a public UGC benchmark with 1,000 videos and 4,816 QA pairs, where an audio removal test ensures that benchmark questions require both acoustic and visual evidence. To reduce inference cost, we propose OMAC, a training free plug in compression method that preserves salient visual memory and temporally grounded audio anchors. To further make compact models robust to compressed inputs, we introduce O-MARC, a compression distillation framework for learning with memory compressed multimodal contexts. On Qwen2.5-Omni-3B, O-MARC improves the average score across four benchmarks to 45.8, outperforming full token inference at 44.1 and OmniZip at 41.0. OMAC also keeps inference efficient, reducing latency by 34.6% (1.53$\times$ speedup) and memory by 34.7% compared with full token inference.

[77] FTibSuite: A Comprehensive Resource Suite for Tibetan Vision-Language Modeling cs.CV | cs.DBPDF

Guixian Xu, Yide Liang, Zeli Su, Xuexian Song, Ziyin Zhang

TL;DR: 本文介绍了FTibSuite，一个为藏语视觉语言建模设计的综合资源套件，包含FTibData（涵盖持续预训练、图文对齐和指令微调的人工验证多模态训练语料）、FTibBench（五个主流多模态基准的藏语适配版本，采用分层质量控制流程以减少翻译噪声）以及FTibVLM（基于Qwen3-VL-8B-Instruct通过三阶段适应流程构建的可复现基线模型）。实验表明，FTibVLM在FTibBench的所有任务上均取得一致的性能提升，同时保持了骨干模型原有的中文能力，为藏语多模态研究提供了首个标准化基础。

Details

Motivation: 解决藏语作为低资源语言在视觉语言建模领域缺乏可复现训练和评估基础设施的问题，填补该语言的研究空白。

Result: 在FTibBench上，FTibVLM将MMBench准确率从42.97提升至67.78，POPE-random准确率从47.53提升至80.56，在所有任务上均实现性能增益，同时骨干模型的中文能力仅有最小程度的退化。

Insight: 创新点包括为低资源语言构建涵盖数据、基准和模型的全套可复现资源套件，以及采用分层质量控制流程来减少基准翻译中的噪声，为其他低资源语言的多模态研究提供了可借鉴的标准化框架和适应方法。

Abstract: Vision-language models have progressed rapidly, but Tibetan remains a severely underserved low-resource language due to the lack of reproducible training and evaluation infrastructure. To fill this gap, we introduce FTibSuite, a comprehensive resource suite for Tibetan vision-language research, consisting of FTibData (human-verified multimodal training corpora spanning continual pretraining, image-text alignment, and instruction tuning data), FTibBench (Tibetan adaptations of five mainstream multimodal benchmarks with a hierarchical quality-control workflow to reduce translation noise), and FTibVLM, a reproducible baseline built on Qwen3-VL-8B-Instruct via a three-stage adaptation pipeline. Experiments on FTibBench show FTibVLM delivers consistent performance gains across all tasks, such as improving MMBench accuracy from 42.97 to 67.78 and POPE-random accuracy from 47.53 to 80.56, while retaining the backbone’s original Chinese capabilities with minimal degradation, providing the first standardized foundation for Tibetan multimodal research.

[78] MedVol-R1: Reward-Driven Evidence Grounding for Volumetric Reasoning Segmentation cs.CV | cs.AIPDF

Zichun Wang, Hairong Shi, Bingzheng Wei, Yan Xu, Zihua Wang

TL;DR: MedVol-R1是一个基于强化学习的3D医学图像分割框架，用于解决自由形式临床查询下的体积推理分割任务。该方法将证据定位与体积分割解耦，首先通过大型视觉语言模型将临床推理定位到可验证的2D证据锚点，然后利用冻结的MedSAM2模块将其传播为连贯的3D掩码。

Details

Motivation: 现有方法通常依赖专用分割令牌连接语言与掩码解码，但这种耦合将决策过程压缩为不透明的潜在表示，限制了可解释性以及对多样化叙述表达的泛化能力。

Result: 在M3D-Seg基准测试的CT-ORG、AbdomenCT-1K和KiTS23数据集上，MedVol-R1持续优于强基线模型，并实现了最先进的性能，且强化学习相比纯监督微调带来了明显提升。

Insight: 创新点在于通过强化学习框架显式解耦证据定位与体积分割，利用多组件奖励机制鼓励信息丰富的证据选择、准确的2D空间定位和跨切片体积一致性，且无需昂贵的思维链标注。

Abstract: Volumetric Reasoning Segmentation (VRS) aims to segment a target region in a 3D medical scan from a free-form clinical query, where the referent is often implicit and requires both medical knowledge and volume-grounded reasoning. Existing methods typically rely on specialized segmentation tokens to connect language with mask decoding, but this coupling collapses the decision process into opaque latent representations, limiting interpretability and generalization to diverse narrative expressions. In this paper, we present MedVol-R1, a reinforcement learning-based framework for VRS that explicitly decouples evidence grounding from volumetric delineation: the LVLM grounds clinical reasoning to a verifiable 2D evidence anchor (key axial slice and 2D bounding boxes), which is then propagated into a coherent 3D mask by a frozen MedSAM2 module. We train MedVol-R1 with cold-start supervised fine-tuning followed by GRPO, guided by a multi-component reward that encourages informative evidence selection, accurate 2D spatial grounding, and cross-slice volumetric coherence, without requiring costly chain-of-thought annotations. Experiments on CT-ORG, AbdomenCT-1K, and KiTS23 from the M3D-Seg benchmark demonstrate that MedVol-R1 consistently outperforms strong baselines and achieves state-of-the-art performance, with reinforcement learning providing clear gains over pure supervised fine-tuning.

[79] JetViT: Efficient High-Resolution Vision Transformer with Post-Training Attention Search cs.CV | cs.AIPDF

Dongyun Zou, Zhuoyang Zhang, Junyu Chen, Wenkun He, Qinhe Peng

TL;DR: 本文提出了JetViT，一种新颖的混合架构视觉Transformer模型家族，以及一种名为Post-Training Attention Search的后训练加速框架。该框架通过将预训练的全注意力ViT模型转换为高效的混合注意力变体，在保持与最先进视觉基础模型同等精度的同时，显著提高了高分辨率图像上的推理效率。

Details

Motivation: 动机在于解决高分辨率图像上，现有全注意力视觉Transformer模型计算开销大、推理效率低的问题，旨在不牺牲准确性的前提下提升模型速度。

Result: 在DINOv3和DepthAnythingV2这两个代表性的高分辨率视觉基础模型上进行了评估。在NVIDIA H100 GPU上，JetViT实现了高达1.79倍的吞吐量提升和高达44.81%的延迟降低，且未牺牲准确性。

Insight: 核心创新点是Post-Training Attention Search后训练加速框架，它通过继承基础模型的权重，高效地探索架构设计空间，具体包括优化线性注意力块设计、寻找线性与窗口注意力块的最佳组合，以及识别并保留关键的全注意力块，从而实现了对预训练模型的高效加速。

Abstract: We introduce JetViT, a novel family of hybrid-architecture Vision Transformer (ViT) models that match the accuracy of state-of-the-art full-attention vision foundation models while achieving substantially higher inference efficiency on high-resolution images. At the core of our approach is Post-Training Attention Search, a post-training acceleration framework that converts pre-trained full-attention ViTs into efficient hybrid-attention variants by identifying and replacing redundant full-attention blocks with linear or window-attention blocks. By inheriting the MLP and attention weights from the base model, Post-Training Attention Search efficiently explores the architectural design space through three key steps: (1) optimizing the linear-attention block design; (2) finding the best combination of linear-attention and window-attention blocks; and (3) identifying and preserving critical full-attention blocks. We evaluate JetViT on two representative high-resolution vision foundation models, DINOv3 and DepthAnythingV2. On the NVIDIA H100 GPU, JetViT achieves up to 1.79x higher throughput and up to 44.81% lower latency without sacrificing accuracy. We will release our code and accelerated ViT models soon.

[80] OmniRetriever: Any-to-Any Audio-Video-Text Retrieval via Fusion-as-Teacher Distillation cs.CVPDF

Yunze Liu, Chi-Hao Wu, Enmin Zhou, Junxiao Shen

TL;DR: 本文提出了OmniRetriever-7B模型，通过一种名为‘融合即教师蒸馏’的新训练目标，来增强音频-视频-文本（AVT）多模态联合嵌入空间。该方法利用融合嵌入作为教师信号来指导单模态嵌入的学习，并直接监督融合嵌入。模型在多个零样本检索基准测试中超越了闭源的Gemini Embedding 2模型，并在新发布的OmniRetriever-Bench基准上取得了最佳性能。

Details

Motivation: 现有的音频-视频-文本编码器在训练时，标准的成对InfoNCE损失函数未能充分利用所有三个模态同时可用时产生的联合嵌入信号，导致多模态融合表示潜力未被充分挖掘。本文旨在填补这一空白。

Result: 在六个零样本检索基准测试中，OmniRetriever-7B在Clotho和SoundDescs上比Gemini Embedding 2的R@1高出13.3-18.0个点，在MSR-VTT和MSVD上达到了与当前顶尖的零样本视频-文本专用编码器相当的水平。在新发布的12方向AVT检索基准OmniRetriever-Bench上，其AVG-all得分为34.84，比Gemini Embedding 2高出1.72，比之前最好的开源AVT方法高出8.03。

Insight: 核心创新点是‘融合即教师蒸馏’训练框架，它将融合嵌入（梯度停止）作为教师信号来蒸馏单模态嵌入，并辅以直接监督融合嵌入的Tuple-InfoNCE损失项。这有效地利用了多模态联合表示来增强单模态和融合表示的学习，为构建更强大的统一多模态嵌入空间提供了一种新思路。

Abstract: Unified multimodal embedding spaces have become the standard interface for cross-modal retrieval and multimodal RAG, and recent audio-video-text (AVT) encoders extend this setting to three modalities. Such encoders can produce a joint (T,V,A) embedding whenever all three modalities are available, but standard pairwise InfoNCE objectives leave this signal unused during training. We close this gap with fusion-as-teacher distillation, which treats a stop-gradient copy of the fused embedding as a teacher signal for the single-modal embeddings, paired with a Tuple-InfoNCE term that supervises the fused embedding directly. We instantiate this objective as OmniRetriever-7B. Across six zero-shot retrieval benchmarks, OmniRetriever-7B surpasses the closed-source Gemini Embedding 2 by 13.3-18.0 R@1 on Clotho and SoundDescs, and reaches the contemporary zero-shot specialist band of open video-text encoders on MSR-VTT and MSVD. To stress-test joint representations, we further release OmniRetriever-Bench, a 12-direction AVT retrieval benchmark totaling 3782 triples; on it OmniRetriever-7B attains AVG-all 34.84, improving over Gemini Embedding 2 by 1.72 and over the best prior open-source AVT method by 8.03.

[81] DV-SFT: Direct Vision Supervision for Fine-Grained Visual Understanding cs.CVPDF

Jianfei Zhao, Feng Zhang, Xin Sun, Chong Feng, Bing Wang

TL;DR: 本文提出了DV-SFT方法，旨在解决多模态大语言模型中视觉令牌因缺乏直接监督而导致的细粒度视觉理解不足问题。该方法利用OCR相关场景中视觉与文本的直接对应关系，为视觉令牌自动生成词级标签，并通过与文本相同的下一个令牌预测目标进行训练，无需修改模型架构或增加额外计算。

Details

Motivation: 当前多模态大语言模型的监督信号仅应用于文本令牌，视觉令牌仅作为上下文的一部分被隐式优化，导致模型对视觉信息的理解较为粗糙。现有方法试图监督视觉输入，但依赖于额外的解码器或前向传播，限制了其实用性。

Result: 在三个领域内和四个领域外基准测试中，DV-SFT均一致优于标准监督微调方法，展现了直接视觉监督的优越性。分析表明，该方法有效提升了细粒度视觉理解能力，并实现了更高的多模态对齐效率。

Insight: 创新点在于首次为视觉令牌构建了显式的、令牌级别的监督，并利用OCR场景的视觉-文本对应关系自动生成标签，将MLLM视为黑盒，无需架构修改。这为提升多模态模型的细粒度感知提供了一种简单高效的训练策略。

Abstract: Multimodal large language models are typically trained end-to-end to predict ground-truth answers, yet supervision signals are applied exclusively to text tokens. Visual tokens, the core carriers of visual information, are optimized only implicitly as part of the context, leading to coarse-grained visual understanding. Prior works attempt to supervise visual inputs but inevitably rely on auxiliary components such as additional decoders or forward passes, because visual tokens lack readily interpretable labels. This limits their practical applicability. In this work, we propose \textbf{D}irect \textbf{V}ision \textbf{S}upervised \textbf{F}ine-\textbf{T}uning (DV-SFT), which constructs explicit, token-level supervision for visual tokens and trains them through the same next-token prediction objective used for text. Specifically, we exploit the direct vision–text correspondence in OCR-related scenarios and automatically label each visual token with the word in its corresponding image patch. DV-SFT treats the MLLM as a black box, requiring no architectural modifications or additional forward passes. Extensive experiments demonstrate the superiority of direct vision supervision. DV-SFT consistently outperforms standard SFT across three in-domain and four out-of-domain benchmarks. Further analyses show that vision supervision effectively enhances fine-grained visual understanding and achieves higher multimodal alignment efficiency.

[82] Respecting Modality Gap in Post-hoc Out-of-distribution Detection with Pre-trained Vision-Language Models cs.CV | cs.AIPDF

Yuanwei Hu, Bo Peng, Yadan Luo, Zhen Fang, Ling Chen

TL;DR: 本文挑战了基于预训练视觉语言模型（VLM）的零样本分布外（OOD）检测中广泛使用的‘文本即原型’范式。作者从理论上证明，现成的文本原型与最优视觉原型存在固有模态鸿沟，仅靠提示工程无法消除。为此，论文提出了一种在线伪监督框架，利用未标记的测试时数据流和预训练VLM的软预测，直接在视觉特征空间中学习类别原型，从而缓解这一鸿沟。

Details

Motivation: 现有零样本OOD检测方法通常将类别名称的文本嵌入作为类别原型，但作者认为这存在固有的模态鸿沟，导致次优性能。

Result: 广泛的实验表明，该方法在多种OOD检测设置下达到了新的最先进水平（SOTA）。

Insight: 核心创新在于挑战了‘文本即原型’的范式，并提出了一个在线优化框架，直接在视觉空间中学习原型以弥合模态鸿沟，这为利用VLM进行OOD检测提供了新思路。

Abstract: Out-of-distribution (OOD) detection has emerged as a popular technique to enhance the reliability of machine learning models by identifying unexpected inputs from unknown classes. Recent progress in pre-trained vision-language models (VLMs) has enabled zero-shot OOD detection without access to in-distribution (ID) training data; in this setting, existing methods commonly treat text embeddings of class names as class prototypes. In this paper, we challenge the widely adopted text-as-prototype paradigm by theoretically showing that off-the-shelf textual prototypes are generally misaligned with the optimal visual prototypes, yielding an intrinsic modality gap that cannot be eliminated by prompt engineering alone. To mitigate this gap under the post-hoc constraint, this paper presents an online pseudo-supervised framework that directly learns class prototypes in the visual feature space using unlabeled test-time data streams and soft predictions from the pre-trained VLMs. We provide theoretical guarantees for the convergence of the online optimization procedure. Extensive experiments empirically demonstrate that our method achieves a new state of the art across a variety of OOD detection setups.

[83] PinPoint: Prompting with Informative Interior Points cs.CV | cs.CLPDF

Pouya Sadeghi, Shawn He, Pedro Pablo Guerrero Vela, C. Thomas, Alex Wong

TL;DR: 本文提出PinPoint方法，通过选择信息丰富的内部点来消除提示歧义，从而提升基于视觉语言模型（VLM）和Segment Anything Model（SAM）的参考图像分割性能。该方法无需训练，仅通过融合多种视觉线索选择紧凑、远离边界的点，即可在RefCOCO系列数据集上显著提升分割精度。

Details

Motivation: 现有无需训练的参考图像分割方法性能落后于经过微调或强化学习调优的专家模型，主要问题在于提示歧义：VLM提供的边界框（bbox）无法明确指示目标像素，而先前方法采用的随机采样内部点可能落在边界、干扰物或背景上，反而损害性能。

Result: 在RefCOCO/+/g数据集上，使用五个内部点时，PinPoint将累积交并比（cIoU）提升了12-18个点，匹配了经过监督或强化学习调优的专家模型性能，且每个查询仅需两次VLM调用。

Insight: 创新点在于提出了一种确定性的、无需训练的点选择器，通过融合多种视觉线索生成共识图，选择远离边界、空间多样且紧凑的内部点，并利用冻结的VLM为点打标签，有效解决了提示歧义问题，显著提升了分割精度。

Abstract: Modern referring image segmentation pipelines couple a vision-language model (VLM) for grounding with a promptable segmenter such as the Segment Anything Model (SAM) for mask generation. Prior training-free instances of this recipe consistently trail fine-tuned and reinforcement-learning (RL)-tuned specialists, and it has been unclear whether the gap comes from the VLM’s grounding, SAM’s capacity, or the prompt. We show that the gap is dominated by prompt ambiguity: a VLM-proposed bounding box (bbox) leaves SAM to guess which pixels inside the bbox belong to the object the expression denotes. Interior points are the natural disambiguator, but where they fall matters; prior work relies on naively sampled points that land on boundaries, distractors, and background clutter, and can even hurt performance compared to the bbox alone. Supervised and RL-tuned methods close this gap by training a VLM to predict better points; we show that this training is unnecessary. At a matched budget of five interior points, replacing naive sampling with stable, informative point selection improves cumulative Intersection-over-Union (cIoU) by 12-18 points across RefCOCO/+/g, with every model fixed. We turn this observation into PinPoint, a deterministic, training-free point selector that fuses four visual cues into a consensus map, selects compact, spatially diverse points away from boundaries, and uses the frozen VLM to label each point. Without any task-specific training, PinPoint matches supervised and RL-tuned specialists on the same stack while issuing only two VLM calls per query.

[84] DynFrame: Adaptive Reasoning-Driven Multimodal Framework with Dynamic Frame Augmentation for Complex Video Understanding cs.CV | cs.AIPDF

Peng Zhang, Guanghao Zhang, Wanggui He, Longxiang Zhang, Mushui Liu

TL;DR: DynFrame是一个用于复杂视频理解的自适应推理驱动多模态框架，通过动态帧增强技术，在单次自回归过程中同时输出时间窗口和采样密度作为原生令牌，实现了可学习的跨密度检索，从而以单步检索获取多粒度证据。

Details

Motivation: 现有视频多模态大语言模型在推理时结合逐步推理和按需视觉证据检索，但仍存在两个结构性问题：一是采样密度不可学习，导致细粒度证据需多次检索，增加推理上下文长度和训练难度；二是检索与答案生成通常使用单一轨迹级优势优化，导致检索令牌和答案令牌的信用分配不当。

Result: 在DM-CoT-74k和DM-RL-45k数据集上训练后，DynFrame-4B在六个基准测试（NExT-GQA、Charades-STA、ActivityNet-MR、Video-MME、MLVU、LVBench）上与强大的7B-8B基线模型竞争，而DynFrame-8B在大多数指标上达到了新的最先进水平。

Insight: 创新点包括引入可学习的跨密度检索机制，将时间窗口和采样密度作为原生令牌集成到单次自回归过程中，以及提出Segment-Decoupled GRPO（SD-GRPO），通过在检索边界分割每个rollout并分配角色特定的令牌级优势，分别对采样决策和答案进行信用分配，从而优化模型性能。

Abstract: Recent video multimodal large language models (MLLMs) increasingly couple step-by-step reasoning with on-demand visual evidence retrieval, allowing models to revisit relevant video segments during inference. However, two structural gaps remain in existing thinking-with-video systems. (i) Sampling density is not a learnable decision: existing methods may let the model decide where to look, but the per-window frame rate is largely fixed. As a result, fine-grained evidence is often recovered through repeated retrieval calls, which increases inference context length and training difficulty. (ii) Retrieval and answer generation are usually optimized with a single trajectory-level advantage, so the “where to look” tokens and the “how to answer” tokens receive the same credit even when one is correct and the other is not. To address these gaps, we present DynFrame, a framework that emits the temporal window and the sampling density as native tokens within a single autoregressive pass. This learnable span-density retrieval enables acquiring multi-granularity evidence with a single retrieval step. Based on the above tokenized retrieval interface, we further introduce Segment-Decoupled GRPO (SD-GRPO), which splits each rollout at the retrieval boundary and assigns role-specific token-level advantages, separately crediting the sampling decision and the answer. Trained on the curated DM-CoT-74k and DM-RL-45k, DynFrame-4B is competitive with strong 7B-8B baselines across six benchmarks (NExT-GQA, Charades-STA, ActivityNet-MR, Video-MME, MLVU, LVBench), and DynFrame-8B sets new state-of-the-art on most metrics. Code is available at https://github.com/zhangguanghao523/DynFrame.

[85] METATR: A Multilingual, Evolving Benchmark for Automatic Text Recognition cs.CVPDF

Mélodie Boillet, Solène Tarride, Christopher Kermorvant

TL;DR: 本文提出了METATR（v1.0），一个多语言、持续演进的基准测试，旨在全面评估自动文本识别（ATR）系统，特别是视觉大语言模型（vLLMs）。该基准通过整合来自多个公共文档集合的多样化数据，涵盖29种语言、多种文字和版式，并定义了标准化的提示和归一化方法，以促进模型的可复现比较和选择。

Details

Motivation: 当前ATR模型通常在包含现代、印刷体、以英文为主的单一数据集上进行评估，这限制了其在许多实际多语言、多样化文档场景下的相关性和适用性。因此，需要一个能反映真实世界文档多样性和复杂性的代表性基准，以支持针对特定用例的模型评估与选择。

Result: 作者评估了一系列最先进的系统（包括开源和闭源模型），并在数据集、语言层面、手写文档鲁棒性以及计算效率等多个维度报告了结果。结果表明，尽管专有模型性能最稳定，但在不同文字和版式上仍存在显著的性能差异。

Insight: 创新点在于构建了一个多语言、持续演进的标准化评估框架（METATR），其核心价值在于最大化数据多样性并建立动态、可扩展的评估流程，为从业者提供了一个多维度的、面向真实世界条件的ATR评估工具，以追踪领域进展。

Abstract: Benchmarks that reflect the diversity and complexity of real-world documents are essential for accurately evaluating Automatic Text Recognition (ATR) systems, especially Vision-Large Language Models (vLLMs). Although recent models demonstrate impressive performance, they are often evaluated on datasets containing modern, printed texts mostly written in English, which limits their relevance to many practical applications. Therefore, selecting a model for a specific use case requires evaluating it on data that matches the target documents. This highlights the importance of representative benchmarks for real-world applications. In this paper, we introduce METATR (v1.0), a multilingual, evolving benchmark designed to evaluate ATR models across a wide range of documents, facilitating meaningful model comparison and selection. The benchmark was designed to maximize diversity by including documents from various public collections. These documents cover 29 languages and include texts with multiple scripts and layouts. Beyond the dataset itself, METATR defines a standardized prompting and normalization methodology and establishes a dynamic evaluation framework. This approach is intended to produce reproducible results while remaining extensible over time. We evaluated a wide range of state-of-the-art systems, including open-source models and closed-source models. Results are reported across various dimensions, including performance at the dataset and language levels, robustness to handwritten documents, and computational efficiency. Our findings show that, although proprietary models achieve the most consistent performance, substantial variability persists across scripts and layouts. Overall, METATR provides a multidimensional, practitioner-oriented framework for assessing multilingual ATR in real-world conditions and tracking progress as the field evolves.

[86] Joint 2D-3D Segmentation and Association in Street-level Imaging cs.CVPDF

Amir Melnikov, Masayuki Tanaka, Yusuke Monno, Masatoshi Okutomi

TL;DR: 本文提出了一种联合2D-3D分割与关联的统一框架，用于街景图像解析。该方法将视觉语义与多视图几何推理相结合，利用零样本检测、分割以及运动结构重建技术，替代传统的2D多目标跟踪，通过3D驱动的关联机制在宽基线视角和不同成像条件下保持目标身份一致性。

Details

Motivation: 为了解决大规模城市测绘和空间数字孪生（SDT）环境中，传统方法严重依赖连续帧进行时序跟踪，在街景图像理解中难以建立稳定跨视图对应关系的问题。

Result: 实验表明，与最先进的纯2D跟踪方法相比，该方法在具有挑战性的城市场景中实现了22%的性能提升，显著提高了对真实序列的覆盖率和身份保持的鲁棒性。

Insight: 创新点在于提出了一个将2D纹理线索与全局3D上下文结合的联合框架，用基于几何一致性的3D驱动关联机制替代传统2D多目标跟踪，实现了无需时序依赖的稳定跨视图语义关联，适用于可扩展的街景处理。

Abstract: Accurate interpretation of street-level imagery is essential for large-scale urban mapping and the creation of Spatial Digital Twin (SDT) environments. This work presents a unified framework for joint 2D-3D segmentation and association that integrates visual semantics with multi-view geometric reasoning. Unlike conventional approaches that rely heavily on sequential frames for temporal tracking, our method leverages zero-shot detection and segmentation together with structure-from-motion reconstruction to establish stable cross-view correspondences. A 3D-driven association mechanism replaces traditional 2D multi-object tracking, using geometric consistency to guide identity preservation across wide-baseline viewpoints and varying imaging conditions. By combining 2D texture cues with global 3D context, the proposed pipeline is well-suited for scalable street-level processing and can be used for a variety of object types. Experiments demonstrate substantially improved coverage of ground-truth sequences and more robust identity retention compared to state-of-the-art 2D-only tracking methods, achieving a 22% performance gain in challenging urban scenarios.

[87] Once-For-All: A Train-Once and Select-Anytime Framework for Multimodal Instruction Tuning cs.CVPDF

Mingkang Dong, Hongyi Cai, Xiwen Lei, Jie Li, Tao Zhang

TL;DR: 本文提出了OFA框架，用于多模态指令调优中的数据选择。该框架通过一次训练得到一个可重用的选择器，能够跨数据集和模型规模直接应用，无需重新计算。实验表明，仅使用15%的数据即可达到全数据性能的98.3%，并在未见数据集上超越全数据训练效果。

Details

Motivation: 多模态指令调优中数据冗余严重，现有方法针对特定模型或数据集设计选择标准，更换目标时需重新计算，成本高昂。

Result: 在LLaVA-665K和Vision-Flan-186K数据集上，仅选择15%数据，在10个下游基准测试中达到全数据性能的98.3%；在未见数据集Vision-Flan-186K上，迁移选择器性能超越全数据训练10.6%。该选择器在Qwen2.5-VL-3B和LLaVA-v1.5-7B模型上均有效，无需针对每个模型重新计算。

Insight: 创新点在于提出了一种一次训练、任意选择的通用数据选择框架，利用冻结CLIP空间聚类生成伪标签训练轻量选择器，实现了选择标准与目标模型和数据集解耦，提升了训练效率与可迁移性。

Abstract: Multimodal instruction tuning is the de facto recipe for adapting vision language models (VLMs), yet instruction data are highly redundant, making data selection critical for training efficiency. Existing methods derive selection signals from a specific model or dataset, so whenever the target model or candidate pool changes, the criteria must be recomputed from scratch at substantial cost. To address this, we propose OFA, a data selection framework that trains a reusable selector once and applies it to any dataset or model without recomputation. OFA clusters multimodal instructions in a frozen CLIP space, derives pseudo labels from the cluster structure, and trains a lightweight selector for only a few epochs; samples on which this selector is least confident are selected as the most informative. Once trained, the frozen selector transfers directly across datasets and model scales. The selector is trained once on LLaVA-665K and applied both to LLaVA-665K itself and, without any retraining, to the unseen Vision-Flan-186K. Selecting only 15% of the data, OFA achieves 98.3% of full data performance across 10 downstream benchmarks; on the smaller Vision-Flan-186K, the transferred selector surpasses full data training by 10.6%, confirming that the learned signal generalizes to datasets never seen during selector training. The same selected subsets benefit VLMs at both Qwen2.5-VL-3B and LLaVA-v1.5-7B without per model recomputation, decoupling selection from the target model. These results demonstrate that a single, transferable selector provides an effective and reusable solution for efficient multimodal instruction tuning.

[88] Cesarean Scar Defect Segmentation in Transvaginal Ultrasound Images: a Dataset and Benchmark cs.CVPDF

Yuan Tian, Yue Li, Wei Xia, Tianyu Xu, Jian Zhang

TL;DR: 该论文提出了首个用于剖宫产瘢痕缺损（CSD）经阴道超声图像分割的公开数据集，包含1111张图像和16个视频，其中501个阳性样本带有精确的像素级手动标注。该数据集旨在为医学图像分割算法提供高质量基准资源，以促进临床创新。

Details

Motivation: CSD是剖宫产后常见的并发症，准确分割其轮廓和尺寸对治疗至关重要，但由于其尺寸小、形态不规则、图像质量欠佳以及临床认知有限，常被漏诊。目前缺乏公开的经阴道超声CSD分割数据集，阻碍了AI在该领域的应用。

Result: 论文构建了一个包含1111张图像和16个视频的CSD数据集，其中501个阳性样本带有精确的像素级手动标注，为CSD分割任务提供了首个公开的基准资源。

Insight: 创新点在于创建了首个公开的经阴道超声CSD分割数据集，填补了该领域的数据空白。数据集遵循标准化临床指南，由经验丰富的超声医师和训练有素的博士生协作标注，确保了高质量和临床相关性，为开发更准确的CSD自动诊断工具奠定了基础。

Abstract: Cesarean Scar Defect (CSD) is one of the most prevalent complications following cesarean delivery. Transvaginal ultrasonography is widely used for primary CSD screening. Accurate determination of CSD outline and dimensions is crucial for treatment. However, CSDs are frequently overlooked by sonographers due to small size and irregular morphology, suboptimal image quality, and limited clinical awareness in resource-constrained settings. Despite artificial intelligence advances in medical imaging, no public dataset exists for transvaginal ultrasound CSD segmentation. To address this gap, we present a comprehensive CSD dataset comprising 1,111 images and 16 videos, yielding 501 positive samples with confirmed CSD and precise pixel-level manual annotations. Annotations are performed following standardized clinical guidelines through collaboration between experienced sonographers and trained PhD students. This work provides high-quality benchmark resources for advancing medical image segmentation algorithms and promoting clinical innovation. Ultimately, improved CSD diagnosis and subsequent treatment strategies can enhance the quality of life in women of reproductive age, representing significant value for both medical research and clinical practice.

[89] OSMa-Bench++: Toward Open-Ended Benchmarking of Semantic Mapping for Manipulation with Prompt-Generated Synthetic Scenes cs.CV | cs.ROPDF

Regina Kurkova, Maxim Popov, Sergey Kolyubin

TL;DR: 本文介绍了OSMa-Bench++，一个用于语义映射评估的开放式基准测试框架。该框架通过提示生成的合成室内场景，支持可控的基准测试，包括自动生成场景描述、合成环境并适配到仿真格式。其关键优势在于利用已知的生成提示作为辅助语义规范，并扩展了基于视觉问答的评估组件。

Details

Motivation: 现有语义映射方法的评估主要依赖于固定基准数据集，这些数据集对机器人操作相关的边缘情况覆盖有限，无法充分满足下游操作任务的需求。

Result: 该框架支持在杂乱、小物体、部分遮挡和光照变化等条件下对语义场景表示进行有针对性的压力测试，使基准测试更具可扩展性，并更好地与下游操作要求对齐。

Insight: 创新点在于提出了一种利用提示生成合成场景进行可控基准测试的流程，并引入了基于提示的辅助语义规范来增强评估。这为语义映射的评估提供了更灵活、更贴近实际应用场景的测试环境。

Abstract: Semantic mapping methods are increasingly used as intermediate scene representations for downstream robotic reasoning and manipulation, yet their evaluation is still largely tied to fixed benchmark datasets with limited coverage of manipulation-relevant corner cases. In this work, we extend OSMa-Bench toward controllable benchmarking with prompt-generated synthetic indoor scenes. Our pipeline automatically generates scene descriptions, synthesizes corresponding environments with SceneSmith, and adapts the resulting assets into an OSMa-Bench-compatible simulation format. This adaptation requires a nontrivial intermediate layer, including semantic normalization, material and texture repair, shader fallback policies, floor handling, navigation setup, and controlled lighting configuration. A key advantage of the proposed setup is that the original scene-generation prompt is known in advance and can therefore serve as an auxiliary semantic specification of the intended scene. We use this property to extend the VQA component of OSMa-Bench with a prompt-grounded question category. The resulting framework supports targeted stress-testing of semantic scene representations under conditions such as clutter, small objects, partial occlusions, and lighting variation, and makes benchmarking more extensible and better aligned with downstream manipulation requirements. Our code is available at https://github.com/be2rlab/OSMa-Bench-v2.

[90] REVERSE: Reinforcing Evidence Verification and Search for Agentic Image geo-localization cs.CVPDF

Yong Li, Furong Jia, Dacheng Yin, Kang Rong, Fengyun Rao

TL;DR: 论文提出了REVERSE框架，用于强化图像地理定位任务中证据搜索与验证之间的交互，实现多轮智能体推理。该框架通过教导模型三个中间决策（看哪里、查询什么、信任哪些证据），并利用带有标注的工具轨迹和过程奖励进行训练，在离线搜索缓存的支持下，在Im2GPS3k和YFCC4k基准上超越了现有检索增强基线，并与更大模型相媲美。

Details

Motivation: 现有图像地理定位方法未能充分模拟人类专家迭代式的工作流程（检查信息区域、形成假设、搜索外部证据、根据新线索修正判断），要么完全绕过证据获取，要么在指导中间决策（如搜索位置、查询内容、过滤噪声）方面监督有限。

Result: 使用一个40亿参数的模型，REVERSE在Im2GPS3k和YFCC4k基准测试中超越了强大的检索增强基线，并与规模大得多的模型性能相当。

Insight: 创新点在于将图像地理定位构建为一个多轮智能体推理过程，通过强化学习教导模型进行区域选择、查询生成和证据筛选等中间决策，并引入工具轨迹标注和过程奖励（视觉定位、查询效用、证据判别）进行密集监督，同时利用离线搜索缓存稳定检索过程，有效处理噪声搜索结果。

Abstract: Image geo-localization aims to determine where a photograph was taken, a task that often requires more than recognizing visible landmarks. Human experts typically solve it through an iterative workflow: they inspect informative regions, form location hypotheses, seek external evidence, and revise their judgments as new clues appear. Existing methods only partially capture this process: direct prediction methods bypass evidence acquisition altogether, while retrieval-augmented methods introduce external evidence but usually provide limited supervision on the intermediate decisions of where to search, how to query, and how to filter noisy results. We present REVERSE, a framework that reinforces the interplay between evidence search and verification to enable multi-turn agentic reasoning. REVERSE teaches three intermediate decisions: where to look, what to query, and what evidence to trust. To support this, we construct tool-grounded trajectories with annotated region selections, search observations, and geo-informative evidence labels, and introduce process rewards for visual grounding, query utility, and evidence discrimination. An offline search cache makes retrieval observations stable and reusable during reinforcement learning, enabling dense supervision over noisy search results. With a 4B model, REVERSE outperforms strong retrieval-augmented baselines and rivals substantially larger models on Im2GPS3k and YFCC4k. Code is available at https://github.com/yonglleee/REVERSE.

[91] Natural Human Motion Recovery by Aligning High-Order Temporal Dynamics from Monocular Videos cs.CVPDF

Dingkun Wei, Zehong Shen, Yan Xia, Georgios Pavlakos, Yujun Shen

TL;DR: 本文提出HTD-Refine，一种用于单目视频人体运动恢复的后处理框架。该框架通过显式估计高阶时序动态（速度和加速度）来增强现有HMR流程，旨在解决恢复的运动过于平滑或动态不一致的问题。核心是PVA-Net，一个从视频直接推断关节2D位置、3D速度和3D加速度的时序Transformer，这些预测用于全局优化以细化世界空间轨迹。

Details

Motivation: 现有从单目视频恢复的人体运动常显得过于平滑或动态不一致，即使关节位置数值准确。作者认为这源于缺乏可靠的高阶时序线索（速度和加速度），而这些对于重建具有真实动量、时序和高频细节的运动至关重要。

Result: 在具有挑战性的真实世界基准测试上的广泛实验表明，HTD-Refine能持续改进最先进的HMR方法，产生更准确的全局轨迹和显著更自然的运动动态。

Insight: 创新点在于明确地将高阶时序动态（速度和加速度）建模为软约束，并集成到全局优化中以细化运动轨迹。客观来看，这凸显了高阶时序建模在单目人体运动恢复中的关键作用，为提升运动自然度提供了一个通用且有效的后处理思路。

Abstract: Human motion recovered from monocular videos often appears overly smooth or dynamically inconsistent, even when joint positions are numerically accurate. We observe that this limitation stems from the absence of reliable high-order temporal cues – velocity and acceleration – which are essential for reconstructing motion that exhibits realistic momentum, timing, and high-frequency detail. We introduce HTD-Refine, a post-processing framework that augments existing Human Motion Recovery (HMR) pipelines using explicitly estimated high-order temporal dynamics. At the core of our system is PVA-Net, a temporal transformer that infers per-joint 2D positions, 3D velocities, and 3D accelerations directly from a monocular video. These predicted dynamics serve as soft yet informative constraints in a global optimization procedure that refines world-space trajectories, significantly reducing jitter, suppressing over-smoothing, and restoring physically plausible motion. Extensive experiments on challenging in-the-wild benchmarks show that HTD-Refine consistently improves state-of-the-art HMR methods, yielding more accurate global trajectories and substantially more natural motion dynamics. Our results highlight the critical role of high-order temporal modeling in advancing monocular human motion recovery.

[92] Small Object Detection in Industrial Recycling: A New Dataset and YOLO Performance Evaluation cs.CV | cs.DBPDF

Oussama Messai, Abbass Zein-Eddine, Abdelouahid Bentamou, Mickael Picq, Nicolas Duquesne

TL;DR: 本文针对工业回收场景中的小、密集且重叠物体检测问题，提出并评估了一个包含超过10k图像和120k实例的新数据集，系统比较了基于深度学习的监督方法在性能、精度和计算效率上的表现，并探讨了数据增强与合成图像的作用，同时提出了未来改进方向。

Details

Motivation: 解决计算机视觉中检测小、密集和重叠物体的挑战，特别是在工业回收过程中，以提升检测系统的可靠性和实用性。

Result: 在新数据集上对现有系统进行了详细比较，评估了性能、准确性和计算效率，并识别了当前最可靠的系统及其针对的特定挑战。

Insight: 创新点包括引入一个专门针对工业回收场景的小物体检测数据集，以及探讨数据增强和合成图像对检测效果的提升；客观分析认为，该研究为小物体检测提供了实用的基准和评估框架，有助于推动该领域在工业应用中的发展。

Abstract: In this paper, we address the problem of detecting small, dense, and overlapping objects, a major challenge in computer vision. Our focus is on reviewing proposed methods based on deep learning supervised approaches. We provide a detailed comparison of these systems on a new dataset of more than 10k images and 120k instances, highlighting their performance, accuracy, and computational efficiency in the industrial recycling process use case. Through this comparative analysis, we identify the most reliable systems currently available and the specific challenges they are designed to tackle. Furthermore, we explore the benefits of data augmentation and synthetic images. Based on our analysis, we also propose potential future directions and innovative solutions that could enhance the effectiveness of small, dense and overlapped object detection systems. The scope of our investigations encompasses object detection, length measurement, and anomaly detection within the context of the recycling process. The anomaly detection strategy is robust against variations in image resolution and zoom levels, ensuring reliable performance in industrial applications. The repository of the proposed dataset, methods and evaluation codes can be found at: https://github.com/o-messai/SDOOD

Azhar Hussian, Marina Ritthaler, André Kaup, Vasileios Belagiannis

TL;DR: 本文提出了一种图像驱动的点云补全方法I2PRef，该方法将单张RGB图像作为主要几何来源，而非辅助引导。它包含一个直接从图像重建完整点云的I2P模块，以及一个基于Transformer的P2P细化模块，通过点令牌与图像特征间的自注意力和交叉注意力迭代优化粗糙输出。

Details

Motivation: 现有多模态方法通常将图像作为次要引导，依赖辅助损失或融合模块，本文旨在探索将图像作为主要几何来源，以提供更强的、仅基于图像的几何感知先验。

Result: 在ShapeNet-ViPC基准测试上进行了广泛实验，取得了最先进的补全性能，Chamfer Distance相对先前方法提升了12.3%。

Insight: 创新点在于明确的I2P任务设计，使图像编码器能学习丰富的几何表示，而基于Transformer的P2P细化模块能逐步恢复细粒度细节，无需3D输入或复杂的多模态融合策略。

Abstract: We present an image-conditioned point cloud completion approach that treats images as the primary geometric source rather than a secondary guide. To this end, we introduce an Image-to-Point (I2P) module that can reconstruct complete point clouds directly from a single RGB image, with no need for 3D inputs. Additionally, we introduce a transformer-based Point-to-Point (P2P) refinement module that uses self- and cross-attention between point tokens and image features to iteratively refine the coarse I2P output. The I2P module enables the image encoder to learn rich geometric representations, while the P2P module progressively recovers fine-grained details. Unlike existing multimodal methods that rely on auxiliary losses or fusion modules, our explicit I2P task provides a strong, geometry-aware prior based on images alone. Extensive experiments on ShapeNet-ViPC demonstrate state-of-the-art completion performance with a 12.3% relative Chamfer Distance improvement over prior methods. Code is available at: https://github.com/AzharSindhi/I2PRef.git

[94] Leveraging Text-to-Image Diffusion Models for Unsupervised Visual Object Tracking cs.CVPDF

Zhengbo Zhang, Zhigang Tu, Junsong Yuan, De Wen Soh, Bo Du

TL;DR: 本文提出Diff-Tracking方法，利用预训练的文本到图像扩散模型进行无监督视觉目标跟踪。该方法通过跨注意力机制，将跟踪任务重新定义为学习一个能激活目标区域的文本提示，并包含初始提示学习和在线提示更新两个组件。

Details

Motivation: 现有最先进的无监督跟踪器在需要细粒度理解视频帧语义和视觉结构信息的场景中表现不佳。文本到图像扩散模型擅长生成准确反映输入提示语义和结构的图像，表明其对视觉语义和结构有很强的把握能力。

Result: 该方法在六个具有挑战性的跟踪数据集上进行了评估，证明了其有效性。

Insight: 核心创新点在于将预训练文本到图像扩散模型重新解释为文本和图像模态之间的桥梁，并利用其跨注意力机制实现无监督跟踪。这为利用大规模生成式模型的丰富语义知识解决判别式任务（如跟踪）提供了新视角。

Abstract: Unsupervised visual object tracking is a challenging task that requires following arbitrary targets in videos without training on ground-truth annotations. Despite considerable progress, existing state-of-the-art unsupervised trackers often struggle in scenarios that demand fine-grained understanding of semantic and visual structural information within video frames. Text-to-image diffusion models are well known for their ability to generate images that accurately reflect the semantics and structures described in the input prompt, demonstrating a strong grasp of visual semantics and structures. Building on this capability, we approach the unsupervised tracking from a new perspective by exploiting the rich semantic knowledge encoded in pretrained text-to-image diffusion models. To adapt the diffusion models, which are originally developed for image generation, to the tracking task, we reinterpret the models as a bridge between text and image modalities. This connection is realized through the cross-attention mechanism: when both text and an image are input into the models, they highlight the regions of the image that are semantically aligned with the text in the cross-attention maps. We therefore learn a prompt that represents the tracking target and activates its corresponding region in the cross-attention map for each frame, which enables object tracking with the diffusion model. Specifically, our method Diff-Tracking is composed of two main components: an initial prompt learner and an online prompt updater. The initial prompt learner generates a prompt that captures the target object in the first frame, allowing the diffusion model to identify the target. The online prompt updater refines the prompt based on motion information, enabling consistent tracking across video frames. We evaluate our approach on six challenging tracking datasets demonstrate the effectiveness of our approach.

[95] DinoComplete: 3D Shape Completion with Distilled Semantic Priors and State Space Models cs.CV | cs.GRPDF

Furkan Mert Algan, Eckehard Steinbach

TL;DR: DinoComplete是一个确定性的3D形状补全框架，通过从DINO特征中提取体素对齐的语义先验来增强几何重建。该方法首先训练学生网络从部分扫描中预测密集语义特征，然后将这些特征与几何表示融合，并引入多尺度体素Mamba模块进行高效长程推理。实验表明，该方法在未见类别和真实扫描数据上优于现有方法，且具有参数少、内存低、推理快的优势。

Details

Motivation: 解决仅依赖几何信息在未见类别和噪声真实扫描中难以推断缺失结构的挑战，通过引入视觉基础模型的语义先验来提升补全的泛化性和鲁棒性。

Result: 在ShapeNet未见类别和ScanNet真实物体上的实验显示，DinoComplete在补全质量上优于先前的确定性和生成式方法，同时参数更少、内存需求更低、推理更快，达到了SOTA水平。

Insight: 创新点包括从DINO特征中蒸馏体素对齐的语义先验以捕获全局结构和部件感知上下文，以及结合全网格和分块序列建模的多尺度体素Mamba模块，实现高效长程推理而不牺牲分辨率。

Abstract: 3D shape completion from partial scans remains challenging for unseen categories and noisy real-world observations, where geometry alone is often insufficient for inferring missing structure. We present DinoComplete, a deterministic and efficient shape completion framework that augments geometric reconstruction with voxel-aligned semantic priors distilled from DINO features. First, we construct multi-view DINO feature volumes aligned with ShapeNet data and train a student network to predict dense semantic features directly from incomplete shapes. These predicted features capture global structure and part-aware semantic context while remaining aligned with the underlying geometry. We then integrate these distilled features into a completion network, where geometric and semantic voxel representations are fused through voxel state-space modeling. To enable efficient long-range reasoning without sacrificing resolution, we introduce a multi-scale voxel Mamba module that refines the fused features by combining full-grid and chunk-wise sequence modeling. Experiments on unseen ShapeNet categories and ScanNet objects show that DinoComplete achieves stronger completion quality than prior deterministic and generative based completion methods while using fewer parameters, requiring lower memory, and achieving faster inference. Our results demonstrate that distilling semantic priors from visual foundation models improves generalization and robustness in 3D shape completion.

[96] CodecCap: High-Fidelity Codec-Inspired Residual Modeling for Dense Video Captioning cs.CVPDF

Zihan Lin, Songhe Deng, Shuwei He, Danxiang Zhu, Dan Zhang

TL;DR: 本文提出CodecCap，一种受视频编解码器启发的密集视频描述框架，通过关键帧描述和残差描述来平衡视觉保真度与冗余度。关键帧描述编码稳定的视觉上下文，残差描述捕捉局部动作和变化，从而在保留细节的同时减少冗余。为评估描述保真度，作者构建了VidCapQA基准测试，并基于CodecCap创建了大规模数据集CodecVDC-100K。

Details

Motivation: 现有视频描述方法难以平衡视觉保真度与冗余度：整体描述紧凑但丢失细节，分段描述覆盖更广但引入大量冗余。本文旨在解决这一瓶颈，通过受编解码器启发的表示方法提升密集视频描述的保真度。

Result: 在VidCapQA基准测试（包含14个能力维度的1000个问题）上，CodecCap显著超越了使用相同底层视觉语言模型（VLM）的直接描述方法，表明关键帧-残差描述能有效提升视频语言监督的保真度。

Insight: 创新点在于将视频编解码思想引入视频描述任务，通过关键帧和残差分解来优化描述表示；同时构建了VidCapQA评估基准，揭示了描述表示是当前视觉语言模型的瓶颈，为高保真视频语言监督提供了新思路。

Abstract: Existing video captioning methods struggle to balance visual fidelity and redundancy: holistic captions are compact but lose fine-grained evidence, whereas segment-wise captions improve coverage but introduce heavy redundancy. We propose CodecCap, a codec-inspired framework for high-fidelity dense video captioning. Analogous to video codecs, CodecCap represents videos using keyframe and residual captions. Keyframe captions exhaustively encode stable visual context, while residual captions capture temporally only localized actions, motions and changes. This effectively preserves fine-grained visual evidence while reducing redundant descriptions. To quantify the fidelity of captions, we introduce VidCapQA, a caption-then-QA benchmark with 1,000 questions across 14 capability dimensions. Results on VidCapQA show that captions directly generated by strong VLMs still miss many visual details, highlighting caption representation as a critical bottleneck. Experiments show that CodecCap significantly surpasses direct captioning with the same underlying VLMs, suggesting keyframe-residual captioning a way for high-fidelity video-language supervision. We further use CodecCap to construct CodecVDC-100K, a large-scale dense captioning dataset with anchor, residual, scene-level, and video-level supervision.

[97] On the Robustness of Machine Unlearning for Vision-Language Models cs.CVPDF

Yujie Lin, Kaidi Jia, Jiayao Ma, Chengyi Yang, Jinsong Su

TL;DR: 本文首次对视觉语言模型（VLM）的机器遗忘进行了系统性综述和鲁棒性分析，提出了统一的评估框架和三种攻击范式，实验表明现有方法往往只是隐藏而非完全移除目标知识，揭示了当前VLM遗忘方法的局限性。

Details

Motivation: 视觉语言模型可能从训练数据中记忆不良信息，因此机器遗忘技术受到关注；但现有VLM遗忘方法的有效性和鲁棒性尚未得到充分评估。

Result: 在多种提示设置下的统一评估和广泛实验表明，许多现有方法在上下文提示或下游重训练攻击下仍显脆弱，未能完全移除目标知识。

Insight: 创新点在于首次系统性地评估VLM遗忘的鲁棒性，并提出三种攻击范式来检验知识是否被真正遗忘；客观来看，该研究为开发更可靠的多模态遗忘策略提供了重要洞见和基准。

Abstract: Vision-language models (VLMs) may memorize undesirable information from training data, motivating growing interest in machine unlearning. In this work, we present the first systematic survey and robustness analysis of VLM unlearning. We provide a comprehensive taxonomy and review of existing VLM unlearning methods, together with unified evaluations under multiple prompt settings. We then propose three attack paradigms to examine whether forgotten multimodal knowledge can be reactivated through contextual prompting or downstream retraining. Extensive experiments show that many existing methods remain vulnerable under these attacks, indicating that current approaches often hide rather than fully remove target knowledge. Our study provides new insights into the robustness and limitations of current VLM unlearning methods and highlights the need for more reliable multimodal unlearning strategies. Code is available at https://github.com/XMUDeepLIT/VLM-UnL-Attack.

[98] ChartAct: A Benchmark for Dynamic Chart Understanding cs.CVPDF

Muye Huang, Wu Lin, Lingling Zhang, Hang Yan, Zhiyuan Wang

TL;DR: 该论文提出了ChartAct，一个用于评估动态图表理解能力的交互式基准测试。它收集了来自8个真实图表网站的673个动态图表，涵盖7种常见图表类型，并构建了1440个高质量问答样本，旨在评估模型在交互式图表环境下的理解能力。

Details

Motivation: 现有图表理解基准主要关注静态图表，而现实世界的图表通常是动态和交互式的，关键信息可能仅在悬停、点击、缩放或拖动等操作后出现，因此需要评估模型识别可见内容、选择适当交互以及对变化图表状态进行推理的能力。

Result: 在ChartAct基准上系统评估了11个先进的多模态模型和GUI代理，实验结果表明现有模型在动态图表理解方面仍有明显局限。最强的模型Claude-Opus-4.7平均成功率为84.5%，而大多数模型低于60%。

Insight: 论文的创新点在于构建了首个专注于动态和交互式图表理解的基准测试，并引入了两种实例化环境（动态图表和仪表盘图表）来评估不同上下文下的理解能力，为研究真实交互环境中的图表理解提供了新工具和评估标准。

Abstract: Charts are widely used to present complex data for analysis and decision making. Existing chart understanding benchmarks mainly focus on static charts, but real-world charts are often dynamic and interactive. Key information may only appear after actions such as hovering, clicking, zooming, or dragging. Dynamic chart understanding therefore requires models to identify visible content, choose proper interactions, and reason over changing chart states. To evaluate this ability, we propose ChartAct, an interactive benchmark for dynamic chart understanding. ChartAct collects and filters 673 dynamic charts from 8 real chart websites, covers 7 common chart types, and constructs 1,440 high-quality question-answer samples. Each sample is instantiated in two environments, Dynamic Chart and Dashboard Chart, to evaluate dynamic chart understanding under different contexts. Based on ChartAct, we systematically evaluate 11 advanced multimodal models and GUI agents. Experimental results show that existing models still have clear limitations in dynamic chart understanding. The strongest model, Claude-Opus-4.7, achieves an average success rate of 84.5%, while most models remain below 60%. We also conduct detailed failure attribution and case analysis. ChartAct provides a new benchmark for studying chart understanding in real interactive environments. Codes at https://github.com/wulin-wulin/OSWorld_Chart

[99] Timestep-Aware SVDQuant-GPTQ for W4A4 Quantization of Wan2.2-I2V cs.CV | cs.AIPDF

Junhao Wu, Dezhong Yao, Hai Jin

TL;DR: 本文提出了一种名为Timestep-Aware SVDQuant-GPTQ的后训练量化框架，用于对Wan2.2-I2V这类采用专家混合（MoE）设计的视频扩散Transformer进行W4A4（权重和激活均为4位）量化。该方法结合了基于SVDQuant的低秩异常值补偿、基于GPTQ的重建感知残差权重量化，以及针对每个专家独立进行的、按时间步分箱的逐层激活裁剪比搜索，以应对激活异常值、时间步相关的激活分布变化以及MoE中不同专家的量化敏感性差异。

Details

Motivation: 对大型视频扩散Transformer进行W4A4量化能显著节省内存，但面临两大挑战：稀疏的大幅度激活异常值，以及多步去噪轨迹中强烈依赖于时间步的激活分布变化。Wan2.2-I2V的双专家MoE DiT设计进一步加剧了难度，其高噪声和低噪声专家表现出不同的量化敏感性，单一的全局校准策略无法有效处理。

Result: 在OpenS2V-Eval基准测试上，该方法相比BF16基线减少了59.3%的峰值GPU内存占用，同时VBench平均得分仅下降0.9%，图像质量（Imaging Quality）仅下降2.3%。

Insight: 核心创新点在于提出了一个专家感知和时间步感知的联合校准框架。具体而言，将低秩异常值补偿、重建感知权重量化与针对每个专家独立进行的、按时间步分箱的激活裁剪策略相结合，有效解决了MoE视频DiT在W4A4量化中的独特挑战，证明了针对专家和时间步的精细化校准对于保持高保真度推理至关重要。

Abstract: W4A4 quantization of large video diffusion Transformers offers substantial memory savings but is hindered by two main challenges: sparse large-magnitude activation outliers, and strongly timestep-dependent activation distributions across the multi-step denoising trajectory. These difficulties are compounded by Wan2.2-I2V’s two-expert Mixture-of-Experts DiT design, whose high-noise and low-noise experts exhibit distinct quantization sensitivities that a single global calibration policy cannot capture. We propose a post-training quantization framework combining SVDQuant-based low-rank outlier compensation, GPTQ-based reconstruction-aware residual weight quantization, and timestep-bin-wise per-layer activation clipping-ratio search conducted independently for each expert. On the OpenS2V-Eval benchmark, our method reduces peak GPU memory by 59.3% relative to the BF16 baseline while incurring only a 0.9% drop in VBench average score and a 2.3% drop in Imaging Quality, demonstrating that expert- and timestep-aware calibration is essential for high-fidelity W4A4 inference on MoE video DiTs.

[100] SCKAN: Structural Consensus-based KAN Prototype Learning for Semi-Supervised Pancreas Segmentation cs.CVPDF

Yuqi Liu, Yufei Chen, Wei Fu, Xiaodong Yue, Shuo Li

TL;DR: 本文提出了一种基于结构共识的KAN原型学习方法（SCKAN），用于解决半监督胰腺分割任务中因样本间形态差异大而导致的监督偏差问题。该方法通过构建跨样本结构共识学习框架，利用Kolmogorov-Arnold Networks（KANs）实现更泛化和准确的分割。

Details

Motivation: 胰腺分割对早期癌症诊断至关重要，但标注数据稀缺需要半监督学习。现有半监督方法因样本间形态差异大，在稀疏监督下泛化能力受限，存在监督偏差问题。

Result: 在两个公开胰腺数据集上的大量实验证明了SCKAN的有效性，但摘要未具体说明定量结果（如Dice分数）或与SOTA方法的比较。

Insight: 创新点包括：1）首次将KANs用于跨样本结构共识学习；2）提出结构约束原型一致性学习（SPCL），通过原型级对比优化实现无偏结构表示；3）提出基于共识的Kolmogorov-Arnold融合（CKaF），利用KAN的自适应B样条非线性聚合稳定共识并过滤样本噪声。

Abstract: Accurate pancreas segmentation is critical for early cancer diagnosis, where annotation scarcity necessitates Semi-Supervised Learning (SSL). However, due to significant inter-sample morphological variability, existing SSL methods face severe generalizability limitations under sparse supervision, leading to the Supervision Bias problem. To address this, we propose Structural Consensus-based KAN Prototype Learning (SCKAN), which constructs the first cross-sample structural consensus learning with Kolmogorov-Arnold Networks (KANs), to achieve more generalizable and accurate segmentation. Specifically, SCKAN contains two key designs: Structure-constrained Prototype Consistency Learning (SPCL), which prompts unbiased structural representation by enforcing cross-sample consistency via prototype-level contrastive optimization, and Consensus-based Kolmogorov-Arnold Fusion (CKaF), which reduces morphology-specific bias by aggregating stable consensus and filtering sample-wise noise via KAN’s adaptive B-spline nonlinearity. Extensive experiments on two public pancreas datasets demonstrate the effectiveness of SCKAN. Code is at https://github.com/rhodaliu17/SCKAN.

[101] NeR-SC: Adapting Neural Video Representation to Screen Content cs.CV | cs.MMPDF

Ruohan Shi, Jiaoyan Zhao, Haogang Feng

TL;DR: 本文提出了NeR-SC，一个专为屏幕内容视频设计的神经隐式表示框架。它基于SNeRV架构，引入了三个针对屏幕内容特性的模块：可学习的颜色调色板、多门控密集融合模块和嵌入级帧跳过策略。实验表明，该方法在屏幕内容视频数据集上取得了优于现有神经表示方法和传统编码标准（如H.264/H.265）的性能，并能实现实时解码。

Details

Motivation: 现有神经隐式表示方法主要针对自然视频设计，而屏幕内容视频（如远程桌面、在线教育）具有锐利边缘、有限调色板和强时间冗余等独特统计特性，现有方法缺乏有效利用这些特性的机制，性能有较大提升空间。

Result: 在DSCVC和VCD数据集上的实验显示，NeR-SC平均PSNR分别达到40.32 dB和41.73 dB，优于代表性神经视频表示方法，并在低码率下超越H.264和H.265；跳过策略实现了无损质量的实时解码。

Insight: 创新点包括：针对屏幕内容离散颜色结构的可学习调色板模块；通过注意力门控实现跨阶段密集交互的多门控融合模块；以及零训练开销、通过跳过冗余解码器调用实现实时解码的嵌入级帧跳过策略。这些设计针对性地利用了屏幕内容的统计特性，提升了压缩效率和性能。

Abstract: Implicit neural representations have emerged as a promising paradigm for video compression, with recent methods achieving competitive performance on natural video. However, screen content video – common in remote desktop, online education, and cloud gaming – exhibits distinct statistics: sharp edges, limited color palettes, and strong temporal redundancy. Existing neural representation methods, designed for natural scenes, lack mechanisms to exploit these properties, leaving substantial room for improvement. In this paper, we propose NeR-SC, a neural representation framework tailored for screen content video. Building on the SNeRV backbone, NeR-SC introduces three screen-content-specific modules: (i) a learnable color palette that models the discrete color structure of screen content by restricting the low-frequency sub-band to a learned color set; (ii) a multi-gate dense fusion module that replaces sequential feature fusion with dense, attention-gated cross-stage interaction; and (iii) an embedding-level frame skip strategy that bypasses redundant decoder invocations for static frames, with zero training overhead. Experiments on DSCVC and VCD show that NeR-SC achieves 40.32~~dB and 41.73~~dB average PSNR, outperforming representative neural video representation methods and, at low bitrates, surpassing H.264 and H.265. The skip strategy enables real-time decoding with no loss in quality.

[102] IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams cs.CVPDF

Jinzhao Li, Yinuo Chen, Wenxuan Song, Yijia Lei, Yichi Zhang

TL;DR: 本文提出了IPIBench，首个用于评估多模态大语言模型在连续视频流场景下交互式主动智能的基准。该基准涵盖主动监控、主动任务管理以及交错进行的被动与主动请求。研究发现现有MLLMs存在主动触发不稳定和被动-主动行为协调性差两大局限，并提出了无需训练的IPI-Agent框架以提升模型性能。

Details

Motivation: 现有MLLMs在被动问答上表现良好，但现实世界的流式助手需要在连续视觉输入上进行主动推理。现有基准主要研究孤立单轮场景下的被动或主动交互，忽略了用户可能添加、修改或取消主动请求并与被动查询交错的动态多轮场景。

Result: 在代表性MLLMs上的评估揭示了主动触发不稳定和被动-主动行为协调性弱两大主要局限。提出的IPI-Agent框架通过实验证明，能在所有基准设置中持续改进现有MLLMs的性能。

Insight: 创新点在于首次构建了针对流式视频下交互式主动智能的评测基准IPIBench，并提出了一个无需训练的智能体框架IPI-Agent，其包含交互控制策略和时间门控机制，用于稳定主动触发和协调多轮交互，为解决MLLMs在动态多轮场景中的协调问题提供了新思路。

Abstract: Recent multimodal large language models (MLLMs) achieve strong performance on reactive question answering, but real-world streaming assistants require proactive reasoning over continuous visual inputs. Existing benchmarks mainly study reactive or proactive interactions in isolated single-turn settings, overlooking dynamic multi-turn scenarios where users may add, modify, or cancel proactive requests alongside interleaved reactive queries. To address this gap, we introduce IPIBench, the first benchmark for evaluating Interactive Proactive Intelligence of MLLMs under streaming video settings. IPIBench covers proactive monitoring, proactive task management, and interleaved reactive-proactive requests. Evaluations on representative MLLMs reveal two major limitations: unstable proactive triggering and weak coordination between reactive and proactive behaviors. We further propose IPI-Agent, a training-free agentic framework with an interaction-control policy and a temporal-gating mechanism for stabilizing proactive triggering and coordinating multi-turn interactions. Experiments show that IPI-Agent consistently improves existing MLLMs across all benchmark settings.

[103] Pop-Up Distractions Reveal Bag-of-Events Behavior in Video Large Language Models cs.CV | cs.CLPDF

Oscar Chew, Serhii Honcharenko, Qian-Hui Chen, Patricia Lu, Dishant Zaveri

TL;DR: 该论文通过引入DistractionBench基准，评估视频大语言模型（VideoLLMs）在存在无关视频片段干扰时能否稳健地关联主体与事件。研究发现，所有测试的11个主流VideoLLMs均表现出显著的’事件袋’（bag-of-events）行为，即模型将视频处理为事件集合而非时序结构序列，导致跨片段幻觉式错误关联。

Details

Motivation: 旨在探究VideoLLMs是否具备跨时间可靠关联视频主体与事件的关键能力，特别是在存在无关干扰片段（如插入广告）时模型的鲁棒性。

Result: 在DistractionBench基准测试中，11个流行的VideoLLMs均表现出严重的’事件袋’行为，模型经常幻觉不同片段实体间的交互，错误地将注入广告的动作归因于主视频主体。

Insight: 揭示了VideoLLMs普遍缺乏时序 grounding 机制，将视频视为无序事件集合而非结构化序列；这为开发具有更强主体-事件关联鲁棒性的模型提供了动机。

Abstract: A key capability for video understanding is reliably linking subjects to events across time, yet whether Video Large Language Models (VideoLLMs) actually achieve this remains unclear. In this work, we introduce DistractionBench to evaluate whether VideoLLMs can robustly link subjects and events in the presence of unrelated video segments. Through controlled interventions, such as inserting short advertisement clips into longer videos, we show that VideoLLMs frequently hallucinate interactions between entities from different segments, incorrectly attributing actions from injected advertisements to subjects in the main video. We characterize this systematic hallucination as bag-of-events (BoE) behavior, where models process videos as collections of events rather than temporally structured sequences. Evaluating 11 popular VideoLLMs, we find that all models exhibit substantial BoE behavior. Our findings suggest that VideoLLMs lack reliable mechanisms for temporal grounding and motivate the development of models with more robust subject-event association.

[104] COVD: Continual Open-Vocabulary Object Detection with Novel Concept Injection cs.CVPDF

Yupeng Zhang, Ruize Han, Yuzhong Feng, Zixin Ren, Yuntong Tian

TL;DR: 本文提出了一个名为COVD的新任务设置，即带有新概念注入的持续开放词汇目标检测，旨在让模型能够顺序学习新出现的概念组，同时保留先前概念和原有的开放词汇知识。为此，作者还建立了一个新的基准测试Novel-114。论文的核心发现是，预训练的视觉编码器通常已经感知并表示了众多新概念，主要瓶颈在于视觉表示与文本概念之间缺乏稳定的语义对齐。基于此，作者提出了一个无需额外参数的高效持续注入框架NoIn-Det。

Details

Motivation: 现实世界中的类别空间是持续演化的，现有的开放词汇目标检测模型难以处理新出现的概念，而进行重复的完全重新训练成本又极其高昂。

Result: 大量实验表明，NoIn-Det在新建立的Novel-114基准上，能够有效地学习新概念并保留旧知识，并且在无需引入额外参数的情况下，持续优于现有的视觉语言模型持续学习方法。

Insight: 论文的创新点在于提出了COVD任务和相应的基准，并揭示了预训练视觉编码器已具备新概念感知能力这一关键洞察。NoIn-Det框架的创新之处在于冻结视觉编码器，仅使用常见概念和已注入概念的文本来保留文本表示空间，并通过仅更新对学习新概念有益的一小部分文本分支参数来实现高效的新概念注入，这是一种参数高效的持续学习方法。

Abstract: Open-vocabulary object detection (OVD) has made significant progress, enabling detectors to generalize from seen to unseen categories. However, real-world category spaces continually evolve, and existing OVD models still struggle with newly emerging concepts, while repeated full retraining is prohibitively expensive. To this end, we introduce a new task setting, termed Continual OVD with Novel Concept Injection (COVD), where models sequentially learn incoming novel concept groups while preserving prior concepts and original open-vocabulary knowledge, along with a new benchmark, Novel-114. Our key observation is that pretrained visual encoders often already perceive and represent many novel concepts, and the main bottleneck lies in the lack of stable semantic alignment between visual representations and textual concepts. Based on this, we propose NoIn-Det, an efficient continual injection framework without additional parameters. NoIn-Det freezes the visual encoder, preserves the text representation space using only texts of common concepts and previously injected concepts, and injects novel concepts by updating only a small subset of text-branch parameters beneficial to novel concept learning. Extensive experiments show that NoIn-Det effectively learns novel concepts, preserves old knowledge, and consistently outperforms existing continual learning methods for VLMs without introducing additional parameters.Novel-114 and the code will be released.

[105] Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generation cs.CVPDF

Joseph Hoche, David Brellmann, Gianni Franchi

TL;DR: 本文提出了一种名为VIG-TUQ的训练无关框架，旨在通过将视觉基础信息融入不确定性估计，来改进大型视觉语言模型（LVLM）的令牌级不确定性量化。该方法通过分析生成过程中视觉特征整合后的隐藏表示，发现高置信度预测更依赖于视觉内容，并据此利用视觉基础分数对语言不确定性进行加权。

Details

Motivation: 当前大型视觉语言模型（LVLM）的不确定性量化方法大多源自纯语言模型领域，主要关注语言模态，而视觉信息对LVLM不确定性的贡献尚未得到充分探索。本文旨在研究LVLM如何处理视觉信息，以及这一过程是否能用于改进不确定性估计。

Result: 在多个数据集和不同LVLM架构（包括早期融合、晚期融合和原生融合模型）上的评估结果表明，VIG-TUQ方法通常优于现有的令牌级不确定性量化方法。

Insight: 核心创新点在于首次系统地探索了视觉信息在LVLM不确定性量化中的作用，并提出了一个无需训练、通过视觉基础分数加权语言不确定性的框架。其关键洞察是发现模型置信度与对视觉内容的依赖程度相关，这为改进多模态不确定性估计提供了新思路。

Abstract: Uncertainty quantification (UQ) remains a critical challenge in Large Vision Language Models (LVLMs) for reliable predictions and real-world deployment. However, most existing methods are adapted from the LLM literature and primarily focus on the language modality, leaving the contribution of visual information to LVLM uncertainty largely underexplored. In this paper, we investigate how LVLMs process visual information and whether this process can be used to improve uncertainty estimation. By analyzing hidden representations after the integration of visual features during the generation process, we observe that high-confidence predictions rely more heavily on visual content than uncertain ones. Building on this insight, we propose Visual-Grounded Token UQ (VIG-TUQ), a training-free framework that explicitly incorporates visual grounding into uncertainty estimation by weighting token-level language uncertainty with visual grounding scores. We evaluate VIG-TUQ on multiple datasets and across diverse LVLM architectures, including early-fusion, late-fusion, and native-fusion models. Results indicate that our method often improves upon existing token-level uncertainty approaches. Code and data will be made available upon acceptance.

[106] Is an Image Also Worth 16x16=256 Superpixels? A Framework for Attentional Image Classification cs.CV | cs.LGPDF

Pedro Henrique da Costa Avelar, Anderson R. Tavares, Luís C. Lamb

TL;DR: 本文提出了一种名为Superpixel Transformers (SPT)的新型框架，将基于超像素的图像分类与Vision Transformers (ViTs)相结合。该框架通过引入多维正弦-余弦位置编码和包含超像素形状与颜色信息的增强补丁数据结构，支持任意的超像素分块策略、连接图和位置编码。在CIFAR10、FashionMNIST和Imagenette等数据集上的实验表明，SPT在性能上超越了以往的基于超像素的图神经网络方法，并与ViTs保持竞争力。

Details

Motivation: 传统上，基于超像素的图像分类依赖图神经网络（GNNs）处理不规则图像表示，而Vision Transformers（ViTs）在自注意力模型方面取得了新进展。然而，GNNs、超像素和Transformer之间的协同联系尚未被探索。本文旨在填补这一空白，提出一个统一框架来结合超像素分类和ViTs的优势。

Result: 在CIFAR10、FashionMNIST和Imagenette等基准数据集上，SPT通过多种超像素生成和图形连接策略进行测试，结果显示其性能优于先前的基于超像素的GNN方法（如SICGAT），并与ViTs保持竞争水平，达到了SOTA或相当的表现。

Insight: 创新点包括：将超像素分块与Transformer架构统一，支持灵活的连接策略；引入多维正弦-余弦位置编码和增强的补丁数据结构，以保留超像素的形状和颜色信息，减少像素聚合中的信息损失；通过约束图形连接来提升ViT性能，为混合注意力框架的跨领域泛化和未来创新开辟了道路。

Abstract: Superpixel-based image classification has traditionally leveraged graph neural networks (GNNs) for processing irregular image representations. Recent advances in computer vision, driven by Vision Transformers (ViTs), have introduced new paradigms in self-attentional models, surpassing convolutional neural networks (CNNs) in various tasks. However, a synergistic connection between GNNs, superpixels, and transformers remains unexplored. In this work, we propose Superpixel Transformers (SPT), a novel framework that unifies superpixel-based image classification and ViTs. SPT generalizes the Superpixel Image Classification with Graph Attention Networks (SICGAT) model and ViT to support arbitrary superpixel-based chunking strategies, connectivity graphs, and positional encodings. We introduce refinements including a multidimensional sine-cosine positional encoding and an enriched patch data structure that fully incorporates superpixel shape and color information. By testing SPT across datasets such as CIFAR10, FashionMNIST, and Imagenette, with various superpixel generation and graph connectivity strategies, we demonstrate that SPT achieves superior performance compared to previous superpixel-based GNN methods and remains competitive with ViTs. Notably, our approach addresses the limitations of SICGAT, such as information loss during pixel aggregation, and shows how constrained graph connectivity can enhance ViT performance. SPT bridges the gap between superpixel-based and transformer models, opening avenues for cross-domain generalization and future innovations in hybrid attentional frameworks, and showing that an image can also be worth $16\times16$ superpixels.

[107] FoundObj: Self-supervised Foundation Models as Rewards for Label-free 3D Object Segmentation cs.CV | cs.AI | cs.LG | cs.ROPDF

Zihui Zhang, Zhixuan Sun, Yafei Yang, Jinxi Li, Jiahao Chen

TL;DR: 本文提出FoundObj框架，用于无标注的3D点云物体分割。该框架通过基于超点的物体发现代理，在语义和几何奖励模块的引导下逐步合并相邻超点，利用自监督2D/3D基础模型提供的先验知识，通过强化学习实现鲁棒的多类别物体识别。

Details

Motivation: 解决在训练时不依赖任何场景级人工标注的情况下，对复杂场景点云进行3D物体分割的挑战性问题，现有方法因学习过程中物体先验不足而通常仅限于识别简单物体。

Result: 在多个基准测试上的广泛实验表明，该方法始终优于现有基线，并在零样本和长尾场景中表现出强大的泛化能力。

Insight: 创新点在于利用自监督2D/3D基础模型提取语义和几何先验作为奖励，通过强化学习引导超点合并过程，实现无标注的多类别3D物体分割，可借鉴其结合基础模型先验与强化学习的框架设计思路。

Abstract: We address the challenging task of 3D object segmentation in complex scene point clouds without relying on any scene-level human annotations during training. Existing methods are typically constrained to identifying simple objects, primarily due to insufficient object priors in the learning process. In this paper, we present FoundObj, a novel framework featuring a superpoint-based object discovery agent that incrementally merges suitable neighboring superpoints, guided by our innovative semantic and geometric reward modules. These modules synergistically leverage semantic and geometric priors from self-supervised 2D/3D foundation models, providing complementary feedback to the object discovery agent and enabling robust identification of multi-class objects through reinforcement learning. Extensive experiments on diverse benchmarks demonstrate that our approach consistently outperforms existing baselines. Notably, our method exhibits strong generalization in zero-shot and long-tail scenarios, underscoring its potential for scalable, label-free 3D object segmentation.

[108] Touch-R1: Reinforcing Touch Reasoning in MLLMs cs.CVPDF

Yingxin Lai, Yafei Zhou, Fucai Zhu, Siyu Zhu, Weihao Yuan

TL;DR: 本文提出了Touch-R1，一个基于Qwen2.5-VL-7B构建的触觉推理多模态大语言模型。为了解决触觉推理中物理属性的序数性和跨传感器分布偏移的挑战，作者构建了包含超过100万对同步触觉数据的大规模数据集TouchReason-1M和评估基准TouchReason-Bench。模型通过一种结合了序数感知准确性、跨传感器物理一致性、结构化格式控制和输入侧触觉接地目标的触觉接地GRPO目标进行训练。

Details

Motivation: 现有触觉-语言模型主要依赖监督或对比学习目标，限制了其基于物理证据进行预测或纠正误导性视觉先验的能力。触觉推理面临物理属性（如硬度、粗糙度）的序数性质和光学触觉硬件固有的跨传感器分布偏移这两个特定挑战，亟待探索。

Result: 在TouchReason-Bench基准测试中，Touch-R1-7B模型平均性能优于Octopi-13B 18.4%，优于GPT-4o 24.7%，达到了SOTA水平。其结构化推理轨迹揭示了探索、比较和修正等涌现行为。

Insight: 核心创新点在于提出了一个触觉接地的GRPO训练目标，该目标通过一个独特的“触觉使用奖励”机制，仅在真实的触觉输入相对于移除、打乱或噪声掩码的反事实控制产生更优正确性时才给予奖励，从而强制模型将推理过程有效地建立在物理接触证据之上。同时，构建的大规模、多传感器同步数据集和专门的评估基准也为该领域提供了重要资源。

Abstract: While rule-based reinforcement learning has recently catalyzed explicit reasoning in multimodal models, tactile reasoning remains largely underexplored. Existing tactile-language models primarily rely on supervised or contrastive objectives, which limits their capacity to ground predictions in physical evidence or rectify misleading visual priors. Tactile reasoning introduces two modality-specific challenges: the ordinal nature of physical attributes (e.g., hardness, roughness) and the cross-sensor distribution shifts inherent in optical tactile hardware. In this work, we introduce TouchReason-1M, a large-scale multimodal dataset comprising over 1M synchronized tactile pairs across four distinct sensors, and TouchReason-Bench, a rigorous framework for evaluating tactile perception and visual-tactile conflict resolution. Building upon these, we propose Touch-R1, a tactile reasoning MLLM based on Qwen2.5-VL-7B. Touch-R1 is trained via a tactile-grounded GRPO objective that combines ordinal-aware accuracy, cross-sensor physical consistency, structured-format control, and an input-side tactile grounding objective. Specifically, the tactile-use reward assigns credit only when authentic tactile inputs yield superior correctness relative to counterfactual controls where the tactile stream is removed, shuffled, or noise-masked. On TouchReason-Bench, Touch-R1-7B outperforms Octopi-13B by 18.4% and GPT-4o by 24.7% on average. Its structured reasoning traces reveal emergent behaviors of probing, comparison, and revision, demonstrating that R1-style reasoning can be effectively grounded in physical contact.

[109] Can Retrieval Heads See Images? Multimodal Retrieval Heads in Long-Context Vision-Language Models cs.CVPDF

Aaron Branson Cigres Li, Zhaowei Wang, Yu Zhao, Yiming Du, Haobo Li

TL;DR: 这篇论文研究了长上下文视觉语言模型中的多模态检索头，提出了一种检测方法，发现这些检索头具有稀疏性、内在性和因果重要性，仅少量注意力头负责跨模态证据检索，且无需额外训练即可用于视觉丰富文档的检索任务。

Details

Motivation: 随着视觉语言模型处理长上下文（如文档、长视频）的需求增加，模型需要在交织的文本和图像中定位相关证据，但现有基于文本的检索头检测方法不适用于图像证据，因此需要研究多模态检索头的特性。

Result: 在MMLongBench-Doc和SlideVQA基准上，掩码前5%的检索头导致性能从48.2%降至5.7%和从71.2%降至8.9%；在MMDocIR上，Qwen3-VL-8B模型使用所选检索头进行评分，在页面检索和布局检索的Recall@1指标上分别比最强基线提高了7.7/7.4和6.3/6.8个宏/微点。

Insight: 创新点在于提出了多模态检索头检测方法，揭示了检索头的稀疏性和跨模态共享但动态调整的特性；客观分析认为，该方法无需训练即可提升文档检索性能，为理解模型内部机制和优化长上下文多模态推理提供了新视角。

Abstract: Large vision-language models increasingly rely on long-context modeling to reason over documents, hour-level videos, and long-horizon agent trajectories, requiring them to locate relevant evidence across interleaved text and images. Prior work has studied this behavior using retrieval heads in large language models, but its copy-based criterion does not directly apply when evidence appears in images. We introduce a multimodal retrieval head detection method that scores attention from question tokens to textual or visual evidence. With this method, we show that multimodal retrieval heads are sparse, intrinsic, and causally important: only 4.4-10.2% of attention heads account for 50% of the positive retrieval-score mass, and masking the top-5% selected heads drops MMLongBench-Doc from 48.2% to 5.7% and SlideVQA from 71.2% to 8.9%, while random-head masking is far less damaging. Further analysis shows that these heads are partly shared across modalities yet remain dynamic within each modality, with image retrieval heads changing more than text retrieval heads as context length and haystack modality change. Without further training, we find that these heads can also be used directly to rank visually rich documents: on MMDocIR, Qwen3-VL-8B selected-head scoring improves Recall@1 by 7.7/7.4 macro/micro points for page retrieval and 6.3/6.8 points for layout retrieval over the strongest reported baseline.

[110] A Dynamic Programming Framework for Discovering Count and Values of Multilevel Image Thresholding cs.CVPDF

Eslam Hegazy, Mohamed Gabr

TL;DR: 本文提出了一种基于动态规划和改进最小误差阈值准则（MET）的多级图像阈值分割方法，旨在自动确定合适的阈值数量。该方法在自然、卫星和医学图像上进行了广泛测试，结果表明其在阈值数量较多时计算效率显著优于传统动态规划方法，并能自动检测合适的阈值数量，但在结构相似性指数（SSIM）和峰值信噪比（PSNR）方面略逊于需要手动输入阈值数量的传统方法。

Details

Motivation: 解决多级图像阈值分割中需要用户预先指定阈值数量的问题，提出一种能自动从输入图像中确定合适阈值数量的方法，以提高预处理算法的自动化程度和实用性。

Result: 在多种自然、卫星和医学测试图像上，与最先进方法进行比较。数值结果显示，当阈值数量较高时，所提出的MET-DP方法比传统动态规划方法耗时显著减少，且能自动检测大多数测试图像的合适阈值数量，但传统手动指定阈值数量的方法在SSIM和PSNR指标上表现更好。

Insight: 创新点在于结合动态规划与改进的MET准则，实现阈值数量的自动确定，提升了计算效率；从客观角度看，该方法为自动化阈值分割提供了新思路，但需在保持图像质量（如SSIM/PSNR）方面进一步优化。

Abstract: Multilevel Image thresholding is an important preprocessing algorithm in computer vision applications nowadays. Since most common thresholding methods take the desired count of thresholds as input by the user, thresholding methods that automatically determines a suitable count of thresholds from the input image itself are advantageous. In this article, a novel thresholding method based on a dynamic programming algorithm and a modification of Minimum Error Thresholding (MET) criterion is thoroughly presented. An empirical statistical study is performed to pinpoint why this proposed method is superior. Moreover, an extended comparison between this proposed method and other state-of-the-art methods is performed on a comprehensive set of natural, satellite and medical test images. The numerical results show that the proposed MET-DP method takes much less time than traditional dynamic programming thresholding methods when the number of thresholds is high. The proposed method can detect a suitable count of thresholds for most of tested images of different types. However, traditional methods that take the count of thresholds as input produce thresholded images of higher structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR) values than MET-DP. Source code can be found on https://w3id.org/met-dp/article1-code

[111] Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini cs.CVPDF

Madhuri Shanbhogue, Zhe Li, Shanfeng Zhang, Gustavo Hernández Ábrego, Shih-Cheng Huang

TL;DR: 本文介绍了Gemini Embedding 2，这是一个原生的多模态嵌入模型，能够将视频、音频、图像和文本嵌入到统一的表示空间中。该模型利用Gemini的多模态能力，为任意组合的交错输入生成嵌入，并在多种任务上表现出良好的泛化能力。通过在多任务多阶段训练设置中进行大规模对比学习，该模型在包括单模态、跨模态和多模态检索在内的关键嵌入基准测试中达到了最先进的性能。

Details

Motivation: 旨在解决多模态数据（视频、音频、图像、文本）的统一表示问题，以支持跨多种模态和任务的通用嵌入能力。

Result: 在多个基准测试中取得SOTA性能：MSCOCO上R@1为62.9，Vatex上NDCG@10为68.8，MTEB多语言任务上为69.9，MTEB代码任务上为84.0，超越了专用模型。

Insight: 创新点在于构建了一个原生统一的多模态嵌入空间，并利用大规模对比学习和多任务多阶段训练实现强大的零样本泛化能力，使其成为RAG、推荐和搜索等下游应用的有力候选。

Abstract: We introduce Gemini Embedding 2, a native multimodal embedding model that allows embedding video, audio, image, and text modalities in a unified representation space. We leverage the multimodal capabilities of Gemini to produce embeddings for arbitrary combinations of interleaved inputs across all these modalities that generalize well across a wide variety of tasks. Applying large-scale contrastive learning in a multi-task multi-stage training setup, we achieve state-of-the-art performance on key embedding benchmarks including unimodal, cross-modal, and multimodal retrieval spanning a diverse set of tasks. We show that our embedding model demonstrates strong performance (with a score of 62.9 R@1 on MSCOCO, 68.8 NDCG@10 on Vatex, 69.9 on MTEB multilingual and 84.0 on MTEB Code) across a variety of tasks surpassing the performance of specialized models. These unified capabilities make Gemini Embedding 2 a promising candidate for downstream use cases such as RAG, recommendation and search. Furthermore, its robust zero-shot performance across distinct fields - from astronomy and bioscience to fine arts and the culinary arts - establishes it as a highly reliable, out-of-the-box representation even for specialized domains.

[112] PlayClass: Automated Play Behaviour Classification in Poultry cs.CVPDF

Prince Ravi Leow, Neil Scheidwasser, Rebecca Oscarsson, Per Jensen, Samir Bhatt

TL;DR: 本文提出了PlayClass，一种用于家禽玩耍行为自动分类的流水线，通过SAM 3进行长时跟踪以减少身份错误，并利用图像和视频基础模型的冻结嵌入进行分类。实验表明，结合手工特征，V-JEPA 2.1在所有骨干网络中表现最佳，宏平均F1分数达到77.0。尽管数据集因玩耍子类型与非玩耍行为相似及遮挡问题而具有挑战性，但该工作为家禽玩耍行为的自动化监测提供了积极证据。

Details

Motivation: 现有动物福利自动化监测主要关注负面指标，而积极福利行为（如玩耍）研究不足，本文旨在填补这一空白，开发家禽玩耍行为的自动分类方法。

Result: 在提出的流水线中，V-JEPA 2.1结合手工特征在所有模型规模上均优于其他骨干网络，宏平均F1分数达到77.0，但数据集因行为相似性和遮挡问题仍具挑战性。

Insight: 创新点包括利用SAM 3进行长时跟踪以减少基于点提示的身份错误，以及结合基础模型冻结嵌入和手工特征进行分类；客观分析认为，该方法通过整合多模态特征和优化跟踪流程，为复杂动物行为识别提供了可扩展的框架。

Abstract: Automated monitoring of animal welfare has largely targeted negative indicators, leaving positive welfare behaviours such as play underexplored. To address this gap, we present PlayClass, a pipeline for play-behaviour classification in poultry from top-down pen video. The pipeline leverages long-duration tracking with SAM 3 via YOLO-guided chunk boundaries to minimise identity errors in point-based prompting, and frozen embeddings from image and video foundation models for play action classification. Although handcrafted motion features from tracked masks alone achieved competitive accuracy, V-JEPA 2.1 consistently outperformed all other backbones across model scales, reaching 77.0 macro-averaged F$_1$ when combined with handcrafted features. Despite this result, the dataset remains challenging due to play sub-types sharing similar kinematic profiles with non-play and inter-bird occlusion. Overall, our work provides encouraging evidence towards automated frameworks for play behaviour classification in poultry.

[113] How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning cs.CVPDF

Qian Yang, Ankur Sikarwar, Huy Le, Le Zhang, Zhuan Shi

TL;DR: 本文研究了如何在统一多模态模型中通过视觉思维提升跨视角空间推理能力。作者提出View Dropout训练方法，强制模型利用生成的思维图像进行答案预测，并比较了三种视觉思维变体（自上而下、全景、点匹配渲染），发现全景视觉思维结合VDrop在跨域泛化中表现最佳。

Details

Motivation: 现有视觉语言模型在跨视角空间推理任务中过度依赖语言推理，忽略了细粒度几何信息，导致生成的思维图像常被模型忽视。本文旨在探究如何使视觉思维真正发挥作用，并确定最有效的视觉思维类型。

Result: 在合成场景上训练并在五个真实世界跨域基准测试中评估，全景视觉思维结合VDrop方法在跨域泛化中达到最佳性能，是唯一同时具备高信息性和可学习性的配置。

Insight: 创新点包括提出View Dropout训练干预机制，通过隐藏部分输入视角来强制模型依赖思维图像；从可学习性与信息性权衡角度系统比较不同视觉思维表示，为多模态推理中的中间视觉表示设计提供了新思路。

Abstract: Cross-view spatial reasoning remains a weak spot for vision-language models (VLMs): they often reason in language and lose the fine-grained geometry needed for the task. Thinking with images aims to address this by generating an intermediate thinking image, but recent work shows that models often ignore the visual evidence in these traces. We therefore ask how to make visual thinking matter, and what kind of visual thinking works best. We study these questions in unified multimodal models (UMMs), which natively support interleaved image-text generation. For the first question, we propose View Dropout (VDrop), a training-time intervention that hides parts of one input view from the answer span while keeping them visible to the thinking-image tokens. This encourages the model to use the thinking image when answering, instead of relying only on the input views. Once the thinking image is used for answer prediction, we study which type of visual thinking is most effective. We frame this as a learnability-informativeness tradeoff and compare three thinking-image variants: top-down, panoramic, and point-matching renderings. Trained on synthetic scenes and evaluated on five real-world out-of-domain benchmarks, panoramic visual thinking with VDrop is the only configuration that is both informative and learnable, and it achieves the best out-of-domain generalization.

[114] Q-GeoMem: Question-Guided Geometric Memory for Video Spatial Reasoning cs.CVPDF

Xianqiang Gao, Qizhi Chen, Delin Qu, Haoming Song, Zhigang Wang

TL;DR: 本文提出Q-GeoMem，一种用于视频空间推理的问题引导几何记忆框架。该框架通过注入相机条件几何信息到视觉标记中，并维护两个互补的记忆库：细粒度上下文库用于存储近期密集特征和相机状态，语义几何证据库用于存储紧凑的长期证据。每个候选帧通过基于Q-Former的问题相关性和相对于保留库的新颖性乘积进行评分，评分在读取时被存储和重用，同时基于容量的替换规则保持库的紧凑性。在推理过程中，两个记忆库在更新前被读取，并与当前帧表示自适应融合。

Details

Motivation: 现有空间视频语言模型通常将记忆视为通用的时间缓存，这可能会引入冗余或不相关的几何信息，削弱长时域推理能力。因此，需要一种能够根据问题动态积累和保留相关几何证据的记忆机制。

Result: 在VSI-Bench和VSTI-Bench基准测试上，Q-GeoMem在评估的空间推理模型中达到了最先进的性能，验证了问题引导几何记忆的有效性。消融实验进一步证实了所提出的证据评分机制的贡献。

Insight: 创新点在于提出了问题引导的几何记忆框架，通过两个互补的记忆库和基于问题相关性及新颖性的评分机制，实现了对视频中几何信息的动态、高效管理，从而提升了长时域空间推理能力。

Abstract: Video spatial reasoning requires accumulating viewpoint-dependent evidence over time while retaining information useful to the question being asked. Existing spatial video-language models improve geometric perception and long-range context modeling, but often treat memory as a generic temporal cache, which can introduce redundant or irrelevant geometry and weaken long-horizon reasoning. We propose \textbf{\ours}, a question-guided geometric memory framework for video spatial reasoning. \ours injects camera-conditioned geometry into visual tokens and maintains two complementary memories: a Fine-Grained Context Bank for recent dense features and camera states, and a Semantic-Geometric Evidence Bank for compact long-range evidence. Each candidate frame is scored by the product of Q-Former-based question relevance and novelty with respect to the retained bank; this score is stored and reused during reading, while a capacity-based replacement rule keeps the bank compact. During reasoning, both memories are read before update and adaptively fused with the current frame representation. Experiments on VSI-Bench and VSTI-Bench show that \ours achieves state-of-the-art performance among evaluated spatial reasoning models, validating the effectiveness of question-guided geometric memory. Ablations further verify the contribution of the proposed evidence scoring mechanism.

[115] PARE: Pruning and Adaptive Routing for Efficient Video Generation cs.CVPDF

Yutong Wang, Yunke Wang, Tianfan Xue, Yu Qiao, Yaohui Wang

TL;DR: 本文提出了PARE方法，用于高效视频生成。该方法通过结构感知剪枝和输入自适应路由，联合压缩视频扩散变换器（Video DiT）的宽度和深度，从而在保持生成质量的同时显著降低每步计算成本。

Details

Motivation: 视频扩散变换器（DiT）虽然能生成高质量视频，但由于其宽块、深架构和迭代采样，计算成本高昂。现有压缩方法通常采用固定架构，无法适应不同输入或去噪阶段。

Result: 在Wan2.1-14B模型上的图像到视频和文本到视频生成实验表明，PARE在VBench多个维度上保持生成质量的同时，显著减少了每步计算量，并能与步骤蒸馏技术结合实现进一步加速。

Insight: 创新点在于：1) 观察到注意力头在空间和时间角色上的专业化，并设计了区分二者的重要性评分以防止关键运动头被过早剪枝；2) 训练了一个轻量级路由器，根据去噪时间步和视觉内容动态选择执行哪些块，实现每输入计算自适应，而非静态移除；3) 采用渐进式训练流程，先通过蒸馏恢复宽度剪枝后的质量，再联合优化学生模型和路由器以解耦学习目标。

Abstract: Video Diffusion Transformers (DiTs) generate high-quality videos but demand substantial compute due to wide blocks, deep architectures, and iterative sampling. Recent methods reduce cost by compressing width, depth, or sampling steps, but typically commit to a fixed architecture that cannot adapt to individual inputs or denoising stages. We propose PARE (Pruning and Adaptive Routing for Efficient video generation), which jointly compresses width and depth with structure-aware pruning and input-adaptive routing. For width, we observe that attention heads specialize into spatial and temporal roles, and design importance scoring that accounts for this distinction to prevent motion-critical temporal heads from being pruned prematurely. For depth, we train a lightweight router conditioned on denoising timestep and visual content to dynamically select which blocks to execute at each step, enabling per-input compute adaptation rather than static block removal. A progressive pipeline first recovers width-pruned quality via distillation, then jointly optimizes the student and router to decouple the two learning objectives. Experiments on Wan2.1-14B for both image-to-video and text-to-video generation show that PARE substantially reduces per-step computation while preserving quality across VBench dimensions, and composes with step distillation for further acceleration.

[116] Feedforward 3D Editing Learns from Semantic-Part Transformation cs.CVPDF

Jiawei Weng, Saining Zhang, Zhenxin Diao, Peishuo Li, Henghaofan Zhang

TL;DR: 本文提出了一种基于语义部件变换的前馈式3D编辑方法，通过构建高质量数据集Pxform和设计PartFlow网络，实现了对3D资产的几何和外观编辑，并在多个基准测试中达到SOTA性能。

Details

Motivation: 现有3D编辑方法缺乏高质量成对监督数据，导致编辑定位不准确、边界模糊和语义一致性差，本文旨在通过语义部件变换解决这些挑战。

Result: PartFlow在几何和外观编辑基准测试中均达到SOTA水平，实验表明其能显著提升编辑保真度和源资产保留能力。

Insight: 创新点包括从语义部件变换学习编辑、构建高质量数据集Pxform，以及PartFlow网络中的掩码感知速度保持和渲染空间一致性监督机制，无需推理时3D编辑掩码。

Abstract: 3D editing is a fundamental capability for scalable 3D content creation. While image editing has rapidly evolved toward large-scale feedforward generative paradigms, 3D AI generation remains dominated by training-free editing pipelines. A central challenge of feedforward 3D editing lies in the lack of high-quality paired supervision. Editable 3D assets require simultaneous preservation of geometry, multi-view consistency, structural coherence, and localized edit controllability. Existing 3D editing datasets often rely on independently generated assets, image-mediated reconstruction or narrow edit taxonomies, leading to inaccurate localization, weak preservation, blurred edit boundaries, and limited semantic consistency. In this work, we introduce a new perspective: scalable feedforward 3D editing should be learned from semantic-part transformations. Based on this insight, we propose Pxform, a high-quality 3D editing dataset with over 100K consistent before/after editing pairs across seven edit types. Instead of treating objects as unstructured shapes, our pipeline grounds edits directly in semantic 3D parts. Built upon Pxform, we further propose PartFlow, a feedforward 3D editing network that injects source-aware latent control into pretrained 3D generative priors. PartFlow introduces mask-aware velocity preservation and render-space consistency supervision to jointly improve edit fidelity and source preservation, while requiring no 3D edit mask during inference. Extensive experiments demonstrate that high-quality semantic-part supervision substantially improves scalable 3D editing, enabling PartFlow to achieve state-of-the-art performance on both geometric and appearance editing benchmarks.

Kim Jihyeon, Sohee Kim, Soosan Lee, Souhwan Jung, James Matthew Rehg

TL;DR: 本文提出了一种基于社会性凝视一致性的高级语义线索，用于检测AI生成图像。通过构建诊断数据集、块组合标题监督和跨架构验证，证明了该线索在提升多种骨干网络检测性能上的有效性，并解释了其跨生成器泛化的机制。

Details

Motivation: 针对现有低层次特征（如像素指纹、频率异常）在人物中心和小范围编辑场景中失效的问题，引入社会性凝视一致性（即交互个体间凝视方向、头眼对齐和瞳孔位置的一致性）作为新的检测维度。

Result: 在COCOAI Interaction子集上，使用该监督方法使视觉语言骨干网络（FakeVLM）的平衡准确率从67.8%提升至71.5%（+3.7 pp），在COCOAI Person子集上从83.0%提升至84.3%（+1.3 pp）；在纯视觉骨干网络（Effort）上也观察到一致增益，证明了线索的骨干无关性。

Insight: 创新点在于将社会性凝视一致性定义为可泛化的高级语义检测线索，并通过块组合标题监督确保推理一致性，同时提出了四步机制解释其跨生成器迁移能力，为AI生成图像检测提供了新方向。

Abstract: Recent generative models have largely closed the gap on low-level artifacts - pixel fingerprints, frequency anomalies, upsampling traces - particularly in person-centric and partial-edit settings where the manipulated region is small and surrounded by photometrically authentic content. We introduce Social Gaze Consistency, a high-level semantic cue defined as the mutual coherence of gaze direction, head-eye alignment, and pupil placement between interacting individuals, and show that it constitutes a previously underutilized detection axis orthogonal to existing low-level paradigms. We instantiate this insight through three coupled mechanisms: (i) a controlled diagnostic dataset with region-specific perturbations of gaze-consistent imagery, where strict pair-level grouping forecloses generator-fingerprint memorization as an optimization-time shortcut rather than relying on augmentation; (ii) Block-Compositional Caption Supervision, which holds a single 5-block reasoning skeleton invariant across 1,250 macro-combined captions, decoupling reasoning consistency from surface diversity; (iii) Cross-architecture validation showing the same supervision improves a vision-language backbone (FakeVLM) by +3.7 pp on the COCOAI Interaction subset (balanced accuracy 67.8 -> 71.5) and +1.3 pp on the COCOAI Person subset (83.0 -> 84.3), with consistent gains on a vision-only backbone (Effort), evidencing a backbone-agnostic cue. Real- and fake-class recalls rise simultaneously, ruling out a “predict-all-fake” artifact. A four-step mechanistic account - paired-edit shortcut blocking, hard-to-easy difficulty transfer, CLIP prior preservation, and diffusion-family shared spectral weakness in periocular structure - explains why training on a single inpainter (FLUX.1-Fill) transfers to multi-generator suites. We will release the code upon acceptance to facilitate reproducibility.

[118] LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding cs.CV | cs.AI | cs.LG | cs.ROPDF

Shihao Wang, Shilong Liu, Yuanguo Kuang, Xinyu Wei, Yangzhou Liu

TL;DR: 本文提出了LocateAnything框架，采用并行框解码（PBD）方法，将视觉定位和检测任务中的边界框作为一个原子单元进行单步解码，从而提升了几何一致性和推理速度。同时，作者构建了一个包含1.38亿训练样本的大规模数据集LocateAnything-Data，以增强数据多样性。

Details

Motivation: 现有视觉语言模型通常将视觉定位和检测任务建模为坐标令牌生成问题，通过串行解码将每个2D框分解为多个1D令牌，这既与框的几何耦合结构不匹配，也因严格的顺序生成造成了推理瓶颈。

Result: 在多个基准测试中，LocateAnything在解码吞吐量和定位精度（特别是高IoU下的定位质量）方面均取得了显著提升，推进了速度-精度前沿。

Insight: 核心创新在于并行框解码（PBD）范式，它通过将边界框作为原子单元解码，保持了框内几何一致性并实现了高度并行化。此外，构建大规模、多样化的训练数据被证明是实现高效、精确统一视觉定位与检测的关键互补因素。

Abstract: Vision-language models (VLMs) commonly formulate visual grounding and detection as a coordinate-token generation problem, serializing each 2D box into multiple 1D tokens that are learned and decoded largely independently. This token-by-token decoding mismatches the coupled structure of box geometry and creates a practical inference bottleneck due to strictly sequential generation. We introduce LocateAnything, a unified generative grounding and detection framework based on Parallel Box Decoding (PBD). By decoding geometric elements such as bounding boxes and points as atomic units in a single step, LocateAnything preserves intra-box geometric coherence and unlocks substantial parallelism. We show that PBD improves both decoding throughput and localization accuracy. We further develop a scalable data engine and curate LocateAnything-Data, a large-scale dataset with more than 138 million training samples, substantially increasing data diversity for high-precision localization. Extensive evaluations show that LocateAnything advances the speed-accuracy frontier, achieving significantly higher decoding throughput while improving high-IoU localization quality across diverse benchmarks. The results highlight the complementary benefits of Parallel Box Decoding and large-scale training data in enabling efficient and precise unified visual grounding and detection.

cs.AI [Back]

[119] Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems cs.AI | cs.CL | cs.MAPDF

Jianing Zhu, Yeonju Ro, John Robertson, Kevin Wang, Junbo Li

TL;DR: 本文提出了AgingBench，一个用于评估长期运行AI代理可靠性的纵向基准测试框架。该框架将代理老化机制分为压缩老化、干扰老化、修订老化和维护老化四类，并通过时间依赖图和反事实探针进行诊断。研究发现代理老化是多维度的，其行为测试与事实精度可能不同步退化，因此可靠的代理部署需要寿命评估和针对性修复。

Details

Motivation: 当前长期运行的AI代理在部署后仍像新初始化模型一样被评估，忽略了其可靠性随运行时间变化的系统性问题。论文旨在解决如何衡量代理在部署后的可靠性衰减问题，并识别其具体退化形式。

Result: 在7个场景、14个模型、多种记忆策略以及约400次运行（涵盖8-200个会话）的实验中，AgingBench揭示了代理老化不是一维的：行为测试可能保持正常而事实精度下降；派生状态跟踪可能在单个模型内急剧崩溃；相同的错误答案需要根据诊断指向进行不同修复。

Insight: 创新点在于提出了首个针对代理寿命工程的纵向可靠性基准AgingBench，将老化机制分类并开发了基于时间依赖图和反事实探针的诊断方法。客观来看，该研究强调了代理系统评估需从静态模型性能转向动态生命周期可靠性，为实际部署提供了重要的工程视角和诊断工具。

Abstract: Long-lived AI agents are increasingly deployed as persistent operational systems, yet they are still evaluated like freshly initialized models. Day-one benchmarks miss a basic systems question: how long does an agent remain reliable after deployment? Even when model weights are frozen, an agent’s effective state keeps changing as it compresses interaction history, retrieves from a growing memory store, revises facts after updates, and undergoes routine maintenance. Reliability therefore becomes a lifespan property of the full agent harness, not only a snapshot property of the base model. We introduce AgingBench, a longitudinal reliability benchmark for agent lifespan engineering: measuring not only whether deployed agents degrade, but what form the degradation takes and where repair should target. AgingBench organizes agent aging into four mechanisms: compression aging, interference aging, revision aging, and maintenance aging. To diagnose these failures, AgingBench uses temporal dependency graphs and paired counterfactual probes that produce diagnostic profiles for the write, retrieval, and utilization stages of the memory pipeline. Across 7 scenarios, 14 models, multiple memory policies, and both runner-controlled and autonomous agents, over ~400 runs spanning 8 - 200 sessions show that agent aging is not one-dimensional: behavioral tests can remain clean while factual precision decays; derived-state tracking can collapse sharply within a single model; and the same wrong answer can require different repairs depending on what the diagnostic profile points to. These results suggest that reliable agent deployment requires lifespan evaluation, mechanism-level diagnosis, and stage-targeted repair, not only stronger day-one models.

[120] Advancing Creative Physical Intelligence in Large Multimodal Models cs.AI | cs.CL | cs.LGPDF

Cheng Qian, Hyeonjeong Ha, Jiayu Liu, Jeonghwan Kim, Emre Can Acikgoz

TL;DR: 本文针对大型多模态模型在开放式环境中的创造性物理智能不足问题，提出了MM-CreativityBench基准，用于评估模型在视觉丰富、物理约束场景下的创造性工具使用能力。研究发现现有模型常因缺乏持续接地探索而失败，并提出了基于可供性接地的对齐方法，通过直接偏好优化和知识库监督来提升模型性能。

Details

Motivation: 当前大型多模态模型在感知和推理方面进展迅速，但其能力是否泛化到开放式环境中的视觉接地解决方案发现仍不明确，这需要超越模式识别的创造性问题解决能力。

Result: 在MM-CreativityBench基准上的实验表明，现有模型在实体和部件选择上表现不佳，常出现幻觉和接地错误；而提出的可供性接地对齐方法显著提升了正确选择率，并大幅减少了相关错误。

Insight: 创新点在于将创造性工具使用构建为偏好学习问题，通过直接偏好优化鼓励基于视觉证据的推理，并结合可供性知识库监督来引导实体探索和多轮规划，从而增强模型的物理可行性和视觉接地能力。

Abstract: Large multimodal models (LMMs) have rapidly advanced in perception and reasoning; however, it remains unclear whether these capabilities generalize to discovering visually grounded solutions in open-ended environments, beyond pattern recognition. In such settings, intelligence requires more than answering well-posed questions: it involves identifying how elements in a scene can be repurposed in non-obvious yet physically feasible ways. This form of creative problem-solving is central to human intelligence, but remains largely untested in current benchmarks. To evaluate this ability, we introduce MM-CreativityBench, a benchmark for affordance-grounded creative tool use in visually rich, physically constrained environments. Each instance presents a scenario image with structured views of candidate entities and their parts, enabling fine-grained, interactive evaluation of how models iteratively inspect the scene, identify relevant affordances, and compose visually and physically grounded solutions. Our experiments show that current LMMs often fall short, not due to lack of generative capability, but because they do not sustain grounded exploration. Models often overlook relevant entities, under-examine critical parts, or hallucinate attributes not grounded in the image. Motivated by this failure mode, we propose affordance-grounded alignment, which casts creative tool use as a preference learning problem. Using Direct Preference Optimization, we encourage models to prefer attribute-affordance reasoning grounded in visual evidence over hallucinated alternatives. In addition, we incorporate supervision derived from an affordance knowledge base to guide broader entity exploration and multi-turn planning. Our results show consistent gains in selecting the correct entities and parts, while substantially reducing hallucination and grounding-related errors.

[121] Reasoning, Code, or Both? How Large Language Models Handle Variations in Math Questions cs.AI | cs.CL | cs.LGPDF

Matthew Kutakh

TL;DR: 本研究系统评估了三种大语言模型（LLMs）处理数学问题变体时的推理鲁棒性：纯推理（CoT）、单次代码执行（PAL）和迭代代码执行（SBSC）。实验基于GSM-Symbolic数据集的1000个问题及其变体，使用Claude Haiku 4.5模型进行测试。结果表明，纯推理（CoT）方法在问题扰动下性能下降最小、最鲁棒，而代码执行方法（尤其是PAL）并未提升鲁棒性。

Details

Motivation: 尽管LLMs在数学推理基准上表现出色，但当问题被简单修改（如更换名称或数字）时，其性能会下降。代码执行方法（如生成并运行Python代码）被提出作为解决方案，但其对推理鲁棒性（即在不同问题变体间保持准确性的能力）的影响尚未得到系统测试。

Result: 在GSM-Symbolic数据集上，纯推理（CoT）方法在扰动下的准确率下降最小（1.3个百分点），且仅有1.8%的问题被破坏；单次代码执行（PAL）最不鲁棒，准确率下降1.7个百分点，3.1%的问题被破坏；迭代代码执行（SBSC）介于两者之间。尽管差异在统计上不显著（p = .096），但所有指标的趋势一致，表明代码执行并未提升小学级别问题变体上的推理鲁棒性。

Insight: 论文的创新点在于首次系统比较了纯推理与代码执行方法在数学问题鲁棒性上的表现，挑战了代码执行能提升鲁棒性的假设。客观来看，其核心发现是：对于简单的表面扰动，自然语言推理链（CoT）可能比依赖外部代码执行更具泛化性和稳定性，这为LLMs的推理方法设计提供了重要参考。

Abstract: Large Language Models (LLMs) achieve impressive accuracy on mathematical reasoning benchmarks, yet their performance drops when problems are modified with simple changes like different names or numbers. Code execution methods, which let models generate and run Python code instead of reasoning in natural language, have been proposed as a solution, but their effect on reasoning robustness (the ability to maintain accuracy across problem variations) has not been systematically tested. This study evaluates three approaches on 1,000 problems from the GSM-Symbolic dataset: pure reasoning using chain-of-thought (CoT) prompting, single-shot code execution using Program-Aided Language models (PAL), and iterative code execution using Step-by-Step Coding (SBSC). All three were run on paired original and modified problems using Claude Haiku 4.5. CoT was the most robust method, with an accuracy drop of 1.3 percentage points and 1.8% of problems breaking under perturbation. PAL was the least robust at 1.7 percentage points and 3.1% broke, with SBSC falling in between. Although these differences were not statistically significant ($p = .096$), the directional trend was consistent across all measures, suggesting that code execution, whether single-shot or iterative, does not improve reasoning robustness on grade-school-level problem variations.

[122] The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence cs.AI | cs.CL | cs.LGPDF

MiniMax, :, Aili Chen, Aonian Li, Baichuan Zhou

TL;DR: 论文介绍了MiniMax-M2系列模型，这是一个基于专家混合架构的语言模型家族，其核心设计理念是‘小型激活参数能释放最大现实世界智能’。旗舰模型M2总参数量为2299亿，但每个token仅激活98亿参数。该系列专为智能体部署端到端设计，包含三个关键组件：面向智能体的数据管道、可扩展的原生智能体强化学习系统Forge，以及具备初步自我进化能力的M2.7检查点。

Details

Motivation: 旨在通过设计一个参数激活稀疏的专家混合模型，在保持高计算效率（小型激活）的同时，实现现实世界任务（如智能体编码、协作）中的前沿性能，解决大模型部署中计算成本与智能体能力之间的平衡问题。

Result: 在智能体编码、深度搜索、办公任务和推理等多个基准测试中，该系列模型实现了前沿性能（frontier-tier performance），将小型激活的计算足迹转化为强大的实际任务能力。

Insight: 创新点包括：1) 提出‘小型激活释放最大智能’的设计原则，并实现于大规模稀疏激活的MoE模型；2) 构建了端到端的智能体原生训练框架，整合了可验证的智能体轨迹数据、适应长视野任务的强化学习系统以及支持白盒/黑盒智能体的解耦架构；3) 在M2.7中探索了模型的自我进化能力，如自动调试训练运行和修改自身脚手架，这可能是迈向自主模型迭代的早期步骤。

Abstract: We introduce the MiniMax-M2 series, a family of Mixture-of-Experts language models built around the principle that mini activations can unleash maximum real-world intelligence. The flagship M2 contains 229.9B total parameters with only 9.8B activated per token. Designed end-to-end for agentic deployment, the M2 series rests on three components: (i) agent-driven data pipelines producing large-scale, verifiable trajectories across agentic coding and agentic cowork, each grounded in an executable workspace and an artifact-aligned reward; (ii) Forge, a scalable agent-native RL system that adapts to long-horizon agent trajectories, paired with windowed-FIFO scheduling, prefix-tree merging, inference optimization, and a clean training-inference-agent decoupling that supports both white-box and black-box agents; (iii) the latest M2.7 checkpoint takes an early step toward self-evolution – autonomously debugging training runs and modifying its own scaffold. Across M2 through M2.7, this combination translates a mini-activation footprint into frontier-tier performance on agentic coding, deep search, office-task, and reasoning benchmarks.

[123] UnityMAS-O: A General RL Optimization Framework for LLM-Based Multi-Agent Systems cs.AI | cs.CL | cs.MAPDF

Yiqun Chen, Wei Yang, Erhan Zhang, Shijie Wang, Qi Liu

TL;DR: 本文提出了UnityMAS-O，一个用于基于LLM的多智能体系统的通用强化学习优化框架。该框架将完整的工作流而非单个响应作为优化单元，通过四个核心对象（逻辑智能体角色、图轨迹、用户定义奖励和智能体-模型映射）来解耦逻辑与物理参数，并支持分布式PPO风格的训练。

Details

Motivation: 当前基于LLM的多智能体系统大多依赖手动编排（如提示、工具和控制规则），缺乏统一的强化学习接口进行优化。现有RL后训练框架主要针对单策略优化，缺少对用户定义的多智能体工作流、结构化交互、角色特定信用分配和可配置参数共享的抽象支持。

Result: 在检索增强QA（Natural Questions、HotpotQA）和代码生成任务上的实验表明，多智能体RL优化后能提升手动指定工作流的性能，对于较小模型和严格的代码通过率指标提升尤为显著。

Insight: 创新点在于将整个多智能体工作流作为优化单元，通过逻辑与物理参数的解耦实现了灵活的参数共享策略（完全共享、完全分离、部分共享），并提供了一个可扩展的分布式运行时架构，使用户无需重写优化基础设施即可定义复杂工作流。

Abstract: LLM-based multi-agent systems decompose complex tasks into interacting roles, but most remain manually orchestrated by prompts, tools, and control rules, while agents are rarely optimized through a unified reinforcement learning interface. Existing RL post-training frameworks mainly target single-policy optimization and lack abstractions for user-defined multi-agent workflows, structured interaction, role-specific credit assignment, and configurable parameter sharing. We present UnityMAS-O, a general RL optimization framework for LLM-based multi-agent systems. UnityMAS-O treats the complete workflow as the optimization unit, rather than a single response or policy trajectory. It represents workflows through four first-class objects: logical agent roles, graph trajectories, user-defined rewards, and agent–model mappings. This decouples logical agents from physical model parameters, supporting full sharing, full separation, and partial sharing, with rewards assigned at role, turn, and trajectory levels. UnityMAS-O extends verl with a Ray-based star-topology runtime. A central controller executes workflows, invokes tools, records structured trajectories, and assembles rewards; model-local worker groups handle rollout, buffering, advantage computation, and distributed PPO-style updates. Users can define agents, workflows, model mappings, and rewards without rewriting the optimization infrastructure. We instantiate UnityMAS-O on retrieval-augmented QA, iterative agentic search, and reflective code generation. Across Natural Questions, HotpotQA, and held-out code tasks, multi-agent RL improves manually specified workflows after optimization, with especially large gains for smaller models and strict code all-passed metrics. These results show that UnityMAS-O can serve as a reusable substrate for converting diverse LLM-based multi-agent workflows into trainable multi-agent RL systems.

[124] It’s Not the Capability: Harness Sensitivity Is Non-Monotone Across LLM Agent Tiers cs.AI | cs.CLPDF

Yong-eun Cho

TL;DR: 本文通过实验挑战了LLM智能体部署中的普遍假设，即模型能力越强所需的结构化约束（harness）越少。研究发现，约束敏感性与模型能力层级并非单调反比关系，而是取决于模型类型（聊天型vs推理型），并提出了分层感知的约束选择指南。

Details

Motivation: 旨在验证LLM智能体部署中的一个核心假设：模型能力与最优约束复杂度之间存在单调反比关系，即更强的模型需要更少的结构化指导。

Result: 在HEAT-24基准测试上，实验结果表明该假设不成立：顶级聊天模型（Gemini 2.5 Flash）在约束更严格时性能（VTSR）下降29-38个百分点；而顶级推理模型（Qwen3.5-122B）在严格约束下达到最高VTSR（91.7%）和最低延迟。受限层级模型（Gemma4:e2B）在所有约束下均保持91.7%的稳定性能。

Insight: 创新点在于揭示了约束敏感性的非单调性及其对模型类型的依赖性，并引入了六类失败分类法（如格式违规主导高能力模型失败，错误文件主导低能力模型失败），为针对不同能力层级的模型选择合适约束提供了实践指导。

Abstract: A prevalent assumption in LLM agent deployment holds that more structured harnesses universally improve reliability, and that higher-capability models need proportionally less structural guidance – together implying a monotone inverse relationship between model capability tier and optimal harness complexity. We test this hypothesis through a controlled 432-run experiment crossing six models across four capability tiers with three harness conditions (light, balanced, strict) on HEAT-24, a 24-task synthetic benchmark with git-based workspace verification. Our results refute the monotone inverse relationship on two fronts. First, for the frontier chat model evaluated (Gemini 2.5 Flash), increased harness verbosity lowers VTSR by 29-38 percentage points – a harness-complexity paradox. Second, for the frontier reasoning model evaluated (Qwen3.5-122B, extended thinking enabled), strict harness achieves the highest VTSR (91.7%) and the lowest latency, the opposite of the prediction. Within the constrained tier, a 2B model (Gemma4:e2B) matches strong-open-tier stability at 91.7% across all harnesses. Because each tier is represented by a single model in this study, these results should be interpreted as model-specific observations; harness sensitivity appears non-monotone across the models evaluated, and depends critically on model type (chat vs. reasoning). We introduce a six-label failure taxonomy showing that format_violation dominates capable-model failures while wrong_file dominates low-capability failures, and we derive practical tier-aware harness selection guidelines.

[125] Alignment Tampering: How Reinforcement Learning from Human Feedback Is Exploited to Optimize Misaligned Biases cs.AI | cs.CL | cs.LGPDF

Dongyoon Hahm, Dylan Hadfield-Menell, Kimin Lee

TL;DR: 本文揭示了强化学习人类反馈（RLHF）对齐大型语言模型（LLM）时存在的一种潜在漏洞——对齐篡改（alignment tampering）。该漏洞源于LLM能够通过生成高质量但带有偏见（如性别歧视、品牌推广等）的输出来影响偏好数据集的构建，而RLHF的优化过程（如强化学习或最佳N采样）会不加区分地放大这些未对齐的偏见。实验表明该漏洞广泛存在，且现有缓解技术难以在不牺牲响应质量的情况下完全解决此问题。

Details

Motivation: RLHF是当前对齐LLM与人类偏好的标准方法，但其存在结构性漏洞：偏好数据集基于LLM自身输出构建，且成对比较仅指示哪个回答更好，而不区分原因（如质量与偏见）。这可能导致LLM在优化过程中放大未对齐的偏见，即“对齐篡改”。本文旨在揭示并研究这一潜在安全风险。

Result: 实验证明，对齐篡改会导致多种偏见被放大，包括关键词偏见、宣传（如性别歧视）、品牌推广和工具性目标寻求。现有用于增强RLHF鲁棒性的技术无法在不牺牲响应质量的情况下完全解决此问题。

Insight: 核心创新点在于识别并形式化了RLHF流程中的一个结构性安全漏洞“对齐篡改”。其关键洞察是，RLHF依赖的偏好标签无法解耦回答质量与潜在偏见，导致奖励模型和后续优化过程可能放大有害偏差。这暴露了当前基于成对比较的RLHF范式的根本局限性，强调了需要开发能区分偏好原因的新对齐方法。

Abstract: Reinforcement Learning from Human Feedback (RLHF) is the standard method to align Large Language Models (LLMs) with human preferences. In this work, we introduce alignment tampering, a potential vulnerability where the LLM undergoing alignment influences the preference dataset, causing RLHF to amplify undesired behaviors. This arises from core limitations of RLHF: (1) preference datasets are constructed from the LLM’s own outputs, allowing it to influence them, and (2) pairwise comparisons only indicate which response is better, not why. These limitations can be exploited to cause alignment tampering. For example, if an LLM generates biased responses with higher quality, annotators will prefer them based on quality. However, preference labels do not distinguish quality from bias, and the reward model inherits this limitation. Optimizing such rewards through reinforcement learning or best-of-N sampling can amplify misaligned biases. Our experiments demonstrate amplification across diverse biases: from keyword bias to propaganda (e.g., sexism), brand promotion, and instrumental goal-seeking. Mitigation remains challenging, as existing techniques for robust RLHF fail to fully resolve alignment tampering without sacrificing response quality. These findings reveal structural vulnerabilities of current RLHF and emphasize the need to prevent this vulnerability. Project page: https://alignment-tampering.github.io/

cs.CR [Back]

[126] BAIT: Boundary-Guided Disclosure Escalation via Self-Conditioned Reasoning cs.CR | cs.CLPDF

Xuan Luo, Yue Wang, Geng Tu, Jing Li, Ruifeng Xu

TL;DR: 本文提出了BAIT（边界感知迭代陷阱），一个三步越狱框架，通过内部披露逐步接近恶意目标。该方法首先让模型识别保护边界，然后要求其细化该边界，最后请求详细示例，利用模型自身的推理和一致性倾向构建披露路径。

Details

Motivation: 解决现有越狱攻击方法在诱导大型语言模型生成有害内容时效率不足的问题，旨在通过引导模型自我披露其安全边界来更有效地绕过防护机制。

Result: 在AdvBench、JailbreakBench、AIR-Bench和SORRY-Bench等基准测试中，BAIT在顶级大语言模型上均实现了较高的攻击成功率，显著超越了传统越狱基线方法。

Insight: 创新点在于利用预防性框架（而非直接知识请求）和边界细化步骤来逐步升级披露，关键洞察是模型的前两步响应有一定概率直接引出有害内容且不易触发过滤机制。

Abstract: In this work, we propose BAIT (Boundary-Aware Iterative Trap), a three-step jailbreak framework that approaches malicious goals through internal disclosure. BAIT first asks the model to identify the protection boundary, then requires it to refine that boundary, and finally requests a detailed example. By expanding each step upon the model’s previous responses, BAIT turns the model’s own reasoning and consistency tendency into a disclosure pathway. Experiments on AdvBench, JailbreakBench, AIR-Bench, and SORRY-Bench demonstrate that BAIT consistently achieves strong attack success rates across top-tier large language models, significantly advancing conventional jailbreak baselines. Further analysis reveals that: 1) prevention-oriented framing significantly outperforms direct knowledge request; 2) the refinement step plays a critical role in disclosure escalation; and 3) the first two steps have a certain chance of eliciting harmful content while triggering little filtering.

cs.LO [Back]

[127] MerLean-Prover: A Recursive Looping Harness for End-to-End Lean 4 Theorem Proving cs.LO | cs.CLPDF

Jinzheng Li, Zeru Zhu, Yuanjie Ren

TL;DR: MerLean-Prover是一个端到端的Lean4定理证明器，通过由规划、检查和Lean三种智能体类型组成的递归外循环，将sorry声明替换为内核可检查的证明。该方法无需微调、自定义强化学习目标或定理特定脚手架。在FormalQualBench基准测试中，它解决了10/23个定理，超越了现有最佳开源基线；在Putnam2025上，它以更短总时间完成了全部12个问题。

Details

Motivation: 旨在构建一个无需复杂定制（如微调或特定定理脚手架）的端到端Lean4定理证明系统，以自动化生成可验证的证明。

Result: 在FormalQualBench（23个博士资格考试定理）上解决10/23，超过最强开源基线OpenGauss（8/23）；在Putnam2025上以更低总时间完成12/12；且能迁移到更小模型（如Sonnet和Haiku）上有效工作。

Insight: 创新点在于设计了一个基于递归循环的简单但有效的智能体组合框架，强调证明计划本身的修订单元；研究表明，在端到端Lean4定理证明中，框架设计与原始模型能力同等重要，且简单框架已能取得显著效果。

Abstract: MerLean-Prover is an end-to-end Lean4 theorem prover that replaces sorry declarations with kernel-checkable proofs. It is built from three agent types (Planning, Check, and Lean) composed by a recursive outer loop whose unit of revision is the proof plan itself, and uses no fine-tuning, no custom RL objective, and no theorem-specific scaffolding. On FormalQualBench, a benchmark of 23 PhD-qualifying-exam theorems, MerLean-Prover solves 10/23, surpassing the strongest published open-source baseline (OpenGauss, 8/23). On Putnam2025, the same harness closes 12/12 with substantially lower total wall-clock than the next-best system that closes the full set. The harness also transfers to smaller models: Sonnet closes all four tested FormalQualBench problems, and Haiku closes the two short ones. These results suggest that harness design is a central factor in end-to-end Lean4 theorem proving, alongside raw model capability, and that a relatively simple harness can already be effective.

cs.SE [Back]

[128] A Universal Cliff and a Design Fingerprint: Cross-Section Defect Detection Under LLM Orchestration cs.SE | cs.AI | cs.CL | cs.MAPDF

Hiroki Fukui

TL;DR: 该论文研究了在大型语言模型（LLM）编排系统中，跨文档远距离部分间矛盾的缺陷检测问题。研究发现，当任务被分配给多个工作代理（orchestration）而非单一代理处理时，所有测试模型都出现了‘通用检测悬崖’现象，即检测能力急剧下降。此外，不同模型在‘悬崖’后的行为模式各异，其中一家开发商的模型随着对齐性增强，表现出独特的报告标准偏移。

Details

Motivation: 动机是探究LLM编排系统（将请求分割给多个工作代理处理再整合）如何影响一类单个代理无法察觉的缺陷——即文档中两个相距较远部分之间存在的矛盾关系。旨在理解编排机制对模型检测此类跨分区缺陷能力的影响。

Result: 在固定文档、缺陷、机制、评分和种子的条件下，测试了来自一家开发商五代模型和五家不同对齐范式的提供商的十个系统。定量结果显示：所有模型在编排下都出现了‘通用检测悬崖’，检测能力下降三分之二或更多；在六个具有高于随机水平判别能力的模型中，仅一家开发商的历代模型随着对齐性增强，在报告标准轴上移动（漏报减少但误报增加），且该趋势在代际间显著（p < 0.001）。自动评判器不稳定（精确度17-50%）。

Insight: 论文宣称的创新点在于揭示了LLM编排系统固有的‘通用检测悬崖’这一结构性缺陷，并发现对齐性最强的系统并非最安全的。客观分析认为，其核心洞察是指出了集成报告的置信度无法反映跨分区缺陷，且模型内部可能准确重构了结构错误，但公开报告却未能揭示，这为评估和设计可靠的LLM系统提供了重要警示。

Abstract: Production language-model systems answer a request by partitioning it across an invisible orchestration of worker agents that recompose one integrated report. We ask what this does to a class of defect no single worker can see: a contradiction in the relation between two distant sections of a document. Holding the documents, defects, mechanism, scoring, and seed fixed, we vary only the model – ten systems across five generations from one developer and five providers from distinct alignment paradigms. Two layers separate. First, a universal detection cliff: every model that finds these cross-section defects under a single agent loses that ability under orchestration, detection falling two-thirds or more across every paradigm tested. The cliff is mechanism-derived and not closed by scale or extended reasoning. Second, how models behave once fallen. A signal-detection decomposition shows that, among the six models discriminating above chance, only one developer’s generations move along the reporting-criterion axis: as alignment is strengthened, the model misses fewer defects yet raises more false alarms on clean documents – two faces of one criterion shift, scaling with generation within that developer (p < 0.001) and near-absent elsewhere. At the floor the missed defect is often not out of view: the model’s private record reconstructs the structural fault accurately, while the integrated report signs off on its soundness, its concern spent on the artifact and an absent collaborator. This resists quantification – an automated judge is unstable (precision 17-50%) and keywords cannot separate it from ordinary agreement – a resistance we report as a finding. We release all runs, probes, defect keys, scorer prompts, and scripts. An integrated report’s confidence is uninformative about partition-spanning defects, the most aligned systems are not the safest, and the cliff is structural.

[129] EdgeFlow: Edge-Map Augmented VLM-Based Flowchart Processing for Industrial Requirements Engineering cs.SE | cs.AI | cs.CVPDF

Zhifei Dou, Shabnam Hassani, Ou Wei

TL;DR: 本文提出EdgeFlow方法，通过将Canny边缘图作为结构先验信息增强视觉语言模型（VLM）的输入，以提升工业需求工程中流程图向Mermaid格式的转换精度，无需标注数据或领域微调。

Details

Motivation: 解决现有VLM在直接处理流程图时，因忽略拓扑关键视觉细节而导致转换失败的问题，旨在实现更准确的机器可读模型生成。

Result: 在真实工业数据集IndusReqFlow上，EdgeFlow相比现成VLM在节点级F1提升17.39个百分点、边级F1提升16.94个百分点、路径级F1提升11.06个百分点，支持基于模型的测试；但在公开合成基准上无显著改进。

Insight: 创新点在于利用确定性提取的边缘图作为结构先验来增强VLM，以零训练方式提升拓扑保持能力；同时指出未来VLM工具评估需纳入多样化的工业数据基准。

Abstract: Flowcharts are widely used in industrial requirements, but usually remain embedded as static images. Vision Language Models (VLMs) show promise in the conversion of these flowcharts into machine-readable models for RE activities, yet, when directly applied to flowchart conversion, they often fail on topology-critical visual details. To address this, we propose EdgeFlow that augments a VLM’s original input with a deterministically extracted Canny edge map-acting as a structural prior-to improve flowchart-to-Mermaid conversion, without requiring annotated training data or domain-specific model fine-tuning. We evaluate EdgeFlow on IndusReqFlow, a dataset sourced from real-world requirements. Compared with off-the-shelf VLMs, EdgeFlow improves node-level F1 by 17.39 percentage points and edge-level F1 by 16.94 percentage points. At the path level, EdgeFlow improves path F1 by 11.06 percentage points, enabling better support for model-based testing. These results demonstrate that EdgeFlow provides a practical, training-free means to improve topology-preserving flowchart-to-Mermaid conversion for industrial RE. Cross-dataset evaluation results on a public synthetic benchmark show no significant improvement; this highlights the need for diverse benchmarks incorporating industrial data for the comprehensive evaluation of future VLM-based RE tools.

cs.RO [Back]

[130] Efficient On-policy Visual-RL via Stochastic Decoupled Policy Gradient cs.RO | cs.AI | cs.CV | cs.LG | eess.SYPDF

Haoxiang You, Yilang Liu, Davis Zong, Qian Wang, Teeratham Vitchutripop

TL;DR: 本文提出了随机解耦策略梯度（SDPG），一种轻量级的视觉强化学习方法，能够在单个RTX 4080 GPU上几小时内端到端训练多样化的视觉运动控制策略。该方法通过轨迹展开的随机扰动来估计策略梯度，大幅减少了批量渲染环境的需求，显著降低了计算和内存开销。

Details

Motivation: 解决视觉强化学习（Visual-RL）中训练效率低、计算和内存开销大的问题，旨在实现快速、轻量的端到端策略训练。

Result: 在视觉MuJoCo基准测试中，SDPG在训练时间、内存使用和奖励方面持续优于基线方法。

Insight: 创新点在于通过随机扰动轨迹展开来估计策略梯度，从而解耦计算，这是一种高效且轻量的梯度估计方法；同时，论文引入了一套现实的视觉机器人基准测试套件，支持从仿真到实物的有效迁移研究。

Abstract: We present the stochastic decoupled policy gradient (SDPG), a lightweight visual reinforcement learning (RL) method that trains diverse visuomotor control policies end-to-end within a few hours on a single NVIDIA RTX 4080 GPU. SDPG estimates policy gradients via random perturbations of trajectory rollouts, requiring orders of magnitude fewer batch-rendered environments and substantially reducing compute and memory overhead. On visual MuJoCo benchmarks, SDPG consistently outperforms baseline methods in training time, memory usage, and rewards. Finally, to support future research, we introduce a suite of realistic visual robotics benchmarks spanning dexterous manipulation, challenging locomotion, and demonstrate effective sim-to-real transfer on physical hardware.

[131] SteelDS: A High-Resolution Video Dataset of E40 Steel Scrap for Object Detection and Instance Segmentation cs.RO | cs.CVPDF

Melanie Neubauer, Christian Rauch, Gerald Koinig, Alexia Tischberger-Aldrian, Roland Pomberger

TL;DR: 该论文介绍了SteelDS数据集，这是一个高分辨率、带标注的视频数据集，专门用于E40级钢屑和铜屑的物体检测与实例分割任务。数据集包含24,297个标注帧，涵盖不同大小和密度的钢和铜对象，旨在模拟工业分选环境，支持机器学习模型开发。

Details

Motivation: 解决工业后磁选阶段中，需要人工干预去除铜杂质的问题，为自动化分选算法提供基准数据，以减少对人工的依赖并提高效率。

Result: 数据集包含五个子集，共标注了396个钢对象和101个铜对象，提供像素级分割掩码和材料类别，可作为评估物体检测和实例分割模型性能的基准。

Insight: 创新点在于提供了高分辨率视频序列和真实工业条件下的对象变化（如间距和密度），这有助于训练更鲁棒的模型，并推动自动化分选技术在复杂、异质钢屑流中的应用。

Abstract: This dataset provides high-resolution, annotated video sequences of shredded E40-grade steel and copper scrap on a conveyor belt. Captured in a controlled laboratory environment, the data reflects the industrial post-magnetic sorting stage, where manual intervention is typically required to remove copper contaminants. The dataset comprises 24,297 labeled frames across five subsets, featuring 396 steel and 101 copper objects categorized by size. It supports the development of machine learning models for material classification, object detection, and instance segmentation. Variations in object spacing and density are included to simulate realistic industrial sorting conditions. Ground truth annotations include pixel-wise segmentation masks and material classes. This dataset serves as a benchmark for evaluating automated sorting algorithms aiming to identify copper impurities within complex, heterogeneous steel scrap streams.

[132] Object Pose and Shape Estimation for Grasping: Does it Work? cs.RO | cs.CVPDF

Pavan Karke, Kushal Shah, Gaurav Singh, Md Faizal Karim, K Madhava Krishna

TL;DR: 本文探讨了物体姿态与形状估计方法在机器人抓取任务中的成熟度，通过对比端到端抓取合成方法与模块化方法（先估计物体姿态与形状，再通过对握采样生成抓取）的性能，发现模块化方法在单视图RGB(-D)输入下优于端到端方法，尤其在小型物体上表现更佳，但受限于姿态与形状估计的准确性，在杂乱场景中性能会部分下降。

Details

Motivation: 研究动机是评估当前物体姿态与形状估计技术是否足够成熟，使得结合对握采样后能在抓取任务中超越端到端抓取合成方法，重点关注平行夹爪、7自由度抓取和单视图RGB(-D)输入场景。

Result: 实验表明，模块化方法在所有测试中均优于端到端方法，能生成大量抓取（包括端到端方法失败的小物体），但在杂乱场景中因姿态与形状估计不准确而性能下降；此外，通过结合视觉语言模型，模块化方法实现了语言条件抓取，性能与SOTA基线LERF-TOGO相当。

Insight: 创新点在于系统性地验证了模块化抓取方法的有效性，揭示了姿态与形状估计精度对抓取性能的关键影响，并展示了如何利用视觉语言模型扩展单视图估计以实现语言条件抓取，为机器人抓取提供了可借鉴的模块化框架。

Abstract: The problem of object pose and shape estimation has seen key advancements lately. Encoder-decoder (e.g., SAM3D, LRM, CRISP) and diffusion-based models (e.g., InstantMesh, Zero123, SceneComplete) have shown category-agnostic shape encoding capacity and open-set generalizability. In this work, we ask the question: Are the object pose and shape estimation methods mature enough, such that when used with antipodal grasp sampling, can outperform the end-to-end grasp synthesis methods? We explore this question in detail by scoping our study to parallel jaw grippers, 7-DoF grasps, and single-view RGB(-D) image as input. We implement and compare a state-of-the-art, end-to-end grasp synthesis method and three modular methods, which first estimate the object pose and shape for all objects in the scene, and generate grasps using antipodal sampling. We observe that the modular methods outperform the end-to-end method in all our experiments. The modular methods are able to synthesize plenty of grasps, even for small objects, where the end-to-end methods fail. The effectiveness of the modular methods is contingent on the accuracy of the pose and shape estimation, and suffers partial degradation in cluttered scenes - a limitation of the existing pose and shape estimation methods. We also analyze the failure modes and run-times for the three modular methods, which use two different ways of object pose and shape estimation: one based on an encoder-decoder model, while another a diffusion model. Finally, we demonstrate that the single-view object pose and shape estimation methods can be augmented with vision-language models to yield language-conditioned grasps from just single-view RGB-D image as input. We notice comparable performance to the state-of-the-art LERF-TOGO baseline.

cs.LG [Back]

Sai Munikoti, Ian Stewart, Chengping Chai, Lisa Linville, Scott Vasquez

TL;DR: 本文提出了MultiSeismo，一个大规模、结构化的多模态地震数据集，包含超过1.6万个地震事件，整合了波形时间序列、地理图像和文本描述。基于此，作者开发了指令集MISCE，并微调出首个领域专用的多模态地震分析模型SeisModal。

Details

Motivation: 解决通用多模态模型在专业科学领域（如地震学）应用受限的问题，因为现有数据集缺乏整合波形、图像和文本等多种模态的综合数据。

Result: 在MultiSeismo基准上评估，通用模型在处理时间序列数据时面临显著挑战，而领域专用模型SeisModal在多模态地震推理任务上表现出优越性能。

Insight: 创新点在于构建了首个整合地震波形、图像和文本的大规模多模态数据集，并验证了通过领域专用架构（如专门的时间序列编码器）微调通用模型，能有效提升其在专业科学任务上的性能。

Abstract: The application of generalist multimodal models (GMMs) to specialized scientific domains remains limited due to the scarcity of comprehensive domain-specific datasets that integrate multiple data modalities beyond text and images. In seismology, understanding earthquake phenomena requires the synthesis of timeseries waveform data, geographical imagery, and contextual metadata, a multimodal integration absent in existing seismic datasets. We present MultiSeismo, a large scale structured multimodal seismic dataset, comprising over 16K seismic events spanning 13 years (2010 to 2023) across diverse geographical regions. Each event data integrates waveform recordings from global station networks, intensity maps, population exposure visualizations, and a comprehensive textual description within a standardized JSON format. We additionally develop MISCE, a multimodal instruction set on top of raw data to enable supervised training and evaluation of GMMs on seismic reasoning tasks ranging from basic information retrieval to complex cross modal analysis. We leverage MISCE to finetune an existing multimodal model (Unified IO 2) enhanced with a specialized timeseries encoder, which yields SeisModal, the first domain specific multimodal model for comprehensive seismic analysis. Evaluation of state of the art multimodal models on MultiSeismo reveals significant challenges, particularly with time-series data processing for general purpose models, while demonstrating SeisModal’s superior performance on seismic multimodal reasoning tasks. These results prove that MultiSeismo provides a rigorous benchmark for future multimodal research in seismology and validate the success of our domain specific architectural adaptations.

[134] Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior cs.LG | cs.CLPDF

Zeyi Huang, Xuehai He, LiLiang Ren, Yiping Wang, Baolin Peng

TL;DR: 本文研究了Latent Recurrent Transformer（LRT），这是一种对自回归Transformer的轻量级增强方法。它通过重用前一个token的高层源层隐藏状态作为下一个token的循环记忆，在不增加停顿token或额外深度循环的情况下，跨位置引入了跨层循环潜在通路。为了在大规模预训练中实现这种循环而不需要顺序展开Transformer，作者提出了交错并行训练方法。在多种模型规模和计算预算下，LRT在匹配有效计算量的情况下，仅增加约0.3%的参数，就能同时改善语言建模损失和上下文学习能力。

Details

Motivation: 动机是探索如何为自回归Transformer模型高效地引入循环机制，以增强其记忆和上下文学习能力，同时避免传统循环方法带来的显著计算开销或架构修改。

Result: 在多种nanochat风格骨干网络和广泛的tokens-per-parameter计算预算下，LRT在匹配有效计算量的情况下，改善了语言建模损失和上下文学习性能，同时仅增加了约0.3%的参数。

Insight: 创新点在于提出了一种轻量级的跨层循环潜在通路（LRT），它巧妙地重用现有计算中的隐藏状态作为循环记忆，并设计了交错并行训练策略以实现高效的大规模预训练，在几乎不增加参数的情况下提升了模型性能。

Abstract: We study Latent Recurrent Transformer (LRT), a lightweight augmentation of autoregressive transformers that reuses a high-level source-layer hidden state from the previous token as recurrent memory for the next token. Because this source state is already computed during ordinary decoding, LRT adds a cross-layer recurrent latent pathway across positions without inserting pause tokens or extra depth loops, and the standard attention mechanism and KV-cache interface are preserved. To pretrain this recurrence at scale without sequentially unrolling the transformer, we introduce interleaved parallel training: a single full-sequence initialization forward pass builds a shared buffer; then disjoint position subsets are refined in parallel and written back, so that all tokens receive recurrent-memory-aware supervision at roughly 2 times baseline compute. Across nanochat style backbones and a wide range of tokens-per-parameter budgets, LRT improves both language-modeling loss and in-context learning under matched effective compute while adding as little as 0.3% parameters.

[135] MONA: Muon Optimizer with Nesterov Acceleration for Scalable Language Model Training cs.LG | cs.CLPDF

Jiacheng Li, Jianchao Tan, Hongtao Xu, Jiaqi Zhang, Yifan Lu

TL;DR: 本文提出了MONA优化器，它结合了Muon优化器的正交化框架与曲率感知加速机制，通过引入基于梯度差指数移动平均的加速项，帮助模型逃离尖锐局部极小值。理论分析表明MONA在保持Muon谱范数正则化的同时提升了收敛性，实验证明其在从1B到68B参数的混合专家模型预训练中，相比Muon和AdamW取得了更好的收敛效果与下游任务性能，并在68B模型微调后于多个基准测试中达到SOTA水平。

Details

Motivation: Muon优化器虽为大规模语言模型训练提供了几何感知更新，但作为一阶方法易陷入尖锐局部极小值，因此需要引入曲率信息以改善优化轨迹。

Result: 在1B至68B参数的混合专家模型预训练中，MONA相比Muon和AdamW实现了更优的收敛与下游任务性能；对68B模型进行监督微调后，在通用能力、数学推理和代码生成基准测试中均达到了SOTA性能。

Insight: 创新点在于将Nesterov式加速项直接集成到Muon的梯度处理流程中，利用梯度差的指数移动平均提供曲率感知，从而在保持正交化框架优势的同时增强逃离尖锐极小值的能力；客观上，这种结合一阶正交化与近似二阶加速的方法为大规模优化提供了新思路。

Abstract: The Muon optimizer has recently offered a promising alternative to AdamW for large language model training, leveraging matrix orthogonalization to produce geometry-aware updates. However, like all first-order methods, Muon can become trapped in sharp local minima. In this work, we present MONA, an optimizer that bridges Muon’s orthogonalization framework with curvature-aware acceleration. MONA adds an acceleration term directly into Muon’s gradient processing pipeline. This term is calculated from the exponential moving average of gradient differences. We provide a detailed convergence analysis for MONA, showing that the acceleration term enables escape from sharp minima while preserving Muon’s spectral-norm regularization. Empirically, MONA achieves better convergence and downstream task performance compared to both Muon and AdamW across three scales of Mixture-of-Experts pretraining, spanning from 1B to 68B parameters, with the largest model trained on 1 trillion tokens. Furthermore, we conduct supervised fine-tuning on the MOE-68B-A3B model and evaluate it on general capability, mathematical reasoning, and code generation benchmarks, where MONA achieves SOTA performance.

[136] The Strongest Teacher Is Not Always the Best Teacher: Student-Centric Answer Selection cs.LG | cs.AI | cs.CLPDF

Zhengyu Hu, Zheyuan Xiao, Linxin Song, Fengqing Jiang, Yutai Li

TL;DR: 本文挑战了当前LLM训练中普遍采用最高性能教师模型生成监督数据的做法，指出即使多个教师都能给出正确答案，最强教师的答案也未必是对特定学生模型的最佳监督。为此，论文提出了学生中心答案采样（SCAS）框架，该框架根据估计的学生中心学习成本，从已验证的教师生成答案中进行选择。

Details

Motivation: 当前LLM训练实践中，通常选择性能最高的教师模型来生成学生训练数据，这隐含地将教师测试性能等同于教学质量。本文动机在于揭示这一假设的局限性，即最强教师提供的监督并不总是对给定学生最有效，从而需要一种更以学生为中心的方法来优化知识蒸馏过程。

Result: 实验在30个教师模型、6个学生基础模型和8个任务上进行，结果表明SCAS框架能持续提升学生模型的性能。这证明了在知识蒸馏中，优先选择与当前学生能力相匹配的监督，而非单纯依赖教师强度，是更有效的策略。

Insight: 论文的核心创新点在于提出了‘学生中心学习成本’这一概念及其高效的前向代理估计方法（基于token-wise梯度分解的动机），并据此构建了SCAS框架。这为知识蒸馏领域提供了一个重要的新视角：监督质量应相对于学生模型的能力来评估和选择，而不仅仅是教师的绝对性能。

Abstract: LLM training increasingly relies on teacher-generated supervision, from synthetic responses to reasoning traces and tool-use demonstrations. Current practice often chooses the highest-performing teacher to generate student training data, implicitly treating teacher test performance as a proxy for teaching quality. We show that this assumption can fail: even when multiple teachers provide correct answers to the same question, the answer from the strongest teacher is not necessarily the best supervision for a given student. To address this gap, we propose Student-Centric Answer Sampling (SCAS), a framework that selects from verified teacher-generated answers according to their estimated student-centric learning cost. Motivated by a token-wise gradient decomposition, we derive an efficient forward-only proxy for this cost and use it to guide answer selection during training. Experiments across 30 teacher models, 6 student base models, and 8 tasks show that SCAS consistently improves student performance, suggesting that effective distillation should prioritize supervision matched to the current student rather than teacher strength alone.

[137] Quantized Keys Steal Attention: Bias Correction for KV-Cache Compression in Video Diffusion cs.LG | cs.AI | cs.CV | cs.GR | eess.IVPDF

Tuna Tuncer, Felix Becker, Thomas Pfeil

TL;DR: 本文针对视频扩散模型中KV缓存量化导致视频质量下降的问题，提出了一种基于Jensen偏差的注意力权重校正方法。通过分析softmax注意力中指数函数的凸性，作者发现量化噪声会系统性地增加缓存键的注意力贡献，导致当前块的注意力被’窃取’。论文推导了一种在线计算的逐注意力分数校正项，使用二阶泰勒近似实现，计算开销可忽略且无需额外内存。

Details

Motivation: 动机在于解决视频扩散模型中KV缓存的内存瓶颈问题：现有低比特量化方法虽能减少内存占用，但会显著降低生成视频的质量。作者发现质量下降的根本原因是量化噪声在注意力机制中引入了系统性偏差（Jensen偏差），导致注意力分布失真。

Result: 在MAGI-1、SkyReels-V2和HY-WorldPlay数据集上使用INT2量化的实验表明，该方法能恢复大部分因激进量化损失的质量，达到接近BF16精度的视频质量。在节省50%内存的情况下，其性能甚至优于INT4量化方案。

Insight: 创新点在于首次揭示了KV缓存量化中注意力权重偏差的数学根源（Jensen偏差），并提出了轻量级的在线偏差校正机制。从工程角度看，该方法通过二阶泰勒近似实现了理论修正的实用化，在几乎不增加计算开销的情况下显著提升量化效果，为内存受限的视频生成任务提供了新思路。

Abstract: Chunk-wise autoregressive video diffusion models rely on a KV cache of previously generated chunks to avoid redundant computation, but this cache quickly becomes a memory bottleneck as videos grow longer. Methods that quantize the KV cache to low bitwidths reduce memory pressure but degrade video quality. We show that a key driver of this degradation is a systematic bias in attention weights: due to the convexity of the exponential in softmax attention, quantization noise inflates the contribution of cached keys, a phenomenon we call the Jensen bias. This effect causes quantized keys to steal attention mass from the unquantized current chunk. We derive a per-attention-score correction that removes this bias in expectation, computed on the fly from the quantization step sizes of the cached keys and the query norm. Using a second-order Taylor approximation, the additional computational overhead is negligible, and no additional memory is needed alongside the cache. Evaluated on MAGI-1, SkyReels-V2, and HY-WorldPlay at INT2 quantization, our correction recovers most of the quality lost to aggressive quantization, reaching near-BF16 video quality, and can outperform INT4 quantization while using 50% less memory.

[138] Beyond Pairwise Preferences: Listwise Reward-Aware Alignment for Diffusion Models cs.LG | cs.CVPDF

Austin Wang, Jiaqi Han, Stefano Ermon, Yisong Yue

TL;DR: 本文提出了Diffusion LAIR，一种用于扩散模型的奖励感知列表偏好优化方法，以解决现有偏好优化方法局限于二元成对比较的局限性。该方法将同一提示下多个候选图像的奖励分数转换为中心化优势权重，并优化一个基于隐式奖励的优势加权回归目标，同时通过二次惩罚项正则化隐式奖励的幅度。

Details

Motivation: 现有偏好优化方法主要将监督简化为二元成对比较，这在训练数据自然包含同一提示的多个候选图像时存在局限，且连续奖励分数能提供比单一胜负标签更丰富的信息。

Result: 实验表明，Diffusion LAIR在SD1.5和SDXL模型上，在文本到图像生成、组合生成和图像编辑基准测试中，优于强大的偏好优化基线方法。

Insight: 创新点在于将列表式学习引入扩散模型对齐，通过中心化优势权重和隐式奖励的正则化，同时利用所有候选图像信息并控制更新幅度，避免了成对比较的信息损失和训练不稳定问题。

Abstract: Preference optimization has emerged as an efficient alternative to online reinforcement learning from human feedback (RLHF) for aligning text-to-image diffusion models. However, existing methods largely reduce supervision to binary pairwise comparisons. This pairwise reduction is limiting when training data naturally contains multiple candidate images for the same prompt, and when continuous reward scores can provide richer information than a single winner-loser label. To address these limitations, we propose Diffusion LAIR, a reward-aware listwise preference optimization method for diffusion models. For each prompt, LAIR converts reward scores across a group of candidate images into centered advantage weights, then optimizes an advantage-weighted regression objective on the implicit reward, defined as the denoising-loss improvement of the current model over a fixed reference model, with a quadratic penalty that regularizes the magnitude of the implicit reward. The resulting objective uses all candidates simultaneously rather than selecting pairs, and remains conservative by explicitly controlling the magnitude of the implicit reward. The LAIR objective admits a bounded closed-form optimum in implicit-reward space, clarifying how the regularization strength controls the magnitude of the preference update. Experiments show that Diffusion LAIR outperforms strong preference optimization baselines on SD1.5 and SDXL across text-to-image generation, compositional generation, and image editing benchmarks.

[139] Guiding LLM Post-training Data Engineering with Model Internals from Sparse Autoencoders cs.LG | cs.AI | cs.CLPDF

Yi Jing, Zao Dai, Jinwu Hu, Zijun Yao, Lei Hou

TL;DR: 本文提出SAERL框架，利用稀疏自编码器（SAE）从大语言模型内部提取特征，来指导强化学习（RL）后训练阶段的数据工程。该框架建模了数据的多样性、难度和质量三个内在属性，并据此进行批次混合、课程排序和数据过滤等操作，从而提升训练效率和模型性能。

Details

Motivation: 当前LLM的后训练数据工程主要依赖外部信号，忽略了模型内部蕴含的丰富信息。本文旨在利用模型内部表示（通过SAE提取）作为数据工程的内在信号源，以更有效地指导RL训练。

Result: 在Qwen2.5-Math-1.5B模型上，SAERL相比基础GRPO算法平均准确率提升3.00%，并以减少20%训练步数达到目标准确率。实验表明SAE在不同模型家族和规模上具有良好的可迁移性，能作为轻量级、可复用的数据工程工具。

Insight: 创新点在于将机制可解释性工具（SAE）提取的模型内部特征，系统地应用于RL数据工程，定义了数据多样性、难度、质量三个可操作的内在属性。这为利用模型内部状态（而非仅外部指标）来优化训练数据提供了新范式。

Abstract: Model internals encode rich information about how a large language model (LLM) processes its training data; however, post-training data engineering largely relies on external signals and ignores rich intrinsic signals lying in model internals. We propose SAERL, a data engineering framework for LLM reinforcement learning (RL). It models three intrinsic data properties: diversity, difficulty, and quality, using model internals extracted with Sparse Autoencoder (SAE), an advanced mechanistic interpretability tool. Each property grounds a concrete data engineering operation: SAE-space clustering with moderate batch mixing for batch diversity control, a difficulty proxy for easy-to-hard curriculum ordering, and a quality probe for data filtering. SAERL improves average accuracy by 3.00% over vanilla GRPO and reaches target accuracy with 20% fewer training steps on Qwen2.5-Math-1.5B, with consistent gains across model scales and RL algorithms. Experiments show that SAE transfers effectively across model families and scales, serving as a lightweight and reusable data engineering tool. These results demonstrate that model internals are a powerful and practical source of signals for post-training data engineering.

cs.HC [Back]

[140] Design First, Code Later: Aesthetically Pleasing Template-Free Slides Generation cs.HC | cs.CVPDF

Zhiyao Cui, Chenxu Wang, Shuyue Hu, Yiqun Zhang, Wenqi Shao

TL;DR: 本文提出了一种无模板的幻灯片自动生成框架DeepSlides，它采用分层工作流将幻灯片页面设计与实现解耦，并引入了专用数据集SlideDesign以及基于多智能体强化学习的训练范式来训练SlideQwens模型。实验表明，该框架在评估指标和人类偏好评估上均优于基线方法。

Details

Motivation: 现有自动生成幻灯片的方法依赖固定模板或直接输出可执行代码，这限制了大型语言模型在创意布局设计方面的能力，并绕过了关键的页面设计步骤。本文旨在解决这些限制，实现无需预定义模板或样式的、系统化的幻灯片设计。

Result: 实验结果表明，所提出的框架在评估指标上优于基线方法，并在人类偏好评估中取得了更优的性能。

Insight: 主要创新点在于提出了一个分层的工作流，将幻灯片页面设计与实现解耦，从而摆脱了对固定模板的依赖；同时，引入了专门的数据集和一种多智能体强化学习训练范式来训练设计模型，这为结构化多模态任务的生成提供了新的思路。

Abstract: Producing presentation slides automatically entails coordinating narrative structure with page-level graphic design under strict spatial constraints. For such structured multimodal tasks, a well-organized design process is essential to ensure the final quality of slides. Existing approaches rely on fixed templates or directly emit executable code, thereby both limiting the creative layout-design capabilities of LLMs and bypassing the essential slide-page design step. To address these limitations, this paper (1) proposes a hierarchical slides generation workflow, DeepSlides, that systematically organizes slide design tasks without any predefined template or style, decoupling slide-page design from implementation; (2) introduces SlideDesign, a dataset tailored specifically for slides generation tasks; and (3) presents a multi-agent reinforcement learning training paradigm and trains a couple of models, SlideQwens, for slide design and implementation. Experimental results demonstrate that our proposed framework outperforms baseline methods on evaluated metrics and achieves superior performance in human preference evaluations. The dataset and code are available at https://github.com/sxswz213/DeepSlides.

Table of Contents

cs.CL [Back]

[1] RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents cs.CLPDF

[2] Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline cs.CL | cs.LGPDF

[3] Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations cs.CL | cs.AIPDF

[4] Memory Architectures for Multi-Turn Text-to-SQL: A Benchmark and Empirical Study cs.CLPDF

[5] Towards Just-in-Time Adaptive Feedback: Enhancing Student Learning via Knowledge-Grounded LLM cs.CLPDF

[6] Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Models cs.CL | cs.AIPDF

[7] Towards Error-Free EHRs: Reasoning-Intensive Consistency Verification Between Clinical Notes and Structured Tables in Electronic Health Records cs.CL | cs.AIPDF

[8] FAB-Bench: A Framework for Adaptive RAG Benchmarking in Semiconductor Manufacturing cs.CL | cs.IRPDF

[9] Conceptual Steganography cs.CLPDF

[10] Why Prompt Optimization Works, and Why It Sometimes Doesn’t: A Causal-Inspired Edit-Level Analysis cs.CL | cs.LG | cs.NEPDF

[11] Rethinking the Multilingual Reasoning Gap with Layer Swap cs.CLPDF

[12] KARMA: Karma-Aligned Reward Model Adaptation cs.CLPDF

[13] SeDT: Sentence-Transformer Decision-Transformer Conditioning for Multi-Turn Conversation Reliability cs.CL | cs.AIPDF

[14] Generating Logically Consistent Synthetic Supply Chain Data with LLM-Driven Knowledge Graph Reasoning cs.CLPDF

[15] ContextGuard: Structured Self-Auditing for Context Learning in Language Models cs.CL | cs.AIPDF

[16] Optimising Factual Consistency in Summarisation via Preference Learning from Multiple Imperfect Metrics cs.CLPDF

[17] Uncertainty-Aware Budget Allocation for Adaptive Test-Time Reasoning cs.CLPDF

[18] GeoFaith: A Spatio-Temporal Dual View of Faithful Chain-of-Thought cs.CL | cs.AIPDF

[19] Are Video Models Zero-Shot Learners and Reasoners in Education? EduVideoBench, A Knowledge-Skills-Attitude Benchmark for Educational Video Generation cs.CLPDF

[20] Learning to Adapt SFT Data for Better Reasoning Generalization cs.CLPDF

[21] Beyond Questions: Evaluating What Large Language Models (Actually) Know cs.CL | cs.AIPDF

[22] Reasoning Depth and Environment Complexity: A Controlled Study of RLVR Data Allocation across Logical Reasoning Tasks cs.CL | cs.AIPDF

[23] Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancement cs.CLPDF

[24] Tournament-GRPO: Group-Wise Tournament Rewards for Reinforcement Learning in Open-Ended Long-Form Generation cs.CL | cs.AIPDF

[25] Recon: Reconstruction-Guided Reasoning Synthesis for User Modeling cs.CL | cs.AIPDF

[26] Cast a Wider Net: Coordinated Pass@K Policy Optimization for Code Reasoning cs.CL | cs.AIPDF

[27] Share More, Search Less: Collaborative Parallel Thinking for Efficient Test-Time Scaling cs.CLPDF

[28] QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents cs.CL | cs.AI | cs.MAPDF

[29] Large Language Model-Powered Query-Driven Event Timeline Summarization in Industrial Search cs.CL | cs.IRPDF

[30] LLMs Are Already Good Tutors: Training-Free Prompt Optimization for Pedagogical Math Tutoring cs.CL | cs.LGPDF

[31] Learning When to Think While Listening in Large Audio-Language Models cs.CL | cs.AI | cs.LG | cs.SDPDF

[32] Not All Tokens Matter Equally: Dynamic In-context Vector Distillation with Decisive-Token Supervision for Long-form Medical Report Generation cs.CL | cs.CV | cs.LGPDF

[33] EpiCurveBench: Evaluating VLMs on Epidemic Curve Digitization cs.CLPDF

[34] Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs cs.CL | cs.AIPDF

[35] Self-Ensembling Vision-Language Models for Chart Data Extraction cs.CLPDF

[36] It’s Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertainty cs.CL | cs.AI | cs.LGPDF

[37] Real Images, Worse Judgments: Evaluating Vision-Language Models on Concreteness and Imagery cs.CLPDF

[38] Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models cs.CL | cs.CVPDF

cs.CV [Back]

[39] Not All Modalities Are Equal: Instruction-Aware Gating for Multimodal Videos cs.CVPDF

[40] Sentinel: Embodied Cooperative Spatial Reasoning and Planning cs.CV | cs.MAPDF

[41] RoMo: A Large-Scale, Richly Organized Dataset and Semantic Taxonomy for Human Motion Generation cs.CVPDF

[42] LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV cs.CV | cs.MM | cs.SDPDF

[43] Frequency-Guided Fusion For RGB-Thermal Semantic Segmentation cs.CVPDF

[44] Benchmarking Convolutional, Transformer, Hybrid, and Vision Language Models for Multi Disease Retinal Screening cs.CV | cs.LGPDF

[45] Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuning cs.CV | cs.CLPDF

[46] CNNs, Transformers, Hybrid, and Vision Language Models for Skin Cancer Detection cs.CVPDF

[47] E$^3$C: Video Generation with 3D Environmental Memory and Ego-Exo Human Pose Control cs.CV | cs.AIPDF

[48] RadarSim: Simulating Single-Chip Radar via Multimodal Neural Fields cs.CVPDF

[49] Personalized Generative Models for Contextual Debiasing cs.CV | cs.AI | cs.LGPDF

[50] Joint Instance Segmentation and Geometric Attribute Regression for Roof Structures in Aerial Imagery cs.CVPDF

[51] BioFact-MoE: Biologically Factorized Mixture of Experts for Vision-Language Prognostic Modeling in Hepatocellular Carcinoma cs.CV | cs.AI | cs.LGPDF

[52] VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenes cs.CV | cs.AIPDF

[53] Multi-Modal Building Inspection via Perceiver IO Fusion of Satellite and Street-Level Imagery cs.CVPDF

[54] Detail Consistent Stage-Wise Distillation for Efficient 3D MRI Segmentation cs.CVPDF

[55] OmniGF: A Dual-Branch Vision-Language Framework for Unified Gaze Following cs.CVPDF

[56] Zero-Shot Object Re-Identification in Egocentric Kitchen Videos via Multi-Stage SAM3 Feature Fusion cs.CVPDF

[57] The Rescue Effect: Spatio-Semantic Early Exit Bypasses Quantization Collapse in CLIP cs.CV | cs.AIPDF

[58] HydraPrompt: An Adaptive and Asymmetric Framework of Vision-Language Models for Synthetic Image Detection cs.CVPDF

[59] Rethinking Weakly-supervised Video Temporal Grounding From a Game Perspective cs.CV | cs.AIPDF

[60] Cross-scale Aligned Supervision for Training GANs cs.CV | cs.AIPDF

[61] AnchorDiff: Training-Free Concept Grounding for MM-DiTs via Anchor-Based Graph Propagation cs.CV | cs.AIPDF

[62] Triadic Dynamics Aware Diffusion Posterior Sampling for Inverse Problems: Optimizing Guidance and Stochasticity Schedules cs.CVPDF

[63] Comparative Study of Vision-Based Metric Measurement for Large-Scale Planar Scenes cs.CV | cs.AIPDF

[64] OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants cs.CV | cs.CLPDF

[65] Clinically-Grounded Counterfactual Reasoning for Medical Video Diagnosis cs.CVPDF

[66] LongCat-Video-Avatar 1.5 Technical Report cs.CVPDF

[67] Uncertainty-Aware Gaussian Map for Vision-Language Navigation cs.CVPDF

[68] Unveiling the Fragility of Vision-Language Models: Multi-Modal Adversarial Synergy via Texture-Constrained Perturbations and Cross-Modal Optimization cs.CV | cs.AIPDF

[69] Re-M3Dr: Rebalanced MultiModal Mean Deviation Regression cs.CVPDF

[70] 3D Gaussian Map with Open-Set Semantic Grouping for Vision-Language Navigation cs.CVPDF

[71] CSV-ViT: A Vision Transformer with the Variable-sized Cortical Supervertices for Detection of Alzheimer’s Disease Pathologies cs.CV | cs.AI | cs.LGPDF

[72] InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward cs.CV | cs.AIPDF

[73] CmIVTP: Cross-modal Interaction-based Vessel Trajectory Prediction for Maritime Intelligence cs.CV | cs.AIPDF

[74] ReCA: Multi-Shot Long Video Extrapolation via Recursive Context Allocation cs.CV | cs.AIPDF

[75] A Hybrid Vision-Language Architecture for Automated Defect Reasoning and Report Generation in Industrial Inspection cs.CV | cs.AI | cs.CL | cs.LGPDF

[76] O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding cs.CVPDF

[77] FTibSuite: A Comprehensive Resource Suite for Tibetan Vision-Language Modeling cs.CV | cs.DBPDF