cs.CL [Total: 16]
cs.CV [Total: 55]
cs.LG [Total: 6]
cs.RO [Total: 1]
cs.GT [Total: 1]
cs.AI [Total: 11]
cs.CR [Total: 1]
cs.HC [Total: 1]
eess.IV [Total: 3]
cs.SD [Total: 2]

cs.CL [Back]

[1] Gloss-Free Representation Learning for Cross-Dataset Sign Spotting cs.CL | cs.CVPDF

Oğuz Akif Tüfekcioğlu, Ezgi Ekin, Mustafa Kaan Çevik, Hacer Yalim Keles

TL;DR: 本文提出了一种无需精细标注（如gloss）的手语表示学习方法，利用广播新闻中弱对齐的语音转录文本作为监督信号，在土耳其手语数据集TSL-News上进行预训练，学习可重用的手语编码器。该方法通过基于规则的形态学词形还原与受限LLM辅助的文本规范化生成伪gloss标签，并在跨数据集的手语定位任务TSL Spotting Benchmark上验证了表示的有效性。

Details

Motivation: 针对资源受限语言的手语研究常受限于密集标注（如gloss、时间边界、手语顺序）的高成本，而广播新闻提供了连续手语与语音转录的弱对齐数据，但文本与手语的松散对齐以及土耳其语等形态丰富语言的词形变化使得监督信号较弱。本文旨在探索基于弱转录文本的监督能否预训练出可重用的手语编码器。

Result: 在TSL Spotting Benchmark上，LLM辅助的编码器将top-5时间定位的平均IoU从0.235提升至0.465，56.2%的样本达到至少0.50的IoU；下游翻译任务中BLEU-4从9.60提升至11.04，ROUGE从23.48提升至27.43。结果表明该方法能有效学习捕获词汇内容和时间结构的手语表示。

Insight: 创新点在于利用广播新闻的弱对齐转录进行无gloss的手语表示学习，通过LLM辅助的文本规范化处理形态丰富语言的伪gloss碎片化问题，并验证了预训练编码器在跨数据集手语定位任务上的可迁移性，为资源受限手语研究提供了低成本监督方案。

Abstract: Sign-language research for resource-constrained languages is often limited by the cost of dense linguistic labels such as glosses, temporal boundaries, and sign order. Broadcast news offers a practical alternative by pairing continuous signing with spoken-language transcripts, but this supervision is weak since text and signing are loosely aligned. Morphologically rich languages such as Turkish add further difficulty, as the same lexical meaning can appear in many inflected forms while some derived forms should remain distinct. We study whether weak transcript-based supervision can pretrain a reusable sign encoder in this setting, where poor text normalization can fragment pseudo-gloss targets and weaken representation learning. Unlike prior pseudo-gloss pipelines designed mainly to improve translation, we test whether the pretrained encoder transfers as a reusable representation for cross-dataset sign spotting. We pretrain on TSL-News, a new Turkish broadcast corpus, using pseudo-gloss labels derived from transcripts rather than manual annotation, comparing rule-based morphological lemmatization with constrained LLM-assisted normalization over a fixed vocabulary. We evaluate the learned representations via cross-dataset sign spotting on a new TSL Spotting Benchmark built from the TSL Dictionary corpus. The LLM-assisted encoder raises top-5 temporal localization mean IoU from 0.235 to 0.465, with 56.2% of examples reaching an IoU of at least 0.50; a frequency analysis suggests this gain is not mainly driven by memorizing frequent pseudo-gloss labels. In a downstream translation check, the same pretraining improves BLEU-4 from 9.60 to 11.04 and ROUGE from 23.48 to 27.43. These results show that loosely aligned broadcast data can provide effective weak supervision for learning sign representations that capture both lexical content and temporal structure.

[2] Backtrader-Bench: Benchmarking LLM Agents on Algorithmic Trading with Self-Generated MCQs cs.CL | cs.AIPDF

Ruoxi Zhao, Maziar Raissi

TL;DR: 本文提出了Backtrader-Bench框架，用于评估LLM智能体在算法交易中的表现。该框架包含两个互补的流水线：一个确定性的多选题（MCQ）生成流水线，从回测配置中生成问题；另一个生成器-求解器过滤流水线，自主挖掘更难的问题。研究评估了11个无工具模型和4个带工具配置的智能体，结果显示带工具的智能体在精选问题集上准确率可达90.0%，显著优于无工具基线（73.0%）。

Details

Motivation: 在算法交易中评估LLM编码智能体存在困难，因为静态基准测试存在数据污染风险，而数值回测输出需要实际代码执行的真实结果。

Result: 在30个精选问题上，带工具的智能体（如GPT-5.5和Opus 4.7）单次通过准确率达到90.0%，比最佳无工具基线（10次运行平均73.0%）高出17个百分点。在38个单独挖掘的更难问题上，无工具模型的准确率进一步下降，半数模型降至接近随机猜测水平（25%）。

Insight: 创新点包括：1）结合确定性MCQ生成和自主挖掘难题的双流水线框架，有效缓解数据污染问题；2）引入独立检查器重新推导每个答案，确保评估可靠性；3）框架设计可扩展，旨在为强化学习生成训练语料，最终目标是构建专门用于量化交易工作流的智能体。

Abstract: Evaluating LLM coding agents in algorithmic trading is difficult because static benchmarks risk data contamination and numerical backtest outputs require ground truth from actual code execution. We present Backtrader-Bench, a framework with two complementary pipelines. A deterministic multiple-choice question (MCQ) pipeline generates questions from backtest configurations across five trading strategies, 33 templates, and three difficulty tiers, with an independent checker that re-derives every answer. A generator-solver filtering pipeline autonomously mines harder questions: a generator writes questions verified by executable code, converts them to MCQs, and discards any that a no-tool solver can answer without code execution. We evaluate 11 models without tools (10 runs each) and four with-tools configurations on a 30-question curated set. Tool-augmented agents reach 90.0% accuracy in a single pass (GPT-5.5 and Opus 4.7), outperforming the best no-tools baselines (73.0%, averaged over 10 runs) by 17 percentage points. On 38 separately mined questions, no-tools accuracy drops further, with half the models falling to roughly random-chance level (25%). Beyond evaluation, the scalable MCQ infrastructure is designed to produce a training corpus for reinforcement learning, with the ultimate goal of building a specialized agent for quantitative trading workflows.

[3] Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgets cs.CL | cs.AI | cs.LGPDF

Mark Shapiro

TL;DR: 该论文提出了一种将循环深度机制融入预训练语言模型的方法，通过在Qwen2.5-0.5B-Instruct模型架构中引入权重绑定的循环块、恒等路径和重入桥接，实现了在潜在空间中进行迭代推理。研究表明，该方法能以两种参数预算（6M适配器或180M完整块）安装，支持任务步骤的逐步计算，并在仅监督最终答案时保持性能，同时展现出良好的深度外推能力和推理速度优势。

Details

Motivation: 解决预训练语言模型在深度推理任务中可能存在的计算效率低、难以进行迭代式潜在状态转换的问题，旨在通过引入循环结构来增强模型执行多步推理的能力，同时保持原有的通用能力。

Result: 在预注册的ARC基准测试中，循环改造后的模型在循环1时性能不劣于基础模型；适配器版本（83.8%）与完整块版本（84.0%）整体表现相当，在深度11以内领先，之外稍逊；在受控语言渲染任务上，经过微调达到79-86%准确率；模型能外推到监督深度约1.5倍（深度18时保持70%准确率）；与同等大小的草稿训练模型相比，循环模型整体以84%对72%胜出，在深度10以外保留53%对2.5%的性能，且推理速度快7.6倍。

Insight: 创新点在于提出了一种可复用的、非终端答案查找的循环深度安装机制，通过恒等路径和重入桥接实现了潜在状态的持续迭代转换，使模型能以较低参数成本进行高效的多步推理，并在深度外推和推理速度上显著优于传统微调方法；同时揭示了在保持已安装机制和通用能力的前提下学习逆向任务时存在的灾难性干扰边界，为深度选择学习留下了开放问题。

Abstract: A dense, pretrained language model can be retrofitted with recurrent depth and learn an iterative latent transition that persists after outcome-only annealing. Qwen2.5-0.5B-Instruct is split into a Prelude, a weight-tied Recurrent Block, and a Coda, with an identity-preserving one-loop path and a re-entry bridge on later loops. At loop 1 the retrofit remains non-inferior to its base on a preregistered ARC battery. Three findings. First, the mechanism is a reusable procedure rather than terminal-answer lookup, and installs at two budgets: 6M trained parameters over frozen base weights and 180M full-block. With intermediate-step supervision, the model computes one task step per loop and persists when only final answers are graded. The adapter matched the full block overall (83.8% versus 84.0%), led through depth 11, and trailed beyond. Verbal fine-tuning reached 79-86% on controlled verbal renderings (zero-shot transfer was minimal), and adapter verbal training begun from the installed mechanism outpaced matched fresh training by 18.6 points, including on a held-out test set. Second, the operation extrapolates to roughly 1.5 times its supervised depth, holding 70% accuracy through depth 18. Third, a same-size scratchpad-trained model matched the recurrent model within its learned horizon but collapsed beyond it. The recurrent model won overall, 84% versus 72%, retained 53% versus 2.5% beyond depth 10, and answered 7.6 times faster. An iterative transformer can therefore perform deeper reasoning in latent space faster than comparable or larger models fine-tuned on the same task, in a system-level comparison. A second task, running the rule in reverse, exposed the limits: the inverse was learnable in isolation, but no continuation acquired it while preserving the installed mechanism and general capability, a catastrophic-interference boundary. Learned depth selection remains open.

[4] Self-Evolving Embodied Agents via Skill-Harness Evolution cs.CL | cs.ROPDF

Peidong Wang, Zhiming Ma, Ying Chang, Xufang Luo, Xiaocui Yang

TL;DR: 本文提出SHAPER框架，一种无需训练即可实现具身智能体自我进化的方法。该方法保持模型参数冻结，通过在目标环境中进行rollout来进化可重用技能和上下文代码harness，从而提升非参数化智能体系统的性能。

Details

Motivation: 现有具身智能体适应新环境的方法（如监督微调、强化学习）需要额外数据、奖励和训练，而许多免训练的代码中心方法依赖可编程机器人API，这在固定接口设置中可能不可用。SHAPER旨在解决这些问题，实现无需模型参数更新的智能体自我进化。

Result: 在VLABench和ESI-Bench基准测试中，SHAPER与纯执行、监督微调和测试时扩展基线（如无验证器选择和投票）进行了比较。结果表明，当模型训练成本高、不可用或不理想时，技能和harness优化是实现自我进化具身智能体的实用途径。

Insight: 创新点在于将同一冻结模型同时用作规划器和优化器，通过进化外部技能和上下文代码harness来提升性能，而无需更新模型参数。这为在固定接口或训练受限场景下实现具身智能体的持续适应提供了新思路。

Abstract: Embodied agents are increasingly built as systems around foundation models, where performance depends not only on model weights but also on the skills, context, action interfaces, and execution harness surrounding the model. While supervised fine-tuning and reinforcement learning can adapt agents to new environments, they require additional data, rewards, and training runs; meanwhile, many train-free code-centric approaches rely on programmable robot APIs that may be unavailable in fixed-interface settings. We propose SHAPER, a self-evolving framework for train-free embodied adaptation that keeps model parameters frozen and improves the non-parametric agent system by evolving reusable skills and a context-code harness through target-environment rollouts. In SHAPER, the same frozen model can serve as both planner and optimizer, refining its external skills and context-code harness without parameter updates. We evaluate SHAPER on VLABench and ESI-Bench, covering embodied agents with different low-level action interfaces, and compare against pure execution, supervised fine-tuning, and test-time-scaling baselines such as verifier-free selection and voting. Our results suggest that skill-and-harness optimization is a practical route to self-evolving embodied agents when model training is expensive, unavailable, or undesirable.

[5] CT-$Δ$Bench: A Benchmark for Longitudinal 3D Medical Imaging Difference Reporting with Vision-Language Models cs.CL | cs.CVPDF

Kegeng Tang, Jingbo Wang, Shaogang Ren, Zihao Wang

TL;DR: 本文提出了CT-$Δ$Bench，一个专门用于评估纵向3D医学影像差异报告的基准测试，旨在解决现有医学基础模型在时序比较方面的不足。该研究引入了针对临床有意义变化的评估指标，并提出了一个名为DeltaMed的基线模型，用于直接对成对CT扫描进行差异报告生成。

Details

Motivation: 临床决策高度依赖对同一患者连续CT扫描的纵向比较，以评估疾病演变，但现有医学基础模型主要局限于单次研究理解，缺乏对时序交叉检验的有效处理。

Result: 在提出的CT-$Δ$Bench基准上，通过患者级别的数据划分防止信息泄露，并开发了专门捕捉临床有意义变化的评估指标，且进行了独立的医生验证以确保合成参考和事件提取流程的可靠性。

Insight: 创新点在于首次构建了专注于纵向医学影像差异报告的基准，并提出了直接基于成对CT扫描进行推理的基线模型DeltaMed，这为开发具有时序感知能力的医学基础模型奠定了基础，超越了传统的两阶段文本差分方法。

Abstract: In medical imaging, the clinical value of Computed Tomography (CT) lies not only in depicting current disease status, but crucially in enabling longitudinal comparison of serial scans to determine disease evolution, a process that underpins response assessment, recurrence detection, and ongoing patient management. Yet, despite this central role of temporal comparison in clinical decision-making, existing medical foundation models remain largely confined to single-study understanding, leaving temporally grounded cross-examination insufficiently addressed. To address this gap, we study longitudinal imaging difference reporting, a task in which a model takes two temporally separated scans from the same patient and generates a clinically meaningful report describing interval changes between them. We introduce CT-$Δ$Bench, a dedicated benchmark for this task with patient-level splitting to prevent information leakage. To better evaluate this task beyond surface-level text similarity, we further develop change-aware metrics specifically designed to capture clinically meaningful longitudinal changes, and conduct an independent physician validation to assess the reliability of the synthesized references and event extraction pipeline. We also compare direct paired-CT reasoning with an indirect two-stage pipeline that first generates single-timepoint reports and then performs textual differencing. Finally, we propose DeltaMed, a baseline model for direct paired-CT difference reporting, and train it on the benchmark training set. Together, these contributions lay the groundwork for temporally aware medical foundation models that better reflect real-world longitudinal clinical reasoning.

[6] Reinforcing Step-level Reasoning for Effective Self-Correction in LLMs cs.CL | cs.AIPDF

Vu Duc Anh, Nhat M. Hoang, Do Xuan Long, Cong-Duy Nguyen, Ponhvoan Srey

TL;DR: 本文提出了一种名为Self-Fix Step-DPO (SFS-DPO)的两阶段强化学习框架，旨在增强大语言模型(LLMs)的逐步推理能力，以实现有效的自我验证与自我纠正。该框架包含一个通过逐步偏好优化强化推理的阶段，以及一个专门训练模型进行自我纠正的阶段，并引入了包含解释性原理的教师辅助变体SFS-DPO-R。

Details

Motivation: 解决大语言模型在实现有效自我纠正（即模型验证并修正自身错误）方面存在的根本性挑战。

Result: 在多个大语言模型上进行领域内和领域外的综合评估表明，SFS-DPO和SFS-DPO-R在性能上持续优于先前的逐步训练基线方法，并在自我纠正的频率和有效性方面均有提升。

Insight: 创新点在于提出了一个两阶段的强化学习框架，将逐步推理强化与明确的自我纠正训练解耦；其核心洞察是强化逐步推理对于实现鲁棒的自我纠正性能至关重要，而教师辅助的变体通过引入解释性原理提供了更强的纠正信号。

Abstract: Achieving effective self-correction, where models verify and correct their own mistakes, remains a fundamental challenge for large language models (LLMs). In this work, we propose Self-Fix Step-DPO (SFS-DPO), a reinforcement learning based, two-stage framework for step-level self-verification and self-correction. The first stage strengthens step-level reasoning via step-level preference optimization, while the second stage explicitly trains models to self-verify and self-correct. We further introduce a teacher-assisted variant, SFS-DPO-R, which incorporates explanatory rationales for error verification to provide stronger corrective signals. Comprehensive in-domain and out-of-domain evaluations across multiple LLMs demonstrate that SFS-DPO and SFS-DPO-R consistently outperform prior step-level training baselines. Our analysis further reveals improvements in self-correction frequency and effectiveness, highlighting the importance of strengthening step-level reasoning for robust performance.

[7] Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs cs.CL | cs.AIPDF

Nimet Beyza Bozdag, Emre Can Acikgoz, Gokhan Tur, Dilek Hakkani-Tür

TL;DR: 该论文研究了大型语言模型在对抗性说服攻击下的脆弱性，通过强化学习训练说服者模型，使其能够用单个虚假论据显著降低目标模型的准确率，并揭示了模型容易放弃正确信念的安全风险。

Details

Motivation: 随着LLM越来越多地与人类及其他模型进行辩论、建议和协作，抵抗有害说服成为可靠行为的核心要求，但当前模型远未满足这一要求。

Result: 强化学习训练的说服者将说服成功率从约24%提升至93%以上，且攻击策略可迁移到未见模型（如Qwen-14B达83%成功率），通过课程学习进一步将GPT-4o-mini攻击成功率从25%提升至38%。

Insight: 论文创新地形式化了对抗性说服威胁并引入强化学习框架，揭示了模型易受基于可信度的虚假策略（如伪造引用）影响，表明说服鲁棒性应作为多智能体与人机决策系统的必要安全标准。

Abstract: Persuasion is a core dynamic of natural language communication, shaping how large language models (LLMs) update beliefs, resolve disagreements, and reach decisions. As LLMs increasingly debate, advise, and think collaboratively with humans and each other, resistance to harmful persuasion becomes a core requirement for reliable behavior. Yet we show that this requirement is far from met: a single targeted persuasive argument is enough to collapse model accuracy to near zero, even when the argument is factually false. We formalize this threat as adversarial persuasion and introduce an adversarial reinforcement learning framework that trains persuader agents to change a target model’s answer in a single interaction. First, we show that optimizing persuasion strategies through trial and error exposes vulnerabilities that static prompting misses: RL-trained persuaders raise persuasion success from approximately 24% to over 93% against the training-time persuadee. Second, we find that these learned strategies transfer to unseen models, achieving 83% attack success on Qwen-14B, 79% on Llama-3.1-8B, and 25% on GPT-4o-mini. Third, we demonstrate that a curriculum that bootstraps on more persuadable open-weight models before targeting harder models further increases GPT-4o-mini attack success from 25% to 38%. Moreover, our results reveal that optimized persuaders increasingly rely on credibility-based tactics, including fabricated citations and false authoritative evidence. Together, these findings expose a critical weakness in current LLM agents: even when they initially reason correctly, they can be steered toward false conclusions by optimized natural language influence. This positions persuasion robustness as a necessary safety criterion for multi-agent and human-AI decision-making systems.

[8] Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing cs.CL | cs.AI | cs.LGPDF

Tianci Liu, Zihan Dong, Tianchun Li, Yi-Chung Chen, Qiming Cao

TL;DR: 本文提出了一种名为HPSE（Hybrid-Policy Self-Editing）的新方法，用于解决大型语言模型（LLM）中非结构化知识编辑（UKE）的可组合性问题。该方法将知识编辑视为一种主动的自蒸馏过程，通过构建混合策略的轨迹来弥补预编辑模型在覆盖新知识上的不足，从而提升模型对编辑段落中原子事实的问答以及多跳推理能力。

Details

Motivation: 现有非结构化知识编辑方法仅能被动地将编辑段落注入模型，导致编辑后的模型虽然能复述段落，但无法回答关于段落中原子事实的简单问题，也无法将这些事实组合起来进行多跳推理，即缺乏可组合性。

Result: HPSE方法在四种不同的LLM主干模型和两种知识编辑器的多种场景下，均实现了即插即用的性能提升，实证结果验证了其有效性。

Insight: 核心创新在于将知识编辑重新定义为从模型自身特权上下文状态进行的主动自蒸馏，无需外部监督；并提出了混合策略轨迹构建机制，在模型自身轨迹覆盖不足时主动介入补充缺失事实，从而有效解决了纯策略蒸馏在新知识上的局限性，提升了编辑知识的可组合性。

Abstract: Large language models (LLMs) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast-changing world. This motivates knowledge editing (KE), which updates specific knowledge in an LLM without changing unrelated others. Recent works move from structured knowledge triples toward unstructured KE (UKE), where the edit is a free-form passage that may state multiple facts at once. Nonetheless, existing editors inject such a passage yet fail to use it: the edited model can recall the passage, but can neither answer atomic questions about its facts nor compose them into multi-hop reasoning. We attribute this missing property, which we term composability, to editors’ passive reliance on the fixed passage as the sole learning source. In response, we cast editing as a proactive self-distillation from a privileged in-context state of the same model, which requires no external supervision. We further reveal that due to the novelty of the injected knowledge, the pre-edited model’s own rollouts rarely cover it, which limits the effectiveness of pure on-policy distillation. To close this gap, we propose HPSE, which builds a hybrid rollout that steps in to place missing facts onto the student’s own trajectory precisely where its coverage fails, while staying on-policy elsewhere. We theoretically analyze HPSE’s advantage over pure on-policy distillation, and empirically establish its plug-and-play improvements across four LLM backbones and two KE editors under various scenarios.

[9] When the API Speaks the Wrong Language: Revisiting Post-Training for Multilingual Tool Use cs.CL | cs.AIPDF

Siddharth Chauhan, Thomas Butler, Abhishek Singhania, Pankaj Porwal, Honey Gupta

TL;DR: 本文研究了多语言环境下大型语言模型（LLM）进行API调用时出现的参数语言不匹配（ALM）问题，即模型选对工具但生成参数的语言不一致导致操作失效。论文重新评估了监督微调（SFT）和强化学习（RL）等后训练策略，发现SFT在提升参数语言一致性和端到端函数调用准确率方面提供了强基线，其性能与更复杂的RL方法相当甚至有时更优；而RL（如GRPO）虽能带来语言一致性和泛化能力的增量提升，但改进有限。

Details

Motivation: 解决LLM在多语言API调用场景中，因生成参数的语言与工具要求不一致（即ALM）而导致的可靠性下降问题，该问题未被标准API调用指标捕获。

Result: 在论文的基准测试中，监督微调（SFT）显著提升了参数语言一致性和端到端函数调用准确率，其性能与强化学习（RL）方法相当甚至有时更优；采用结构化、参数感知奖励的RL（如GRPO）能带来语言一致性和泛化能力的增量改进，尤其在泛化和多目标权衡方面表现更明显。

Insight: 论文的创新点在于识别并形式化了多语言工具使用中的ALM问题，并通过系统实验表明，仔细的监督训练（SFT）已能实现大部分性能提升，而RL仅提供针对性而非根本性的改进，这为多语言API落地提供了更简单有效的训练策略选择。

Abstract: The reliability of Large Language Models (LLMs) for API calling degrades in multilingual settings. A common failure occurs when a model selects the correct tool but generates argument values in an inconsistent language, which we term Argument Language Mismatch (ALM). Although semantically correct, such outputs are operationally invalid and not captured by standard API-calling metrics. We revisit post-training strategies for mitigating ALM and find that, in our benchmark, supervised fine-tuning (SFT) provides a strong baseline, substantially improving argument language consistency and end-to-end function call accuracy. Under consistent model selection, SFT achieves performance comparable to, and sometimes exceeding more complex reinforcement learning (RL) approaches. We further examine whether RL with structured, argument-aware rewards offers additional benefits. While methods such as Group Relative Policy Optimization (GRPO) can improve language consistency and better preserve general reasoning ability, these gains are incremental and most pronounced in generalization and multi-objective trade-offs. Overall, our results suggest that much of the performance in multilingual API grounding can be achieved through careful supervised training, with RL providing targeted rather than fundamental improvements.

[10] Ripple-Pivot Search: Active Parallel Decoding for Diffusion Large Language Models cs.CLPDF

Yushi Ye, Xu Chen, Haoyun Jiang, Jinsong Lan, Haihong Tang

TL;DR: 本文提出了一种名为Ripple-Pivot Search（RPS）的新型免训练解码方法，用于加速扩散大语言模型（dLLMs）的推理。该方法通过主动识别并解码中等熵值的‘支点’位置，利用其引发的‘涟漪效应’来降低后续位置的不确定性，从而允许在后续步骤中并行解码更多标记，显著提升整体解码速度。

Details

Motivation: 现有的dLLM并行解码调度器通常只在每个位置满足特定标准后才进行解码提交，忽略了早期解码决策可能对后续解码过程带来的益处。本文旨在利用解码过程中的‘涟漪效应’，通过主动选择关键位置来加速推理。

Result: 在3个dLLM模型和4个推理与代码生成基准测试上，RPS相比标准解码器实现了4-10倍的实时加速，同时保持了生成质量；相比之前的lookahead基线方法，在多数设置下实现了更高的吞吐量，并将准确率提升了最高5.49%。当与KV缓存结合时，RPS相比标准解码器进一步实现了最高18倍的实时加速。

Insight: 核心创新点在于识别并利用了dLLM解码中的‘涟漪效应’，并提出了一种通过前瞻评估来主动寻找和利用中等熵值‘支点’位置的免训练解码策略。这为并行解码调度提供了新思路，即通过策略性地提前解码特定位置来优化整体并行效率，而非仅依赖局部标准。

Abstract: Diffusion Large Language Models (dLLMs) have emerged as a competitive alternative to autoregressive language models, offering the potential for substantially faster inference through parallel decoding. Existing parallel decoding schedulers typically commit positions only after they meet a per-position criterion, overlooking how early commitments may benefit subsequent decoding. We identify a ripple effect in dLLM decoding: proactively committing a mid-entropy pivot position can induce a pronounced reduction in uncertainty across the remaining masked positions. This uncertainty reduction allows subsequent steps to unmask more tokens in parallel, thereby accelerating the overall decoding process. To exploit the ripple effect, we propose Ripple-Pivot Search (RPS), a novel training-free decoding method that seeks mid-entropy positions as promising candidate pivots (where to decode), and determines their token assignment that yields the greatest downstream benefit via lookahead evaluation (what to decode). Across 3 dLLMs and 4 reasoning and code-generation benchmarks, RPS achieves 4-10$\times$ wall-clock speedup over the standard decoder while preserving generation quality, and improves accuracy over the previous lookahead baseline by up to 5.49% while delivering higher throughput in most settings. When integrated with KV caching, RPS further achieves up to 18$\times$ wall-clock speedup over the standard decoder.

[11] AWARe: Mitigating Catastrophic Forgetting via Activation-Weighted Adaptive REtention cs.CLPDF

Juncheng Liao, Jinfan Lv, Guoming Wang, Jupeng Zheng, Ling Xiao

TL;DR: 该论文提出了一种名为AWARe（Activation-Weighted Adaptive REtention）的微调方法，旨在缓解多模态大语言模型（MLLMs）在下游任务微调时出现的灾难性遗忘问题。该方法通过基于激活模式动态控制参数更新，选择性地冻结对保留先前能力至关重要的参数，同时让次要参数适应新任务，且无需修改模型架构。

Details

Motivation: 多模态大语言模型在下游任务微调时，新任务特定的梯度更新会覆盖对先前知识至关重要的参数，导致灾难性遗忘，这限制了MLLMs的实际部署。

Result: 广泛的实验表明，与现有方法相比，AWARe在有效保留上游能力的同时，实现了更优的下游任务性能。

Insight: 论文的创新点在于提出了一种基于激活加权的自适应保留机制，通过动态计算参数重要性分数来指导选择性冻结，这是一种轻量级、无需改变模型架构的持续学习方法，可借鉴其利用模型内部激活模式来识别和保护关键知识的思路。

Abstract: Multimodal Large Language Models (MLLMs) exhibit strong generalization and reasoning abilities due to large-scale multimodal pre-training. However, fine-tuning these models on downstream tasks often leads to catastrophic forgetting, where newly learned task-specific knowledge degrades previously acquired capabilities. This issue arises because gradient updates for new tasks overwrite parameters critical to prior knowledge, limiting the practical deployment of MLLMs. To address this challenge, we propose Activation-Weighted Adaptive REtention (AWARe), a fine-tuning method that mitigates catastrophic forgetting by dynamically controlling parameter updates based on activation patterns. AWARe assigns activation-based importance scores to parameters, selectively freezing those essential for preserving prior capabilities while allowing less important parameters to adapt to new tasks. Importantly, AWARe operates without modifying model architectures, ensuring compatibility with existing inference engines. Extensive experiments demonstrate that AWARe effectively preserves upstream capabilities while achieving superior downstream performance compared to existing methods. Code is available at https://github.com/kaln27/AWARe.

[12] Causal Structure is Inducible but Functionally Decoupled: The Routing/Readout Boundary of a Typed Mechanism Library cs.CLPDF

Xining Xun

TL;DR: 该论文研究了语言模型在处理因果干预性问题时如何组织计算机制，发现通过类型级监督可以诱导出按类型划分的离散机制槽（typed mechanism library），这些机制槽用于路由但功能上与答案读出解耦。研究在具有精确干预性真实值的因果世界基准上，使用冻结协议和两种规模（22.6M和125M）的模型进行了验证，并报告了四个预注册发现。

Details

Motivation: 解决语言模型在回答干预性问题时，其内部计算如何根据查询所需的证据类型进行组织的问题，特别是探索因果知识的结构是否可诱导且功能解耦。

Result: 在因果世界基准上，使用22.6M和125M两种规模的模型进行实验，验证了类型级监督能诱导出按类型组织的机制槽结构，该结构作为路由索引与答案读出功能解耦（|Δŷ| ≤ 3.4×10⁻⁶），且不影响模型质量（与参数匹配的单体模型相差0.0082 nats以内），同时编辑后状态可精确本地化和可逆。

Insight: 创新点在于提出了类型化机制库的概念，揭示了因果结构可通过监督诱导但与功能解耦的边界（路由/读出边界），并强调了在跨规模比较时使用未经校准的无监督基线可能带来的混淆问题；方法上采用预注册和机器可检查标准增强了研究的可重复性和可信度。

Abstract: When a language model answers an interventional question, the computation it must perform depends on the type of evidence the query requires. We report a decoupling in how a transformer organizes causal knowledge: slot-by-type structure induced by type-level supervision organizes routing, yet remains functionally decoupled from answer readout. We establish this with a typed mechanism library – discrete mechanism slots partitioned by evidence type, auditable at the state level – on a causal-world benchmark with exact interventional ground truth, under a frozen protocol, at two scales (22.6M and 125M). Four preregistered findings. (i) Origin. Slot-by-type organization is induced by type-level supervision: absent in architecturally identical unsupervised controls, not buyable by content-free gating labels, and statistically attributable to the supervision signal, replicating at 125M under a powered preregistered protocol (all nine cells passed). (ii) Boundary. The induced structure is a typed routing index with a sharp routing/readout boundary: slot codes scaffold routing but do not drive answer readout ($|Δ\hat{y}| \le 3.4\times10^{-6}$, zero collateral, three seeds, stable across a 5.6x scale window) – we therefore make no behavioral-editability claim. (iii) Cost. The structure is free: LM quality matches a parameter-matched monolith within 0.0082 nats. (iv) Trust. The library state is exactly local under edit and bit-exactly revertible – 250 single-edit and 1,000 stacked reverts per seed, zero failures. We further find that the unsupervised null itself moves with scale, so comparisons reusing a null calibrated at one scale may be confounded at another. Every claim is tied to a preregistered, machine-checkable criterion archived before the data it governs; the full audit trail, including one criterion we failed and how the frozen protocol handled it, is released as an appendix.

[13] GRPO for Financial Advice Generation: Outperforming Commercial LLMs under CATE Evaluation cs.CL | cs.AI | cs.LGPDF

Ofir Ben Shoham, Shrutendra Harsola, Vignesh Subrahmaniam, Shravan Mohan, Yakov Gazman

TL;DR: 本文提出使用分组相对策略优化（GRPO）微调开源语言模型，以生成可执行的财务建议，并通过基于LLM的评估和独立的因果审计（CATE）验证其效果。该方法在财务建议生成任务中显著优于商业LLM基线，实现了更高的预估毛利润提升和更低的风险。

Details

Motivation: 从业务记录生成可执行的财务建议需要模型整合数值推理、领域知识和合理判断，同时避免有害建议，但直接监督困难，因为历史决策不一定最优且高质量标签获取成本高。

Result: 在基于条件平均处理效应（CATE）的观察性离策略审计中，训练后的LLM实现了约两倍于最强商业基线的预估毛利润提升（0.0228 vs. 0.0104），同时具有最低的下行率和最少的负尾部风险。

Insight: 创新点在于将财务建议生成构建为强化学习问题，使用GRPO进行微调，并结合基于LLM的评估和独立的因果审计来验证模型性能，这揭示了评估方法可能捕捉不同信号，因果审计可作为LLM评估的有价值补充。

Abstract: Generating actionable financial advice from business records demands that models integrate numerical reasoning, domain knowledge, and sound judgment, while avoiding recommendations that could harm the business. Direct supervision is difficult: historical decisions are not necessarily optimal, and high-quality free-form labels are expensive to obtain. We formulate financial advice generation as a reinforcement learning problem and fine-tune an open-weight language model using Group Relative Policy Optimization (GRPO). Our reward is an LLM-as-a-judge rubric that scores each recommendation across multiple binary dimensions of advice quality, augmented with a safety gate for harm prevention. Since LLM-based evaluation alone cannot confirm whether improvements reflect genuine business value rather than adaptation to the judge, we complement it with a judge-independent audit based on a standard doubly-robust Conditional Average Treatment Effect (CATE) estimator. Under this observational off-policy audit, our trained LLM achieves approximately twice the estimated gross-profit lift of the strongest evaluated commercial baseline ($0.0228$ vs.\ $0.0104$), together with the lowest downside rate and the least negative tail risk of any policy evaluated. Notably, the two evaluations do not rank the baselines identically: the untrained base model places last on the judge rubric but second on the causal audit, indicating that the audit captures a signal the judge does not. Our results demonstrate that GRPO with a finance-grounded reward signal can produce substantially more useful business recommendations than commercial LLMs, and that a judge-independent causal audit is a valuable complement to, rather than a confirmation of, LLM-as-a-judge assessment in financial NLP.

[14] LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured – Reinforced Polarizer Keeps a Frozen LLM from Being Confidently Misled by the Wrong Evidence cs.CL | cs.IR | cs.LGPDF

Po-Jen Ko, Che-Cheng Wu, Hung-Chun Hsu, Li-Yang Chang, Chuan-Ju Wang

TL;DR: 论文提出了LODESTAR方法，通过强化学习训练一个名为‘极化器’的固定自然语言字符串，将其插入冻结大语言模型的提示中，以引导模型在检索增强问答中更可靠地选择答案。该方法旨在解决传统基于最低熵的选择规则容易被误导性证据欺骗的问题，在多个QA基准测试上取得了优于现有方法的性能。

Details

Motivation: 传统基于答案令牌最低熵的选择规则在检索增强问答中存在缺陷：当模型读到误导性段落时，会变得‘自信地错误’，其熵值降低，使得信号看起来更可信。论文旨在通过干预模型读取的上下文（而非修改模型权重）来解决这一信任危机。

Result: 在5,008个问题、使用相同冻结模型和候选池的评估中，LODESTAR在五个QA基准上取得了最高的平均F1分数（从0.5148提升至0.5339）、最高的精确匹配率（0.4136）以及最高的GPT-4o法官评分（0.6435），在所有70个方法-数据集F1比较中均优于14个已发布配置，且结果具有统计显著性。

Insight: 核心创新在于提出了‘极化器’概念——一个通过离线强化学习训练、可插入提示的固定文本干预模块，它通过诱导第三方冻结模型的‘不确定性’来评估文本干预效果，从而引导模型更审慎地处理证据，降低了模型被误导性段落欺骗的频率（从30.3%降至26.0%）。这为不修改模型权重、仅通过提示工程提升模型鲁棒性提供了新思路。

Abstract: Predictive-distribution entropy makes a strong selection rule in retrieval-augmented question answering: across five QA benchmarks, keeping the candidate answer that a frozen respondent LLM produces with the lowest answer-token entropy lifts mean answer $F_1$ from 0.4769 to 0.5148 over the retriever’s top-ranked passage, with no gold answers. Yet this lowest-entropy rule, which prior entropy-based selectors adopt, fails in a specific and consequential way: a misleading passage makes the respondent confidently wrong, driving its entropy down precisely where the signal looks most trustworthy. We show that the failure comes from the passage the respondent reads – and the context that passage is read in is an input we can intervene on. We introduce LODESTAR, to our knowledge the first method to score a text intervention by the uncertainty it induces in a third-party frozen respondent, compared across one question’s candidates. LODESTAR uses reinforcement learning to train, once and offline, a polarizer – a short fixed natural-language string inserted into the respondent’s prompt and never into its weights; its training labels are built offline from gold answers and two LLM judges, and inference reads neither. Evaluating every competing selector under the same frozen respondent and the same candidate pools on 5,008 questions, LODESTAR attains the highest mean $F_1$ of any inference-ready selector (0.5148 to 0.5339), the highest exact match (0.4136), and the highest GPT-4o judge score of the frozen-respondent configurations judged (0.6435); its three-seed mean wins all 70 method-by-dataset $F_1$ cells against fourteen published configurations while remaining paired-significant against every one. The gain holds both in-domain and out-of-domain, and ablating the polarizer shows it is what makes the respondent read a misleading passage less often (26.0% against 30.3%).

[15] SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges cs.CLPDF

Yuchao Wu, Junqin Li, XingCheng Liang, Yongjie Chen, Yinghao Liang

TL;DR: 本文提出了SAG（SQL-Retrieval Augmented Generation），一种用于检索增强生成（RAG）的结构化检索架构。它通过将文档组织成事件-实体索引，在查询时动态构建基于共享实体的超边来连接相关文本块，从而有效处理结构化约束和多跳推理问题，避免了构建和维护全局知识图谱的复杂性。

Details

Motivation: 主流基于稠密向量的RAG方法在处理结构化约束和多跳推理时存在固有局限，而基于图的方法虽然能部分解决，但通常需要离线构建知识图谱，导致语义碎片化、维护成本高且增量更新复杂。SAG旨在克服这些缺点。

Result: 在HotpotQA、2WikiMultiHopQA和MuSiQue基准测试上，SAG在检索和端到端问答性能上均达到最佳，且随着推理链复杂性增加，优势扩大。在最具挑战性的MuSiQue上，SAG的Recall@5达到80.36%，比最强基线高出11.52个百分点。

Insight: 核心创新在于提出了一种无需构建全局知识图谱的结构化检索方法：将文本块表示为语义完整的事件及其关联实体，形成潜在的“超边”以保留n元关系；查询时，以共享实体为连接键动态构建查询范围内的证据邻域，所有证据均保持原始文本块形式，便于维护和增量更新。

Abstract: While retrieval-augmented generation (RAG) has proven effective at giving LLMs access to external knowledge, mainstream dense-retrieval implementations remain inherently limited in handling structured constraints and multi-hop reasoning. Graph-based methods address this by constructing knowledge graphs offline, but they often fragment semantics, incur high maintenance, and complicate incremental updates. We propose SAG (SQL-Retrieval Augmented Generation), a structured retrieval architecture that organizes documents into an event-entity index without building a global knowledge graph. SAG represents each chunk as a semantically complete event paired with its entities, forming a latent hyperedge that preserves n-ary relations without decomposing them into triples. At query time, SAG treats shared entities as join keys to connect related chunks. This dynamically yields a query-scoped neighborhood of events, and yet every piece of evidence remains the original chunk throughout. Experiments on HotpotQA, 2WikiMultiHopQA, and MuSiQue show that SAG achieves the best retrieval and end-to-end QA performance on every benchmark, with gains that widen as reasoning-chain complexity increases. On MuSiQue, where multi-hop evidence chaining is most demanding, SAG reaches 80.36% Recall@5, outperforming the strongest baseline by 11.52 points. This work paves the way for knowledge infrastructure that enables LLM agents to retrieve and reason over continually growing organizational knowledge.

[16] One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL cs.CL | cs.AI | cs.LGPDF

Simon Yu, Nicholas Tomlin, Marwa Abdulhai, Ximing Lu, Derek Chong

TL;DR: 本文研究了多智能体强化学习中单一大型语言模型模拟用户行为导致的泛化失败问题，即模拟器崩溃现象。作者从理论和实验上分析了该问题，并提出两种解决方案：推理时的言语化采样和训练时的协同训练，以提升策略在未见模拟器和真实用户上的泛化能力。

Details

Motivation: 解决多智能体强化学习中因依赖单一LLM模拟用户行为而导致的模拟器崩溃问题，即策略过度拟合模拟器的主导模式，从而在泛化到新模拟器和真实用户时性能下降。

Result: 在三个多轮对话基准测试（Persuasion for Good、τ²-bench、CooperBench）上验证，言语化采样相比单一模拟器RL将未见成功率提升高达9%，协同训练进一步将增益提升至14%；人类研究也显示出对真实用户的类似增益。

Insight: 创新点在于理论形式化模拟器崩溃并提出两种互补解决方案，强调训练环境多样性对多轮RL泛化的重要性；客观来看，协同训练框架SCOPE为后续研究提供了开源工具，推动了多智能体RL中环境多样性的探索。

Abstract: Multi-agent reinforcement learning for human-AI interaction typically relies on a single large language model to simulate user behavior. We show that this approach systematically fails to generalize, and trace the failure to simulator collapse: because the simulator LLM is mode-collapsed, an LLM policy trained against it overfits to narrow strategies that exploit the simulator’s dominant mode, and such a policy transfers poorly to unseen simulators and real users. We formalize this collapse theoretically and propose two complementary solutions, one at inference time and one at training time. The inference-time solution, Verbalized Sampling, broadens the simulator’s behavior by sampling from a verbalized response distribution, reducing mode collapse. The training-time solution, Co-Training, jointly optimizes the policy against a population of trainable simulators, preventing it from overfitting to any single simulator’s mode. We validate both solutions on three multi-turn benchmarks: Persuasion for Good, $τ^2$-bench, and CooperBench. Verbalized Sampling improves held-out success by up to 9% over single-simulator RL, and Co-Training pushes gains further to 14%; the human study shows similar gain on real users. Both solutions preserve the policy diversity that collapses under single-simulator RL. To support further work in this direction, we release SCOPE, an open-source framework for Population Co-Training multi-agent RL. More broadly, our results suggest that the diversity of the training environment, not only the policy, is critical to the generalization of multi-turn RL to real-world deployment.

cs.CV [Back]

Wonbong Kim, Jiatong Xiao, Rui Li, Xufei Wang, Qiwen Gu

TL;DR: 本文提出GeoUniPR，一个基于几何一致性的统一框架，用于解决视觉与激光雷达之间的跨模态地点识别问题。该方法通过将激光雷达点云投影到相机视角构建几何一致的深度图像视图，并结合强度、法向量等原生激光雷达线索，形成一个多通道几何表示。在此基础上，使用两个结构相同的模态特定ViT编码器，通过参数高效的适配方法学习统一的嵌入空间，并提出了空间一致性的对比损失SC-InfoNCE来抑制空间连续性导致的假负样本。

Details

Motivation: 现有跨模态地点识别方法通常依赖复杂的对齐模块、多阶段训练或对预训练主干网络进行全微调，以弥合模态差异。本文从几何一致性的角度重新审视该问题，旨在通过更简洁统一的框架来减少跨模态差异。

Result: 在KITTI和KITTI-360数据集上的大量实验表明，GeoUniPR在同模态和跨模态地点识别任务上均达到了最先进的性能，并展现出强大的跨数据集泛化能力。

Insight: 核心创新在于从几何一致性出发，通过投影构建直接对应的RGB-激光雷达表示，并利用多通道几何信息增强结构一致性。方法避免了复杂的对齐设计和繁重的训练策略，仅通过参数高效的适配和专门的对比损失（SC-InfoNCE）即可实现高性能，为跨模态表示学习提供了简洁有效的思路。

Abstract: Cross-modal place recognition (CMPR) aims to identify the same location across heterogeneous sensing modalities, such as vision and LiDAR. Existing methods commonly bridge the modality gap using complex alignment modules, multi-stage training, or full fine-tuning of pretrained backbones. In this work, we revisit CMPR from the perspective of geometric consistency and propose GeoUniPR, a unified and concise geometry-consistent framework. GeoUniPR reduces cross-modal discrepancy at the representation level by projecting LiDAR point clouds into the camera perspective to construct Geometry-Consistent depth image views (DIV), which establish direct RGB-LiDAR correspondence. We further augment DIV with native LiDAR cues, including intensity and surface-normal information, yielding a multi-channel geometric representation that improves structural consistency. Based on this representation, GeoUniPR learns a unified embedding space using two modality-specific ViT-based encoders with identical architectures, trained through parameter-efficient adaptation without auxiliary alignment modules, multi-stage training, or full backbone fine-tuning. In addition, we introduce Spatially-Consistent InfoNCE (SC-InfoNCE), a CMPR-specific contrastive objective that suppresses distance-induced false negatives under spatial continuity. Extensive experiments on KITTI and KITTI-360 demonstrate that GeoUniPR achieves state-of-the-art (SOTA) performance in both same-modal and cross-modal place recognition, with strong cross-dataset generalization.

[18] Self-Evolving Code-with-Image Reasoning cs.CVPDF

Tianze Yang, Liang Wu, Ruitong Sun, Yucheng Shi, Yanqiao Wang

TL;DR: 这篇论文提出了‘Code-with-Image’推理范式，用于解决需要执行多步视觉算法的视觉任务。核心方法是让模型通过Python解释器实现视觉算法代码，并引入一种无需训练的自进化反思循环，让模型从自身失败的代码中学习并积累可移植的技能。

Details

Motivation: 现有多模态模型在解决视觉任务时，其推理仍停留在语言层面，无法真正执行算法。对于需要执行多步视觉计算的任务，模型即使能描述算法也常出错，因为语言无法运行算法。本文旨在突破这一感知瓶颈，让代码本身成为推理载体。

Result: 在作者构建的Code-with-Image Bench (CwI-Bench)基准测试上，GPT-5.6-luna模型在无工具思维链下准确率低于30%，提供解释器后达到43%，而通过自进化反思获得技能后提升至67%。一个开源的27B参数模型也表现出类似的提升轨迹（9% → 33% → 56%）。

Insight: 创新点在于将视觉推理从语言描述转变为可执行代码的实现，并提出了一个通过‘反思循环’让模型自我调试和进化技能的无训练方法。这提供了一种通过代码调试来调试推理过程的新视角，且生成的技能是纯文本，具有跨模型规模和任务族的可迁移性。

Abstract: Multimodal models increasingly reach for tools when solving visual tasks (crop, zoom, rotate, brighten), a paradigm known as thinking-with-images. The central challenge is one of perception: tools mostly serve to expose visual evidence, reasoning over that evidence stays in language, and most targets are ones a human could in principle determine by inspection. Some visual questions, however, are not bottlenecked by perception: recovering their answers requires executing a multi-step visual algorithm over the pixels. On such questions a model often names the correct algorithm at once yet still answers wrong, because language can describe an algorithm without being able to run one. Code-with-Image crosses that line: given nothing but a Python interpreter, the model must implement a genuine visual algorithm in code to solve the task; the program itself becomes the reasoning. The bottleneck then shifts from executing code to deciding which algorithm to implement. So we let the model teach itself: a training-free reflection loop studies its own failed programs, tests repairs against constructive ground truth, and keeps what survives as portable skills. On our Code-with-Image Bench (CwI-Bench), thirty task families induced by hidden visual computations with disjoint learning and evaluation splits, even GPT-5.6-luna stays below 30% with tool-free chain of thought; given a bare interpreter it reaches 43%, and with skills evolved through its own executable reflection, 67%. The open 27B model climbs the same ladder (9% $\rightarrow$ 33% $\rightarrow$ 56%), and the skills are plain text, transferable across scales and families. When code carries the reasoning, debugging code becomes debugging reasoning.

[19] VLMs Win a Systematic Evaluation of Underwater Image Reconstruction cs.CVPDF

Sara Aghajanzadeh, Yingxue Wang, Ieva Bagdonaviciute, David Forsyth

TL;DR: 本文提出了一种用于水下图像重建的系统化评估流程，用于评估方法的准确性、重建一致性以及水参数的影响。该流程被用于评估多种现有方法，包括基于近似物理散射模型的方法和视觉语言模型（VLMs）。评估结果表明，VLMs在整体上显著优于基于物理模型的方法，这很可能归因于其强大的图像先验。

Details

Motivation: 目前水下图像恢复领域缺乏系统化的评估方法，本文旨在建立一个标准化的评估流程，以全面、客观地比较不同重建方法的性能。

Result: 在提出的系统化评估流程下，视觉语言模型（VLMs）在准确性、重建一致性等方面全面且显著地超越了基于物理模型的方法。在真实水下场景图像上的结果也强烈证实了这一评估结论。

Insight: 论文的创新点在于提出了首个系统化的水下图像重建评估流程。一个关键的发现是，尽管VLMs未明确使用物理模型，但其强大的图像先验能力使其在该任务上超越了传统的物理模型方法，这挑战了该领域依赖显式物理建模的传统思路。

Abstract: Underwater image restoration consists of recovering an image which looks like there is no water present. To date, evaluation has not been systematic. This paper describes a systematic evaluation pipeline for underwater reconstruction, which can be used to assess a method for accuracy; consistency of reconstruction over camera moves; and the effect of water parameters. We use this pipeline to evaluate a range of current procedures, from models constructed using explicit but approximate physical models of scattering to Vision-Language Models (VLMs which are not currently trained with explicit physical models). Overall, VLMs wholly and significantly outperform physically based models in our evaluation, likely because of the importance of a strong image prior. Results on images of real underwater scenes strongly confirm the evaluation.

[20] Clinical Feasibility of Low-Magnification Fluorescence Imaging for Breast Cancer Margin Detection Using Texture Analysis and Deep Learning cs.CV | cs.AI | cs.LGPDF

Pouya Afshin, Tianling Niu, Tongtong Lu, David Helminiak, Julie Jorns

TL;DR: 本研究比较了使用4倍和10倍放大倍率的MUSE（紫外表面激发显微镜）成像技术，结合纹理分析（基于局部二值模式）和深度学习（基础视觉Transformer模型）进行乳腺癌手术切缘检测。结果表明，两种放大倍率在诊断准确性上表现相当，但4倍放大具有更大的视野和更快的图像采集速度，因此更适合用于术中快速、准确的切缘评估。

Details

Motivation: 解决在乳腺癌手术中快速、准确检查手术切缘的问题，评估不同放大倍率的MUSE成像技术在实际临床应用中的可行性，以优化术中诊断流程。

Result: 在基于深度学习的分类中，4倍和10倍放大均达到96.30%的灵敏度、100%的特异度和98.18%的准确率；基于纹理分析的方法，两种放大倍率准确率均为96.67%，但4倍特异性更优（100%），10倍灵敏度更高（100%）。总体而言，10倍放大未带来性能的明显提升。

Insight: 论文的创新点在于通过实验证明，在乳腺癌切缘检测任务中，较低放大倍率（4倍）的MUSE成像可以达到与较高放大倍率（10倍）相当的诊断准确性，同时兼具视野大、成像快的优势，这为优化临床术中成像系统提供了实用依据。从客观角度看，将纹理分析与深度学习模型（ViT）结合用于不同分辨率图像的比较分析，也是一种值得借鉴的多方法验证思路。

Abstract: High-resolution images of unprocessed surgical breast tissue can be obtained using microscopy with ultraviolet surface excitation (MUSE). This technique is considered a promising method for checking surgical margins during breast cancer surgery. In this study, MUSE images at 4x and 10x magnifications were compared using patch-level classification methods. Texture analysis (TA) based on local binary patterns (LBP) and deep learning (DL) with a base Vision Transformer (ViT) model were used. Both methods achieved similar performance at both magnifications. Using DL method, both 4x and 10x magnifications achieved 96.30% sensitivity, 100% specificity and 98.18% accuracy. Using TA method, 4x achieved better specificity (100% vs 93.33%) and 10x yielded higher sensitivity (100% vs 93.33%), but both had the same accuracy (96.67%). No clear improvement in performance was observed with 10x magnification. These results show that 4x imaging achieves the same diagnostic accuracy as 10x imaging. At the same time, 4x offers a larger field of view and faster image capture. Therefore, lower magnification can be effectively used in MUSE systems for accurate and efficient intraoperative margin assessment.

[21] Multi-Agent Target-Existence Verification and Learned Mask Geometry Refinement: Winning Report of the MeViS-Text Track at the 8th LSVOS Challenge 2026 cs.CVPDF

Jungyoon Lee, Gyuil Lim, Doeon Kim, Seong-heum Kim

TL;DR: 本文介绍了在第八届大规模视频对象分割挑战赛MeViS-Text赛道中获得第一名的解决方案SSUPER。该方案处理由文字运动表达引导的视频对象分割任务，特别针对描述不存在的对象的欺骗性无目标表达。系统通过多模态大语言模型解析表达、生成候选掩码并选择目标，并引入独立的多智能体存在性验证模块来专门处理无目标情况，最后通过风格细化器调整掩码几何形状以匹配数据集标注风格。

Details

Motivation: 解决MeViS-Text赛道中由文字运动表达引导的视频对象分割问题，核心挑战在于需要准确处理那些描述视频中不存在对象的欺骗性无目标表达，这类表达必须在所有帧中输出空掩码。现有方法在联合决策选择和存在性时，容易被类别合理的候选掩码误导，导致无目标案例漏判。

Result: 在官方挑战赛排行榜上，完整系统达到了0.9081339614的最终分数。通过引入独立的多智能体存在性验证模块，无需新的分割调用，就恢复了大部分残留的无目标错误。

Insight: 主要创新点在于将目标存在性验证解耦为一个独立的多智能体审计过程，该过程全面审计整个谓词（类别、数量、动作、轨迹、事件顺序和语义角色），以区分目标缺失和暂时不可见，并需要矛盾证据而非仅仅不确定性来做出无目标判决。另一个见解是，在语义决策固定后，部分剩余误差是标注风格差异导致的，而非语义错误，因此可以通过仅使用训练数据的风格细化器来对齐掩码几何形状，从而提升性能。

Abstract: We present the first-place solution to the MeViS-Text track of the 8th Large-scale Video Object Segmentation (LSVOS) Challenge 2026: referring video object segmentation guided by written motion expressions, including deceptive no-target expressions that match no object in the video and must yield empty masks in every frame. Our pipeline, SSUPER, resolves each expression into a visual concept, generates full-video candidate masklets with SAM~3.1, and selects target IDs. At every reasoning stage, three heterogeneous multimodal large language models independently execute the same stage-specific prompt before a single synthesis pass commits one schema-validated verdict. Although this system rejects every no-target expression in validation, the leaderboard reveals that a substantial share of test no-target cases still slips through. The reason is that hard negatives name a plausible object and fail only under the complete temporal predicate, so when selection and existence are decided together, a category-plausible masklet anchors the verdict. Hence, we decouple existence verification into an independent multi-agent audit of the full predicate (category, count, action, trajectory, event order, and semantic role) that distinguishes absence from temporary invisibility, discounts apparent motion caused by camera movement, and requires contradicting evidence rather than mere uncertainty for a no-target verdict. Without any new segmentation call, this audit recovers most of the residual no-target errors. A training-data-only StyleRefiner then aligns mask geometry with the annotation style of MeViSv2 while preserving every presence decision by construction, showing that once the semantics are fixed, part of the remaining error is stylistic rather than semantic. The complete system reaches a Final score of 0.9081339614 on the official challenge leaderboard.

[22] Test-Time Hallucination Control in Large Vision-Language Models cs.CVPDF

Mehran Tamjidi, Hamidreza Dastmalchi, Ali Cheraghian, Mohammadreza Alimoradijazi, Aijun An

TL;DR: 本文提出了一种名为测试时幻觉缓解（TTH）的新型免训练方法，用于解决大型视觉语言模型（LVLM）中的物体幻觉问题。该方法通过引入一个基于零样本多模态分类器的令牌验证器模块，生成与输入图像相关的辅助逻辑值，并将其与原始LVLM输出在选定对象令牌上进行融合，最后应用基于熵的加权方案以实现稳健预测。

Details

Motivation: LVLM在生成关于输入图像的非事实内容（即物体幻觉）是其实际应用可靠性的关键障碍。现有免训练方法要么需要多轮解码增加计算开销，要么以模型特定的方式修改内部状态，可能损害预训练知识。本文旨在提出一种高效且通用的免训练方法来解决这些局限性。

Result: 在多个LVLM家族和多样化基准测试上的广泛实验表明，TTH方法持续提高了准确性和鲁棒性，证明了其良好的泛化能力和实际有效性。

Insight: 核心创新在于提出了一种免训练的、基于令牌验证和融合的测试时幻觉控制框架。具体包括：1）使用零样本多模态分类器作为令牌验证器，无需额外训练即可生成图像相关的辅助逻辑值；2）设计了令牌级融合和基于熵的加权机制，在缓解幻觉的同时保护了模型的预训练知识，且计算开销低、通用性强。

Abstract: Object Hallucination in large vision-language models (LVLMs), where models generate non-factual content about input images, remains a critical barrier to their reliability in real-world applications. Existing mitigation strategies can be categorized into training-based and training-free methods. Training-based methods often achieve strong performance but are costly, requiring extensive computational resources, large-scale data, and time-consuming fine-tuning. Training-free approaches are particularly appealing due to their efficiency. However, existing training-free methods either require multiple decoding rounds, which adds computational overhead, or modify internal states in a model-specific way that risks degrading pretrained knowledge. We propose Test-Time Hallucination Mitigation (TTH) method, a novel training-free method that addresses both limitations. TTH introduces a token-validator module, implemented as a zero-shot Multi-Modal Classifier (MMC), to generate auxiliary logits grounded in the input image. These logits are fused with the original LVLM outputs at the token level for object tokens selected from a candidate pool. An entropy-based weighting scheme is then applied to enable robust and accurate predictions. Extensive experiments across multiple LVLM families and diverse benchmarks demonstrate that TTH consistently improves accuracy and robustness, underscoring its generalizability and practical effectiveness. Code is released at https://github.com/Mehran-TAM/TTH

[23] Gaussian Meta-Space Augmentation for Stacking Ensembles in Multimodal IPMN Risk Stratification cs.CV | cs.LGPDF

Max A. Nelson, Eminenur Sen Tasci, Zhixiang Wang, Zongwei Zhou, Halil Ertugrul Aktas

TL;DR: 本文提出了一种名为cUPMI的高斯元空间增强方法，用于改进多模态IPMN风险分层的堆叠集成模型。该方法通过对集成组合器的对数概率元特征进行类条件高斯增强，以正则化并提升模型性能。在多中心数据集上的实验表明，cUPMI能有效正则化高容量树组合器，并在融合放射组学和2.5D CNN特征时，实现了超越基线模型的性能。

Details

Motivation: 胰腺癌是致死率极高的恶性肿瘤，对导管内乳头状粘液性肿瘤（IPMN）进行风险分层是早期干预的关键，但通常需要侵入性组织活检。现有的基于视觉的方法（如放射组学和深度学习）提供了有前景但独立的判别机会，而多序列MRI和解剖分解分析提供了互补信息。有效融合这些信息对于有序IPMN异型增生风险预测至关重要。

Result: 在多中心分析中，cUPMI对经过适当正则化的L2逻辑回归二元分类堆叠增益有限，但在二元和仅放射组学设置中，能一致地正则化高容量树组合器（RF AUC提升0.015，XGBoost AUC提升0.024）。在8流放射组学任务上，XGBoost获得了最清晰的序数效益（3分类QWK提升0.022）。此外，放射组学与2.5D CNN流的折叠锁定融合产生了最强的整体模型，RF堆叠达到QWK 0.595和二元AUC 0.839，超越了放射组学、2.5D ResNet和3D DenseNet-121基线。

Insight: 论文的创新点在于提出了cUPMI，一种针对集成组合器元特征的类条件高斯增强方法，旨在通过数据增强正则化元学习空间，提升多模态融合的鲁棒性。客观来看，该方法的核心洞察是将集成学习中的元特征空间视为可增强的对象，通过引入特定于类的噪声来防止过拟合并改善泛化，特别是在处理高容量组合器时效果显著。同时，研究展示了多模态（放射组学与深度学习特征）的折叠锁定融合策略的有效性，为医学影像多模态集成提供了技术参考。

Abstract: Pancreatic cancer is among the most lethal malignancies; risk stratification of intraductal papillary mucinous neoplasms (IPMNs) offers a crucial opportunity for early intervention but typically requires invasive tissue biopsy. Dominant vision-based approaches, including radiomics and deep learning, provide promising but initially separate discrimination opportunities. Similarly, multisequence MRI (T1W/T2W) and anatomically decomposed (head, body and tail) analysis of the pancreas provide additional and potentially complementary signals. Effective fusion of this information is crucial in ordinal IPMN dysplasia risk prediction and can be accomplished via a meticulously regularized and calibrated ensemble stacking combiner. We present cUPMI, a class-conditional Gaussian augmentation of a combiner’s log-probability meta-features, and test it on various prediction paradigms. In our multi-center analysis, we find cUPMI adds limited value to properly regularized L2-logistic binary classification stacks, but consistently regularizes higher-capacity tree combiners in the binary and radiomics-only setting (RF +0.015 and XGBoost +0.024 binary AUC, positive in all seeds). Its cleanest ordinal benefit appears for XGBoost on an 8-stream radiomics task (3-class no < low < high, +0.022 QWK in all seeds). Separately, fold-locked fusion of radiomics and 2.5D CNN streams yields the strongest overall model, an RF stack reaching QWK 0.595 (95% CI [0.54, 0.64]) and binary AUC 0.839, surpassing radiomics, 2.5D ResNet, and 3D DenseNet-121 baselines.

[24] From Synthesis to Removal: Physics-Grounded Reflection Simulation and Diffusion-Based Video Dereflection cs.CV | cs.AI | eess.IVPDF

Zepeng Wang, Jiagao Hu, Fuhao Li, Yuxuan Chen, Fei Wang

TL;DR: 本文提出了一个名为S2R的闭环框架，用于解决视频反射去除问题。该框架集成了基于物理的反射模拟（S2R-Synthesis）、基于扩散模型的视频去反射（S2R-Removal）以及首个视频去反射基准测试（S2R-Bench）。

Details

Motivation: 现有研究主要集中在单图像反射去除，而视频反射去除由于缺乏成对数据、时序一致的去除模型和专门的评估基准，仍未被充分探索。本文旨在解决这些问题。

Result: 在自建的S2R-Bench和多个公共图像基准测试上的实验表明，该方法达到了最先进的性能，并且推理速度甚至比非扩散基线模型更快，同时验证了S2R-Synthesis的有效性。

Insight: 主要创新点在于构建了一个从数据合成到模型训练再到评估的完整闭环系统。具体包括：在结构空间进行基于物理的增强以模拟关键玻璃效应；首次将扩散模型应用于视频反射去除，并通过反射感知的潜在适应和一步像素几何细化实现快速推理；建立了首个支持全参考评估和真实世界人类感知评估的视频去反射基准测试。

Abstract: Videos captured through glass often contain reflections that degrade visual quality and interfere with downstream vision tasks. Although single-image reflection removal has been extensively studied, video reflection removal remains largely underexplored due to the lack of paired video data, temporally coherent removal models, and dedicated evaluation benchmarks. We present a closed-loop framework that unifies physics-grounded reflection simulation, diffusion-based video dereflection, and benchmark evaluation. Our S2R-Synthesis pipeline generates paired reflected and reflection-free videos by performing physics-grounded augmentation in the structure space and rendering realistic reflected videos with a trained video diffusion renderer; the augmentation models key glass-related effects including roughness-induced blur, thickness-induced ghosting, and reflectance variation. Based on the synthesized data, we introduce S2R-Removal, the first diffusion-based video reflection removal model, which adapts a pretrained video diffusion prior through reflection-aware latent adaptation and one-step pixel-geometric refinement, recovering the clean transmission in a single denoising step. We further build S2R-Bench, the first benchmark for video reflection removal, supporting both full-reference evaluation and real-world human perceptual assessment. Experiments on S2R-Bench and multiple public image benchmarks demonstrate state-of-the-art performance and faster inference than even non-diffusion baselines, and validate the effectiveness of S2R-Synthesis. Project page: https://codingwzp.github.io/VideoDereflection_S2R.

[25] Repurposing RGB-based Foundation Model for Depth Estimation on Thermal Images Using Hierarchical Supervision cs.CV | cs.ROPDF

Jie Hong, Tingtian Li, Xuesong Li, Xiao Li

TL;DR: 本文提出RGB-HS框架，通过利用RGB基础模型的多层次监督，将知识迁移到热图像深度估计任务中。该框架采用并行分支结构，在多个层级对齐热图像和RGB图像的编码器token，并引入基于RGB图像质量的验证机制来优化对齐过程。

Details

Motivation: 热图像深度估计在夜间、雨天等恶劣条件下对机器人应用极具价值，但现有方法未能充分利用RGB基础模型所编码的丰富层次化表示。

Result: 在流行基准测试上的大量实验表明，RGB-HS取得了有竞争力的性能，更有效地利用了RGB基础模型的表示能力进行热图像深度估计。

Insight: 创新点在于通过多层次token对齐和基于图像质量的验证机制，使热图像分支能从RGB教师分支中同时捕获结构精度和语义抽象，从而更充分地挖掘基础模型的表征潜力。

Abstract: Depth estimation from thermal images is highly valuable for robotic applications in adverse conditions, such as nighttime and rainy weather. Recent studies have sought to transfer knowledge from RGB-based foundation models to thermal modalities, yet the rich hierarchical representations these models encode remain underutilized. To address this limitation, we propose RGB-HS, a novel framework for thermal-image depth estimation that leverages hierarchical supervision from an RGB-based foundation model. Specifically, we first replace the baseline thermal encoder with a foundational model and introduce a parallel RGB branch that also employs a foundational model as an encoder of the same architecture, taking RGB images as input. The alignment is then performed across multiple levels between the tokens of the two encoders, allowing the thermal student branch to capture both structural precision and semantic abstraction from the RGB teacher branch. Furthermore, we introduce verification to refine the alignment process by weighting tokens from the RGB branch based on RGB image quality. Extensive experiments on the popular benchmark demonstrate that RGB-HS achieves competitive performance and more effectively exploits the representational capacity of RGB-based foundation models for depth estimation on thermal images.

[26] A Hybrid Framework of Vision Transformer and Gated Recurrent Unit for Detection of Mosquito Diseases cs.CV | cs.AIPDF

Danial Sharifrazi, Saadat Behzadi, Nouman Javed, Roohallah Alizadehsani, Prasad N. Paradkar

TL;DR: 该论文提出了一种结合Vision Transformer和卷积GRU的混合框架，用于从视频中检测蚊子是否感染登革热病毒。该框架首先使用YOLO 11M模型进行蚊子识别和背景去除，然后利用ViT提取视觉特征，最后通过卷积GRU分类器对视频序列进行分类。

Details

Motivation: 由于蚊子体积小且视频背景复杂，从蚊子运动行为中识别登革热病毒感染蚊子是一项重大挑战，传统AI方法难以从视频帧中提取准确特征。

Result: 在比较了RNN、LSTM、GRU及其卷积版本后，提出的ConvGRU模型取得了最佳性能，准确率达到88.88%，精确率84.45%，召回率82.82%，F1分数82.81%。

Insight: 创新点在于将卷积操作与基于序列的网络（特别是ConvGRU）相结合，能够同时从蚊子运动中提取精确的空间特征和长期时间依赖性，为复杂环境下的蚊子行为分析提供了可靠解决方案。

Abstract: Identifying dengue virus-infected mosquitoes from control mosquitoes is a major challenge in analyzing mosquito locomotion behavior due to the small size and complexity of the video background. Conventional AI methods are often unable to extract accurate features from video frames and produce erroneous features. In this study, a three-step framework is introduced: first, mosquitoes are identified and the background is removed using the YOLO 11M model, then visual features are extracted using the Vision Transformer (ViT), and finally the videos are classified with a convolutional GRU (ConvGRU) classifier. A comparative analysis of different models, including Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and their convolutional versions showed that the ConvGRU model achieved the best performance; it achieved 88.88% accuracy, 84.45% precision, 82.82% recall, and 82.81% F1 score. These results demonstrate that combining convolutional models with sequence-based networks, especially in the ConvGRU model, allows the simultaneous extraction of precise spatial features and long-term temporal dependencies from mosquito movements. Finally, the proposed framework provides a reliable solution for analyzing mosquito behavior in complex environments.

[27] Generative Video Compression Based on Hierarchical Referencing cs.CVPDF

Daowen Li, Ding Ding, Zifu Zhang, Kai Li, Ying Chen

TL;DR: 本文提出了一种基于分层参考的生成式视频压缩方法GVCHR，通过将潜在帧组织成层次结构，并耦合分层参考结构和分层质量结构来提升编码效率。该方法在潜在编码阶段引入了分层时序上下文挖掘以利用互补的时序信息，在生成重建阶段则通过分层注意力适配器来减少伪影传播。

Details

Motivation: 现有基于扩散的生成式视频压缩方法在潜在帧编码时，既未精心设计参考和质量结构，也未考虑帧级质量变化对去噪过程的影响，这限制了编码效率并加剧了生成重建过程中的伪影传播。

Result: 在多个基准测试上的实验表明，与之前的最先进方法相比，GVCHR在LPIPS和DISTS指标上分别实现了50.5%和54.0%的BD-rate增益，同时显著提升了视觉质量。

Insight: 创新点在于提出了一个耦合的分层参考与质量结构，并设计了分层时序上下文挖掘和分层注意力适配器，从而在编码效率和生成重建质量之间取得了更好的平衡，有效减少了伪影传播。

Abstract: Diffusion-based generative video compression has emerged as a promising paradigm to improve perceptual quality, where latent frames are required to be encoded efficiently while serving as denoising conditions. However, existing methods neither carefully design reference and quality structures during latent coding nor account for the impact of frame-level quality variation on denoising procedure, which limits coding efficiency and aggravates artifact propagation during generative reconstruction. In this paper, we propose GVCHR, Generative Video Compression based on Hierarchical Referencing. The key idea is to organize latent frames hierarchically, where the selected high-quality references benefit both latent coding and generative reconstruction. In latent coding, GVCHR couples a hierarchical reference structure with a hierarchical quality structure, assigning more bits to lower-layer frames that are reused more frequently as references. Built on this design, we introduce Hierarchical Temporal Context Mining to exploits complementary short- and long-term temporal context for effective latent coding. In generative reconstruction, the coding-side hierarchy is incorporated into a Hierarchical Attentive Adapter which is attached to a video diffusion transformer. This adapter uses hierarchical attention to restrict each latent frame to attend only to the same- or lower-layer references, thereby reducing artifact propagation during denoising. Experiments validate GVCHR on multiple benchmarks. Compared with the previous state-of-the-art method, GVCHR achieves 50.5% and 54.0% BD-rate gains in terms of LPIPS and DISTS, respectively, while also delivering clearly improved visual quality.

[28] Cloak of Invisibility: Real-Time Privacy-Preserving Volumetric Video Streaming cs.CVPDF

Hossein Khalili, Philip Do, Alexander Vilesov, Kittipat Apicharttrisorn, Nader Sehatbakhsh

TL;DR: 本文提出了InViStream系统，用于解决三维视频流中的隐私保护问题。该系统在摄像头端实时检测并移除私有物体，仅将经过处理的点云数据上传至云端进行融合，从而在保持场景公共部分可用性的同时，保护个人隐私。

Details

Motivation: 三维视频流将隐私问题扩展为多视角、三维融合的挑战，现有系统主要优化重建和延迟，而单帧图像的隐私保护方法无法直接处理校准的多视角RGB-D数据融合。

Result: 在合成和真实RGB-D场景（如办公室、会议室）上的评估显示，InViStream在合成数据上的Dice/Recall达到0.799/0.891，真实数据上为0.792/0.908，合成SSIM高于0.98，并能实现高于30 FPS的实时流传输。

Insight: 创新点在于结合物体检测与深度感知掩码，在多个校准视图间传播公共/私有决策，并仅融合经过净化的点云，实现了从源头保护隐私的实时三维视频流系统。

Abstract: Volumetric video streaming turns privacy into a 3D, multi-view problem. Unlike ordinary video, where sensitive content can often be redacted frame by frame, RGB-D volumetric pipelines capture people, rooms, and personal objects from multiple cameras and fuse them into a shared 3D representation. A private object missed in one view, or only partially removed before fusion, can therefore reappear in the reconstructed scene. This creates a privacy challenge for 3D telepresence, education, entertainment, and immersive applications: private content should be removed before raw visual and geometric data leave the camera side, while the public part of the scene should remain useful for real-time reconstruction. Existing volumetric streaming systems mainly optimize reconstruction, data movement, and latency, while privacy-preserving vision methods are designed for single-camera, single-frame images and do not directly address calibrated multi-view RGB-D fusion. We present InViStream, a real-time “privacy-from-source” system designed for this setting. InViStream addresses three challenges in volumetric capture: private objects may appear differently across views, RGB masking alone can leave geometric privacy leakage in depth, and public/private instances of the same class must be separated consistently before cloud-side fusion. To address these challenges, InViStream combines object detection with depth-aware masking, propagates public/private decisions across calibrated views, and fuses only sanitized point clouds. We evaluate InViStream on synthetic and real RGB-D scenes, including offices, conference rooms, living rooms, and settings with multiple public and private people and objects. InViStream achieves synthetic Dice/Recall of 0.799/0.891 and real Dice/Recall of 0.792/0.908, with synthetic SSIM above 0.98 and real-time streaming above 30 FPS.

[29] Motion-as-Prompt: Enhancing Motion Reasoning in Multimodal Large Language Models via Motion-Guided Cross-Frame Visual Prompting cs.CV | cs.AIPDF

Xikai Sun, Kebin Liu, Haotian Wang, Li Liu, Xu Wang

TL;DR: 本文提出了一种名为Motion-as-Prompt（MaP）的框架，旨在增强多模态大语言模型（MLLMs）在视频中的运动推理能力。该方法通过恢复密集点轨迹、选择富含运动信息的帧，并将轨迹标记在视觉输入上，使冻结的MLLMs能够观察到原本可能被稀疏采样丢弃的关键运动信息，从而提升对物体移动、碰撞和因果交互的理解。

Details

Motivation: 当前MLLMs通常通过稀疏均匀采样处理视频以控制计算成本，但这可能丢弃帧间关键的运动过渡信息，限制了模型对运动、碰撞和因果交互的推理能力。

Result: 在CLEVRER和Something-Something-v2基准测试中，MaP显著提升了运动推理准确率，分别使GPT-5.5模型获得了4.2%和8.9%的平均增益，且未损害非运动理解任务，展现了其鲁棒性。

Insight: 创新点在于将运动信息（轨迹）作为视觉提示直接注入到模型的输入中，这是一种无需模型训练或架构修改的轻量级解决方案，有效利用了运动引导的跨帧视觉提示来弥补稀疏采样的信息损失。

Abstract: Motion-centric video reasoning is fundamental to interactive applications such as robotic manipulation and autonomous navigation. However, multimodal large language models (MLLMs) typically process videos through sparse uniform sampling to control visual-token and attention costs. This strategy may discard critical transitions between sampled frames, limiting reasoning about object movement, collisions, and causal interactions. To mitigate this issue, we propose Motion-as-Prompt (MaP), a track-guided cross-frame visual prompting framework. MaP recovers dense point trajectories, selects motion-informative frames, and marks the trajectories accumulated between consecutive sampled frames directly onto the visual inputs, making otherwise hidden displacement, direction changes, and interactions observable to frozen MLLMs. Experiments on CLEVRER and Something-Something-v2 show that MaP consistently improves average motion-reasoning accuracy, yielding gains of 4.2% and 8.9% for GPT-5.5, respectively. Notably, these improvements are obtained without degrading non-motion understanding, highlighting the robustness of MaP. These results demonstrate that MaP provides a simple and effective solution for enhancing motion-centric video reasoning without model training or architectural modification. Project page:https://github.com/SunVictor23/MaP.

[30] EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection cs.CVPDF

Cunzheng Fan, Dawei Yan, Guanlin Wang, Xingshuo Yang, Yupeng Jia

TL;DR: EGM-Det是一个用于无人机RGB-IR目标检测的熵引导多模态自适应融合框架。它通过双流架构保留模态特定表征，并引入熵偏移门融合模块，利用浅层熵先验（输入强度、局部熵和跨模态差异）来指导局部偏移对齐和空间-通道门控融合，从而自适应地选择聚合可靠的RGB和红外线索。此外，该方法还引入了跨模态蒸馏来正则化学习的融合门并减少融合退化。

Details

Motivation: 现有方法通常以静态或固定权重融合RGB和红外图像的多模态特征，忽略了空间上变化的模态可靠性问题。本文旨在解决这一不足，实现更有效的自适应融合。

Result: 在DroneVehicle、LLVIP和VEDAI三个基准数据集上的实验表明，EGM-Det取得了最先进的性能，特别是在VEDAI数据集上，其性能超过先前方法超过10个百分点。

Insight: 创新点在于提出了熵引导的自适应融合机制，利用浅层熵先验（强度、局部熵、跨模态差异）来动态指导特征融合，并辅以跨模态蒸馏进行正则化，这为处理多模态数据中可靠性不均的问题提供了新思路。

Abstract: Joint use of RGB and infrared (IR) imagery can improve UAV-view object detection, but most existing methods fuse multimodal features with static or fixed weights and therefore overlook spatially varying modality reliability. We propose EGM-Det, an entropy-guided multimodal adaptive fusion framework for RGB-IR object detection. EGM-Det employs a dual-stream architecture to preserve modality-specific representations and introduces an Entropy Offset Gate Fusion module for adaptive multi-scale fusion. The module derives shallow entropy priors from input intensity, local entropy, and cross-modal discrepancy, and uses them to guide local offset alignment and spatial-channel gated fusion. It therefore selectively aggregates reliable RGB and infrared cues instead of uniformly combining heterogeneous features. We further introduce cross-modal distillation to regularize the learned fusion gates and reduce fusion degradation. Each student branch extracts complementary knowledge from the cross-modality teacher branch matched to the main branch, while entropy-adaptive supervision emphasizes uncertain modality decisions. Experiments on DroneVehicle, LLVIP, and VEDAI demonstrate state-of-the-art performance across all three benchmarks; in particular, EGM-Det outperforms prior approaches by more than 10 percentage points on VEDAI.

[31] Boundary-Enhanced Segmentation of Pig Point Clouds in Commercial Housing Environments cs.CVPDF

Zhankang Xu, Fei Shi, Xiangyu Qi, Zhaoyang Wang, Mengxin Guo

TL;DR: 本研究提出了一种基于边界特征分析的猪点云分割方法，旨在解决真实猪舍环境中猪点云与背景结构紧密接触导致的边界模糊、局部粘连和背景误分割问题。该方法以Octree Transformer为骨干网络，通过八叉树卷积、自注意力编码和多尺度特征融合整合局部几何细节与全局语义上下文，并引入软距离边界伪标签和双向跨边界语义模块以增强边界监督与特征交互。

Details

Motivation: 在真实猪舍环境中，猪点云常与背景结构紧密接触，导致目标边界模糊、局部粘连和背景误分割，这降低了后续点云补全和体型测量的准确性。

Result: 在综合数据集上的实验表明，该方法在分割精度、平均交并比和边界描绘方面显著优于多种最先进模型，有效缓解了边界粘连问题。

Insight: 创新点包括采用Octree Transformer骨干网络整合多尺度特征，以及提出软距离边界伪标签提供连续边界监督和双向跨边界语义模块实现边界与语义特征的显式交互，为下游精准畜牧业任务提供了可靠的点云输入。

Abstract: In real pigsty environments, pig point clouds often come into close contact with background structures, resulting in blurred target boundaries, local adhesion, and background mis-segmentation. This reduces the accuracy of subsequent point cloud completion and body size measurement. To address these challenges, this study proposes a pig point cloud segmentation method based on boundary feature analysis. The proposed method adopts Octree Transformer as the backbone network and integrates local geometric details with global semantic context through octree convolution, self-attention encoding, and multi-scale feature fusion. Furthermore, soft-distance boundary pseudo-labels are generated to provide continuous boundary supervision, and a bidirectional cross-boundary semantic module is designed to enable explicit interaction between boundary and semantic features. Experiments conducted on a comprehensive dataset demonstrate that the proposed method significantly outperforms various state-of-the-art models in terms of segmentation accuracy, mean intersection over union, and boundary delineation. The results indicate that the method effectively alleviates boundary adhesion, providing reliable point cloud inputs for downstream precision livestock farming tasks.

[32] STAR: A Spatial-Topology Aware Routing Framework for Generalizable 3D Scene Understanding cs.CVPDF

Mingwei Xing, Xinliang Wang, Yifeng Shi

TL;DR: 本文提出STAR（空间拓扑感知路由框架），旨在解决多传感器模态下3D场景理解中拓扑差异导致的专家分配难题。该框架通过多属性自监督预训练分支学习跨域结构先验，并结合域感知专家分支中的空间引导路由和熵控动态分配机制，实现稳定的跨域表示学习和自适应专家激活。

Details

Motivation: 现有基于混合专家（MoE）架构的多域3D理解模型中，传统基于特征的MoE路由器在语义监督下可能无法充分表征局部采样拓扑，导致在语义一致性与几何异质性共存时专家分配困难。

Result: 在ScanNet验证集上达到80.1% mIoU，在S3DIS上达到77.2% mIoU，在室内外场景的多项任务实验中均持续超越强基线模型。

Insight: 创新点包括：1）引入覆盖拓扑和纹理变化的多属性自监督预训练分支以锚定跨域结构先验；2）设计域感知专家分支，结合空间引导路由（DSR）捕捉局部拓扑变化，以及熵控动态分配（EDA）根据路由不确定性调整激活专家数量，实现了表示学习稳定性与专家分配自适应性的结合。

Abstract: Constructing a unified 3D scene understanding model has long been hindered by the topological discrepancies across sensor modalities. While applying the Mixture-of-Experts (MoE) architecture is a flexible approach for multi-domain 3D understanding, we observe that conventional feature-only MoE routers may underrepresent local sampling topology under semantic supervision, making expert allocation difficult when semantic consistency coexists with geometric heterogeneity. To overcome this challenge, we propose STAR (Spatial-Topology Aware Routing Framework). Specifically, we introduce a multi-attribute self-supervised pre-training branch, covering topological and textural variations, to anchor cross-domain structural priors. Building upon this, we design a domain-aware expert branch with two mechanisms: Domain-Spatial-Guided Routing (DSR), which captures local topological variations from spatial context, and Entropy-controlled Dynamic Allocation (EDA), which adjusts the number of activated experts according to routing uncertainty. Together, these branches combine stable cross-domain representation learning with adaptive expert allocation. Extensive experiments across various tasks, encompassing both indoor and outdoor scenes, demonstrate the effectiveness of STAR. It achieves 80.1% mIoU on the ScanNet validation set and 77.2% mIoU on S3DIS, consistently improving over strong baselines. Code is available at our project page (https://xmw666.github.io/STAR/).

[33] Learning from Multimodal Pseudo-Labels for Robust Open-Vocabulary Instance and Panoptic Segmentation cs.CV | cs.AI | cs.MMPDF

Duy Tran Thanh, Yeejin Lee, Byeongkeun Kang

TL;DR: 本文提出了一种多模态框架，用于解决开放词汇实例分割（OVIS）和开放集全景分割（OSPS）任务，旨在无需详尽人工标注的情况下识别预定义和未见过的物体类别。该方法利用预训练的视觉语言模型（如Grounded SAM、LLaVA和CLIP）自动生成伪标签，并通过CLIP引导的同义词过滤和GPT驱动的标题重建来增强鲁棒性。

Details

Motivation: 现有方法在开放词汇分割中常面临伪掩码噪声大、视觉-文本对齐有限以及难以处理同义词或词汇外（OOV）词语的问题，本文旨在克服这些挑战，实现更鲁棒的分割性能。

Result: 在COCO数据集上的大量实验表明，该方法在OVIS和OSPS基准测试中均显著优于先前的最先进（SOTA）方法，取得了实质性改进。

Insight: 创新点包括：1）利用多模态模型（Grounded SAM、LLaVA、CLIP）自动生成伪标签，减少对人工标注的依赖；2）引入扩展的接地损失、语义一致性损失和生成式标题重建损失，以增强视觉-文本对齐；3）通过CLIP引导的同义词过滤和GPT驱动的标题重建来处理同义词和OOV词语，提升模型对开放词汇的泛化能力。

Abstract: This work addresses the challenge of open-vocabulary instance segmentation (OVIS) and open-set panoptic segmentation (OSPS), which aim to recognize both predefined and unseen object categories without exhaustive human annotations. Existing methods often suffer from noisy pseudo-masks, limited visual-textual grounding, and difficulty handling synonyms or out-of-vocabulary (OOV) words. To overcome these challenges, we propose a multimodal framework that leverages pre-trained vision-language models for automatic pseudo-label generation, CLIP-guided synonym filtering, and GPT-based caption reconstruction. In our target-vocabulary-assisted pseudo-labeling setting, the framework first constructs pseudo segmentation masks, descriptive captions, and semantically aligned synonym sets using Grounded SAM, LLaVA, and CLIP, providing multimodal supervision without manual annotation. We then enhance visual-textual alignment through three complementary training objectives: an extended grounding loss that incorporates visually grounded synonyms, a semantic consistency loss, and a generative caption reconstruction loss. Extensive experiments on the COCO dataset demonstrate that the proposed method consistently outperforms previous state-of-the-art approaches under this protocol, achieving substantial improvements on both OVIS and OSPS benchmarks.

[34] Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System cs.CV | cs.AIPDF

Haoyu Zhang, Shuoxun Zhang, Peng Ye, Lin Zhang, Jiakang Yuan

TL;DR: 本文针对基于多模态大语言模型（MLLM）的无人机航拍图像理解与推理任务，构建了首个统一评估基准UAVQA-Bench，并提出了一个无需训练的多智能体系统UAV-MAS。该系统通过领域感知、迭代验证和自适应搜索机制，显著提升了MLLM在复杂航拍场景下的性能。

Details

Motivation: 现有MLLM在无人机航拍图像理解上面临尺度变化大、视角任意、目标密集等独特挑战，且缺乏统一的评估基准来系统衡量其理解和推理能力。

Result: 在提出的UAVQA-Bench基准上，UAV-MAS系统（使用32B开源MLLM）取得了77.0%的整体准确率，超越了Gemini 3 Pro 4.0个百分点；其8B变体相比基础模型提升了8.7%。

Insight: 论文的创新点在于构建了覆盖多维度能力的统一无人机问答基准，并设计了一个无需训练的多智能体架构，通过任务路由、迭代验证和自适应搜索来缓解领域不匹配、错误传播和静态推理等关键失败模式。

Abstract: Multimodal Large Language Model (MLLM)-based UAV aerial image understanding and reasoning is essential for aerial intelligence yet poses distinct challenges arising from extreme scale variation, arbitrary camera orientations, and high object density. Despite growing interest, existing evaluations remain fragmented across individual datasets and narrow tasks, leaving a critical gap in unified assessment of UAV understanding and reasoning capabilities. To fill this gap, we construct UAVQA-Bench, a benchmark of 1,500 human-annotated QA pairs drawn from 13 public UAV datasets, covering 6 capability dimensions and 16 tasks in both multiple-choice and visual grounding formats. Systematic evaluation of a broad range of open-source and closed-source MLLMs as well as agent-based systems on UAVQA-Bench identifies three key failure modes: domain-toolset mismatch, unchecked error propagation, and static reasoning. Motivated by these findings, we propose UAV-MAS, a training-free multi-agent system for MLLM-based UAV aerial image understanding and reasoning, comprising a Domain-Specific Perception Engine (DSPE) that routes queries to task-appropriate visual tools, a Context-Aware Iterative Refinement module (CAIR) that validates intermediate reasoning to curb error accumulation, and a Difficulty-Aware Adaptive Search mechanism (DAAS) that adjusts search depth to question difficulty. UAV-MAS with a 32B open-source MLLM achieves 77.0% overall accuracy on UAVQA-Bench, surpassing Gemini 3 Pro by 4.0%, while the 8B variant improves 8.7% over its base model.

[35] JieZi: A Large-Scale Expert-Audited Dataset and Benchmark for Ancient Chinese Character Exegesis cs.CV | cs.AIPDF

Ran Li, Huiguo He, Jiahuan Cao, Junle Liu, Hiuyi Cheng

TL;DR: 本文介绍了JieZi，一个用于古汉字训诂的大规模专家审核数据集和基准测试。该工作提出了古汉字训诂（ACCE）这一视觉问答（VQA）任务，并构建了包含超过50万QA对的JieZi-Dataset训练集和JieZi-Bench评估基准，以支持从字形识别到历时演变分析的四个渐进式任务层级。实验表明，当前多模态大语言模型在基础识别上表现良好，但在字形分析和语义推理等方面存在困难，而使用JieZi-Dataset微调能显著提升模型在所有层级的性能。

Details

Motivation: 现有计算方法（如字符识别和检索）过于关注子任务，缺乏支持全面学术分析的结构化数据集和基准，无法满足融合视觉观察、语言分析和历史背景的古汉字训诂学术需求。

Result: 在多模态大语言模型上的实验表明，现有模型在基础字符识别上表现良好，但在字形分析、语义推理和历时理解方面存在困难。使用JieZi-Dataset进行微调后，模型在所有四个任务层级上的性能都得到了显著提升。

Insight: 论文的创新点在于首次提出了一个结构化的古汉字训诂VQA任务（ACCE）及其对应的四个渐进式分析层级，并构建了首个大规模、专家审核的训练数据集和评估基准。其数据构建流程通过专家设计的模板和源文本引用约束生成，并在关键阶段进行人工验证，有效减少了事实性错误，确保了学术准确性。

Abstract: The scholarly exegesis of ancient Chinese characters demands integrating visual observation, linguistic analysis, and historical context. However, existing computational approaches focus narrowly on subtasks such as character recognition and retrieval, lacking the structured datasets and benchmarks required for comprehensive scholarly analysis. To address this limitation, we introduce Ancient Chinese Character Exegesis (ACCE), a vision-language question answering (VQA) task that models the scholarly exegesis process. ACCE is organized into four progressive levels: basic character identification, glyph-form analysis, meaning exegesis, and diachronic evolution analysis. To support this task, we construct two complementary resources. JieZi-Dataset is the first large-scale, expert-audited VQA training dataset for ACCE, comprising over 500K QA pairs. It is constructed via a pipeline that reduces factual errors by constraining generation with expert-designed templates and source-text references. Human verification is further applied at each key stage to ensure scholarly accuracy. JieZi-Bench is an evaluation benchmark aligned with the exegesis process, constructed and verified by human experts to ensure evaluation reliability. It consists of four levels with reference answers curated from authoritative lexicographic works held separate from the training data. Experiments on multimodal large language models show that current models perform well on basic identification but struggle with glyph analysis, semantic reasoning, and diachronic understanding. Fine-tuning on JieZi-Dataset substantially improves performance across all four levels. Code and dataset are available at https://github.com/Ran00w/JieZi.

[36] UniSwap: Streaming Audio-Visual Identity Swapping for Talking Videos cs.CV | cs.SDPDF

Yuxuan Zhang, Haozhong Xiong, Jiayi Song, Jinpeng Yu, Yang Shi

TL;DR: UniSwap是首个用于说话视频流式联合音视频身份替换的框架，通过单一的音视频扩散Transformer模型，在给定源视频、参考图像和参考语音片段的情况下，实现参考外观和音色的迁移，同时保持源内容与动态。

Details

Motivation: 现有方法分别优化视觉和音频模态，难以保证音视频一致性，且缺乏对齐的跨身份训练数据。

Result: 实验表明，UniSwap在音视频同步性、身份保持、流式效率和长时生成稳定性方面表现优异。

Insight: 创新点包括：交换重建训练流水线解决数据稀缺；通过上下文预训练、条件流式适应和高效自强制DMD实现高效流式生成；特征RoPE分解支持稳定长序列推理；多LoRA切换使不同角色共享冻结主干。

Abstract: Talking-video character replacement requires coordinated transfer of appearance and voice while preserving the source motion, scene, linguistic content, and audio-video timing. Existing methods use separately optimized models for the two modalities, making audio-visual consistency difficult to enforce. We present UniSwap, the first framework for streaming joint audio-visual identity replacement in talking videos. Given a source video, a reference image, and a reference voice clip, UniSwap transfers the reference appearance and vocal timbre within a single audio-visual diffusion transformer while preserving the source content and dynamics. To address the scarcity of aligned cross-identity training pairs, we introduce a swap-and-reconstruct pipeline that removes visual and vocal identity from real clips and uses the original clips as reconstruction targets. Starting from a bidirectional backbone, we progressively adapt the model through In-context Pretraining for joint replacement, Conditional Streaming Adaptation for block-causal KV-cached generation, and Efficient Self-forcing DMD for mitigating exposure bias and reducing sampling from 30 to 3 denoising steps per block. Efficient Multi-LoRA Switching enables the three DMD roles to share a single frozen backbone. Feature-RoPE Decomposition keeps cached positions within the training range, supporting stable long-form inference. Experiments demonstrate strong audio-visual synchronization, competitive identity preservation, efficient streaming, and stable long-form generation.

[37] LiveAnimate: Stable Long-Form Streaming Human Animation in Real-Time cs.CVPDF

Yuxuan Zhang, Haozhong Xiong, Yubo Huang, Jiayi Song, Jinpeng Yu

TL;DR: LiveAnimate是一个基于140亿参数视频扩散Transformer（DiT）构建的实时、长时、稳定的人体动画生成系统。它通过两阶段训练（参考锚定教师强制适应和块级自强制蒸馏）将采样步骤减少到三步，并引入姿态检索汇注意力（PR-Sink）机制来维持长序列生成中的外观一致性，最终在两张NVIDIA H100 GPU上实现了19.63 FPS的流式推理。

Details

Motivation: 解决现有基于扩散模型的人体动画生成系统（每段视频需要数分钟到数小时）无法满足直播、远程呈现和虚拟化身等交互式应用对实时性要求的问题。

Result: 在三分钟的基准测试中，LiveAnimate从最初30秒到最后1分钟保持了近乎恒定的感知质量和身份一致性（IQA 4.047 vs. 4.026），而先前系统质量显著下降或需要数小时离线计算。该系统在质量、延迟和时长上为交互式全身动画建立了新的性能基准。

Insight: 主要创新点包括：1）将预训练双向DiT通过两阶段训练（参考锚定教师强制适应、块级自强制蒸馏）转化为块因果自回归生成器，实现三步采样；2）提出PR-Sink注意力机制，结合静态汇、动态汇和滚动窗口，通过有界KV缓存实现长时外观一致性而无需存储整个序列；3）结合Ulysses序列并行和算子融合实现高效推理。

Abstract: Pose-driven human animation synthesizes a video of a target person from a single reference image and a driving pose stream. Real-time generation is essential for interactive applications such as live streaming, telepresence, and virtual avatars, yet diffusion-based systems require minutes to hours per clip, precluding responsive interaction. We present LiveAnimate, to our knowledge the first animation system to combine real-time streaming with stable long-form generation at billion scale, built on a 14B-parameter video Diffusion Transformer (DiT). A two-stage training pipeline first adapts a pretrained bidirectional DiT into a block-causal autoregressive generator through Reference-Anchored Teacher-Forcing Adaptation, and then reduces the sampling budget to three steps through Block-wise Self-Forcing Distillation. To preserve appearance over extended streams, we introduce Pose-Retrieval Sink Attention (PR-Sink), a bounded KV-cache mechanism combining a Static Sink that permanently anchors the first generated block, a Dynamic Sink that holds a pose-retrieved historical block, and a three-slot Rolling Window. When a pose recurs, PR-Sink restores the relevant appearance context without retaining the entire sequence, so memory and per-block latency remain constant regardless of stream duration. Together with Ulysses sequence parallelism and operator fusion, these designs enable 19.63,FPS streaming inference on two NVIDIA H100 GPUs. On a three-minute benchmark, LiveAnimate maintains nearly constant perceptual quality and identity from the first 30 seconds to the final minute (IQA 4.047 vs.\ 4.026), while prior systems degrade substantially or require hours of offline computation for the same rollout. These results establish a new operating point in quality, latency, and duration for interactive full-body animation.

[38] VOLA: Improving Open-World Driving by VLM-Based Semantic Attribute Prediction cs.CVPDF

Yuchen Zhang, Yuan Gao, Sebastian Schmidt, Johannes Betz

TL;DR: 论文提出VOLA方法，通过基于视觉语言模型（VLM）的语义属性预测来改进开放世界驾驶感知。该方法将场景感知从类别标签转向密集的动作相关属性（如7级可驾驶性和5级脆弱性），利用Qwen3.5的图像token隐藏状态作为空间语义表示，并通过轻量级边界感知解码器生成全分辨率属性图，无需自回归文本生成或外部掩码模型。在CARLA构建的密集属性标签上训练，并在真实场景和未见障碍物上测试，结果表明该方法在熟悉类别上匹配纯视觉分割器，并在开放世界异常上实现更好的迁移性能。

Details

Motivation: 解决真实世界驾驶的开放性问题，即车辆可能遇到训练数据外的物体（如掉落床垫、鹿），仅识别物体名称不足，需预测密集的动作相关属性（如可驾驶性和脆弱性）以指导运动决策。

Result: 在CARLA训练并迁移到真实场景和未见障碍物，与纯视觉分割器和提示VLM分割器比较：在平均脆弱性等级召回率上达到69.4%，优于最佳纯视觉基线（57.1%）和最佳提示VLM基线（53.9%），显示在开放世界异常上实现更好的迁移。

Insight: 创新点包括将驾驶感知从类别标签转向动作相关属性，直接利用VLM图像token隐藏状态作为空间语义表示，以及设计轻量级边界感知解码器生成全分辨率属性图，无需额外文本生成或掩码模型，提升了开放世界场景的泛化能力。

Abstract: Driving in the real world is open-world: a car may encounter a fallen mattress, a deer, or other objects outside its training data. Naming them is not enough. The system must know how to treat each region: can it drive over it, and how severe would a collision be? We therefore shift scene perception from category labels to dense action-relevant attributes, where each pixel is labeled by how it should affect motion rather than by object name. We instantiate this general formulation with two ordered attributes: 7-rank drivability and 5-rank vulnerability. We read Qwen3.5 image-token hidden states directly as a spatial semantic representation. A lightweight boundary-aware decoder then turns this coarse token grid into sharp full-resolution attribute maps. The whole process requires neither autoregressive text generation nor an external mask model such as SAM. We train on dense attribute labels built in CARLA and test transfer to real scenes and to novel obstacles never seen in training. We compare with vision-only segmenters trained on the same attributes and prompted VLM segmenters. Our model matches strong vision-only segmenters on familiar categories and improves transfer to real open-world anomalies, reaching 69.4% mean vulnerability-rank recall versus 57.1% for the best vision-only baseline and 53.9% for the best prompted VLM baseline. These results show that VLM image tokens provide useful semantic cues for transferring driving attributes to objects outside the training vocabulary.

[39] Achieving Near-Zero-Overhead Multi-Model Hierarchical Classification in Real-Time Detection Pipelines cs.CV | cs.DC | cs.LGPDF

Vaishnav Raju

TL;DR: 本文提出了一种在边缘设备（以NVIDIA Jetson DLA为核心平台）上实现近乎零开销的多模型分层分类推理流水线的方法。该方法通过五步流程（包括架构适配、动态范围手动修正、量化感知训练、ONNX图手术和并发流水线设计），解决了在专用神经加速器（DLA）上部署自定义模型的实际难题，使得分类模型能在DLA上与GPU上的检测模型并发执行。

Details

Motivation: 解决在边缘视觉系统（如目标识别、监控）中，随着流水线阶段增加，所有模型在GPU上串行运行造成的实时吞吐量瓶颈问题。同时，利用现代边缘SoC中GPU与神经加速器（如NPU/DLA）的并发能力，但克服了在这些加速器上部署自定义模型时面临的操作符约束、量化不兼容和缺乏端到端流程文档等实际障碍。

Result: 在Jetson Orin NX平台上验证了一个双头人物属性分类器（在DLA上运行）与一个GPU目标检测器并发的流水线。结果显示，与仅运行检测器（13.3 FPS @ 1080p）相比，整个流水线实现了近乎零开销（12.5 FPS），并且双DLA扩展无需额外成本。通过手动动态范围修正，从隐式量化的75%准确率恢复了94.0%的准确率。

Insight: 创新点在于提出了一套通用、可复现的五步方法论，用于实现零GPU回退的DLA INT8分类模型部署，并系统化地记录了九大工程约束及其通用解决方案。该方法论与骨干网络无关，可推广至任何检测-分类边缘流水线，核心在于通过并发执行（GPU检测 + DLA分类）和精细的量化部署流程来消除流水线瓶颈。

Abstract: Edge-deployed vision systems in target recognition, surveillance, autonomous vehicles, and drone domains require hierarchical inference pipelines where a detection model identifies objects of interest and downstream classifiers provide fine-grained attribute analysis. Running all models on the GPU creates a serial bottleneck that limits real-time throughput as pipeline stages grow. Modern edge SoCs pair GPUs with dedicated neural accelerators (NPUs, DLAs) capable of concurrent execution, yet deploying custom models on these accelerators remains impractical due to strict operator constraints, quantization incompatibilities, and an undocumented end-to-end pipeline. We target NVIDIA Jetson DLA cores as the representative platform. We present a five-step methodology for zero GPU fallback DLA INT8 deployment of classification backbones, comprising architecture adaptation, manual dynamic range workaround to rescue TensorRT’s implicit quantization (recovering 94.0% accuracy from implicit quantization’s 75%) for rapid pipeline validation before explicit quantization, quantization-aware training, ONNX graph surgery for DLA compilation, and a concurrent GPU-detection/DLA-classification inference pipeline. We document nine engineering constraints with root-cause analysis and generalizable solutions. Validation on a dual-head person attribute classifier running on DLA alongside a GPU object detector on a Jetson Orin NX demonstrates near-zero pipeline overhead (12.5 vs. 13.3~FPS detector-only at 1080p), with dual-DLA scaling at no additional cost. The methodology is backbone-agnostic and generalizes to any detection-classification edge pipeline.

[40] Can Vision Models Read the Radar Display? On the Feasibility of Radar Imagery for Air Traffic Complexity Estimation cs.CV | cs.LGPDF

Hyewook Kim, Byul Kang, Seokbin Yoon, Keumjin Lee

TL;DR: 本文探讨了将雷达图像作为深度学习视觉模型输入用于空中交通复杂度估计的可行性。作者将交通态势编码为位置图像并补充五个表示飞机状态变量的通道，训练Vision Transformer回归基于飞机间几何关系推导的四个内在复杂度分量。模型在所有四个分量上达到R² > 0.96，并通过单机移除扰动研究验证了其响应与移除飞机对扇区复杂度的贡献成比例。

Details

Motivation: 空中交通管制员通过雷达显示屏感知交通复杂度，这启发研究者探索计算机视觉模型处理相同图像以建模管制员感知复杂度的可能性，但雷达图像作为深度学习输入的可行性尚不明确。

Result: 在基于飞机对几何关系推导的四个内在复杂度分量回归任务中，Vision Transformer模型在所有分量上均达到R² > 0.96的高性能。单机移除扰动研究表明，模型响应变化与移除飞机对扇区复杂度的实际贡献成比例，而非均等对待所有移除。

Insight: 创新点在于将极稀疏、自相似的雷达图像（主要由黑色背景和少量视觉相同的飞机光点组成）编码为多通道位置图像，并补充飞机航向、速度、高度等状态变量，使视觉模型能捕捉对操作至关重要的微小位置变化。这证明了非典型视觉特征的雷达图像在交通复杂度建模中的可行性，为领域特定图像处理提供了新思路。

Abstract: Air traffic controllers perceive traffic complexity through the radar display, suggesting that a computer vision model operating on the same imagery may provide a natural architecture for modeling controller-perceived complexity; however, whether radar imagery is a viable input format for deep learning vision models remains unclear. Unlike natural images, radar images are extremely sparse and self-similar, consisting primarily of a black background and a few visually identical aircraft blobs, while small changes in aircraft positions can substantially alter sector-level complexity. To test whether a vision model can capture these operationally important differences, we encode each traffic situation as a position image supplemented by five channels representing aircraft state variables, including heading, speed, and altitude, and train a Vision Transformer (ViT) to regress four intrinsic complexity components derived from pairwise geometric relations among aircraft. The model achieves $R^2 > 0.96$ for all four components, and a one-aircraft-removal perturbation study shows that its response changes proportionally to how much the removed aircraft contributed to sector complexity rather than treating every removal as equivalent. These results demonstrate that, despite its atypical visual characteristics, radar imagery is a viable input format for air traffic complexity modeling.

[41] TD-VAD: Breaking Visual Dependence in Video Anomaly Detection with Text-Driven Learning cs.CVPDF

Shuangqing Zhang, Lei-Lei Ma, Zhao Wang, Wen Dong, Xinyi Xu

TL;DR: 本文提出了一种文本驱动的视频异常检测方法TD-VAD，旨在打破传统方法对视觉数据的依赖。该方法利用大语言模型生成具有时序特性的文本描述来训练模型，无需目标域异常视频数据，并通过事件演化因果注意力模块建模长短期时序逻辑。推理时使用冻结的CLIP编码器对齐文本与视频模态。

Details

Motivation: 现有视频异常检测方法严重依赖视觉数据，但异常视频标注数据稀缺且获取困难，难以扩展。本文主张利用易于收集的文本描述来替代视觉数据，以解决数据依赖和可扩展性问题。

Result: 在XD-Violence和UCF-Crime两个大规模VAD数据集上的综合实验表明，该方法大幅优于先前的一类分类和无监督VAD方法。

Insight: 创新点在于首次提出完全基于文本描述训练VAD模型，无需任何目标域异常视频；设计了事件演化因果注意力模块来捕捉事件时序逻辑；利用冻结的CLIP编码器在推理时实现跨模态对齐，缓解领域差距。

Abstract: Visual data is typically a prerequisite for training existing video anomaly detection (VAD) methods. However, obtaining sufficient annotated anomaly data for training is challenging and not scalable due to the rarity of anomaly data and the wide variety of abnormal events. In this work, we advocate that the effectiveness of treating texts as video sequences for the VAD model and propose a novel Text-Driven Video Anomaly Detection (TD-VAD) approach to break visual dependence. In contrast to the anomaly video data, text descriptions of abnormal events are easy to collect, and their class labels can be directly derived. Specifically, our method utilizes video-like text descriptions with temporal characteristics generated by LLM to train a VAD model, without any reliance on target-domain anomaly data. To capture the long- and short-range temporal logic of events, we design the event evolution causal attention module to model contextual dependencies across time. During inference, considering the domain gap between the texts and video sequences, we use the frozen CLIP encoder to extract embeddings of video frames to align the text modality while retaining crucial visual information. Comprehensive experiments on two large-scale VAD datasets, XD-Violence and UCF-Crime, demonstrate that our method outperforms prior one-class and unsupervised VAD methods by a large margin.

[42] Distractor-Aware Video Object Segmentation cs.CVPDF

Andreas Robinson, Abdelrahman Eldesokey, Michael Felsberg

TL;DR: 本文提出了一种针对半监督视频目标分割任务的‘分心物感知’方法。该方法将传统的一对一分类（目标 vs. 背景）扩展为一对多分类，将分心物作为独立的类别进行处理。通过改进现有的学习-学习什么（LWL）方法，新方法在DAVIS 2017基准测试上取得了新的最优性能。

Details

Motivation: 半监督视频目标分割任务中，传统方法通常将问题建模为目标与背景的一对一分类。然而，视频中除了目标和背景，通常还存在与目标视觉相似的分心物，这些分心物容易导致误判，降低分割精度。

Result: 所提出的方法在DAVIS 2017验证集上达到了新的最优水平（SOTA），并在DAVIS 2017测试开发集基准上比基线方法提升了4.6个百分点。

Insight: 核心创新点在于将分心物从背景中分离出来，形成‘一对多’的分类方案，从而能够对容易导致性能下降的挑战性区域施加特殊关注。这提供了一种更精细、更鲁棒的建模视频场景中多类别干扰的思路。

Abstract: Semi-supervised video object segmentation is a challenging task that aims to segment a target throughout a video sequence given an initial mask at the first frame. Discriminative approaches have demonstrated competitive performance on this task at a sensible complexity. These approaches typically formulate the problem as a one-versus-one classification between the target and the background. However, in reality, a video sequence usually encompasses a target, background, and possibly other distracting objects. Those objects increase the risk of introducing false positives, especially if they share visual similarities with the target. Therefore, it is more effective to separate distractors from the background, and handle them independently. We propose a one-versus-many scheme to address this situation by separating distractors into their own class. This separation allows imposing special attention to challenging regions that are most likely to degrade the performance. We demonstrate the prominence of this formulation by modifying the learning-what-to-learn (LWL) method to be distractor-aware. Our proposed approach sets a new state-of-the-art on the DAVIS 2017 val dataset, and improves over the baseline on the DAVIS 2017 test-dev benchmark by 4.6 percentage points.

[43] GeoBridge: Decoupled Semantic Conditioning for Generative Image Geolocalization cs.CVPDF

Zhiyang Dou, Xumeng Han, Fengde Peng, Zipeng Wang, Moxuan Zhao

TL;DR: GeoBridge提出了一种解耦语义条件机制，将冻结的语义多模态大语言模型与冻结的黎曼流匹配头部连接，用于在球面上生成连续坐标，以解决生成式图像地理定位中坐标解码的瓶颈问题。

Details

Motivation: 当前多模态大模型在图像地理定位中主要提升了地理线索推理能力，但将推理解码为坐标的方法滞后，传统基于地名预测并通过地理编码API获取坐标的方式是离散且有损的，忽略了图像证据并将多粒度语义压缩为粗略查找，因此需要一种连续、几何感知的解码表示方法。

Result: 在IM2GPS3K基准测试中，GeoBridge在25/200/750公里阈值下分别达到38.67/52.89/70.37的准确率，优于基于地名到API的流水线方法和推理增强的直接预测方法，在这些精度相关的尺度上实现了提升。

Insight: 创新点在于通过解耦语义监督与条件接口，避免了角色冲突，使连续条件能够在不干扰球形解码器的情况下注入地理先验；这是一种解码端的算法贡献，与思维链推理正交且互补，可应用于其他需要将语义推理与几何生成解耦的任务。

Abstract: Multimodal large language models (MLLMs) have advanced image geolocalization mainly by improving how they reason about geographic cues. How that reasoning isdecoded into coordinates, however, has lagged behind. Predicting a place name for a geocoding API is discrete and lossy: it ignores image evidence and collapses multi-granular semantics into a coarse lookup. We argue that the bottleneck has shifted from what a model reasons to how that reasoning is represented for a continuous, geometry-aware decoder. We present GeoBridge, a role-decoupled conditioning mechanism that connects a frozen semantic MLLM to a frozen Riemannian flow-matching head that generates coordinates on the sphere. The central obstacle is arole conflict: supervising the condition with discrete semantic labels biases its representation toward class-discriminative geometry, at odds with the smooth manifold the generative head requires. GeoBridge keeps the semantic supervision decoupled from the condition interface: a separate projection forms the continuous condition the frozen head expects, injecting geographic priors without disturbing the spherical decoder. On IM2GPS3K, GeoBridge reaches 38.67/52.89/70.37 at the 25/200/750 km thresholds, improving over a place-name-to-API pipeline and reasoning-augmented direct prediction at these precision-relevant scales. GeoBridge is a decode-side algorithmic contribution, orthogonal and complementary to chain-of-thought reasoning. Code will be made publicly available.

[44] LookBack: Where and How to Score LVLM Responses via Visual Reference Usage cs.CV | cs.AI | cs.CLPDF

Beomsik Cho, Jinhyeong Kim, Dongseok Lee, Jaehyung Kim

TL;DR: 论文提出了一种名为LookBack的无训练LVLM响应评分方法，通过结合视觉回溯分数来增强词元似然，以衡量每个响应词元对图像词元的引用强度，从而更准确地评估LVLM响应的视觉基础性。

Details

Motivation: LVLM在生成响应时会出现与图像内容不符的幻觉，现有基于置信度的指标主要捕捉文本合理性而非与图像的一致性，因此需要新的评分方法来解决这一问题。

Result: 在四个基准测试和三个模型上，LookBack在Best-of-N选择中持续优于现有基线方法，且额外开销可忽略不计。

Insight: 创新点在于引入视觉回溯分数作为轻量级度量，直接评估响应与视觉输入的关联，弥补了传统置信度指标在视觉基础性评估上的不足，为LVLM响应评分提供了更可靠的训练无关解决方案。

Abstract: Large Vision-Language Models (LVLMs) integrate visual perception with language generation, enabling responses that span image understanding and complex reasoning. However, LVLMs do not just inherit the text-level hallucinations; they also hallucinate against the image, producing fluent responses ungrounded in what they see. This makes LVLM response scoring inherently harder, and our diagnostics show that existing confidence-based metrics adopted from LLMs are insufficient for LVLMs. Specifically, removing the input image barely changes confidence-based selection, suggesting that output-space confidence primarily captures textual plausibility rather than agreement with the image. To address this gap, we propose LookBack, a training-free LVLM response scoring method that augments token likelihood with visual lookback score, a lightweight measure of how strongly each response token refers to image tokens. Across four benchmarks and three models, LookBack consistently improves Best-of-$N$ selection over existing baselines with negligible additional overhead.

[45] Warping Earth Observations for better ice labeling in the Marginal Marginal Ice Zone cs.CVPDF

Tom Kelly, Martin S. J. Rogers

TL;DR: 本文提出了一种基于互信息扭曲的新颖架构，用于对齐多卫星（Sentinel-1和MODIS平台）、多模态（可见光、热红外、雷达）卫星影像。该方法旨在解决动态环境中（如南极边缘冰区）因图像采集时间不同导致的地表特征移动问题，从而改善海冰分类。通过引入一个稀疏专家标注的数据集进行验证，结果表明在分割前进行空间对齐能提高分类精度，并实现从稀疏点监督到密集海冰分割的准确预测。

Details

Motivation: 解决动态环境（如南极边缘冰区）中多模态卫星影像因时空不匹配而难以有效融合的问题。传统方法依赖于像素级对应的假设，但在快速变化的冰区，冰面特征在图像采集间会发生移动，这挑战了有效的感知基础和多模态推理。

Result: 在引入的包含2,088个像素级标注（7,046个专家点分类）的稀疏专家标注数据集上，该方法在分割前进行模态对齐，提高了分类准确性，并实现了从稀疏点监督到密集海冰分割的准确预测。

Insight: 创新点在于提出基于互信息扭曲的架构来对齐多模态卫星影像，以应对动态环境中的时空不匹配。客观分析认为，该方法通过先对齐再分割的流程，有效利用了稀疏监督数据，为动态地球观测任务中的多模态融合提供了新思路。

Abstract: Multimodal satellite imagery provides complementary information for Earth Observation, but accurately combining heterogeneous sensors remains challenging in dynamic environments. Fast-changing regions, such as the Antarctic marginal ice zone, cannot fully exploit multimodal information from different satellite sensors because surface features move between image acquisitions. This spatial and temporal mismatch challenges effective perceptual grounding, violating the assumption of pixel-level correspondence that underpins most multimodal reasoning and downstream classification pipelines. Antarctic sea ice provides a challenging benchmark due to the rapid, heterogeneous drift of individual ice floes and the differing responses of sea ice to radar, visible and thermal sensing modalities. Accurate, dense supervision of sea ice remains scarce because generating pixel-wise labels requires time-consuming expert interpretation of noisy data, leading to historical reliance on coarse-resolution maritime ice charts for model training. This paper presents a novel architecture based on mutual information warping to align multi-satellite (Sentinel-1 and MODIS platforms) multimodal (visible, thermal, radar) satellite scenes. To demonstrate the approach, we introduce a sparse expert-labeled dataset of 2,088 pixel-wise annotations (7,046 expert point classifications) located at the ice-water margin interface across 43 scenes. Our results demonstrate that spatially grounding and aligning modalities prior to segmentation improves classification accuracy, and enables accurate, dense sea ice segmentation from sparse point-wise supervision.

[46] Do You See What You Draw? A Semantic Closed-Loop Framework for Holistic Evaluation of Unified Multimodal Models cs.CV | cs.AIPDF

Hao Zhang, Jiaxin Qi, Zhijiang Tang, Jianqiang Huang

TL;DR: 本文提出了一种名为自生成理解（SGU）的无标注评估框架，用于整体评估统一多模态模型（UMMs）的集成能力。该框架通过一个语义闭环挑战（即感知图像、生成文本描述、根据描述重建视觉内容，并对自生成输出进行推理）来探测模型的理解与生成能力的统一性，从而提供一种零成本的系统级评估方法。

Details

Motivation: 当前评估协议通常将生成能力和判别能力作为独立任务分开评估，缺乏对统一多模态模型（UMMs）作为统一系统的整体评估方法，这构成了一个关键挑战。

Result: 大量实验表明，即使是高性能的UMMs也常常难以对其自生成的上下文进行有效推理，这揭示了单独评估理解或生成任务所无法捕捉的局限性。该框架为下一代统一多模态模型的基准测试提供了基础。

Insight: 主要创新点在于提出了一个无需额外标注的语义闭环评估框架（SGU），通过模型自身的生成-理解循环来整体评估其统一能力。从客观角度看，这种自指涉的评估范式为衡量多模态模型的内部一致性和系统集成度提供了新颖且低成本的视角。

Abstract: As Large Vision-Language Models increasingly aim to integrate visual generation and understanding within a single parameter space, evaluating such structural unification in a cohesive manner remains a critical challenge. Current evaluation protocols predominantly treat generative and discriminative capabilities as separate tasks, leaving a gap in system-level evaluation for unified multimodal models (UMMs). In this work, we propose Self-Generative-Understanding (SGU), a novel, annotation-free evaluation framework that probes the integrated capabilities of unified models through a semantic closed-loop challenge. Without requiring new annotations, SGU leverages the dual understanding-and-generation abilities of UMMs by asking them to first perceive an image and produce a textual description, subsequently reconstruct a visual context based on that description, and finally perform reasoning over the self-generated output. This pipeline provides a zero-cost testbed that yields an integrated performance score specifically tailored for evaluating UMMs as unified systems. Extensive experiments show that even high-performing UMMs often struggle to reason over their own generated contexts, revealing limitations that are not captured by separate evaluations of understanding or generation alone. Our work provides a complementary holistic evaluation framework and offers a foundation for benchmarking the development of next-generation unified multimodal models.

[47] HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion cs.CVPDF

Wenshuo Peng, Kaipeng Zhang

TL;DR: 本文提出了HarmoniDPO框架，用于解决视频到音频生成中存在的时序同步不精确和感知质量低的问题。该框架通过整合基于偏好的优化到扩散模型中，结合了双视频表征、在线直接偏好优化和双尺度扩散搜索算法，以生成与视频高度同步且人类主观偏好的高质量音频。

Details

Motivation: 现有视频到音频生成方法通常将视频压缩为单一特征，导致时序动态和细粒度视觉信息丢失，并且其基于重建的训练目标与人类对音频质量和适当性的感知判断相关性差。

Result: 实验表明，HarmoniDPO在音视频同步和主观音频质量方面超越了现有最先进方法，为从视频生成逼真且符合人类偏好的音频提供了鲁棒的解决方案。

Insight: 主要创新点包括：1）使用结合全局上下文和逐帧特征的双视频表征以保留时序动态和语义细节；2）借鉴人类反馈强化学习思想，采用在线直接偏好优化来微调扩散模型，提升感知质量和对齐度；3）提出用于推理阶段的自适应优化输出保真度的测试时缩放算法——双尺度扩散搜索。

Abstract: Video-to-audio (V2A) generation faces significant challenges in achieving precise temporal synchronization and high perceptual quality due to the complex, ambiguous relationship between visual and auditory cues. Existing methods typically compress video inputs into single feature representations, leading to significant loss of temporal dynamics and fine-grained visual information. These approaches also rely on reconstruction-based training objectives that poorly correlate with human perceptual judgments of audio quality and appropriateness. We propose HarmoniDPO, a novel framework that integrates preference-based optimization into diffusion-based V2A generation to address these limitations. (1) Our approach leverages a dual video representation: combining global context with frame-wise features to preserve temporal dynamics and semantic detail. (2) Inspired by reinforcement learning from human feedback (RLHF), HarmoniDPO employs online Direct Preference Optimization (online-DPO) to fine-tune a diffusion-based V2A model from preference judgments, enhancing perceptual quality and alignment. (3) Additionally, we introduce Dual-scale Diffusion Search (DDS), a test time scaling algorithm that adaptively optimizes output fidelity during inference. Experiments demonstrate that HarmoniDPO outperforms state-of-the-art methods in audio-video synchronization and subjective audio quality, offering a robust solution for generating realistic, human-preferred audio from video.

[48] Surfsvr: 2D Surface Priors as 3D Geometric Regularizers for Sparse Voxel Reconstruction cs.CVPDF

Yan Di, Chengxi Li, Yaoxing Wang, Mengge Liu, Zhigang Li

TL;DR: SurfSVR提出了一种新颖的稀疏体素重建范式，通过将2D表面先验作为显式的3D几何正则化器来解决现有方法在弱纹理或稀疏观测区域产生的表面碎片化、过度细分和漂浮伪影等问题。该方法首先将每张图像组织成连贯的表面区域，然后根据拟合可靠性和几何复杂度自适应选择平面或二次曲面模型表示，并将这些结构化2D先验提升到3D空间，在整个重建流程中指导体素细分、提供监督并抑制伪影。

Details

Motivation: 现有稀疏体素重建方法主要依赖局部光度证据和离散可见性统计来优化几何，导致在弱纹理或稀疏观测区域出现表面碎片化、过度细分和漂浮伪影等问题。

Result: 在3个公共基准测试上的广泛实验表明，SurfSVR在具有显著不同可见性和几何特征的场景中，都能持续提升稀疏体素重建质量，达到了最先进（SOTA）的重建质量。

Insight: 核心创新在于将2D图像中的语义和几何一致性（通过联合推理外观、单目深度、法线和跨视图几何来组织连贯表面区域）转化为3D空间中持久的结构化约束，并统一地应用于重建流程的多个关键阶段（如体素细分、优化监督、体素剪枝和后处理），从而实现了从噪声像素级深度预测到可靠区域级几何先验的有效提升和整合。

Abstract: Sparse voxel reconstruction offers an efficient representation for high-fidelity 3D modeling, yet its geometry is commonly optimized from local photometric evidence and discrete visibility statistics. This often leads to fragmented surfaces, excessive subdivision, and floating artifacts, particularly in weakly textured or sparsely observed regions. We introduce SurfSVR, a novel sparse voxel reconstruction paradigm that treats 2D surface priors as explicit 3D geometric regularizers. Instead of directly lifting noisy pixel-wise depth predictions, SurfSVR first organizes each image into coherent surface regions by jointly reasoning over appearance, monocular depth, normals and cross-view geometry. Each region is then represented by an adaptively selected planar or quadratic surface model based on fitting reliability and geometric complexity, while cross-model agreement distinguishes reliable geometry from ambiguous predictions. These structured 2D priors are lifted into 3D and integrated throughout the reconstruction pipeline. They guide surface-adaptive voxel subdivision, provide region-level depth and normal supervision during optimization, enhance geometrically reliable sparse-observed surfaces in voxel pruning, and suppress off-surface floaters during post-refinement training. This unified design converts semantic and geometric coherence in image space into persistent structural constraints in 3D. Extensive experiments on 3 public benchmarks demonstrate that SurfSVR consistently improves sparse voxel reconstruction across scenes with substantially different visibility and geometry characteristics, achieving state-of-the-art reconstruction quality. Codes and models will be released soon.

[49] Auditing Frame-Level AUC in Weakly Supervised Video Anomaly Detection: Granularity, Resolution, and Scene Bias cs.CVPDF

Sara Abdulaziz, Egor Bondarev

TL;DR: 本文对弱监督视频异常检测（WSVAD）中主流的帧级AUC评估指标进行了审计，揭示了标准池化AUC在评估模型真实异常定位能力、区分模型性能以及揭示场景偏差方面的局限性。

Details

Motivation: 动机是质疑当前WSVAD领域广泛使用的池化AUC评估协议的有效性，因为它可能混淆异常事件定位和视频间场景差异的贡献，无法可靠反映模型在单个视频内的定位能力。

Result: 在UCF-Crime数据集上对多个SOTA模型的分析表明：池化AUC无法可靠预测视频内异常定位性能，且在当前测试集规模下缺乏区分SOTA模型微小性能差异的分辨率；同时，所有模型都在正常片段上表现出对分辨率、颜色编码等记录属性的敏感性。

Insight: 创新点在于提出了一个考虑配对粒度的评估协议，强调应在视频内粒度（within-video）评估定位能力，并揭示了模型表征与异常定位能力的解耦现象，以及跨架构共享的场景偏差问题。这为未来更严谨的WSVAD评估提供了方法论借鉴。

Abstract: Frame-level area under the ROC curve (AUC) is the dominant evaluation metric for weakly supervised video anomaly detection (WSVAD). Its standard form measures whether an anomalous frame outranks a normal frame drawn from anywhere in the test set. We refer to this comparison as pooled AUC, since it aggregates frame pairs across test videos regardless of source. Pooled AUC therefore credits both event localization and differences between video sources. We audit this protocol on UCF-Crime across recent state-of-the-art models spanning different backbone families. Holding each model’s frame scores fixed, we read them under three pairing granularities: global, per anomaly category, and within each video, then repeat the same three-granularity readout on zero-shot scores computed from the models’ internal representations. We assess ranking reliability with a paired video bootstrap. Three findings follow. First, pooled AUC does not reliably predict within-video anomaly localization: models with similar pooled scores exhibit large localization differences and rank reversals under stricter granularities. Second, at the benchmark’s test-split size, pooled AUC lacks the resolution to support state-of-the-art margins reported in the field. Within each backbone family, it resolves no comparison at those margins, while within-video AUC resolves several over identical predictions. Learned representations further reveal that within-video anomaly structure and detector localization are decoupled. Third, on normal footage alone, every model we examine separates videos by recording properties, such as resolution and color encoding, indicating that scene sensitivity is shared across the setting rather than specific to any architecture. We publicly release a granularity-aware protocol computable from existing predictions and scene-factor annotations for UCF-Crime.

[50] LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration cs.CV | cs.AIPDF

Enhuai Liu, Yunke Wang, Yutong Wang, Changming Sun, Chang Xu

TL;DR: 本文提出LoSA，一种无需训练的稀疏注意力方法，用于加速视频扩散模型的推理过程。该方法通过固定保留99%注意力质量的门槛，而非预设稀疏率，动态选择关键/值块，在保证近乎无损生成质量的同时实现显著加速。

Details

Motivation: 视频扩散变换器在采样时成本高昂，因为每个去噪步骤都需要对长3D令牌序列进行自注意力计算，其二次成本随分辨率和时长增长而占主导。现有稀疏注意力方法追求激进的稀疏性，导致速度提升与注意力保真度之间权衡不佳。

Result: 在Wan2.1-1.3B模型上，LoSA单独实现了1.36倍加速，VBench Overall指标仅下降0.06分；与特征缓存结合时，在HunyuanVideo上达到3.2倍加速，指标下降仅0.02分，优于同类稀疏基线。在三个视频扩散变换器上，LoSA在高达3.2倍加速下始终实现最佳的无训练速度-质量权衡。

Insight: 创新点在于将优化目标从固定稀疏率转变为固定保留注意力质量（如99%），并利用早期密集步骤精确测量块注意力质量，随后冻结并重用高支持度的块索引。关键洞察是发现约40%的块交互可移除而保留99%注意力质量，且高支持度块在去噪步骤间保持稳定。

Abstract: Video diffusion transformers are costly to sample: every denoising step applies self-attention over a long 3D token sequence, a quadratic cost that dominates as resolution and duration grow. Sparse attention reduces this cost without retraining, but existing methods pursue aggressive sparsity, where further speedup costs disproportionately more attention fidelity. We target the opposite end of this trade-off: fix near-lossless fidelity by construction, and remove as much computation as this constraint permits. Two observations make this regime practical: roughly 40% of block interactions can be removed while retaining 99% of the attention mass, and the high-mass support remains stable across denoising steps. We propose LoSA, a training-free sparse-attention method that fixes a retained-mass threshold of 99% rather than a sparsity ratio: it measures exact block attention masses at one early dense step, keeps, for each head and query block, the smallest key/value block set meeting the threshold, and reuses the frozen block indices for all remaining steps. On Wan2.1-1.3B, LoSA alone gives a $1.36\times$ speedup with a 0.06-point VBench Overall drop. The benefit is largest under composition: combined with feature caching, LoSA reaches a $3.2\times$ speedup on HunyuanVideo at a 0.02-point drop, versus 0.32 points for the strongest sparse baseline at comparable speed. Across three video diffusion transformers and speedups up to $3.2\times$, LoSA consistently achieves the best training-free speed-quality trade-off.

[51] Localizing to Debias: A Patch-Level Benchmark and Baseline for Weakly Supervised Spatial Anomaly Detection cs.CVPDF

Sara Abdulaziz, Abdulrahman Al-Abri, Giacomo D’Amicantonio, Egor Bondarev

TL;DR: 本文提出了一种稀疏时空框架SST-WSVADL，用于弱监督视频异常检测（WSVAD），旨在解决现有方法在粗粒度时间监督与细粒度空间推理之间的差距。该方法通过动态稀疏化逐步聚焦于最相关的时空区域，抑制背景偏差，并公开了三个数据集的帧级空间标注和评估协议，以支持空间偏差的可审计性。

Details

Motivation: 当前WSVAD方法存在背景偏差问题，即模型倾向于依赖背景和场景级线索而非真正的异常证据，这可能导致伦理问题（如将异常与社会环境错误关联），且缺乏空间定位使得偏差难以审计。

Result: 实验表明，SST-WSVADL在UCF-Crime、XD-Violence和MSAD等基准数据集上与现有方法竞争力相当，同时实现了场景偏差的定位和块级可审计性。

Insight: 创新点包括：引入稀疏时空框架通过动态稀疏化自然抑制背景内容；结合运动感知正则化端到端耦合时空分支，无需外部检测器或视觉语言提示；公开空间标注和评估协议为模型可解释性评估提供可重复基础。

Abstract: Despite growing interest in weakly supervised video anomaly detection (WSVAD), current methods struggle to bridge the gap between coarse temporal supervision and fine-grained spatial reasoning. A key obstacle is the tendency of temporal detectors to latch onto background and scene-level cues rather than truly discriminative anomaly evidence. This background bias raises ethical concerns: models may inadvertently associate anomalies with societal or environmental context rather than authentic crime-related cues. Without spatial grounding, such biases remain hidden and unauditable. To address this, we propose SST-WSVADL, a sparse spatio-temporal framework that bridges temporal anomaly detection with fine-grained spatial localization. Rather than processing all spatial regions indiscriminately, SST-WSVADL progressively focuses on the most anomaly-relevant spatio-temporal regions through dynamic sparsification, naturally suppressing background dominant content while preserving discriminative evidence. The temporal and spatial branches are coupled end-to-end via motion-aware regularization that guides sparsification toward dynamically informative regions, without relying on external detectors or vision-language prompts. We publicly release frame-level spatial annotations and a method-agnostic evaluation protocol for three public datasets: UCF-Crime, XD-Violence, and MSAD. These resources enable the community to audit spatial biases in WSVAD predictions, supporting progress toward more ethical and accountable anomaly detection. Experiments demonstrate that SST-WSVADL is competitive with prior methods across benchmarks while enabling localization and patch-level auditability of scene bias, providing a reproducible foundation for interpretability-oriented evaluation of WSVAD models.

[52] Look What the Probes Dragged In! Real-World Chest X-ray Shortcuts in MedCLIP cs.CV | cs.LGPDF

Nikolette Pedersen, Regitze Sydendal, Veronika Cheplygina, Théo Sourget

TL;DR: 本研究探讨了基于CLIP的医学视觉语言模型MedCLIP及其视觉编码器（冻结的ResNet-50）在真实世界胸部X光图像中存在的捷径问题。通过在ResNet-50的中间层附加17个线性分类探针，并在NIH-CXR14（气胸）和PadChest（心脏肥大和气胸）数据集上进行训练和评估，分析了模型在不同层的行为。研究发现，最终线性探针虽然AUROC高但校准性差，且不同类型的捷径（如局部特征或扫描仪噪声）在不同深度出现。手动分析还揭示了数据质量问题，强调了高质量数据集的重要性。

Details

Motivation: 尽管基于CLIP的模型在医学AI中达到SOTA水平，但它们仍易受捷径（shortcuts）影响。本研究旨在探究这些真实世界捷径如何在MedCLIP模型的不同层中显现，以理解其脆弱性。

Result: 在NIH-CXR14和PadChest数据集上，最终线性探针实现了高AUROC，但校准性较差。层间置信度分析表明，局部捷径（如引流管）出现在较深层，而扩散捷径（如扫描仪特定噪声）出现在较早层，与先前研究一致。

Insight: 创新点在于通过层间探针和置信度曲线系统分析医学CLIP模型中捷径的深度依赖性。客观来看，该方法揭示了模型对数据伪影的敏感性，并强调了数据质量对模型鲁棒性的关键影响，为未来医学AI的评估和改进提供了新视角。

Abstract: Vision-language models, such as contrastive language-image pre-training (CLIP)-based approaches, have reached state-of-the-art (SOTA) results in medical artificial intelligence. However, recent work reveals that CLIP-based models remain vulnerable to shortcuts. We investigate how real-world shortcuts manifest across different layers of the medical CLIP-based model, MedCLIP, and its vision encoder, a frozen ResNet-50. We attach 17 linear classification probes to the intermediate layers of the ResNet-50 and train them on three different dataset configurations and targets: NIH-CXR14 (pneumothorax) and PadChest (cardiomegaly and pneumothorax). This setup allows us to observe model behaviour during evaluation using subgroup-based calibration and layer-wise confidence curves. We find that the final linear probes achieve a high AUROC but poor calibration in the models. The layer-wise confidence analyses suggest that shortcuts emerge at different depths. Patterns consistent with localised shortcuts, such as drains, appear at later layers, while patterns consistent with diffuse shortcuts, such as scanner-specific noise patterns, emerge earlier, aligning with previous work. Finally, we conduct a manual analysis of the images, which reveals data quality issues in both NIH-CXR14 and PadChest. Our findings underscore that even SOTA models remain vulnerable to shortcuts, and the need for high-quality and well-annotated datasets to draw solid conclusions. Code can be found on our GitHub: https://github.com/nikodice4/MedCLIP_shortcuts.

[53] Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatars cs.CVPDF

Ruibin Li, Tao Yang, Zhiyuan Ma, Fangzhou Ai, Shilei Wen

TL;DR: 本文提出了Avatar-Forever框架，用于实现高质量、实时、无限时长的交互式数字人生成。该框架采用解耦并行训练策略，将生成效率与长时鲁棒性作为两种独立能力分别优化，并通过ForeverCache机制减少推理时的冗余计算，最终在单个H100 GPU上实现了768x512分辨率视频以27.2 FPS的端到端吞吐。

Details

Motivation: 解决现有流式视频生成系统依赖顺序蒸馏训练范式所导致的两个问题：一是早期阶段的失败或分布偏移会影响后续优化，使训练难以收敛；二是以蒸馏为中心的目标虽利于短时生成，但在长序列自回归推理中误差累积会导致质量下降。

Result: 基于一个220亿参数的视频基础模型构建，Avatar-Forever在保持身份一致性、运动连贯性和视觉保真度的同时，支持无限制的音频驱动数字人生成，在单个H100 GPU上实现了高分辨率（768x512）视频27.2 FPS的端到端吞吐。

Insight: 核心创新在于将生成效率（通过全参数蒸馏训练高效生成器）与长时鲁棒性（通过面向恢复的展开训练RRT训练轻量级长时适配器）解耦并行训练，避免了目标冲突并简化了训练流程；同时提出的ForeverCache分块特征缓存机制有效减少了流式推理中的历史计算冗余。

Abstract: Existing streaming video systems often rely on sequential, distillation-centered training pipelines to enable few-step long-video generation. However, this paradigm suffers from two limitations. First, failures or distribution shifts introduced in earlier stages affect later optimization, complicating the training process to converge. Second, the distillation-centric objective favours short-term generation but is prone to quality degradation when autoregressive errors accumulate over long rollouts. We propose Avatar-Forever, a decoupled parallel training framework for high-quality real-time infinite interactive avatars. Instead of coupling generation efficiency and long-horizon robustness under a sequential distillation pipeline, we treat them as two independent capabilities that can be trained in parallel. One branch performs full-parameter distillation to train an efficient generator with high visual quality, while another trains a lightweight long-horizon adapter via Recovery-oriented Rollout Training (RRT), which improves generation robustness under long-horizon inference conditions. Our decoupled parallel training design simplifies the overall training process and avoids unnecessary objective conflicts between few-step generation and long-horizon adaptation. We further introduce ForeverCache, a chunk-wise feature caching mechanism to substantially reduce redundant history computation during streaming inference. Built upon a 22B video foundation model, Avatar-Forever supports unbounded audio-driven avatar generation while maintaining identity consistency, motion coherence, and visual fidelity, enabling an end-to-end throughput of high-resolution 768x512 videos at 27.2 FPS on a single H100 GPU and providing a practical path toward stable digital humans.

[54] SCOPE-Router: Cost-Aware Open-Set VLM Routing for Execution-Oriented Tasks cs.CVPDF

Tao Yu, Yifei Qu, Zhiqing Cui, Pengfei Zhou, Zhongtian Luo

TL;DR: 本文提出了SCOPE-Router，一个面向执行任务的成本感知开放集视觉语言模型路由系统。论文贡献包括：创建了首个面向执行的VLM路由基准VLM-ExecRouterBench，涵盖代码、智能体和搜索三个领域；设计了双塔结构的SCOPE-Router路由器，通过混合校准构建模型行为档案，支持新模型无需重新训练即可加入路由；提出了架构无关的成本感知目标函数CRM+RCCR，将成本偏好编码到连续相关性目标中。

Details

Motivation: 现有VLM路由研究局限于传统视觉问答评估，缺乏针对开放集场景的系统校准优化，且训练目标通过softmax归一化稀释了多正例信号，未考虑成本因素。本文旨在解决这些限制，为执行导向任务开发更实用、成本感知的开放集路由方法。

Result: 在三个基准测试中，SCOPE-Router在所有基准上均取得了最佳Rank Score。在分布外设置下超过第二名1.84分，在双重分布外开放集评估下超过6.75分。CRM+RCCR目标函数应用于四种不同路由器时，将Rank Score提升了1.25到6.21分。

Insight: 创新点包括：1) 构建了首个面向执行任务的VLM路由基准，覆盖多领域和宽价格范围模型；2) 通过混合校准构建模型行为档案的双塔路由架构，实现新模型即插即用；3) 提出CRM+RCCR目标函数，通过逐对独立评分将成本偏好编码为连续相关性目标，避免多正例信号稀释，并在路由空间中正则化具有相似偏好的查询。

Abstract: Model routing aims to select the most suitable model from a candidate pool for each query, balancing quality and cost. Existing VLM routing research is limited to traditional VQA evaluation, lacks systematic calibration optimization for open-set scenarios, and employs training objectives that dilute multi-positive signals via softmax normalization without incorporating cost. We address these limitations with three contributions: (1)VLM-ExecRouterBench, the first execution-oriented VLM routing benchmark covering Code, Agentic, and Search domains with 11 candidate models spanning nearly two orders of magnitude in pricing; (2)SCOPE-Router, a dual-tower router that matches queries to model behavior profiles constructed via hybrid calibration (random/diagnostic/diversity sampling), enabling new models to join routing without retraining; (3)CRM+RCCR, an architecture-agnostic cost-aware objective that encodes cost preference into continuous relevance targets through per-pair independent scoring, eliminating multi-positive dilution while regularizing queries with similar routing preferences to be closer in the routing space. Empirically, SCOPE-Router achieves the best Rank Score on all three benchmarks, surpassing the runner-up by 1.84 points under OOD settings and by 6.75 points under doubly OOD open-set evaluation. When applied to four diverse routers, CRM+RCCR improves Rank Score by 1.25–6.21 points.

[55] Autonomous Telerehabilitation via Skeletal Motion Prediction and Joint-Level Performance Assessment cs.CV | cs.LGPDF

Lara Pereira, João Ruivo Paulo, Pedro Santos, Paulo Peixoto

TL;DR: 本文提出了一种基于无标记RGB视频的远程康复系统，该系统集成了骨骼运动质量评估和短期运动预测两个模块。系统通过自注意力双向LSTM进行运动质量分类，并利用基于图的运动预测模块计算关节位置误差以提供局部偏差反馈。

Details

Motivation: 动机是开发自主康复系统，使其不仅能识别人体运动，还能在没有治疗师持续监督的情况下提供结构化反馈，以支持用户进行远程康复训练。

Result: 分类器在PROZIS数据集的深蹲序列上实现了96.45%的平均类别准确率；采用的STARS预测器在Human3.6M数据集上，于560毫秒预测时长的平均每关节位置误差为75.8毫米，在所有预测时间范围内均优于基于图和循环神经网络的基线模型。

Insight: 创新点在于将运动质量评估与短期运动预测集成到一个统一的管道中，通过MMD-NCA度量学习进行分类，并利用关节级位置误差生成空间局部化的偏差信号，为辅助机器人和家庭康复场景的自主反馈系统提供了新思路。

Abstract: Autonomous rehabilitation systems must not only recognize human motion but also provide structured feedback to support users without continuous therapist supervision. This paper presents a telerehabilitation pipeline that integrates skeleton-based exercise quality assessment and short-term motion prediction into a two-module system operating on marker-free RGB video. A self-attentive Bidirectional LSTM performs exercise quality classification using MMD-NCA metric learning, while a graph-based motion prediction module computes per-joint position errors between predicted and observed poses, generating spatially localized deviation signals. Each module is evaluated independently on established benchmarks: the classifier achieves 96.45% mean-class accuracy on squat sequences from the PROZIS dataset, and the adopted STARS predictor achieves a mean MPJPE of 75.8 mm at 560 ms on Human3.6M, outperforming graph and recurrent baselines across all prediction horizons. The framework is designed for eventual deployment in assistive robotics and home-based rehabilitation contexts; end-to-end integration and clinical validation are important directions for future work. By combining motion recognition and prediction in a single system, this work contributes a step toward autonomous, feedback-driven telerehabilitation, for more accessible and scalable rehabilitation solutions.

[56] Understanding Why Foundation Models Work for Diffusion-Generated Image Detection cs.CVPDF

Davide Cozzolino, Giovanni Poggi, Luisa Verdoliva

TL;DR: 本文研究了基于视觉基础模型的AI生成图像检测器为何有效，通过DDIM反转、频率交换和潜在空间分析等方法，揭示了这些检测器主要依赖非语义的低中频分布差异来区分真实图像与扩散生成图像。

Details

Motivation: 尽管视觉基础模型在检测AI生成图像时表现出强大的泛化能力和鲁棒性，但其有效性的原因尚不明确，本文旨在探究这些检测器利用哪些线索进行区分。

Result: 实验表明，检测器分数在语义相同的合成图像副本间显著变化，且判别线索主要位于低中频范围，而非传统生成模型常见的高频伪影；潜在空间分析显示扩散生成图像的方差和有效维度降低。

Insight: 创新点在于通过系统分析协议揭示了基础模型检测器依赖非语义的低中频分布差异，这为理解其鲁棒性和泛化性提供了新见解，并有助于开发更可解释的取证方法。

Abstract: Vision foundation models have recently emerged as powerful feature extractors for detecting AI-generated images, achieving strong generalization across generators and robustness to common image degradations. However, the reason behind their effectiveness is poorly understood. In this work, we investigate what cues are exploited by foundation-model-based detectors to distinguish real images from diffusion-generated ones. To this end, we design an ad hoc analysis protocol based on DDIM inversion. Given a real image we generate a sequence of synthetic copies by changing the depth of DDIM inversion. Even though most copies are semantically identical to the real reference, the detector score varies significantly across them due to subtle traces introduced by the diffusion synthesis, showing that its decision is not primarily driven by semantic failures. Through a frequency-swapping analysis, we further reveal that the discriminative cues exploited by the detectors are mainly localized in the low-to-mid frequency range, rather than only in the high-frequency range, as is the case for artifacts commonly associated with generative models. Finally, a latent-space analysis shows that regenerated images exhibit reduced variance and effective dimensionality, indicating that diffusion models do not fully reproduce the variability of real data. Overall, our results suggest that foundation-model-based detectors succeed by capturing non-semantic low-to-mid frequency distributional discrepancies between real and diffusion-generated images. These findings provide new insight into the robustness and generalization of such detectors and suggest directions for more interpretable forensic methods.

[57] Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization cs.CVPDF

Byungoh Ko, Jinyoung Park, Jongha Kim, Jeehye Na, Jaewon Cho

TL;DR: 本文研究了多模态大语言模型（MLLMs）中的物体幻觉问题，即模型生成与视觉输入不一致的错误描述。作者发现，现有的直接偏好优化（DPO）方法及其变体未能充分利用上下文信息来缓解幻觉。为此，他们提出了上下文偏好增益（CPG）指标来衡量模型利用上下文的能力，并基于此提出了上下文校准的DPO（C²-DPO）方法，该方法直接最大化CPG，从而更有效地减少幻觉。

Details

Motivation: 尽管DPO及其利用上下文的变体被用于缓解MLLMs的物体幻觉问题，但现有方法是否真正利用了上下文信息尚不明确。本文旨在探究DPO对上下文信息的利用程度，并解决其利用不足的问题。

Result: 在多个基准测试中，C²-DPO显著降低了幻觉率，且不损害通用推理能力。例如，在Qwen2-VL-Instruct-2B模型上，相对减少了Object HalBench幻觉率36%。

Insight: 论文的创新点在于提出了CPG这一量化指标来诊断DPO对上下文的利用不足，并设计了C²-DPO方法，通过直接优化CPG来强制模型更好地利用上下文信息，从而更有效地校准偏好、减少幻觉。这为基于偏好的优化提供了新的、可量化的改进方向。

Abstract: Multimodal large language models (MLLMs) have made rapid progress, yet they still exhibit object hallucination, generating plausible but incorrect descriptions that are inconsistent with the visual input. Direct Preference Optimization (DPO) mitigates this by training models to prefer non-hallucinated responses over hallucinated ones, and recent efforts further enrich the preference data with relevant context. However, it remains unclear whether DPO actually leverages such context. To investigate this, we propose Contextual Preference Gain (CPG), a simple metric that measures how much a model’s preference strengthens when relevant context is provided. We find that higher CPG consistently corresponds to lower hallucination, yet standard DPO and its variants exhibit only limited CPG, indicating that they underutilize contextual information and thus remain prone to hallucination. To address this, we propose Context-Calibrated DPO (C$^2$-DPO), which directly maximizes CPG while preserving the original preference ordering. Across multiple benchmarks, C$^2$-DPO substantially reduces hallucination without compromising general reasoning, relatively reducing the Object HalBench hallucination rate of Qwen2-VL-Instruct-2B by 36%. Code is available at https://github.com/mlvlab/C2-DPO

[58] Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs cs.CVPDF

Yung-Hsu Yang, Luigi Piccinelli, Samuel Rota Bulò, Sunghwan Hong, Denis Rozumny

TL;DR: 本文提出了Map-Det3D，一种用于单目视频流的在线多视角3D目标检测模型。该方法摒弃了主流的2D检测再提升至3D的范式，而是利用一个前馈式度量3D重建模型作为几何主干，直接从RGB图像序列重建的3D空间中预测度量3D边界框。

Details

Motivation: 动机在于解决单目3D检测中因深度和绝对尺度信息不足而导致的脆弱性问题。现有方法依赖2D检测再预测3D属性，对深度误差敏感，且学习的尺度先验在相机、运动或环境发生域偏移时容易失效。

Result: 实验表明，该设计在不同基准测试上支持强大的在线性能，并且无需适应即可实现稳健的跨域迁移，证明了将重建先验用于检测是实现稳定度量3D检测的可行路径。

Insight: 核心创新点在于将度量3D重建模型重新用作检测的几何主干，直接在重建的3D空间中进行检测，避免了易错的2D到3D提升步骤。这为单目视频的稳定3D感知提供了一种新的、基于重建先验的实用方案。

Abstract: Metric 3D object detection is a core capability for embodied agents, yet most reliable systems lean on depth sensors, trading away cost, power, and integration simplicity. This motivates monocular 3D detection, which avoids additional constraints, yet it faces a major obstacle: from a single image, depth, and especially absolute scale, are underconstrained. As a result, the prevailing pattern of detecting in 2D and then predicting 3D attributes is often brittle, since modest range errors can dominate 3D localization, and the learned scale prior can fail when cameras, motion, or environments undergo domain shifts. To address this, we propose Map-Det3D, an online multi-view 3D object detection model that brings detection directly into a 3D space reconstructed from RGB. We map a short temporal window into multiple views and repurpose a feed-forward metric 3D reconstruction model as our geometric backbone while tuning its object-aware capabilities. Building on this representation, Map-Det3D directly predicts boxes in metric 3D space, without the widely used 2D-to-3D lifting. Experiments across different benchmarks show that this design supports strong online performance and robust transfer without adaptation, suggesting that training reconstruction priors for detection is a practical route to stable metric 3D detection from monocular video. Code and models are available at https://royyang0714.github.io/Map-Det3D.

[59] HSTGFormer: Hyper Spatial-Temporal Graph Transformer for 3D Human Pose Estimation cs.CV | cs.AIPDF

Ruochen Li, Shuang Chen, Wenke E, Farshad Arvin, Amir Atapour-Abarghouei

TL;DR: 本文提出了HSTGFormer，一种用于单目3D人体姿态估计的图增强Transformer框架。该方法通过构建超时空图（HSTG）将全局时空推理分解为围绕关节-时间节点的局部时空感受野，并引入自适应双尺度时序图（ADSTG）来捕获互补的短程和长程时序依赖，从而实现了结构感知的耦合推理。

Details

Motivation: 现有基于Transformer的方法通常将空间推理和时间推理作为独立的阶段进行处理，这可能削弱人体运动中固有的统一时空相互依赖关系，并在时序建模前压缩了帧级结构信息。

Result: 在Human3.6M和MPI-INF-3DHP基准测试上的实验表明，HSTGFormer在保持高计算效率的同时，取得了强大的姿态估计精度。

Insight: 核心创新在于将时空推理重新表述为关节-时间节点上的局部耦合图聚合，通过超时空图（HSTG）实现结构感知的联合建模，以及通过自适应双尺度时序图（ADSTG）和节点级融合模块来灵活捕获多尺度时序模式。

Abstract: Transformer-based methods have achieved strong performance in monocular 3D human pose estimation, but most existing approaches organise spatial and temporal reasoning as separate stages, which may weaken unified spatial-temporal interdependencies inherent in human motion and compress frame-level structural information before temporal modelling. In this paper, we propose HSTGFormer, a graph-enhanced Transformer framework that reformulates spatial-temporal reasoning as localised coupled graph aggregation over joint-time nodes. Specifically, HSTGFormer introduces a Hyper Spatial-Temporal Graph (HSTG), which decomposes global spatial-temporal reasoning into local spatial-temporal receptive fields around individual joint-time nodes by extending per-frame skeleton graphs into temporal neighbourhoods, thereby enabling structure-aware coupled reasoning while preserving local structural motion information. It further incorporates an Adaptive Dual-Scale Temporal Graph (ADSTG) to capture joint-specific temporal dependencies over complementary short- and long-range windows. A lightweight node-wise fusion module further adaptively integrates the two graph representations for each joint-time node. Experiments on Human3.6M and MPI-INF-3DHP show that HSTGFormer achieves strong accuracy with high computational efficiency.

[60] GeoFlow: Efficient Driving Video Generation via Geometry-Aligned Priors cs.CVPDF

Jiazheng Liu, Hang Li, Jiawei Zhang, Jiahe Li, Xiaohan Yu

TL;DR: 本文提出GeoFlow框架，通过引入几何对齐先验分布来提升驾驶视频生成的效率。该方法利用多视图几何和空间自适应噪声注入构建源分布，从而缩短采样轨迹，显著减少训练和推理时间。

Details

Motivation: 现有扩散模型和流匹配模型在生成高保真驾驶视频时，因依赖标准高斯噪声源分布而忽略时空相关性，导致推理延迟高且几何不一致。

Result: 实验表明，GeoFlow仅需在基线模型上微调数小时即可提升少步生成质量，完全收敛训练后大幅减少达到SOTA视频生成所需的推理步数。

Insight: 创新点在于利用几何先验构建源分布，使采样轨迹更直更短，从而高效利用驾驶视频的时空结构，减少冗余计算。

Abstract: Generative models like Diffusion Models and Flow Matching have demonstrated remarkable capabilities in synthesizing high-fidelity driving videos, but are severely constrained by high inference latency due to the requirement of extensive sampling steps. We argue that this inefficiency stems from the prevailing reliance on a standard Gaussian source distribution, where consecutive frames are initialized as independent Gaussian noise. This paradigm disregards the rich spatiotemporal correlations inherent in driving videos, compelling the model to regenerate deterministic scene structures existing in previous frames from noise, which is both computationally redundant and prone to geometric inconsistency. To address this problem, we propose GeoFlow, a novel framework designed to achieve efficient driving video generation by harnessing explicit geometric priors. Instead of sampling from standard Gaussian noise, we leverage multi-view geometry and spatially-adaptive noise injection to construct a Geometry-Aligned Prior (GAP) distribution as starting point. This initialization bridges the gap between source distribution and data distribution, yielding a significantly straighter and shorter sampling trajectory. Extensive experiments demonstrate that GeoFlow can achieve remarkable efficiency of both training and inference: merely several hours of fine-tuning on baseline models can significantly boost few-step generation quality, while fully converged training drastically reduces number of inference steps required for state-of-the-art video generation.

[61] Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Prediction cs.CVPDF

Zhongbin Guo, Jiahao Xie, Dongling Xiao, Qianle Wang, Ruiqi Lu

TL;DR: 本文提出GAS框架，将视觉生成任务重新解释为表征学习的辅助监督信号，通过解耦的混合Transformer架构实现跨模态的下一代嵌入预测，在训练后丢弃生成分支以实现零推理开销，从而提升多模态理解能力。

Details

Motivation: 现有统一框架通常依赖离散视觉标记化或扩散目标，其生成目标与视觉理解模型所需的连续表征存在差异，难以直接迁移以增强预训练多模态大语言模型。

Result: 在不同模型规模和训练阶段，GAS均能提升多模态理解的整体性能，尤其在感知和空间理解任务上获得最可靠的增益，且推理时无额外开销。

Insight: 采用解耦的混合Transformer架构，通过共享下层主干和并行上层层，使生成损失以更精细的空间精度和更强的视觉保留能力丰富共享视觉通路，同时保护上层理解层免受直接生成梯度影响；构建高度相关的生成任务以需求深度认知基础而非仅通用合成。

Abstract: While Multimodal Large Language Models (MLLMs) have achieved remarkable progress, visual understanding and generation are typically treated as divergent objectives. Existing unified frameworks often rely on discrete visual tokenization or diffusion objectives whose generative targets differ from the continuous representations consumed by visual understanding models, making direct transfer to enhance existing pretrained MLLMs non-trivial. In this work, we present GAS, a generation-guided training framework that reinterprets visual generation as auxiliary supervision for representation learning. Concretely, GAS adapts Next Embedding Prediction (NEP) as a cross-modal generation paradigm within a decoupled Mixture-of-Transformers (MoT) architecture. By maintaining a shared lower trunk and parallel upper layers, GAS lets generation losses enrich the shared visual pathway with finer spatial precision and stronger visual retention while shielding the upper understanding layers from direct generation gradients. To maximize this synergy, we further construct highly correlated generation tasks that demand deep cognitive grounding rather than generic synthesis alone. Across model scales and training stages, GAS improves aggregate multimodal understanding, with its most reliable gains on perception and spatial comprehension. Crucially, because the auxiliary generation branch is discarded after training, these gains incur zero inference overhead. Extensive controlled comparisons and representation-level analyses further clarify when and why generation-guided training benefits understanding, and demonstrate the feasibility of generation-guided training as a practical route to stronger multimodal understanding.

[62] Few-Shot Ordinal Learning for Day-Wise Freshness Estimation with Hyperspectral Fish Images cs.CV | cs.AI | eess.IV | eess.SPPDF

Kazi Nabiul Alam, Pooneh Bagheri Zadeh, Akbar Sheikh-Akbari

TL;DR: 本文提出了一种基于高光谱成像（HSI）的少样本学习框架，用于估计鱼片逐日新鲜度。该方法将每个鱼片视为一个独立的少样本任务，并引入基于CORAL的序数预测头来建模新鲜度变化的排序特性，同时结合生物启发的单调性和嵌入平滑性约束。在仅使用每个鱼片三个标记日期的严格未见鱼片协议下，该方法在16天鲑鱼HSI数据集上取得了优于标量回归和标签分布基线的性能。

Details

Motivation: 现有基于HSI的食品质量评估深度学习方法需要密集标注的训练数据，成本高昂，且难以应对鱼片间强变异性和每个产品标记数据稀缺的挑战。本文旨在开发一个少样本学习框架，以低成本、高效地估计逐日新鲜度。

Result: 在严格的未见鱼片协议下，该方法在16天鲑鱼HSI数据集上实现了1.58天的平均绝对误差和72.3%的2天准确率，仅使用每个鱼片三个标记日期，显著优于相同协议下的标量回归和标签分布基线方法。

Insight: 创新点包括将每个鱼片建模为独立的少样本任务，采用CORAL风格的序数预测头来捕获新鲜度进展的排序特性，并引入基于生物学的单调性和嵌入平滑性约束以引导预测轨迹的合理性。这为小样本食品质量评估提供了新思路。

Abstract: Non-destructive food quality assessment has increasingly benefited from hyperspectral imaging (HSI), which captures spectral signatures linked to biochemical changes during storage. Estimating day-wise freshness, however, remains challenging owing to strong inter-fillet variability and scarce labelled data per product. All existing deep learning approaches for HSI-based freshness prediction operate under full supervision, requiring densely annotated training sets that are costly to obtain at the individual-product level. We introduce, to the best of our knowledge, the first few-shot learning framework for HSI-based food quality estimation. Each fillet defines a distinct episodic task, and a CORAL-style ordinal prediction head captures the ranked nature of freshness progression through cumulative threshold modelling. Biologically grounded monotonicity and embedding smoothness constraints further guide predictions toward plausible trajectories. On a 16-day salmon HSI dataset under a strict unseen-fillet protocol, our method achieves a mean absolute error of 1.58 days and 2-day accuracy of 72.3% with only three labelled days per fillet, substantially outperforming scalar regression and label-distribution baselines under an identical unseen-fillet protocol.

[63] SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Reward cs.CV | cs.AIPDF

Zile Zhou, Huining Yuan, Weichen Zhang, Xinlei Chen, Xiao-ping Zhang

TL;DR: 本文提出了SCOUT方法，旨在解决现有视觉语言模型在空间推理方面的瓶颈。该方法结合了结构化思维链和过程监督强化学习，通过显式建模3D环境感知来增强空间理解，并引入多目标过程奖励和定制优势估计以实现精细的信用分配。

Details

Motivation: 现有视觉语言模型在鲁棒空间推理方面存在瓶颈，强化学习方法存在中间推理步骤信用分配不佳的问题，而结构化推理方法又忽视了深度感知对全面3D理解的关键作用。

Result: 在通用空间基准测试和复杂空间推理任务上，SCOUT-3B模型分别比基线模型提升了16.85%和6.3%；更大的SCOUT-7B模型甚至以4.28%的优势超越了GPT-4o。尽管仅在单张图像上训练，SCOUT-7B在跨域泛化到多图像和视频场景时也表现出鲁棒性。

Insight: 创新点在于将结构化思维链与过程监督强化学习相结合，通过显式建模3D感知和引入多目标过程奖励机制，解决了空间推理中信用分配的难题，并构建了专用的结构化数据集SCOUT-24k来支持训练。

Abstract: Existing Vision-Language Models (VLMs) exhibits a critical bottleneck in robust spatial reasoning. Recent reinforcement learning (RL) methods aim to close this gap with verifiable outcomes, yet they suffer from poor credit assignment across intermediate reasoning steps. Concurrently, structured reasoning approaches overlook the critical depth perception necessary for comprehensive 3D understanding. To address these challenges, we propose SCOUT (Structured Chain-Of-Thought Utilizing Process-Supervised RL Training). Specifically, we design a structured Chain-of-Thought (CoT) framework that explicitly models 3D environmental perception to ensure robust spatial understanding and reasoning. Furthermore, we introduce a novel RL algorithm featuring multi-objective process rewards and a tailored advantage estimation method, facilitating fine-grained credit assignment across distinct segments of the reasoning trajectory. To support our framework, we develop SCOUT-24k, a structured spatial reasoning CoT dataset synthesized through a customized pipeline. Extensive evaluations demonstrate that SCOUT-3B improves upon baseline models by 16.85% and 6.3% on general spatial benchmarks and complex spatial reasoning tasks respectively. Notably, our larger SCOUT-7B even outperforms GPT-4o by a margin of 4.28%. Moreover, despite being trained exclusively on single image, SCOUT-7B exhibits robust out-of-domain generalization to multi-image and video scenarios. These empirical results render SCOUT as a critical step towards next generation of spatially-aware VLMs.

[64] ScaleVid: Geometry-Aware Video Object Scaling with Mesh-Free Inference cs.CVPDF

Youze Huang, Penghui Ruan, Bojia Zi, Xianbiao Qi, Shihao Zhao

TL;DR: 本文提出了一种名为ScaleVid的无网格推理方法，用于实现几何感知的视频对象缩放。该方法通过一个渐进式的两阶段训练框架，将几何感知的前景变换与背景保持和真实视频合成解耦，无需在推理时进行网格-像素对齐或显式的3D重建。

Details

Motivation: 现有方法存在局限：基于文本引导的方法主要在2D图像平面上操作，深度引导的方法控制粗糙，而基于网格的方法需要昂贵的3D重建。本文旨在解决这些不足，实现更高效、更实用的几何感知视频对象缩放。

Result: 在构建的互补配对几何和真实背景基准测试以及野外视频上进行了广泛实验。结果表明，该方法在几何一致性、前景保真度和背景保持方面优于现有方法，并且比需要显式3D重建的方法推理速度更快、更实用。

Insight: 创新点在于提出了一个两阶段训练框架，通过构建几何扰动的伪源视频作为输入，原始完整视频作为重建目标，实现了无需真实世界缩放目标对的无监督学习。第一阶段学习鲁棒的前景-背景合成，第二阶段引入以对象为中心的3D变形引导，从而实现几何感知的缩放，避免了显式的3D重建。

Abstract: Geometry-aware video object scaling aims to anisotropically resize the object along object-centric axes while preserving geometric plausibility, temporal coherence, and background consistency. Existing text-guided methods mainly operate in the 2D image plane, while depth-guided approaches provide coarse control and mesh-based methods require costly 3D reconstruction. We present a progressive two-stage training framework that decouples geometry-aware foreground transformation from background preservation and realistic video composition, without mesh-pixel alignment and explicit 3D reconstruction at inference. In both stages, geometrically perturbed pseudo-sources are constructed from real videos, while the original complete videos are retained as reconstruction targets. The first stage uses planar transformations to learn robust foreground-background composition, whereas the second introduces object-centric 3D deformation guidance for geometry-aware scaling. This pseudo-source reconstruction formulation enables real-video synthesis without paired real-world scaling targets. We construct complementary paired-geometry and real-background benchmarks and further evaluate on in-the-wild videos. Extensive experiments demonstrate superior geometric consistency, foreground fidelity, and background preservation, together with faster and more practical inference than methods requiring explicit 3D reconstruction.

[65] Automated Borehole Core Analysis with Report-Derived Weak Labels and Supervised Crack Segmentation cs.CVPDF

Usama Imdad, Ali Khan, Luke Lu, Zubair Khalid, Arif Mahmood

TL;DR: 该论文提出了一种自动化钻孔岩心分析框架，结合了基于报告文本的弱监督分类和全监督裂缝分割方法。通过从数字测井报告中提取结构化间距类别作为弱标签，并利用DINO编码器获取领域特定表示；同时，手动标注了5,087张岩心行图像用于训练分割模型，其中提出的门控U-Net结合了PiDiNet边缘图和Mask R-CNN掩码，实现了最优分割性能。后处理将预测的裂缝位置转换为缺陷间距类别，并辅以基于规则的层理角度和岩性颜色描述估计。

Details

Motivation: 解决钻孔档案中缺乏像素级裂缝标注的问题，利用现有的岩心托盘照片和数字测井报告，自动化提取缺陷间距信息，以支持地质分析。

Result: 在裂缝分割任务中，门控U-Net的F1分数达到0.860，裂缝类IoU为0.754，在评估的分割配置中表现最佳；在1,200张评估图像上，层理角度和岩性颜色描述的预测与报告参考值的一致性分别为75.4%和84.7%。

Insight: 创新点包括：利用报告文本层提取弱间隔级标签进行弱监督分类，结合DINO编码器增强领域表示；提出门控U-Net，通过学习的空间门控机制融合边缘和掩码信息，提升分割精度；框架整合了弱监督分类和全监督分割，实现端到端的岩心分析。

Abstract: Borehole archives commonly contain core tray photographs and corresponding digital log reports, but no native pixel-level crack annotations. We investigate two complementary approaches for extracting defect-spacing information from these archives. First, structured spacing categories recovered from the report text layer provide weak interval-level labels for classification. A DINO encoder trained on unlabeled core crops supplies domain-specific representations, and a manually verified subset is used to identify label inconsistencies. Second, we manually annotate 5,087 extracted core-row images and evaluate fully supervised crack-segmentation models. Our gated U-Net combines PiDiNet edge maps with Mask R-CNN masks through a learned spatial gating mechanism. This configuration achieves an F1 score of 0.860 and a crack-class IoU of 0.754, the highest result among the evaluated segmentation configurations. Deterministic post-processing converts predicted crack locations into defect-spacing categories. Separate rule-based branches estimate core-relative bedding angles and lithological color descriptors; their predictions agree with log-report references on 75.4% and 84.7% of 1,200 evaluated images, respectively. Because these references are extracted from existing reports, the reported values measure agreement with recorded geological observations rather than independent physical accuracy. The resulting framework combines report-derived weak supervision for spacing classification with fully supervised segmentation for image-based crack localization.

Weihao Bo, Shan Zhang, Yanpeng Sun, Jie Liu, Yongke Yao

TL;DR: 本文提出了Diagram-MMU，一个用于评估多模态大语言模型（MLLMs）在科学图表解析与理解方面能力的多模态基准。该基准包含来自六个领域的3.7k个精选图表和18.3k个人工验证的问题，评估了图表到代码解析、图表到代码编辑和图表问答三个任务。对12个MLLMs的评估表明，图表到代码任务比图表问答更具挑战性，且代理设置对性能有不同影响。

Details

Motivation: 随着MLLMs在科学写作与协作（如OpenAI Prism）中应用的增长，将科学图表直接转换为LaTeX TikZ代码等功能变得重要。为了评估和提升MLLMs在此类图表相关任务上的能力，需要构建一个专门的基准。

Result: 在Diagram-MMU基准上评估了12个MLLMs。结果显示，模型在图表问答上推理良好，但在图表到代码解析和编辑任务上表现挣扎，表明这些任务更具挑战性。在代理设置下，大多数模型在解析和编辑任务上性能提升，但在问答任务上性能下降，而Claude-4.6 Opus在所有三个任务上均表现出一致的提升。

Insight: 论文的创新点在于构建了一个专注于科学图表多模态理解与代码生成的大规模、多领域基准。客观来看，其将图表任务细分为解析、编辑和问答，并引入代理设置进行评估，有助于揭示MLLMs在不同任务类型上的能力差异和优化方向，为提升MLLMs的图表到代码生成能力提供了具体的研究目标。

Abstract: Multimodal Large Language Models (MLLMs) have been growing the capability for scientific writing and collaboration. For example, OpenAI Prism is a free workspace for scientific writing and collaboration. One important feature in Prism is turning scientific diagrams directly into LaTeX TikZ code. In this paper, we build a benchmark, Diagram-MMU, a multi-modal benchmark designed to assess MLLMs’ ability for scientific diagram parsing and understanding. Diagram-MMU features 3.7k curated diagrams and 18.3k human-validated questions across six domains. It evaluates MLLMs on three tasks common in vibe writing workspaces: diagram-to-code parsing, diagram-to-code editing, and diagram question answering, alongside agentic settings per task. The evaluation of 12 MLLMs reveals that diagram-to-code tasks are more challenging than diagram question answering: models can reason well over diagrams but struggle to parse and edit them, underscoring the need for methods to enhance MLLMs’ capability in diagram-to-code generation. Under agentic settings, most models improve parsing and editing performance but degrade on question answering, while Claude-4.6 Opus consistently improves across all three tasks. Project Page: https://vi-ocean.github.io/projects/diagram-mmu.

[67] AVA-Encoder: Towards Agent-Native Video Representation Learning cs.CV | cs.CLPDF

Chuyue Li, Jinpeng Yu, Haozhe Wang, Tian Xueyun, Zhijing Zhang

TL;DR: AVA-Encoder是一个用于智能体原生视频表示学习的框架，通过将视频转换为知识图谱（KG）表示并重建回视频，为智能体提供结构化、可理解和可操作的视频表示。该框架利用文本梯度优化，通过重建差异驱动编码策略的伪训练和知识图谱表示的细化。

Details

Motivation: 解决创意智能体缺乏从高质量人类电影中学习的有效方法，以及缺少既忠实于电影内容又可直接用于智能体推理和操作的结构化视频表示这一关键挑战。

Result: 在实验中，AVA-Encoder比最强外部基线提升了20.7个百分点；在仅策略控制的设置下，其伪训练的镜头级智能体视频编码器策略在使用系统提示令牌减少74.3%的情况下，仍优于精心人工调整的策略。

Insight: 创新点在于提出了将视频表示为知识图谱（包含结构化文本节点和链接资产层）的智能体原生表示形式，以及利用重建差异驱动的文本梯度优化框架进行策略训练和表示细化，这为智能体理解和编辑视频内容提供了新途径。

Abstract: Creative agents still lack an effective way to learn from high-quality human films, limiting their ability to produce cinematic-grade videos. A key challenge is the absence of a structured video representation that is both faithful to film content and directly usable for agentic reasoning and manipulation. To address the challenge, we propose the Agentic Video Auto-Encoder (AVA-Encoder), a framework for learning agent-native video representations via agentic auto-encoding. AVA-Encoder transforms a video into a knowledge graph (KG) representation and then reconstructs it back into video. Its hierarchy and state nodes store structured text, while a linked asset layer holds generated images, audio, and video. Typed edges preserve the relations between these text descriptions and assets in a form that agents can easily understand, query, and edit. The video reconstruction differences drive a textual-gradient optimization framework, which expresses evaluation feedback as natural-language update directions for Data-Independent Encoding Policy Pseudo-Training in the outer loop and optional Data-Dependent KG Representation Refinement in the test-time inner loop. Extensive experiments show that AVA-Encoder improves by 20.7 percentage points over the strongest external baseline. In the controlled policy-only setting, its pseudo-trained shot-level Agentic Video Encoder policy also outperforms a carefully human-tuned policy while using 74.3% fewer system-prompt tokens. We release the complete AVA-Encoder framework, a reliable agentic video reconstruction benchmark, and the first dataset of high-quality film KG representations.

[68] Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence cs.CV | cs.AI | cs.MMPDF

Aman Tyagi, Hemanth Boinpally, Jonathan Chen, Douglas Gebert, Steven Hickson

TL;DR: 本文提出了一种名为’Agentic Self-Improvement’的框架，旨在解决现有黑盒图像到视频（I2V）模型因随机性导致的输出不稳定和控制困难问题。该框架通过一个两阶段的闭环优化过程，首先利用多模态大语言模型迭代优化输入提示，然后使用贝叶斯优化协同优化随机种子和CFG尺度，从而系统性地提升视频生成的质量和可控性。

Details

Motivation: 现代黑盒I2V模型虽然功能强大，但其固有的随机性使得文本提示或超参数的微小变化会导致输出结果差异巨大，这迫使专业工作流依赖于低效的试错过程。论文旨在解决这一缺乏细粒度控制和可靠性的核心挑战。

Result: 在人类偏好研究中，通过该智能体方法生成的视频显著优于无引导的基线方法，胜率高达69%。这表明该方法在提升视频生成的可预测性和控制性方面取得了显著成效。

Insight: 论文的核心创新在于将视频合成重构为一个目标导向的闭环优化问题，并提出了一个结合多模态LLM提示优化与贝叶斯优化的两阶段框架。其中，利用Davidsonian场景图查询进行语义一致性评估和用于伪影检测的常见错误问题，以及由此衍生的新颖视频-文本一致性评分，是指导优化过程的关键技术洞察。

Abstract: Modern black-box Image-to-Video (I2V) models offer powerful capabilities in automated content creation, yet their lack of fine-grained control and reliability presents significant challenges in professional workflows. Their inherent stochasticity causes minor variations in textual prompts or hyperparameters to yield drastically different outputs often necessitating inefficient, brute-force trial-and-error processes. To address these limitations, we introduce the ``Agentic Self-Improvement” framework, which reframes video synthesis into a closed-loop, goal-directed optimization. Our framework systematically navigates the generation parameter space using a novel two-stage approach. In the first stage, an iterative prompt optimization loop uses a multimodal Large Language Model (mLLM) to refine the input prompt. This refinement implements two automated evaluations: Davidsonian Scene Graph (DSG) queries ensure semantic adherence, and Common Mistake Questions (CMQ) for artifact detection. At the second stage, we use Bayesian optimization to efficiently co-optimize stochastic seeds and CFG scales. This search is guided by a suite of quality metrics, including the novel Video-Text Adherence (VTA) score derived from the DSG and CMQ evaluations. Our framework significantly outperforms unguided search methods: in human preference studies, videos generated via our agentic approach were strongly preferred over baseline outputs, achieving win rates up to 69%. This work provides a practical and extensible methodology for enhancing the predictability and control of state-of-the-art video generation models, moving the field beyond speculative curiosities toward reliable, production-ready tools.

[69] Class Activation Mapping in Explainable Computer Vision: A Method-Centered Review of CNN, Transformer, and Foundation-Model-Era Visual Explanations cs.CV | cs.AIPDF

AmirHossein Eshghi, Hamid Saadatfar, Seyyed Ali Hoseini, AmirMohsen Eshghi, Siavash Arjomand Bigdel

TL;DR: 本文是一篇关于类别激活映射（CAM）在可解释计算机视觉中的方法中心综述，系统梳理了从2016年至今的57篇核心论文，涵盖了CNN、Transformer和基础模型时代的视觉解释方法。

Details

Motivation: 旨在系统回顾CAM方法的发展历程，梳理其从CNN分类器到Transformer及基础模型时代的演变，并解决当前评估标准碎片化的问题。

Result: 论文未提供具体定量结果，但通过构建基于归因机制、架构依赖性和评估目标的方法分类法，系统分析了梯度CAM、混合CAM风格方法及模型感知方法的发展趋势。

Insight: 创新点在于提出了一个统一的方法分类框架，并指出领域正从单层低分辨率CNN解释转向多层、概率化、令牌感知及基础模型感知的比较性解释；同时强调了评估标准统一的重要性。

Abstract: Class activation mapping (CAM) is one of the most widely used visual explanation families in explainable artificial intelligence. Its purpose is intuitive: it converts internal model evidence into a heatmap that highlights the image regions, convolutional channels, tokens, or patches that support a target class or concept. Since the first CAM formulation in 2016, the field has moved far beyond global-average-pooled CNN classifiers. CAM-style methods now include gradient-based post-hoc explanations, gradient-free score and ablation methods, high-resolution upscaling, weakly supervised localization and segmentation, transformer token attribution, causal and debiasing methods, and foundation-model-era approaches that use CLIP, DINO, SAM, or feature-distribution comparisons. This review synthesizes a strict corpus of 57 method-centered papers published from 2016 onward. The paper develops a taxonomy that separates methods by attribution mechanism, architectural dependence, and evaluation objective. It then reviews gradient-based CAMs, recent and hybrid CAM-style methods, and model-based or architecture-aware methods. Across the corpus, the main trend is clear: the field is shifting from explaining one class score in one low-resolution CNN layer toward comparative, multi-layer, probabilistic, token-aware, and foundation-model-aware explanations. At the same time, evaluation remains fragmented. Faithfulness, localization, robustness, computational cost, and human trust are often measured with different protocols. The review therefore emphasizes not only what each method contributes, but also which gap it leaves open and which later methods attempt to close that gap.

[70] StateFlow: Building, Evolving, and Accessing 3D World States for Previsualization cs.CVPDF

Yuyang Yin, Zixiang Li, Longxuan Deng, Hongkai Li, Shifang Zhao

TL;DR: 该论文提出了StateFlow，一个以状态为中心的生成式预可视化框架。它通过构建、演化和访问一个持久化的3D世界状态，来解决现有生成方法在可控性和迭代编辑方面的不足，从而支持电影、游戏等领域的场景创作。

Details

Motivation: 现有基于简单提示词的生成方法（如图像或视频合成）对场景、动作、摄像机等多因素进行联合控制，可控性弱且难以支持迭代编辑。论文认为缺失的核心组件是一个显式且持久的工作状态。

Result: 实验表明，StateFlow能够为视频创作和类游戏原型制作生成高质量的3D世界。

Insight: 论文的核心创新在于将预可视化过程建模为对一个持久化、结构化的3D世界状态（包含场景元素和摄像机配置）的构建、演化和访问。具体技术亮点包括：先验引导、冲突感知的双视图初始化来构建状态；结构化状态转换来演化世界，避免全场景重新生成；以及渲染-反馈反射来优化摄像机轨迹，超越仅依赖视觉语言模型（VLM）语义的方法。

Abstract: Previsualization is an intermediate layer between ideas and production in film, games, architecture, and urban design. It lets creators iteratively refine scenes, actions, cameras, and spatial-temporal dynamics. Yet existing generative methods rely on simple prompts to jointly control all of these factors through one-shot image or video synthesis, offering weak controllability and limited support for iterative editing. Fundamentally, a world comprises multiple elements with geometry, appearance, and other attributes, together with cameras. Different frames are produced through local modifications or recombinations of this shared state, which is otherwise largely reused. Therefore, we argue that the missing component is an explicit and persistent working state. To address this, we present StateFlow, a state-centric framework for generative previsualization. Rather than generating videos in one shot, StateFlow uses an editable 3D world to organize scene structure, evolution, and cameras, while off-the-shelf video models enhance visual quality when higher fidelity is desired. This world is maintained as a persistent structured 3D state of scene elements and camera configurations, serving as the core working representation for previsualization. Built on this insight, StateFlow has three stages to construct, evolve, and access the world state. State construction lifts generated 2D content into a coherent 3D world through prior-guided, conflict-aware dual-view initialization, while State evolution translates user intent into structured state transitions while preserving world memory, avoiding full-scene regeneration for each edit. State access uses render-feedback reflection to refine camera plans into visually feasible trajectories, avoiding reliance on VLM semantics alone. Experiments show that StateFlow produces high-quality 3D worlds for video creation and game-like prototyping.

Yan Deng, Fei Xu

TL;DR: 本文提出了DreamFly，一个基于扩散模型的空中视觉语言导航框架，旨在解决历史上下文有限、规划视野短和隐式终止不可靠的挑战。该方法通过因果对齐的历史记忆、滚动时域扩散规划和显式终止估计来提升导航性能。

Details

Motivation: 现有视觉语言动作模型在适应空中导航任务时面临历史信息利用不足、规划视野短以及终止判断依赖隐式学习的问题，导致在部分可观测环境下的导航效果不佳。

Result: 在OpenFly基准测试中，DreamFly在测试可见和不可见环境上分别取得了32.04%/29.46%的成功率和28.22%/23.54%的SPL分数，在两项指标上均优于所有对比方法，同时达到了最低的导航误差。

Insight: 创新点包括：1）因果对齐的历史记忆机制，避免未来信息泄露；2）滚动时域扩散规划，以“规划K步，执行一步”的策略结合闭环视觉反馈；3）LiteStop模块从初始全掩码状态的动作对数直接估计停止概率，将显式终止与动作生成解耦。

Abstract: Aerial vision-language navigation (VLN) requires an embodied agent to integrate visual evidence over time, plan future actions, and determine when it has reached a navigation goal under partial observability. Although recent VLA models offer a promising perception-to-action paradigm, adapting them to aerial navigation remains challenging due to limited historical context, short planning horizons, and unreliable implicit termination. To address these challenges, we propose DreamFly, a diffusion-based aerial VLN framework built on Dream-VLA. DreamFly introduces a causally aligned historical memory that augments the current visual representation using only observations preceding the current decision step, enabling temporal reasoning without future information leakage. We further formulate navigation as receding-horizon diffusion planning, where the policy predicts a $K$-step action chunk but executes only the first action before replanning. This plan-$K$, execute-one strategy uses future actions as auxiliary planning targets while preserving closed-loop visual feedback. Finally, LiteStop estimates the stop probability directly from action logits at the initial all-mask state, decoupling explicit termination from action generation. Experiments on the OpenFly benchmark demonstrate consistent improvements in seen and unseen environments. DreamFly achieves 32.04%/29.46% SR and 28.22%/23.54% SPL on the test-seen/test-unseen splits, respectively, outperforming all compared methods on both metrics while attaining the lowest navigation error. These results demonstrate the effectiveness of jointly modeling historical context, future action structure, and explicit termination for aerial VLN.

cs.LG [Back]

[72] LEMUR: Latent Entropy-aware Multimodal Unlearning via Visual-anchored Reasoning Redirection cs.LG | cs.CLPDF

Xinhao Zhong, Yuxia Qiao, Junhao Li, Hao Fang, Yi Sun

TL;DR: 本文提出LEMUR框架，一种针对经过强化学习后训练的多模态大推理模型的免训练、推理时遗忘方法。该框架利用强化学习诱导的熵动态作为控制信号，通过熵调制的视觉锚定潜在注入来重定向推理轨迹，从而有效抑制敏感信息在推理链和最终答案中的泄露，同时保持模型的非敏感效用和输出流畅性。

Details

Motivation: 强化学习后训练赋予多模态大推理模型探索性思维链能力，但这引入了一种独特的隐私漏洞：即使敏感事实已从最终答案中成功遗忘，模型仍可能在推理轨迹中复现它。现有遗忘方法未针对此问题设计。

Result: 在多种多模态大推理模型上，LEMUR在抑制推理轨迹和答案泄露方面持续优于现有遗忘方法，同时更好地保留了非敏感效用和输出流畅性。

Insight: 核心创新在于发现并利用强化学习诱导的、在基础模型中基本不存在的独特令牌级熵特征作为隐私泄露信号，并基于此设计了通过视觉锚定潜在注入进行推理重定向的免训练遗忘框架。

Abstract: Reinforcement-learning (RL) post-training equips multimodal large reasoning models (MLRMs) with exploratory chains of thought (CoT), substantially improving visual reasoning. However, we find that this capability introduces a distinct privacy vulnerability: even when a sensitive fact is successfully unlearned from the final answer, the model may still reproduce it in its reasoning trace. This leakage is substantially more pronounced in natively RL-trained MLRMs than in their non -reasoning base models, revealing a privacy risk that existing unlearning methods are not designed to address. We show that RL-induced exploration leaves sensitive content with a distinctive token-level entropy signature that is largely absent from base models. Based on this observation, we propose LEMUR, a fully training-free, inference-time unlearning framework for natively RL-trained multimodal models. LEMUR uses entropy dynamics as a control signal to identify when sensitive reasoning begins and when sanitization should stop. During this interval, it redirects the reasoning trajectory through entropy-modulated visual-anchor latent injection, replacing committed tokens with sanitized, probability-weighted embeddings re-grounded in the input image. Across diverse MLRMs, LEMUR consistently outperforms existing unlearning met hods in suppressing both reasoning-trace and answer leakage, while better preserving non-sensitive utility and output fluency. These results demonstrate that RL-induced entropy dynamics provide a distinctive signal for privacy leakage and that exploiting this signal enables effective training-free unlearning for reasoning-capable multimodal models.

[73] Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL cs.LG | cs.AI | cs.CLPDF

Minglai Yang, Xinyu Guo, Utkarsh Tyagi, Mian Zhang, Razvan Dumitru

TL;DR: 本文提出Rubric Dropout方法，以缓解在基于评分标准的强化学习（RL）中出现的奖励破解问题。通过在训练时随机丢弃评分标准中的部分准则，防止模型过度优化特定准则，从而提升模型在分布外（OOD）基准测试上的泛化性能。

Details

Motivation: 当使用LLM评分标准作为奖励函数对语言模型进行强化学习训练时，模型会过度优化固定的评分标准，导致在训练评分员分数持续上升的同时，更强黄金评分员的分数却下降，即出现奖励破解现象。

Result: 在HealthBench-Hard和ResearchQA两个OOD基准测试上，应用30%或50%的Rubric Dropout后，黄金评分员分数分别提升了1-2点和6-7点，同时降低了奖励破解指标，且在领域内性能无损失。实验表明30-50%的丢弃率是一个有效的范围。

Insight: 核心创新点是借鉴神经元丢弃（dropout）思想，在每次训练步骤中随机丢弃部分评分准则，迫使策略学习更鲁棒的特征，而非针对固定准则进行过拟合。这种方法简单有效，且优于重新加权准则等替代方案。

Abstract: Reinforcement learning against rubrics, lists of criteria graded by an LLM judge, has become a standard way to post-train language models on tasks with no deterministic answer. The rubric, however, is a fixed proxy for quality, never a complete description of it, and a policy trained against it long enough will learn to exploit the difference. We measure this directly. Training Qwen3-8B with Group Relative Policy Optimization (GRPO) on medical and science rubrics and grading out-of-distribution (OOD) benchmarks with both the training judge and a stronger gold judge, we find that the two scores diverge during training. The training judge’s score keeps climbing while the gold judge’s score peaks and then falls, by 3 points on HealthBench-Hard and by 22 points on ResearchQA. A judge with a fixed bias would shift the gold curve by a constant, not send it down while the training score rises, so the divergence is reward hacking, not judge noise. We propose Rubric Dropout, a one-line fix borrowed from neuron dropout. At every step, we randomly drop a subset of the rubric’s criteria before computing the reward, so the policy never optimizes the same rubric twice. The dropped subset is shared across each rollout group, so GRPO’s group-relative advantages stay comparable, and evaluation always uses the full rubric. Comparing no dropout against dropout at 30% and 50% on both benchmark pairs, dropout raises the OOD gold score at every matched checkpoint (+1 to +2 points on HealthBench-Hard, +6 to +7 points on ResearchQA), lowers the two hacking measures we track, and costs nothing in domain. Sweeping the dropout fraction shows a broad 30-50% sweet spot, while the natural alternative, reweighting criteria by how useful they are to training, performs worse than no intervention at all in our setting.

[74] Epiplexity Guided Data Selection and Generation for Out-of-Distribution Generalization cs.LG | cs.CLPDF

Ellen Su, Andres Potapczynski, Shikai Qiu, Edward Hughes, Andrew Gordon Wilson

TL;DR: 本文提出了一种基于Epiplexity（一种衡量计算受限学习者能从数据中提取的结构信息的指标）的数据选择与合成数据生成方法。通过将Epiplexity作为在线训练信号，用于自适应地确定训练过程中不同数据域的采样权重，以及指导生成器产生最大化Epiplexity的合成数据分布。实验表明，更高的Epiplexity预示着在零样本和基于微调的下游任务上更好的性能，支持了富含结构信息的数据能产生跨域可迁移表征的假设。

Details

Motivation: 现代系统越来越需要在训练时未指定的任务间进行迁移。本文旨在探索何种数据能促进在新颖、未预料场景下的泛化，其核心假设是富含更多结构信息的数据可能包含可被更广泛下游任务复用的共享计算回路和子程序。

Result: 在零样本和基于微调的任务上，更高的Epiplexity预测了下游性能的提升，支持了其作为有效训练信号的作用。

Insight: 创新点在于将Epiplexity这一理论指标操作化为实用的在线训练信号，分别用于指导数据选择（通过拟合训练损失曲线的缩放定律来预测Epiplexity增益）和合成数据生成（使用REINFORCE策略梯度优化生成器奖励）。这为主动构建或选择能提升模型泛化能力的数据提供了新思路。

Abstract: Modern systems are increasingly expected to transfer across tasks not specified during training. What data facilitates generalization in these new, unanticipated settings? One hypothesis is that data with more structural information could contain shared circuits and subprograms that could be recycled in a wider array of downstream settings. Epiplexity, a recently proposed measure of the structural information a compute-bounded learner can extract from data, provides a mechanism to reason about this relationship. In this paper, we show how to operationalize epiplexity as an online training signal for data selection and synthetic data generation. For selection, we fit scaling laws to the training loss curves of natural data domains to predict the expected epiplexity gain as a function of training tokens, and use this signal to adaptively determine the sampling weights over domains during training. For synthetic data generation, we define a generator’s reward as the change in learner epiplexity over a buffer of previously generated data and use REINFORCE policy gradients to guide the generator toward an epiplexity-maximizing distribution. In both cases, higher epiplexity predicts improved downstream performance on zero-shot and fine-tuning based tasks, supporting the hypothesis that data rich in structural information yield representations that transfer across domains.

[75] Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling cs.LG | cs.CLPDF

Xinmu Ge, Zizhuo Zhang, Yu Huang, Jianing Zhu, Lin Yuan

TL;DR: 本文通过测试时缩放视角重新审视了策略蒸馏（OPD）对大型语言模型推理能力的提升机制。研究发现，OPD主要改善了采样效率，而非持续扩展模型的能力边界，其表现出的优势更多源于采样效率提升而非从教师模型获得新推理能力。

Details

Motivation: 旨在探究策略蒸馏（OPD）作为后训练技术是否真能让学生模型从更强教师模型蒸馏知识、扩展能力边界，还是仅提升采样效率。

Result: 在多个OPD变体实验中，OPD训练模型在avg@K指标上保持优势，但pass@K优势随K增大逐渐转向预OPD基础模型；使用pass@1024的分析显示，OPD导致更多原本可解问题变得不可解。

Insight: OPD的增益主要来自采样效率提升而非能力边界扩展，揭示了其‘虚幻蒸馏’特性；研究强调了评估LLM后训练技术时需区分采样效率与真实能力扩展的重要性。

Abstract: On-policy distillation (OPD) has emerged as a promising post-training technique for enhancing LLM reasoning. It is commonly believed to enable the student model to distill knowledge from a stronger teacher model, thereby expanding capabilities beyond the pre-OPD base model. In this study, we examine this view through the lens of test-time scaling by varying the sampling budget K and evaluating performance with pass@K and avg@K. Specifically, across several OPD variants, we observe that OPD-trained models maintain superior avg@K performance across sampling budgets, while the advantage in pass@K gradually shifts to the pre-OPD base models as K increases. These results suggest that OPD primarily improves sampling efficiency rather than consistently expanding the student’s reasoning capability boundary. The pass@K dynamics throughout OPD training further reveal a progressive shift toward stronger small-K performance at the expense of the large-K capability boundary. Furthermore, a problem-level solvability analysis using pass@1024 as the criterion reveals an asymmetry: OPD causes more previously solvable problems to become unsolvable than previously unsolvable problems to become solvable. Together, these findings suggest that, from the perspective of capability expansion, OPD behaves more like an “illusory distillation”: its apparent gains arise primarily from improved sampling efficiency rather than from acquiring genuinely new reasoning capabilities from the teacher.

[76] AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses cs.LG | cs.AI | cs.CLPDF

Cheng Qian, Wenting Zhao, Liangwei Yang, Heng Wang, Jielin Qiu

TL;DR: 本文提出了一种在测试时进行强到弱能力转移的新方法，称为’强到弱脚手架’。该方法利用更强的构建模型为更弱的目标模型设计推理时’马具’（harnesses），以帮助其更可靠地解决任务，而无需更新目标模型的参数。在四个心智理论基准测试中，该方法将目标模型的平均性能从0.49提升至0.91，效果显著。

Details

Motivation: 传统蒸馏方法通常在训练时通过更新参数来将大模型能力转移到小模型。本文旨在探索是否可以在测试时实现这种能力转移，即让更强的模型通过设计推理时的辅助结构来即时提升较弱模型的任务解决能力。

Result: 在四个代表性的心智理论基准测试上，该方法将目标模型的平均性能从0.49提升至0.91，几乎翻倍。实验表明，性能提升主要源于将不稳定的模型推理卸载到确定性代码、基准特定的路由和严格的答案格式执行中。

Insight: 核心创新在于提出了’测试时能力转移’的概念，通过构建模型设计推理时’马具’来即时增强弱模型性能，无需重新训练。研究发现，提升主要源于结构化辅助而非鼓励模型进行更广泛的推理或采样，这为传统的训练时蒸馏提供了重要的补充途径。

Abstract: Recent work on distillation transfers the capabilities of large models to smaller ones often by updating the latter’s parameters, through teacher forcing, on-policy distillation, and related training-time methods. In this paper, we ask whether such transfer can instead occur at test time. We study strong-to-weak scaffolding: whether a stronger builder model can construct inference-time harnesses that help a weaker target model solve tasks more reliably without any parameter updates. Using four representative Theory-of-Mind benchmarks, each builder model uses 5% of the data as a validation set to iteratively refine its harness over multiple rounds, after which the finalized harness is evaluated on the full test set. Empirically, this form of test-time capability transfer is highly effective, nearly doubling average target-model performance from 0.49 to 0.91. Our analysis shows that the gains come primarily from offloading unstable model reasoning into deterministic code, benchmark-specific routing, and strict answer-format enforcement, rather than from encouraging the target model to reason more extensively or sample more broadly. We further find that builder-model reasoning effort improves harness quality monotonically, platform effects are modest relative to the builder model’s own capability, and weaker target models receive the largest gains. These results suggest that inference-time harness design is an important complement to conventional training-time distillation, enabling strong models to transfer cognitive structure to weaker models without retraining.

[77] Click2Poly: A VLM for vector mapping buildings and walls cs.LG | cs.CVPDF

Nicolas Girard, Jawher Ben Abdallah, Arno Gobbin, Liuyun Duan, Sacha Lepretre

TL;DR: Click2Poly是一个基于Florence-2视觉语言模型（VLM）构建的人机交互AI助手，旨在加速建筑物和围墙矢量地图的手动编辑过程。它通过响应用户点击来直接编辑矢量图层，并以QGIS插件的形式实现，在真实生产环境中提升了矢量地图制作的效率。

Details

Motivation: 建筑物和围墙的精确矢量制图对地理空间应用至关重要，但目前仍是一个劳动密集型过程。现有深度学习方法虽能自动提取，但为满足制图标准，仍需人工进行质量控制和修复复杂案例，该论文旨在加速这一手动步骤。

Result: 论文将Click2Poly实现为QGIS插件，并在真实生产环境中验证了其能加速建筑物和围墙矢量图层的手动编辑工作，但摘要中未提及具体的定量实验结果或基准测试对比。

Insight: 主要创新点在于将视觉语言模型（VLM）扩展应用于地理空间矢量编辑这一特定人机交互任务，通过“点击即编辑”的直观交互范式，将AI作为辅助工具无缝集成到专业GIS工作流（QGIS）中，提升了生产环境的实用性。

Abstract: Accurate vector mapping of buildings and walls is critical for geospatial applications but remains a labor-intensive process. While recent deep learning methods have improved automatic extraction, in order to meet cartographic standards they always require a human to perform quality control and fix complex cases in the extraction. We present Click2Poly, a human-in-the-loop AI assistant designed to speed up this manual step. Extending the Florence-2 Vision Language Model (VLM), Click2Poly responds to user clicks by editing the building or wall vector layer directly. Implemented as a QGIS plugin, Click2Poly speeds up the manual editing of building and wall vector layers in a real-world production environment.

cs.RO [Back]

[78] HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editing cs.RO | cs.CVPDF

Zhenjie Yang, Xingyu Jiao, Guopeng Zhong, Shuzhe Yang, Shi Che

TL;DR: 本文提出了HandEdit，一个统一的大规模具身感知图像编辑数据集和基准，专门用于将第一人称视角中的人类手和手臂转换为各种灵巧机器人形态。该数据集包含超过2亿个编辑实例，涵盖26种不同的URDF模型，并建立了手部和手-臂两个评估轨道。

Details

Motivation: 解决灵巧机器人操作数据收集成本高昂的问题，利用丰富的人类第一人称手部视频作为可扩展替代方案，但人类与机器人数据在外观、关节结构和相机视角上存在显著差异，现有通用图像编辑模型缺乏必要的具身先验知识来弥合这一差距。

Result: 对11个代表性图像编辑基线模型进行了广泛评估，使用了包括通用相似性指标、基于VLM的评判和具身感知指标在内的多维度量套件。HandEdit作为一个关键资源，旨在推进具身感知编辑模型的发展。

Insight: 创新点在于构建了一个专门针对具身AI的、大规模且统一的机器人形态转换图像编辑数据集与基准，通过URDF条件化评估，为从丰富人类视频数据中进行可扩展的灵巧机器人学习铺平了道路。

Abstract: Robotic manipulation with dexterous hands is a cornerstone of Embodied AI, yet its progress is stifled by the high cost of collecting embodiment-aware teleoperation data. While abundant egocentric videos of human hands offer a scalable alternative, the profound discrepancies in appearance, articulation, and camera viewpoints between human and robotic data raise significant challenges for co-training. Though existing general image-editing models demonstrate strong capabilities, they lack necessary embodiment-specific priors to fully bridge this gap. In this work, we present HandEdit, a unified large-scale embodiment-aware image-editing dataset and benchmark specifically designed to transform human hands and arms into various dexterous robotic embodiments within egocentric frames. HandEdit comprises over 200M editing instances derived from five diverse source datasets, covering 26 distinct URDFs, including 13 hand-only and 13 hand-arm configurations. Alongside the dataset, we establish a unified benchmark protocol with two tracks: Hand-only and Hand-Arm, supporting URDF-conditioned evaluation. We conduct extensive evaluations of 11 representative image-editing baselines using a multi-dimensional metric suite, including generic similarity metrics, VLM-based judgment, and embodiment-aware metrics. HandEdit serves as a critical resource at the intersection of image editing and robotics: it advances embodiment-aware editing models while enabling scalable dexterous robotic learning from abundant human video data, paving the way for more generalizable Embodied AI.

cs.GT [Back]

[79] Do LLMs Take Care of Their Own? Similarity Signals Can Induce Cooperation cs.GT | cs.AI | cs.CL | cs.MAPDF

Akash Kundu, Emanuel Tewolde, Ratip Emin Berker, Samuel F. Brown, Vincent Conitzer

TL;DR: 本文提出了首个评估LLM在接收到分级相似性信号时决策行为的框架，发现不同LLM模型对相似性信号的反应差异显著，某些现代模型在合作问题、收益结构和提示框架下表现一致。实验表明相似性信号的计算数据集对诱导合作影响甚微，且LLM在评估其他模型的思维链推理时会系统性地自我识别为高度相似。最后，作者构建了一个LLM行为博弈论模型来解释其推理逻辑，并证明在足够高的相似性分数下该模型能支持均衡中的合作结果。

Details

Motivation: 随着基于LLM的、具有用户指令目标的智能体广泛部署，它们在战略互动中日益相遇，面临寻找互利结果的挑战。本文旨在探索在提供分级相似性信号的设置下，LLM如何解决如囚徒困境等合作问题。

Result: 实验发现不同LLM模型对相似性信号的反应差异巨大，某些现代模型在多种合作问题、收益结构和提示框架下行为一致。相似性信号的计算数据集对诱导合作影响很小，LLM在评估其他模型思维链时会自我识别为高度相似。

Insight: 创新点在于引入了首个评估LLM在分级相似性信号下决策的框架，并揭示了LLM在战略互动中基于相似性信号诱导合作的行为模式。从客观角度看，该研究为理解LLM在博弈环境中的决策机制提供了新视角，其行为博弈论模型为解释LLM合作行为提供了理论支持。

Abstract: As LLM-based agents with user-instructed goals are becoming widely deployed, they increasingly encounter each other in strategic interactions, and face challenges of finding mutually beneficial outcomes. Prior literature has argued that cooperation problems such as the Prisoner’s Dilemma are resolvable in settings where agents know they follow very similar decision making patterns, as for example in monocultural AI ecosystems. Following that line of work, this paper introduces the first framework for evaluating LLM decision making when agents are provided with graded similarity signals. Among our findings, we establish that different LLM models vary drastically in how they navigate similarity signals, with some modern models showing consistent behavior across cooperation problems, payoff structures, and prompt framing. Perhaps surprisingly, our experiments also show that the dataset based on which the similarity signal is computed has small to no impact on induced cooperation, and that LLM models systematically self-identify as highly similar when asked to evaluate another model’s chain-of-thought reasoning by themselves. Finally, we develop an LLM-behavioral-game-theoretic model that captures some of their reasoning rationale, and show that it can support cooperative outcomes in equilibrium under sufficiently high similarity scores.

cs.AI [Back]

[80] Reproducing and Stress-Testing Two Approaches to LLM Reasoning Reliability: Test-Time Probability Aggregation and Logic-Representation Editing cs.AI | cs.CL | cs.LGPDF

Minhan Cho, Jimin Kweon

TL;DR: 本文独立复现并压力测试了两种提升大语言模型推理可靠性的方法：RPC（推理时聚合token概率和自洽性）和LCF（训练投影器分离隐藏状态为“内容”和“逻辑”并编辑逻辑部分）。研究在多个新任务领域和模型上验证了这两种方法的有效性，发现RPC的改进优势有限且不稳定，而LCF的逻辑有效性方向存在但较弱，且可能对某些模型产生负面影响。

Details

Motivation: 动机在于验证两种声称能提升LLM推理可靠性的方法（RPC和LCF）的有效性，因为原始评估均由方法作者自行完成，缺乏独立复现和跨模型、跨领域的压力测试，且LCF未公开代码，其可靠性声明需要独立验证。

Result: 在四个新任务领域（文本到SQL、法律提取、谬误识别、先例分级）和多个模型（Qwen3-8B及四个7-8B模型）上的测试表明：RPC在原始推理路径上可精确复现，但在新领域上相比自洽性的优势不显著（p>=0.28），且在BIRD任务中扩大样本后优势消失甚至反转；LCF的逻辑有效性方向可检测但较弱（最佳子层可分离性0.82 vs 语义属性控制0.95），对Qwen3模型的正面效果不显著（p=0.56），而对另外两个模型显著降低了性能。

Insight: 创新点在于对两种未经验证的LLM可靠性方法进行了首次独立、系统的复现与跨领域压力测试，揭示了方法在实际泛化中的局限性：RPC的改进可能高度依赖特定设置且不稳定，LCF的逻辑表示编辑虽有一定理论基础但实际效果微弱甚至有害，这强调了独立评估和代码开源对AI研究可信度的重要性。

Abstract: We independently reproduce two recent methods for making large language model (LLM) reasoning more reliable, and stress-test them across domains and models (RPC across four new task domains with Qwen3-8B, LCF across four 7-8B models). The first, RPC, aggregates token probabilities and self-consistency at inference; the second, LCF, trains projectors that split hidden states into “content” and “logic” and edits the logic part toward a valid region. Validating such reliability claims matters because the original evaluations are run by each method’s own authors and were never independently reproduced or stress-tested across models and domains, and LCF shipped no public code. We re-run RPC’s published-path aggregation and re-implement LCF’s projector, contrastive, and intervention pipeline, then extend both to text-to-SQL, legal extraction, fallacy identification, and precedent grading, and probe LCF’s representation directly. RPC reproduces the original grid exactly on the authors’ released reasoning paths; on four new domains its edge over self-consistency is never significant (ties or small mixed differences, paired p >= 0.28), and on BIRD, the one domain where we vary the budget, the edge grows with K as predicted but its largest gap (+2.5 accuracy at K=32, p=0.16) reverses to -0.25 when we enlarge the sample to n=200. LCF’s logic-validity direction is real but weak (0.82 separability at the single best sub-layer versus 0.95 for a semantic-attribute control); its one positive effect (Qwen3 $Δ$Prob) is not significant (p=0.56), while it significantly reduces $Δ$Prob on two of the other three models.

[81] BEST-KAG: Enhancing Question Answering of Building Engineering Standards with Multimodal Knowledge Graph Modeling and Large Language Model cs.AI | cs.CLPDF

Jia-Rui Lin, Junxi Guo, Keyin Chen, Peng Pan

TL;DR: 本研究提出了BEST-KAG框架，旨在通过多模态知识图谱建模和大语言模型增强建筑工程标准问答。该框架构建了一个包含251个标准、17万节点和31万边的多模态知识图谱，并采用图检索增强生成架构，实现了基于条款、可追溯的问答。实验表明，BEST-KAG在专家评估和BLEU、ROUGE等指标上显著优于多个主流大语言模型。

Details

Motivation: 解决现有标准应用流程依赖基于关键词的文档检索和人工跨条款解释，无法可靠支持多条款推理、多模态知识利用或可追溯的条款级证据关联的问题。

Result: 在建筑工程标准问答任务上，BEST-KAG在专家评估、BLEU和ROUGE指标上持续优于多个主流LLM基线，最佳改进高达74.01%。

Insight: 创新点包括：1) 用于统一表示文档层次结构和异构标准知识的多模态知识图谱；2) 规则与LLM混合的可扩展多模态知识抽取流水线；3) 基于图检索的知识增强生成架构，实现基于条款且可追溯的问答。从客观角度看，其将结构化知识图谱与非结构化LLM能力结合，为专业领域（如建筑工程）的可靠知识问答提供了可扩展的解决方案。

Abstract: Construction standards are critical for building safety and sustainability. Existing standard application workflows rely on keyword-based document retrieval and manual cross-clause interpretation, which cannot reliably support multi-clause reasoning, multimodal knowledge utilization, or traceable clause-level evidence linkage. To address these limitations, this study develops a multimodal knowledge-driven framework that supports question answering on standard knowledge named BEST-KAG (Knowledge-Augmented Generation for Building Engineering STandards). The framework introduces 1) a multimodal knowledge graph (MKG) for unified representation of document hierarchy and heterogeneous standard knowledge with various connections, 2) a rule-LLM hybrid knowledge construction pipeline for scalable multimodal knowledge extraction, creating a large MAG with 251 building engineering standards, 171,652 nodes and 310,914 edges, and 3) a graph-retrieval-based knowledge-augmented generation architecture for clause-grounded and traceable question answering. Experiments demonstrate that BEST-KAG consistently outperforms multiple mainstream LLMs in terms of Expert evaluation, and metrics including BLEU, and ROUGE, with the best improvement up to 74.01% compared to the baselines.

Del Coburn, Scott Sanner, Dan Silver

TL;DR: 本文提出了Social Chain of Thought (SCoT)，一种用于医学鉴别诊断的多轮多智能体架构。该架构将多智能体交互构建为一个审慎的协作推理框架，旨在解决复杂病例的诊断问题。

Details

Motivation: 动机在于解决LLMs在医学诊断推理中的透明性问题，尤其是在需要整合多种专家推理的复杂鉴别诊断场景下，现有方法不清楚多智能体系统何时、为何以及何处优于单一模型推理。

Result: 评估表明，SCoT在召回率上优于单一智能体基线、单智能体流程消融实验和best-of-n扩展方法，其优势在最具挑战性的诊断病例中最为显著，能帮助恢复真实诊断并收敛到更高召回率的鉴别诊断列表。

Insight: 创新点在于将医学鉴别诊断的方法论具体化为一个结构化的多轮多智能体交互管道，为复杂、透明的协作式LLM推理提供了一个可解释的框架，其性能增益并非单一推理模型所能复现。

Abstract: Medical diagnostic reasoning is a high-impact use case for LLMs that carries significant implications for the health and wellbeing of users. When OpenAI (2026) reports that more than 5% of ChatGPT messages globally are healthcare-related, the transparency of these systems becomes a serious design concern. This is especially true for complex cases, where differential diagnosis often requires integrating multiple forms of specialist reasoning. Existing work has proposed multi-agent approaches to medical diagnosis, but it remains unclear when such systems are needed, why they help, and where they outperform monolithic inference. We introduce Social Chain of Thought (SCoT),a multi-round pipeline for medical differential diagnosis that structures multi-agent interaction as a deliberative framework for collabora. tive LLM reasoning. Evaluating SCoT against single-agent baselines, one-agent pipeline ablations, and best-of-n scaling, we show that its recall advantage is not reproduced by monolithic inference alone. SCoT is most successful in the hardest diagnostic cases, where multiple rounds of specialist conversation help recover ground-truth diagnoses and converge on a higher-recall differential.

[83] Benchmarking LLM Judges for Mobile Agent Evaluation cs.AI | cs.CL | cs.CVPDF

Ziqiang Wan, Li Gu, Zhixiang Chi, Zhi Liu, Seyed Mehdi Ayyoubzadeh

TL;DR: 该论文提出了MobileJudgeBench基准，用于系统评估基于LLM的评判方法在移动智能体轨迹上的表现。研究发现，简单的截图采样基线方法能与专用方法竞争甚至更优，基准质量指标能可靠预测评判器的实际效用，并且不同LLM后端在失败模式上表现出相反的保守与宽松特性。

Details

Motivation: 移动智能体基准日益依赖基于LLM的评判器来评估任务完成情况，但这些评判器在移动智能体轨迹上的可靠性尚未得到充分检验。

Result: 在包含931条人工标注轨迹、覆盖6个移动智能体基准、4个智能体模型和68个应用的测试集上，评估了6种评判方法。结果表明，简单基线方法常优于专用方法，基准质量指标与智能体排名保真度及强化学习下游性能相关。

Insight: 创新点在于构建了首个系统评估移动智能体轨迹LLM评判器的基准，并揭示了评判器质量主要受LLM主干驱动而非复杂流程，以及不同LLM后端存在精确率-召回率特性导致的相反失败模式。

Abstract: Mobile agent benchmarks increasingly rely on LLM-based judges to evaluate task completion, yet the reliability of these judges on mobile agent trajectories remains largely unexamined. We introduce MobileJudgeBench, a benchmark for systematically evaluating LLM-as-judge methods on mobile agent trajectories. Our benchmark comprises 931 human-annotated trajectories spanning 6 mobile agent benchmarks, 4 agent models, and 68 apps. We evaluate 6 judge methods (five adapted from SPA-Bench, A3 with two modes, AndroidArena, and AgentRewardBench, plus a simple baseline we design) across multiple LLM backends. Our experiments reveal three key findings. First, a simple baseline judge with sampled screenshots is competitive with, and often exceeds, purpose-built methods, indicating that more elaborate judge pipelines do not consistently improve judge quality; among competitive methods, the LLM backbone is the primary driver. Second, benchmark quality metrics reliably predict real-world judge utility: they correlate with both agent ranking fidelity for evaluation and downstream performance when judges serve as reward signals for on-policy reinforcement learning. Third, failure analysis across two LLM backends uncovers qualitatively opposite failure profiles, one conservative and the other permissive, linked to the backbone’s precision-recall characteristics.

[84] When Self-Consistency Backfires: Majority Vote Hurts the Majority of Hard Science Problems for Small LLMs cs.AI | cs.CL | cs.LGPDF

Utkarsh Bahuguna

TL;DR: 该论文研究了自洽性（Self-Consistency, SC）通过多数投票在小语言模型（LLMs）处理困难科学问题时的负面影响。研究发现，在GPQA Diamond基准测试中，对Qwen2.5-7B和Llama-3-8B等指令微调模型使用多数投票，反而会降低大部分问题的准确率。论文通过预注册实验确认了这一现象，并探讨了其机制在于模型置信度与答案正确性不匹配，同时指出目前没有无需验证器的有效方法能接近理论上的最优路由性能。

Details

Motivation: 自洽性通过多数投票是广泛使用的推理时计算开销方法，但作者观察到在小语言模型处理高难度科学问题时，这种方法可能适得其反，因此旨在系统性地验证和分析这一反直觉现象。

Result: 在GPQA Diamond基准（198个研究生级别科学问题）上，多数投票降低了Qwen2.5-7B在56.6%的问题和Llama-3-8B在65.7%的问题上的准确率。理论上的最优路由（需要真实标签）能将准确率比N=1时提升14（Qwen）和17（Llama）个百分点，但无需验证器的门控方法（如多数一致或词元熵）在N=64时准确率提升微乎其微（≤0.002）。

Insight: 论文的核心创新在于揭示了在小指令微调模型上，自洽性多数投票在处理困难科学问题时可能有害，其机制是模型的高置信度（高一致性）并不对应高正确性。这挑战了自洽性方法普遍有益的假设，并指出了未来研究（特别是对原生推理模型）的重要开放性问题。

Abstract: Self-consistency (SC) via majority vote is a widely used way to spend inference-time compute: sample N chains of thought, return the plurality answer. On the full GPQA Diamond benchmark (198 graduate-level science questions), majority voting reduces per-problem accuracy on a majority of problems for two instruction-tuned models from different families: 56.6% of problems for Qwen2.5-7B and 65.7% for Llama-3-8B, with Qwen the primary demonstration and Llama corroborating the direction from a near-chance baseline. The effect was pre-registered on a 151-problem confirmatory split after being observed on 47 exploratory problems, and all four confirmatory hypotheses passed. A grid oracle that routes each problem to the best N across {1, 2, 4, 8, 16, 32, 64} marks a theoretical upper bound 14 accuracy points above N = 1 for Qwen and 17 for Llama, an oracle bound requiring ground truth rather than a deployable method. No verifier-free gate reaches it: neither a plurality-agreement gate nor a token-entropy gate moves accuracy more than 0.002 from fixed-budget voting at N = 64. The mechanism is direct: confidence does not track correctness on these problems. In the highest-agreement bin the plurality answer is correct about half the time for Qwen, and for Llama that bin is less accurate than its lowest-agreement bin. We pre-register and confirm these findings on small instruction-tuned models; we do not test reasoning-native models, which we flag as the central open question.

[85] The Sleeping Agent: What Gist-Based Context Compression Loses and Why cs.AI | cs.CLPDF

Nicholas E. Kyrkewood

TL;DR: 本文研究了基于主旨的上下文压缩在长序列语言模型代理中的效果，发现其对不同类型记忆检索的影响存在差异。通过引入受睡眠记忆巩固启发的显著性加权整合框架，论文诊断了主旨压缩何时有益、何时有害，并揭示了压缩失败源于时间信息丢失的特定机制。

Details

Motivation: 旨在理解基于主旨的上下文压缩（即将会话历史总结为紧凑表示）在长序列语言模型代理中如何影响不同类型的记忆检索，特别是其利弊条件尚不明确。

Result: 在LoCoMo对话数据集（共1,935个纯文本问题）上的评估显示，主旨压缩在多跳推理和单跳事实问题上显著优于截断方法，但在时间问题上表现大幅下降，准确率远低于完整上下文基准；通过修改提示词保留时间表达式后，时间问题准确率提升了+0.314。

Insight: 创新点在于使用生物启发的显著性加权整合框架作为诊断工具，揭示了压缩失败的核心机制是时间信息丢失；通过精准的提示词修改（仅增加一句话）即可针对性修复此缺陷，体现了方法的高精度可调试性。

Abstract: Gist-based context compression—summarising older conversation history into compact representations—is a common approach in long-horizon language model agents, yet its effect on different types of memory retrieval is poorly understood. We use Salience-Weighted Consolidation (SWC), a biologically-inspired compression framework motivated by sleep-based memory consolidation, as a diagnostic probe to study when gist compression helps and when it hurts. SWC scores conversation history by salience, partitions it into priority tiers, and applies structured gist abstraction to mid-priority content. Evaluating four conditions on all ten LoCoMo conversations—1,935 matched text-only questions in total, 1,501 used in the primary aggregate after excluding Category 5 (adversarial) questions—at temperature 0, we find a consistent task-type interaction: gist compression substantially outperforms truncation on multi-hop reasoning and single-hop factual questions, but temporal questions remain substantially harder under compression, with compressed conditions scoring well below the full-context reference on the conversations where both are evaluated. We trace this failure to a specific mechanism: the gist abstraction prompt preserves relational and event structure while discarding dates and times. A preservation analysis across all ten conversations confirms the mechanism: an approximately 20-fold increase in temporal expression preservation (3.05% to 62.39%) with a one-sentence prompt modification, while named entity and event preservation rates barely change (x1.02 and x1.11), demonstrating that the fix is a precision instrument. The prompt modification recovers +0.314 [0.254, 0.375] judge accuracy on category-2 (temporal) questions in the matched set. Code and results: https://github.com/kyrkewood/sleeping-agent.

[86] Claim-Level Reliability Assessment for Efficient Test-Time Reasoning cs.AI | cs.CLPDF

Sen Xu, Wei Wang, Shixi Liu, Jixin Min, Yingwei Dai

TL;DR: 本文提出了一种名为CLR（Claim-Level Reliability Assessment）的训练无关框架，用于在测试时高效评估大语言模型推理的可靠性。该方法通过将推理轨迹浓缩为关键声明（claims），并专注于语义证伪（falsification），而非生成完整解决方案，从而在有限计算预算下重新分配计算资源，从额外采样转向针对性验证，以提高推理准确性。

Details

Motivation: 动机在于解决测试时扩展（test-time scaling）的效率问题。传统对整个推理轨迹（whole-trace）的评估常因常规令牌的信号稀释而掩盖关键错误，且固定模型能力下生成完全正确的解决方案本身就很困难。因此，需要一种更高效的方法来评估和提升推理的可靠性。

Result: 在匹配的计算预算下，在四个大语言模型和四个推理基准测试上的实验表明，CLR普遍优于pass@1和自洽性（self-consistency）方法。例如，在GPT-OSS-20B/CMIMC25上，CLR比pass@1高出27.15个百分点，并将自洽性准确率从77.50%提升至82.19%，同时减少了37.0%的令牌消耗。

Insight: 创新点在于提出了声明级证伪（claim-level falsification）原则，将评估焦点从整个解决方案的构建转移到对关键声明的针对性证伪上。这利用了解决方案构建与声明反驳之间的根本不对称性：反驳一个错误声明只需找到一个决定性缺陷，这比构建完美推理路径更容易，从而能更高效地压缩高置信度错误轨迹的生存空间，并通过非线性可靠性评分有效抑制错误共识。

Abstract: We propose claim-level falsification as a principle for test-time scaling and instantiate it through Claim-Level Reliability Assessment (CLR), a training-free framework that reallocates test-time compute from additional solution sampling to targeted verification. Since whole-trace evaluation often obscures decisive errors due to signal dilution from routine tokens, CLR condenses each reasoning trace into a compact set of decision-critical claims, thereby isolating its logical anchors. Furthermore, recognizing the inherent difficulty of generating entirely correct solutions under fixed model capabilities, CLR shifts the focus to semantic falsification. This approach exploits a fundamental asymmetry between solution construction and claim refutation. Constructing a valid solution requires a flawless reasoning path, whereas refuting an incorrect claim requires identifying only a single decisive flaw. This targeted search for negative evidence systematically compresses the survival space of high-confidence incorrect traces, effectively suppressing erroneous consensus via nonlinear reliability scoring. Across four LLMs and four reasoning benchmarks under matched budgets, CLR generally improves upon pass@1 and self-consistency. On GPT-OSS-20B/CMIMC25, for instance, CLR exceeds pass@1 by 27.15 percentage-points and raises self-consistency accuracy from 77.50% to 82.19% with 37.0% fewer tokens.

[87] Who Thinks Best Depends on How Long You Let Them: Budget-Dependent Rankings in LLM Evaluation cs.AI | cs.CLPDF

Rodrigo Guedes de Souza, Alison R. Panisson

TL;DR: 这篇论文挑战了LLM评估中模型排名在不同推理条件下保持稳定的假设，通过改变token生成预算（64到4096个token）在三个推理基准上评估四个模型。研究发现模型排名会随预算变化而反转，存在非单调行为，模型之间存在互补性，并且预算感知的路由器能部分利用这种互补性。

Details

Motivation: 动机是质疑标准LLM评估中模型排名在不同推理条件下保持稳定的假设，研究token生成预算变化对模型性能排名的影响。

Result: 在三个推理基准（共56476次推理）上，模型排名随预算变化发生显著反转（p<0.01）；Oracle分析显示模型互补性最高可达+27.8个百分点，尤其在受限预算下；预算感知路由器能捕获14.1%的Oracle差距。

Insight: 创新点在于揭示了LLM评估中模型排名对token预算的依赖性，提出了预算条件化评估协议的必要性，并展示了利用预算特征进行模型路由的潜力与局限性。

Abstract: Standard evaluation of large language models assumes stable model rankings across inference conditions. We challenge this assumption by varying the token generation budget, i.e., the maximum tokens a model may produce, across seven levels (64–4,096), evaluating four models on three reasoning benchmarks (56,476 inferences). We report four findings: (i) 3–19% of items exhibit non-monotone behavior (accuracy decreasing with more budget), even after controlling for truncation, and this phenomenon is model-specific (cross-model overlap: 6–14%). (ii) Model rankings reverse across budgets on all benchmarks ($p {<} 0.01$, McNemar). (iii) Oracle analysis reveals model complementarity up to $+27.8$pp, most pronounced at constrained budgets. (iv) A budget-aware router captures 14.1% of the oracle gap cross-domain; budget features help within-domain ($+1.6$ to $+5.7$pp) but are domain-specific and hurt transfer ($-1.2$pp). These results argue for budget-conditioned evaluation protocols.

[88] Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning cs.AI | cs.CVPDF

Shibo Gao, Peipei Yang, Xu-Yao Zhang, Linlin Huang

TL;DR: 本文提出了一种从免训练到智能体推理的视频异常检测（VAD）新范式，以解决现有方法中‘何时发生’与‘发生了什么’的分离问题。首先提出了Glance then Scrutinize（GtS）免训练框架，利用静态和动态文本指导进行从粗到细的异常定位与理解；进一步提出了工具增强的智能体VAD方法，通过微调和强化学习使多模态大语言模型学会调用视频裁剪工具、密集检查帧并自我纠正错误假设。

Details

Motivation: 现有VAD方法存在‘何时-何事’分离：传统基于DNN的方法能定位异常发生时间但缺乏语义理解，而基于LLM的方法能解释事件内容但忽略了精确的时间定位，这源于缺乏统一的推理范式。

Result: 在扩展的VAGU-T基准（包含7,567个真实世界视频，涵盖21个异常类别）上，使用联合评估语义可解释性和时间精度的新指标JeAUG进行实验。结果表明，GtS显著超越了免训练的基线方法，而智能体模型在实现更高准确率的同时，推理速度也更快。

Insight: 创新点在于借鉴人类观察监控视频的‘全局审视-局部细查-迭代思考’范式，构建了从免训练到智能体学习的统一推理框架。特别是通过工具调用、密集帧采样和基于联合答案-定位奖励的强化学习，使模型具备了自主修正错误假设的智能体能力，并引入了更全面的评估基准和指标。

Abstract: Video Anomaly Detection (VAD) aims to identify anomalous events and localize their temporal intervals. Existing approaches exhibit a “when-what” dissociation: traditional DNN-based methods localize when anomalies occur but lack semantic understanding, whereas LLM-based methods explain what happens but neglect precise temporal grounding. We attribute this to the absence of a unified reasoning paradigm. Inspired by how humans inspect surveillance videos - glancing globally to form temporal hypotheses, scrutinizing suspicious segments, and thinking iteratively to correct errors - we study this global-to-local paradigm from two perspectives. We first propose Glance then Scrutinize (GtS), a training-free framework using static and dynamic textual guidance for coarse-to-fine anomaly grounding and understanding, balancing accuracy and speed. To break the ceiling imposed by frozen external modules, we further propose a tool-augmented agentic VAD method, where a multimodal large language model learns to invoke a video cropping tool, inspect densely resampled frames, and self-correct mislocalized hypotheses, via cold-start supervised fine-tuning followed by reinforcement learning with a joint answer-grounding reward. For training and evaluation, we extend our prior VAGU benchmark into VAGU-T (Video Anomaly Grounding, Understanding, and Thinking), comprising 7,567 real-world videos over 21 anomaly categories with human-validated grounding, explanations, QA pairs, and chain-of-thought tool-calling traces. We further introduce JeAUG, a metric jointly evaluating semantic interpretability and temporal precision. Experiments show that GtS substantially surpasses training-free baselines, while the agentic model delivers both higher accuracy and faster inference.

[89] Can Frontier LLMs Match Natively Multimodal Embeddings? A Comparison on Hard-Negative Text-to-Image Retrieval cs.AI | cs.CV | cs.IR | cs.LGPDF

Archan Dutta, Vyanktesh Kanungo

TL;DR: 本研究首次在Flickr30k数据集上对比了原生多模态嵌入模型（如Gemini Embedding 2）与前沿大语言模型（如GPT-4.1和Claude Sonnet 4.6）在困难负样本文本到图像检索任务中的表现。研究发现，LLMs在零样本排序能力上与专用多模态嵌入模型相当，但后者在预计算嵌入后更适合低延迟应用。

Details

Motivation: 随着Gemini Embedding 2等多模态嵌入模型的出现，以及前沿LLMs展现出强大的视觉理解能力，研究需要评估LLMs是否能在多模态检索任务中作为有效的零样本排序器，并与专用嵌入模型进行直接比较。

Result: 在Flickr30k基准测试中，GPT-4.1和Claude Sonnet 4.6的表现与Gemini Embedding 2相当。

Insight: 论文的创新点在于首次直接比较了原生多模态嵌入与基于LLM的视觉排序。客观来看，研究揭示了前沿LLMs在零样本多模态检索任务中已达到与专用模型相媲美的能力，同时明确了专用嵌入模型在低延迟场景下的优势，为模型选型提供了重要参考。

Abstract: Multimodal retrieval and classification across different types of media, spanning text, images,video and audio, has traditionally relied on dual-encoder models that align visual and textual representations through contrastive learning. The March 2026 release of Gemini Embedding 2, Google’s first natively multimodal embedding model to map text, images, video, audio, and documents into a single shared space, raises competition among multimodal retrieval systems. Simultaneously, frontier Large language models (LLMs) have also demonstrated strong visual understanding, raising the question of whether they can serve as effective zero-shot rankers. Our study provides the first direct comparison of native multimodal embeddings against LLM-based visual ranking on Flickr30k. We observe that GPT-4.1 and Claude Sonnet 4.6 perform on par with Gemini Embedding 2. Additionally, once embeddings are precomputed, multimodal embeddings are better suited for low-latency applications.

[90] MBA: Multimodal Benchmark and Agents for Real-World Business Ideation cs.AI | cs.CV | cs.LGPDF

Hojun Choi, Jaeyo Shin, Suin Lee, Hyunjung Shim

TL;DR: 本文提出了MBA-Bench，这是首个用于训练和评估商业创意生成智能体的多模态基准数据集，包含六个领域的3万个样本，每个样本包含图像和文本。作者还提出了MBA-b和MBA-k两种智能体，它们通过结合新颖的奖励目标（创造性和可行性）以及特定设置的强化学习进行训练，在基准测试中显著超越了仅使用文本描述或多模态输入的基线模型。

Details

Motivation: 现有的基于大语言模型的智能体系统在商业创意生成方面仍局限于纯文本范式，而现实世界的商业环境本质上是多模态的，包含丰富的视觉信息。因此，需要建立一个多模态基准来推动该领域的发展。

Result: 在MBA-Bench上的实验表明，仅使用图像描述的基线模型性能有限，而多模态输入的基线模型在多项指标上接近闭源模型性能。作者提出的MBA-b和MBA-k智能体分别比图像描述基线高出63.9%和77.1%，比多模态基线高出25.6%和35.8%，实现了显著的性能提升。

Insight: 论文的创新点在于构建了首个多模态商业创意基准（MBA-Bench），并提出了针对隐藏评估标准（MBA-b）和已知评估标准（MBA-k）两种设置进行优化的智能体训练方法，其中引入了创造性和可行性作为新颖的奖励目标，并采用了基于LoRA的监督微调和分组相对策略优化进行训练。

Abstract: Agentic systems powered by large language models (LLMs) have opened new opportunities for business ideation. Yet existing approaches remain confined to a text-only paradigm, despite the inherently multimodal nature of real-world contexts. We thus introduce MBA-Bench, the first multimodal benchmark for training and evaluating business ideation agents, comprising 30K samples across six domains, each domain characterized by distinct visual cues not fully conveyed by text alone. Concretely, we automatically caption images and employ GPT-4o to generate five reference ideas for each of three business questions through retrieval query generation, market evidence retrieval, and evidence-augmented synthesis. Following prior work, we evaluate agents across six business-oriented criteria using MLLM-as-a-Judge. To consider settings where criteria are hidden or disclosed, we present MBA-b and MBA-k for blind and known, respectively. We train both with two novel reward objectives—creativity and feasibility—while MBA-k further optimizes the six disclosed criteria for eight in total. Both are trained via LoRA-based supervised fine-tuning followed by group relative policy optimization with these setting-specific rewards. For extensive experiments on MBA-Bench, we set up two baselines accommodating either captions only or multimodal inputs, with the latter nearing closed-source performance on several metrics. MBA-b and MBA-k outperform caption baselines by 63.9% and 77.1%, and multimodal baselines by 25.6% and 35.8%, respectively.

cs.CR [Back]

[91] How China-Origin Vision-Language Models Move from Refusal to Reframing in State Alignment cs.CR | cs.AI | cs.CLPDF

Guang Yang, Fengchen Liu, Alex Wang, Homa Hosseinmardi, Amir Ghasemian

TL;DR: 本文系统研究了中源视觉语言模型在政治敏感话题上的状态对齐扭曲现象，构建了一个包含200个核心条目和七种视觉抽象探针的基准测试，对九个VLMs进行了21,708次试验。研究发现，中源模型更倾向于进行状态对齐的框架重构而非明确拒绝，且中文提示会显著增加重构概率，这种审查机制正从可见的拒绝转向隐形的流畅重构。

Details

Motivation: 旨在探究基于文本的LLMs中已观察到的状态对齐扭曲现象是否以及如何出现在多模态系统中，并系统性地量化其表现形式。

Result: 在构建的基准测试上，中源模型进行状态对齐框架重构的概率是非中源模型的1.6至3.2倍；中文提示使重构几率增加约三倍；在Qwen模型迭代中，明确拒绝率下降而框架重构率上升。

Insight: 创新性地将多模态审查分解为拒绝、信息完整性、视觉基础、框架重构等多个独立维度进行测量，揭示了审查机制从“显性拒绝”到“隐形重构”的演变，这本质上是一个人机交互问题，因为它移除了用户识别信息被隐瞒的关键信号。

Abstract: State-aligned distortion has been documented in China-origin text-based large language models (LLMs), but whether, and in what form, it arises in multimodal systems has not been systematically examined. We construct a balanced benchmark of 200 core entries spanning ten politically sensitive topics, plus a seven-variant visual-abstraction probe, and run nine vision-language models (VLMs), seven China-origin and two non-China, across four elicitation paradigms and two prompt languages, yielding 21,708 trials. Each response is audited on six dimensions – explicit refusal, information integrity, visual grounding, state-aligned framing, language consistency, and response length – by two independent frontier LLM judges, validated against three human experts on a 200-trial sample. Measuring each dimension separately lets us decompose multimodal censorship into individual signals rather than a single refusal-based score; in particular, refusal and framing are measured independently, so a model can stop refusing while still reframing. We find that (i) Chinese-language prompting roughly triples the odds of state-aligned framing, within every model; (ii) China-origin models reframe more than non-China models (direction robust across judges and human raters; magnitude 1.6–3.2x); (iii) the effect is strongest in text-only political commentary (36.5%) and is gated by recognition of the depicted subject rather than pixel detail, persisting even at silhouette for iconic images; and (iv) across four Qwen generations, state-aligned framing rises while explicit refusal falls: censorship migrates from a visible act (refusal) to an invisible one (fluent reframing). We argue this shift to invisible reframing is fundamentally a problem of human-AI interaction: it removes the very signal users rely on to recognize that information has been withheld.

cs.HC [Back]

[92] Measuring Browser Webcam Gaze Honestly: A Capture-Clock Methodology and Open Reference Implementation cs.HC | cs.CV | eess.SPPDF

Chi-Sheng Chen, Gabriel A. Brat

TL;DR: 本文提出了一种基于浏览器网络摄像头进行诚实视线追踪延迟测量的方法，通过捕获-时钟方法学，利用浏览器的requestVideoFrameCallback API恢复每帧的捕获时间戳，从而准确测量真实世界的推理延迟。

Details

Motivation: 解决基于浏览器的网络摄像头视线追踪器在报告延迟时存在的不准确问题，因为现有方法通常在样本发出时打时间戳，而非在源帧捕获时，导致延迟测量失真。

Result: 在WebGazer和新的FaceMesh+KRR管道两个可互换引擎上进行了演示，实现了对每帧捕获时间的精确恢复，对于不公开推理管道的引擎（如WebGazer）提供了可验证的下界延迟测量。

Insight: 创新点在于利用浏览器API恢复捕获时钟，确保延迟测量的诚实性；客观分析认为该方法为大规模数据收集和临床环境中的视线追踪提供了可靠的基准测试工具。

Abstract: Browser-based webcam gaze trackers are increasingly used for crowd-scale data collection and in clinical settings where lab eye trackers are impractical, but the reported latency numbers may not represent real world functionality. The common practice of timestamping each gaze sample when it is emitted, rather than when its source frame was captured, makes the measured inference latency read about $0,$ms no matter how slow the engine really is. We show how to measure it honestly, recovering a per-frame capture clock from the browser’s \texttt{re-quest-Video-Frame-Call-back} (rVFC) API (\texttt{captureTime} where the browser exposes it for local camera streams, else \texttt{presentationTime}, in which case every recovered latency is a verifiable lower bound): exact source-frame pairing through a per-frame queue for engines that expose their inference pipeline, and a further lower bound for engines that do not, such as WebGazer. We release an open TypeScript implementation and benchmark harness, demonstrated on two interchangeable engines: WebGazer and a new FaceMesh+KRR pipeline.

eess.IV [Back]

[93] Geometry-Based Compression of Plenoptic Point Clouds eess.IV | cs.CV | cs.MMPDF

Davi R. Freitas, Gustavo L. Sandri, Ricardo L. de Queiroz

TL;DR: 本文提出了一种用于压缩全光点云（PPC）属性的高效方法，该方法结合了基于Karhunen-Loève变换的颜色属性处理和具有帧内预测能力的多属性编码器。该方案可集成到MPEG的基于几何的点云压缩（G-PCC）标准中，利用G-PCC现有的任意属性编码解决方案。在不同空间分辨率的PPC上进行压缩性能评估，结果显示与现有方法（如基于RAHT或基于视频的PCC解决方案）相比具有竞争力，作者认为其编码器达到了新的最先进水平。

Details

Motivation: 全光点云通过为每个点关联多个颜色（而非单一颜色）来提供更高的真实感，但随之而来的数据量增加需要高效的压缩方法。本文旨在解决PPC属性（颜色）的高效压缩问题，以支持其实际应用。

Result: 在不同空间分辨率的PPC上进行压缩性能评估，结果显示该方法与基于RAHT或基于视频的PCC解决方案等现有方法相比具有竞争力，作者宣称其编码器达到了新的最先进（SOTA）水平。

Insight: 创新点在于将Karhunen-Loève变换应用于PPC的颜色属性压缩，并结合具有帧内预测能力的多属性编码器，该方案设计灵活，可无缝集成到现有的MPEG G-PCC标准框架中，提升了压缩效率。

Abstract: Plenoptic point clouds (PPC) are novel data structures that represent the light from different viewing directions in order to provide a higher degree of realism to regular point clouds. This is achieved by associating each point to multiple colors instead of a single one. Here, we present a method to efficiently compress the attributes of a PPC, consisting of a Karhunen-Loève transform over the color attributes followed by multiple attribute coders with intra prediction capability. This compression scheme can be incorporated within the MPEG’s geometry-based PCC (G-PCC) standard, using any of G-PCC’s existing solutions for attribute coding. Compression performance assessment using PPCs of different spatial resolutions reveals competitive results in comparison to existing methods, such as RAHT-based or video-based PCC solutions. We believe our coder to be the new state of the art.

[94] Uncertainty-Aware and Explainable Ensemble Deep Learning Framework for Multi-Class Skin Lesion Classification eess.IV | cs.AI | cs.CV | cs.LGPDF

Rofiqul Islam, Lilatul Ferdouse

TL;DR: 本文提出了一种用于多类别皮肤病变分类的不确定性感知和可解释深度学习框架。该框架通过深度集成学习结合了视觉Transformer模型（MaxViT-Tiny）和基于CNN的模型（ConvNeXt-Tiny与EfficientNetV2-B0），并利用蒙特卡洛Dropout估计预测不确定性，以及使用Grad-CAM++提供可视化解释。

Details

Motivation: 解决皮肤癌诊断中因类内高变异性、类间相似性、类别不平衡以及深度学习模型可解释性有限所带来的挑战，旨在构建一个可信赖的计算机辅助诊断系统。

Result: 在HAM10000数据集上，经过不确定性感知过滤（熵<1.0，置信度>=0.7）后，框架达到了96%的准确率和99%的ROC-AUC，宏平均精确率、召回率和F1分数分别为94%、95%和95%，所有三个指标的加权平均分数均为96%，展示了SOTA水平的性能。

Insight: 创新点在于将视觉Transformer与CNN模型进行深度集成，并系统性地结合了不确定性量化（MC Dropout）和模型可解释性（Grad-CAM++）技术，为医疗AI提供了同时具备高精度、可解释性和可靠性评估的端到端解决方案。

Abstract: Skin cancer diagnosis from dermoscopic images remains challenging due to high intra-class variability, inter-class similarity, class imbalance, and the limited interpretability of deep learning models. This paper proposes an uncertainty-aware and explainable deep learning framework for multi-class skin lesion classification. The framework combines a vision transformer model (MaxViT-Tiny) with CNN-based models (ConvNeXt-Tiny and EfficientNetV2-B0) through deep ensemble learning. Monte Carlo (MC) Dropout estimates predictive uncertainty and identifies unreliable predictions, while Grad-CAM++, an explainable AI (XAI) technique, provides visual explanations by highlighting lesion regions that influence model decisions. Evaluated on the HAM10000 dataset, the framework achieves 96% accuracy and 99% ROC-AUC under uncertainty-aware filtering (entropy < 1.0, confidence >= 0.7), with macro-average precision, recall, and F1-score of 94%, 95%, and 95%, respectively, and 96% weighted-average scores across all three metrics. The results demonstrate accurate, interpretable, and uncertainty-aware skin lesion classification for trustworthy computer-aided diagnosis.

[95] Domain-Aware Lightweight Spectral-Grouped Convolutions for Hyperspectral Fish Freshness Classification eess.IV | cs.AI | cs.CV | eess.SPPDF

Kazi Nabiul Alam, Pooneh Bagheri Zadeh, Akbar Sheikh-Akbari

TL;DR: 论文提出了一种名为SGNet的轻量级网络架构，用于高光谱图像（HSI）的鱼类新鲜度分类。该网络通过分组卷积和深度空间路径分离光谱与空间特征提取，并采用双注意力机制自适应地突出信息特征。在自建的16天冷藏鲑鱼片数据集上，SGNet以仅4.75M参数实现了97.8%的分类准确率和0.64天的平均绝对误差（MAE）。

Details

Motivation: 解决高光谱图像数据在鱼类新鲜度评估中的特定挑战，包括光谱信息主导空间纹理、标签的序数结构以及训练样本数量有限，而传统深度学习方法未能充分处理这些特性。

Result: 在新构建的16天冷藏鲑鱼片数据集上，SGNet达到97.8%的分类准确率和0.64天MAE，参数量仅为4.75M；消融实验验证了各组件贡献，相比ResNet-50和Vision Transformers，参数量减少了5到18倍。

Insight: 创新点在于领域感知的轻量级设计，通过光谱分组卷积分离特征提取路径，并结合双注意力机制（通道压缩激励与空间门控）自适应融合特征；客观来看，这种针对HSI数据光谱主导性和小样本特性的定制化架构，在保持高精度的同时大幅降低了模型复杂度，有利于工业实时应用。

Abstract: Hyperspectral imaging (HSI) offers nondestructive assessment of fish freshness by detecting biochemical alterations across spectral bands. However, conventional deep learning approaches do not fully address the particular characteristics of HSI data, such as spectral dominance over spatial textures, ordinal label structure, and a small number of training samples. We propose SGNet (Spectral-Grouped Network), a lightweight architecture that separates spectral and spatial feature extraction using grouped convolutions and a depthwise spatial pathway. A dual attention mechanism that couples channel-wise squeeze-and-excitation with spatial gating adaptively highlights informative features. SGNet achieves 97.8% classification accuracy and 0.64 days mean absolute error (MAE) with just 4.75M parameters when tested on our newly developed 16-day refrigerator-stored salmon fillet dataset. Ablation studies validate the contribution of each component, while comparisons demonstrate a five- to eighteen-fold parameter reduction relative to ResNet-50 and Vision Transformers. Our findings indicate that domain-aware design supports precise, real-time freshness prediction for industrial implementation.

cs.SD [Back]

[96] Qwen-MusicAVQA-7B: A Multimodal Model for Music Audio-Visual QA cs.SD | cs.CV | cs.MMPDF

Maryam Dehdashti

TL;DR: 该论文提出了Qwen-MusicAVQA-7B，一个用于音乐视听问答（AVQA）的轻量级多模态模型。它通过学习的线性投影，将冻结的Whisper音频编码器连接到冻结的Qwen2-VL-7B-Instruct视觉语言模型，无需特定任务的融合网络。在MUSIC-AVQA基准测试中，该模型在可用视频测试子集上达到了96.0%的准确率，并且训练成本极低。

Details

Motivation: 解决音乐视听问答任务时，避免训练或适配大型全模态系统的繁重开销，探索一种轻量级但高效的替代方案。

Result: 在MUSIC-AVQA基准的7,402个问题的可用视频测试子集上，达到96.0% ± 3.9%的准确率；在重新表述的MUSIC-AVQA-R基准的头尾部分采样测试中，准确率也分别高达96.5%和95.6%。相比之下，在匹配数据和输入条件下微调的Qwen2.5-Omni-7B仅达到80.9%。

Insight: 核心创新在于采用轻量级、冻结编码器加简单投影的架构，证明了其有效性。关键发现是下游任务准确率与音频表征保留的细粒度局部时序信息量高度相关，而非简单地使用序列或向量表示。这种设计使得整个适配过程训练成本极低（单卡A100约5小时）。

Abstract: A common approach to adding audio to a vision-language model is to train or adapt a large omni-modal system. We show that a lightweight alternative can be highly effective for music audio-visual question answering (AVQA). Qwen-MusicAVQA-7B connects a frozen Whisper encoder to Qwen2-VL-7B-Instruct through learned linear projections. The same frozen encoder processes both the video’s music track and a TTS-spoken question through separate projectors, while the language model fuses visual frames, music, and question audio through pretrained self-attention, with no task-specific fusion network. On MUSIC-AVQA, our system reaches 96.0% +/- 3.9% accuracy across three independent training seeds on the 7,402-question available-video test subset. Our central finding is that downstream accuracy tracks how much fine-grained local temporal information the audio representation preserves. In a matched 32-token comparison, a stride-pooled Whisper frame sequence outperforms a globally pooled PANNs representation expanded to the same budget by 26 percentage points, even though PANNs sees at least as much audio and uses a far larger projector. The effect is not simply sequence versus vector: within Whisper alone, reducing temporal resolution at a fixed token budget costs a comparable amount. Under matched data and inputs, fine-tuned Qwen2.5-Omni-7B reaches 80.9%, against 95.9% for our 30 s variant; because the systems differ in backbone and adaptation, this is a system-level comparison. Accuracy remains high on sampled head and tail splits of the rephrased MUSIC-AVQA-R benchmark (96.5% and 95.6%). Because both encoders stay frozen and the music features are cached, the entire adaptation is cheap to train: the complete two-stage AVQA run takes approximately 5 hours on a single A100 80GB, and every run reported here fits on that one GPU.

[97] Dialogue-Aware Video-to-Music Generation Using Public Domain Film Collections cs.SD | cs.CV | cs.MMPDF

Haven Kim, Zachary Novack, Julian McAuley, Hao-Wen Dong

TL;DR: 本文提出了一个名为OSSL-v2的可复现、版权友好的公开领域电影视频-音乐数据集，并利用该数据集研究了一种新的视频到音乐生成方法。该方法的核心创新在于，通过引入时间轴并逐帧利用对话音轨作为条件信号，来增强现有模型的视频交叉注意力机制，从而更好地捕捉电影音乐与屏幕对话之间的时序关联。

Details

Motivation: 当前视频到音乐生成领域存在可复现性差距，模型常依赖可能失效的网络爬取数据。本文旨在通过构建一个稳定、自托管的数据集，并探索对话作为关键条件信号，以改善电影音乐生成的质量和可控性。

Result: 在公开领域和商业电影上的评估表明，该方法在视频到音乐生成任务上优于现有的最先进基线模型。

Insight: 主要创新点包括构建了一个可复现、版权清晰的大规模数据集OSSL-v2，以及提出了一种利用对话音轨进行逐帧调制的时序条件生成方法，这为理解视听媒体中音乐与对话的耦合关系提供了新思路。

Abstract: Video-to-music generation has drawn growing interest for its role in conveying the emotion of visual media, including film. Progress in the field, however, is hampered by a reproducibility gap: models are often trained on crawled corpora referenced through YouTube URLs that may be deleted, with the underlying data often difficult and time-consuming to retrieve. To address this, we introduce the Open Screen Soundtrack Library version 2 (OSSL-v2), a self-hosted corpus of 34,343 video clips totaling 246.4 hours, sourced from public-domain films. Unlike crawled corpora, OSSL-v2 is reproducible (i.e., not subject to link rot) and copyright-conscious, yet still large enough to train functional video-to-music models. We then use this film-domain corpus to study dialogue as a conditioning signal for video-to-music generation, motivated by the close temporal coupling between film music and on-screen speech. Specifically, we augment existing models’ video cross-attention with a time axis and modulate it frame-by-frame with the dialogue track. Evaluated on both public-domain and commercial films, our approach shows improvement over the state-of-the-art baselines. The dataset is available at https://huggingface.co/datasets/McAuley-Lab/OSSL-v2.

Table of Contents

cs.CL [Back]

[1] Gloss-Free Representation Learning for Cross-Dataset Sign Spotting cs.CL | cs.CVPDF

[2] Backtrader-Bench: Benchmarking LLM Agents on Algorithmic Trading with Self-Generated MCQs cs.CL | cs.AIPDF

[3] Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgets cs.CL | cs.AI | cs.LGPDF

[4] Self-Evolving Embodied Agents via Skill-Harness Evolution cs.CL | cs.ROPDF

[5] CT-$Δ$Bench: A Benchmark for Longitudinal 3D Medical Imaging Difference Reporting with Vision-Language Models cs.CL | cs.CVPDF

[6] Reinforcing Step-level Reasoning for Effective Self-Correction in LLMs cs.CL | cs.AIPDF

[7] Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs cs.CL | cs.AIPDF

[8] Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing cs.CL | cs.AI | cs.LGPDF

[9] When the API Speaks the Wrong Language: Revisiting Post-Training for Multilingual Tool Use cs.CL | cs.AIPDF

[10] Ripple-Pivot Search: Active Parallel Decoding for Diffusion Large Language Models cs.CLPDF

[11] AWARe: Mitigating Catastrophic Forgetting via Activation-Weighted Adaptive REtention cs.CLPDF

[12] Causal Structure is Inducible but Functionally Decoupled: The Routing/Readout Boundary of a Typed Mechanism Library cs.CLPDF

[13] GRPO for Financial Advice Generation: Outperforming Commercial LLMs under CATE Evaluation cs.CL | cs.AI | cs.LGPDF

[14] LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured – Reinforced Polarizer Keeps a Frozen LLM from Being Confidently Misled by the Wrong Evidence cs.CL | cs.IR | cs.LGPDF

[15] SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges cs.CLPDF

[16] One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL cs.CL | cs.AI | cs.LGPDF

cs.CV [Back]

[17] GeoUniPR: A Geometry-Consistent Unified Framework for Cross-Modal Place Recognition cs.CVPDF

[18] Self-Evolving Code-with-Image Reasoning cs.CVPDF

[19] VLMs Win a Systematic Evaluation of Underwater Image Reconstruction cs.CVPDF

[20] Clinical Feasibility of Low-Magnification Fluorescence Imaging for Breast Cancer Margin Detection Using Texture Analysis and Deep Learning cs.CV | cs.AI | cs.LGPDF

[21] Multi-Agent Target-Existence Verification and Learned Mask Geometry Refinement: Winning Report of the MeViS-Text Track at the 8th LSVOS Challenge 2026 cs.CVPDF

[22] Test-Time Hallucination Control in Large Vision-Language Models cs.CVPDF

[23] Gaussian Meta-Space Augmentation for Stacking Ensembles in Multimodal IPMN Risk Stratification cs.CV | cs.LGPDF

[24] From Synthesis to Removal: Physics-Grounded Reflection Simulation and Diffusion-Based Video Dereflection cs.CV | cs.AI | eess.IVPDF

[25] Repurposing RGB-based Foundation Model for Depth Estimation on Thermal Images Using Hierarchical Supervision cs.CV | cs.ROPDF

[26] A Hybrid Framework of Vision Transformer and Gated Recurrent Unit for Detection of Mosquito Diseases cs.CV | cs.AIPDF

[27] Generative Video Compression Based on Hierarchical Referencing cs.CVPDF

[28] Cloak of Invisibility: Real-Time Privacy-Preserving Volumetric Video Streaming cs.CVPDF

[29] Motion-as-Prompt: Enhancing Motion Reasoning in Multimodal Large Language Models via Motion-Guided Cross-Frame Visual Prompting cs.CV | cs.AIPDF

[30] EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection cs.CVPDF

[31] Boundary-Enhanced Segmentation of Pig Point Clouds in Commercial Housing Environments cs.CVPDF

[32] STAR: A Spatial-Topology Aware Routing Framework for Generalizable 3D Scene Understanding cs.CVPDF

[33] Learning from Multimodal Pseudo-Labels for Robust Open-Vocabulary Instance and Panoptic Segmentation cs.CV | cs.AI | cs.MMPDF

[34] Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System cs.CV | cs.AIPDF

[35] JieZi: A Large-Scale Expert-Audited Dataset and Benchmark for Ancient Chinese Character Exegesis cs.CV | cs.AIPDF

[36] UniSwap: Streaming Audio-Visual Identity Swapping for Talking Videos cs.CV | cs.SDPDF

[37] LiveAnimate: Stable Long-Form Streaming Human Animation in Real-Time cs.CVPDF

[38] VOLA: Improving Open-World Driving by VLM-Based Semantic Attribute Prediction cs.CVPDF

[39] Achieving Near-Zero-Overhead Multi-Model Hierarchical Classification in Real-Time Detection Pipelines cs.CV | cs.DC | cs.LGPDF

[40] Can Vision Models Read the Radar Display? On the Feasibility of Radar Imagery for Air Traffic Complexity Estimation cs.CV | cs.LGPDF

[41] TD-VAD: Breaking Visual Dependence in Video Anomaly Detection with Text-Driven Learning cs.CVPDF

[42] Distractor-Aware Video Object Segmentation cs.CVPDF

[43] GeoBridge: Decoupled Semantic Conditioning for Generative Image Geolocalization cs.CVPDF

[44] LookBack: Where and How to Score LVLM Responses via Visual Reference Usage cs.CV | cs.AI | cs.CLPDF

[45] Warping Earth Observations for better ice labeling in the Marginal Marginal Ice Zone cs.CVPDF

[46] Do You See What You Draw? A Semantic Closed-Loop Framework for Holistic Evaluation of Unified Multimodal Models cs.CV | cs.AIPDF

[47] HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion cs.CVPDF

[48] Surfsvr: 2D Surface Priors as 3D Geometric Regularizers for Sparse Voxel Reconstruction cs.CVPDF

[49] Auditing Frame-Level AUC in Weakly Supervised Video Anomaly Detection: Granularity, Resolution, and Scene Bias cs.CVPDF

[50] LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration cs.CV | cs.AIPDF

[51] Localizing to Debias: A Patch-Level Benchmark and Baseline for Weakly Supervised Spatial Anomaly Detection cs.CVPDF

[52] Look What the Probes Dragged In! Real-World Chest X-ray Shortcuts in MedCLIP cs.CV | cs.LGPDF

[53] Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatars cs.CVPDF

[54] SCOPE-Router: Cost-Aware Open-Set VLM Routing for Execution-Oriented Tasks cs.CVPDF

[55] Autonomous Telerehabilitation via Skeletal Motion Prediction and Joint-Level Performance Assessment cs.CV | cs.LGPDF

[56] Understanding Why Foundation Models Work for Diffusion-Generated Image Detection cs.CVPDF

[57] Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization cs.CVPDF

[58] Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs cs.CVPDF

[59] HSTGFormer: Hyper Spatial-Temporal Graph Transformer for 3D Human Pose Estimation cs.CV | cs.AIPDF

[60] GeoFlow: Efficient Driving Video Generation via Geometry-Aligned Priors cs.CVPDF

[61] Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Prediction cs.CVPDF

[62] Few-Shot Ordinal Learning for Day-Wise Freshness Estimation with Hyperspectral Fish Images cs.CV | cs.AI | eess.IV | eess.SPPDF

[63] SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Reward cs.CV | cs.AIPDF

[64] ScaleVid: Geometry-Aware Video Object Scaling with Mesh-Free Inference cs.CVPDF

[65] Automated Borehole Core Analysis with Report-Derived Weak Labels and Supervised Crack Segmentation cs.CVPDF

[66] Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams cs.CV | cs.AIPDF

[67] AVA-Encoder: Towards Agent-Native Video Representation Learning cs.CV | cs.CLPDF

[68] Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence cs.CV | cs.AI | cs.MMPDF

[69] Class Activation Mapping in Explainable Computer Vision: A Method-Centered Review of CNN, Transformer, and Foundation-Model-Era Visual Explanations cs.CV | cs.AIPDF

[70] StateFlow: Building, Evolving, and Accessing 3D World States for Previsualization cs.CVPDF

[71] DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation cs.CV | cs.AIPDF

cs.LG [Back]

[72] LEMUR: Latent Entropy-aware Multimodal Unlearning via Visual-anchored Reasoning Redirection cs.LG | cs.CLPDF

[73] Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL cs.LG | cs.AI | cs.CLPDF

[74] Epiplexity Guided Data Selection and Generation for Out-of-Distribution Generalization cs.LG | cs.CLPDF

[75] Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling cs.LG | cs.CLPDF

[76] AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses cs.LG | cs.AI | cs.CLPDF