cs.CL [Total: 12]
cs.CV [Total: 48]
cs.RO [Total: 2]
cs.MA [Total: 1]
cs.GR [Total: 1]
cs.LG [Total: 2]
cs.AI [Total: 6]

cs.CL [Back]

[1] From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontier cs.CL | cs.AIPDF

Eric Jiang, Xiao Liang, Yikai Zhang, Yingjia Wan, Mengting Li

TL;DR: 这篇立场论文主张AI4Math系统需要从预定义问题求解器转向能够应对前沿数学挑战的研究代理，系统回顾了当前领域在数据集、自动形式化和证明合成方面的进展，并指出了现有系统在作为数学研究代理时的核心局限性。

Details

Motivation: 当前基于大语言模型的定理证明器虽然在形式化证明生成方面取得了显著成功，但本质上仍局限于处理定义良好的数学问题，无法有效应对前沿研究数学（如发现新定理或解决开放猜想）的开放性、欠规范和多重抽象性挑战。

Result: 论文未提供具体的定量实验结果，而是通过系统性综述和局限性分析，为AI4Math的未来发展绘制了战略路线图。

Insight: 创新点在于提出了从求解器到研究代理的范式转变，并系统性地识别了现有系统在数据集、关系结构、数学探索、工具生态系统和人机协作等方面的核心限制，为构建能进行严格形式化数学推理的前沿数学研究代理指明了方向。

Abstract: Recent developments in AI for Mathematics (AI4Math), especially Large Language Model (LLM)-driven theorem provers, has achieved remarkable success in formal proof generation for well-defined mathematical problems through Interactive Theorem Proving (ITP) languages. However, current systems remain fundamentally limited in tackling frontier research mathematics, such as discovering new theorems or resolving open conjectures, which are often open-ended, under-specified, and involve multiple layers of abstraction. We argue that the next leap in AI4Math systems requires a decisive shift from predefined problem-solvers to research agents that can address frontier mathematical challenges with rigorous formal mathematical reasoning. In this position paper, we provide a systematic review of the field, covering datasets, auto-formalization, and proof synthesis. More importantly, we identify core limitations of existing systems in serving as mathematical research agents, examining issues across datasets, relational structure, mathematical exploration, tool ecosystem, and human-AI collaboration, outlining a strategic road-map for the future of AI4Math.

[2] DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment cs.CLPDF

Xinyu Geng, Xuanhua He, Sixiang Chen, Yanjing Xiao, Fan Zhang

TL;DR: 论文提出了DeepSearch-World，一个可验证的确定性环境，包含42万个多跳QA任务，并基于此构建了DeepSearch-Evolve自蒸馏框架，用于训练网络搜索智能体。该框架通过迭代的轨迹生成、过滤、数据混合和微调，使智能体能够自我进化，在多个基准测试中达到有竞争力的性能。

Details

Motivation: 现有方法中，监督微调依赖于固定的教师蒸馏轨迹，而稀疏奖励强化学习对长程交互的监督较弱，因此需要一种能让智能体从自身经验中持续改进的训练框架。

Result: 在未使用更强大模型蒸馏的情况下，DeepSearch-World-9B在BrowseComp上达到31.2%，在GAIA上达到61.5%，在HotpotQA上达到93.4%，与开源智能体相比具有竞争力。

Insight: 创新点在于构建了一个可验证的确定性环境，支持进度验证、基于事实的反思和失败恢复等关键认知行为，并通过自蒸馏框架实现了智能体的可扩展自我进化，为长程网络智能体的研究提供了新途径。

Abstract: Training tool-use agents to improve from their own experience remains challenging, as supervised fine-tuning relies on fixed teacher-distilled trajectories, while sparse-reward reinforcement learning provides weak supervision for long-horizon interactions. We present DeepSearch-Evolve, a self-distillation framework for web agents built on DeepSearch-World, a deterministic and verifiable environment with reproducible search and page-reading tools. DeepSearch-World contains 420K multi-hop QA tasks constructed from entity-level random walks and supports key agentic cognitive behaviors useful for self-evolving, including progress verification, grounded reflection, and failure recovery. DeepSearch-Evolve iteratively performs trajectory generation, filtering, data mixing, and fine-tuning to train stronger agents. Without distillation from more capable models, DeepSearch-World-9B achieves competitive performance compared with open-source agents, reaching 31.2% on BrowseComp, 61.5% on GAIA, and 93.4% on HotpotQA, showing that verifiable environments enable scalable self-evolution for long-horizon web agents. We will release the environment, 420K training pool, validation set, model, and code to facilitate future research on self-improving deep search agents.

[3] Hallucination Self-Play: Bootstrapping Reinforced Detector via Evolved Generator cs.CL | cs.LGPDF

Shiping Yang, Shining Liang, Weihao Liu, Wenbiao Ding, Linjun Shou

TL;DR: 该论文提出了一种名为Hallucination Self-Play（HSP）的新框架，用于解决大语言模型（LLM）输出中真实性幻觉的检测难题。该框架通过让检测器和生成器这两个角色（均从同一基础模型初始化）进行自我博弈来相互促进：检测器评估输出的真实性，而生成器则产生越来越难以检测的幻觉响应。通过强化学习从AI反馈（RLAIF）和基于规则的强化学习，两者迭代进化，从而在无需外部监督的情况下提升小模型的检测性能。

Details

Motivation: 当前识别LLM生成输出中的真实性幻觉面临高质量标注数据稀缺的挑战。现有方法依赖先进LLM合成训练数据，但将生成器视为静态组件，限制了检测器的迭代改进能力。

Result: 在RAGTruth基准测试和两个模型系列上的实验表明，所提出的HSP框架能够逐步增强一个小型LLM，使其在无需外部监督的情况下，达到甚至超越先进LLM的检测性能。

Insight: 核心创新点在于提出了一个检测器与生成器协同进化的自博弈框架，通过强化学习实现两者的迭代优化，从而在数据稀缺情况下自举提升幻觉检测能力。这为利用模型自身生成对抗性数据以持续改进特定任务性能提供了新思路。

Abstract: Identifying faithfulness hallucinations in LLM-generated outputs remains challenging due to the scarcity of high-quality annotated data. Recent work relies on advanced LLMs to synthesize training data, including rationales, labels, and hallucinated claims. However, these methods treat the generator as a static component, limiting iterative improvement of the detector. To address this limitation, we introduce Hallucination Self-Play (HSP), a novel framework that enables the detector to bootstrap with an evolved generator. HSP involves two roles initialized from the same base model, a detector that assesses the faithfulness of model outputs, and a generator that produces increasingly hard-to-detect hallucinated responses. Specifically, the detector is first fine-tuned on human-labeled data and then employed as a reward model to train the generator via reinforcement learning from AI feedback (RLAIF). In turn, the evolved generator synthesizes hallucination data to further optimize the detector through rule-based reinforcement learning. Experiments on RAGTruth benchmark and two model families demonstrate that the proposed framework can progressively enhance a small LLM to match or even outperform advanced LLMs without external supervision. Our code is available at https://anonymous.4open.science/r/Hallucination-Self-Play-50B5 .

[4] When Implausible Tokens Get Reinforced: Tail-Aware Credit Calibration for LLM Reinforcement Learning cs.CL | cs.AI | cs.LGPDF

Xiuyi Lou, Zicheng Xu, Yu-Neng Chuang, Hoang Anh Duy Le, Zhaozhuo Xu

TL;DR: 本文针对大语言模型强化学习中广泛使用的无评论家方法存在的均匀信用分配问题，提出了尾部感知信用校准方法。该方法通过计算尾部风险分数来识别并抑制对上下文错误但概率低的尾部令牌的正面信用强化，从而避免错误推理行为的无差别增强。

Details

Motivation: 动机在于解决无评论家强化学习方法中均匀信用分配导致的‘正面信用污染’问题，即上下文错误的低概率尾部令牌与合理的令牌获得相同的正面信用，导致有缺陷的推理行为被不加区分地强化。

Result: 在三个大语言模型和八个基准测试上的实验结果表明，TACO方法持续优于GRPO风格的基线方法，并提高了训练稳定性，支持在长视野强化学习中实现持续的性能提升。

Insight: 创新点在于提出了一种尾部感知的信用校准机制，通过结合局部生成上下文计算尾部风险分数，以区分意外的稀有性和不确定性驱动的探索，从而在不完全移除梯度的情况下调整风险令牌的正面信用，使有用的稀有模式得以积累强化，同时渐进地抑制偶然噪声。

Abstract: Reinforcement learning (RL) has achieved remarkable success in enhancing the reasoning capabilities of large language models (LLMs). However, widely used critic-free RL methods rely on uniform credit assignment, broadcasting the same advantage to all tokens regardless of their differences. We identify a critical failure mode of this design, which we refer to as Positive-Credit Contamination: low-probability tail tokens that are contextually erroneous receive identical positive credit to plausible ones within the same trajectory, resulting in the indiscriminate reinforcement of flawed reasoning behavior. To mitigate this issue, we propose Tail-Aware Credit calibratiOn (TACO), a method that calibrates uniform credit assignment to suppress undesirable positive updates. TACO first computes a tail-risk score that incorporates the local generation context to assess each token’s risk of falling into the unreliable tail, distinguishing unexpected rarity from uncertainty-driven exploration. TACO then uses this score to tune positive credit for risky tokens without removing their gradients entirely, so that recurring useful rare patterns can accumulate reinforcement while incidental noise is progressively dampened. Experimental results across three LLMs and eight benchmarks show that TACO consistently outperforms GRPO-style baselines. Notably, TACO improves training stability, supporting sustained performance gains in long-horizon RL. The source code is available at: https://github.com/xiuyilou/TACO.

[5] Can We Trust LLM’s Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Framework cs.CL | cs.AIPDF

Riccardo Revalor, Jalees Rehman, Debjit Pal

TL;DR: 本文提出了GRAPHEVAL，一个基于图的推理框架，用于量化大型语言模型（LLM）推理中的不确定性、一致性和鲁棒性。该框架引入了图推理一致性分数（GRCS）作为新的不确定性量化指标，以及图自一致性（GSC）解码策略，旨在评估和提升推理过程的忠实性，而非仅关注最终答案的一致性。

Details

Motivation: 现有解码策略（如自一致性）仅评估最终答案的一致性，而忽略了中间推理步骤的逻辑有效性，导致可能无法检测到LLM推理中的缺陷和不忠实推理。本文旨在解决如何可靠地量化LLM推理中的不确定性、如何选择更忠实的推理路径以及推理拓扑在对抗条件下的鲁棒性这三个基本问题。

Result: 研究发现，GRCS是唯一一个在不同规模模型（无论是能力更强的还是较小的模型）中都与推理忠实性持续负相关的指标。GSC解码策略在较小模型中揭示了自一致性因不忠实的“幸运猜测”而被高估的程度，同时在能力更强的模型中保持或提高了准确性。对抗性中心点消融实验表明，GSC选择的路径是关键路径，偏离它会降低推理忠实性，并在特定情况下导致准确性下降。

Insight: 创新点在于将不确定性量化重新定义为整体的推理忠实性问题，并提出了基于图的GRCS指标来量化推理空间的语义-结构共识，以捕捉病态的模式崩溃和自信的幻觉。此外，提出的GSC解码策略通过基于中心点的选择，在忠实性和准确性之间进行权衡，为评估和改进LLM的推理逻辑提供了新的框架和工具。

Abstract: Large-Language Models (LLMs) can be prone to flawed and unfaithful reasoning that decoding strategies like Self-Consistency (SC) fail to detect as they evaluate only final-answer agreement while ignoring the logical validity of intermediate steps. This raises three fundamental questions: How can we reliably quantify uncertainty in LLM reasoning? Can semantic, structural, and causal awareness select more faithful reasoning compared to naïve majority voting? and How robust is reasoning topology under adversarial conditions? To address these questions, we introduce GRAPHEVAL, a graph-based reasoning framework that re-frames uncertainty quantification (UQ) as a holistic reasoning fidelity problem. We propose a novel UQ metric, Graph Reasoning Coherence Score (GRCS), that quantifies semantic-structural consensus of the reasoning space and captures pathological mode collapse and confident hallucinations. We find that GRCS is the only metric that is consistently negatively correlated with reasoning faithfulness across both more capable and smaller models. Additionally, we introduce Graph Self-Consistency (GSC), a medoid-based decoding strategy that trades nominal accuracy for reasoning fidelity, exposing the degree to which SC is inflated by unfaithful lucky guesses in smaller models, while preserving or improving accuracy in more capable ones. Finally, through adversarial medoid ablation, we demonstrate that the GSC-selected path acts as a “load-bearing path” and forcing models away from it degrades reasoning faithfulness and, in targeted cases, causes drops in accuracy.

[6] What LLM Forecasters Know but Don’t Say: Probing Internal Representations for Calibration and Faithfulness cs.CL | cs.AIPDF

Raphaël Sarfati, Pratyush Ranjan Tiwari, Siddharth Boppana, Christopher J. Earls, Srikar Varadaraj

TL;DR: 本文研究大型语言模型在预测任务中的校准和忠实性问题，发现通过探测模型内部表示可以显著改善预测校准度，并揭示思维链推理可能无法忠实反映预测依据。实验表明内部表示能作为‘谎言探测器’追踪行为变化，且预测决策在推理开始前已基本确定，从而提出基于预推理激活的令牌节省策略。

Details

Motivation: 针对微调用于预测的LLM存在校准不佳且思维链推理可能无法忠实反映预测证据的问题，探索内部表示是否能为校准和忠实性提供更直接的窗口。

Result: 在OpenForesight基准上，对Eternis-Forecaster 8B等模型的中间激活训练表示池化探针，实现了显著更好的校准；探针作为谎言探测器在84%的情况下能预测行为变化方向；通过预推理激活分布路由问题可节省30-47%的生成令牌且无精度损失。

Insight: 创新点在于利用内部表示而非思维链来评估和改善模型预测的校准与忠实性，揭示了预测决策在推理前已基本固定的现象，并提出了基于预推理激活的高效令牌节省方法，为校准、审计和分流语言模型预测器提供了实用工具。

Abstract: Large language models fine-tuned for forecasting can be accurate yet poorly calibrated, and their chain-of-thought (CoT) reasoning may not faithfully reflect the evidence behind a forecast. We ask whether internal representations offer a more direct window into both. Working with Eternis-Forecaster 8B on OpenForesight, we train representation-pooling probes on intermediate activations and find they achieve substantially better calibration; a result that also holds for GLM-4.7-Flash and GLM-4.5-Air. We then assess CoT faithfulness through evidence ablation and diversionary injection: removing an influential source in the prompt often changes the model’s forecast while leaving the reasoning trace untouched. The same probes function as lie detectors: their activations track behavioral shifts far better than the reasoning trace does, and they also predict the direction of change in 84% of cases, including when the CoT conceals the perturbation’s influence. Finally, forced answering reveals that forecasts are largely fixed before reasoning begins: a single pre-reasoning pass recovers the committed answer and confidence, and routing questions by the spread of this pre-set answer distribution saves 30-47% of generated tokens, with no loss of accuracy. Together, these results establish probing internal representations as a practical tool for calibrating, auditing, and triaging language model forecasters and reasoning models more broadly.

[7] MASTE: A Multi-Agent Pipeline for Zero-Shot Aspect Sentiment Triplet Extraction cs.CLPDF

Ao Hong, Lehang Wang, Zhirun Yue, Mingxin Wang, Zihan Wang

TL;DR: 本文提出MASTE，一种用于零样本方面情感三元组提取的多智能体流水线方法。该方法将ASTE任务分解为四个顺序阶段，由专门化的智能体处理不同子任务，实现了无需训练数据的零样本性能。在四个基准测试上的实验表明，MASTE显著优于零样本和思维链提示的LLM基线，缩小了与全监督方法的差距。

Details

Motivation: 解决大型语言模型在ASTE任务上零样本性能有限的问题，因为单次生成需要模型同时确定跨度边界、观点分组和情感极性，而现有的少样本上下文学习和思维链提示改进有限且依赖领域数据或精心设计的提示。

Result: 在四个ASTE基准测试上，MASTE大幅优于相同骨干网络的零样本和思维链LLM基线，在不使用任何标注三元组的情况下，缩小了与全监督方法的差距。

Insight: 创新点在于将ASTE任务分解为顺序子任务并由专门化智能体处理的多智能体流水线设计，实现了训练无关的零样本泛化能力；客观分析认为其模块化分解和显式条件传递是提升复杂信息抽取任务零样本性能的有效策略。

Abstract: Aspect Sentiment Triplet Extraction (ASTE) requires jointly identifying (aspect, opinion, sentiment) triples from a given review sentence. While large language models (LLMs) achieve strong zero-shot performance on many NLP benchmarks, their effectiveness on ASTE remains limited, as single-pass generation forces the model to determine span boundaries, opinion grouping, and sentiment polarity in a single decoding step. Common remedies, such as few-shot in-context learning and chain-of-thought prompting, offer only marginal improvements and rely heavily on either in-domain demonstrations sampled from labeled training data or carefully engineered reasoning prompts, neither of which is broadly available in zero-shot deployment. Inspired by the classical agent paradigm, we propose MASTE, a multi-agent pipeline for zero-shot Aspect Sentiment Triplet Extraction. MASTE decomposes ASTE into four sequential stages, where specialized agents handle different compositional subtasks with explicit conditioning on prior outputs. This design enables entirely training-free zero-shot ASTE and generalizes across different backbones and datasets. Extensive experiments on four ASTE benchmarks show that MASTE substantially outperforms zero-shot and chain-of-thought LLM baselines under the same backbone, narrowing the gap to fully supervised methods without using any labeled triplets. Code is available at https://github.com/Hankerlove/MASTE.

[8] Diarization-Guided Qwen-ASR Adaptation for Multilingual Two-Speaker Conversational Speech cs.CLPDF

Hao Wu, RongQi Han, Zhen Wang, Wei Liang, Wei Xu

TL;DR: 本文介绍了为MLC-SLM 2026挑战赛任务1设计的系统，用于处理多语言双人对话语音。该系统结合了模块化说话人日志前端和经过挑战赛适配的Qwen3-ASR-1.7B识别器，通过说话人日志引导音频分段，并按语言/区域分组后由适配的ASR模型解码。ASR适配采用监督全微调、基于TTS合成语音的LoRA微调以及基于WER/CER奖励的GRPO强化学习三阶段策略。

Details

Motivation: 解决多语言双人对话场景下，端到端ASR系统在说话人分离和识别精度上的挑战，通过结合说话人日志前端与ASR适配，提升整体转录性能。

Result: 在官方开发集上，系统平均tcpMER为23.70，相比原始Qwen-ASR-1.7B降低6.83个绝对点；在最终评估集上平均tcpMER为17.97。消融实验表明监督微调贡献最大，合成语音LoRA适配和强化学习进一步提升了鲁棒性。

Insight: 创新点在于将说话人日志前端与多阶段ASR适配（全微调+合成数据LoRA+强化学习）相结合，形成模块化流水线；客观分析认为，利用TTS合成数据增强和针对幻觉、重复等问题的强化学习惩罚机制，是提升多语言对话ASR鲁棒性的有效策略。

Abstract: This paper describes our self-designed system for Task 1 of the MLC-SLM 2026 Challenge for multilingual two-speaker conversational speech. The system combines a modular speaker diarization front end with a challenge-adapted Qwen3-ASR-1.7B recognizer. The diarization front end performs voice activity detection, subsegment generation, CAMPPlus speaker embedding extraction, two-speaker spectral clustering, and RTTM-based audio segmentation. The resulting speaker-attributed segments are grouped by language or region and decoded by the adapted ASR model. For ASR adaptation, we first perform supervised full fine-tuning on the official training data, then apply LoRA fine-tuning with synthetic speech generated by a three-pipeline TTS-based synthetic speech augmentation framework, and finally refine the model using GRPO reinforcement learning with rewards based on WER/CER and penalties for hallucination, repetition, and length deviation. On the official development set, the full system achieves an average tcpMER of 23.70, reducing the error rate by 6.83 absolute points relative to the released Qwen-ASR-1.7B performance. On the final evaluation set, the system achieves an average tcpMER of 17.97. Ablation results show that supervised fine-tuning provides the largest gain, while synthetic-speech LoRA adaptation and reinforcement learning further improve robustness.

[9] When Synthetic Speech Is All You Have: Better Call GRPO cs.CL | cs.AIPDF

Shashi Kumar, Yanis Labrak, Hasindri Watawana, Sergio Burdisso, Esaú Villatoro-Tello

TL;DR: 本文研究了在隐私受限领域（如银行业）中，基于LLM的自动语音识别（ASR）系统面临真实语音数据稀缺的问题，因此转向使用合成语音（TTS）进行模型适应。论文发现，与传统的监督微调（SFT）相比，采用无评论者的强化学习方法——组相对策略优化（GRPO）能更有效地利用合成语音数据，显著降低词错误率（WER）。

Details

Motivation: 在受监管领域（如银行业），收集真实语音数据成本高且受法律限制，因此合成语音成为ASR模型适应的主要资源。然而，合成语音与真实录音存在声学不匹配，现有工作局限于监督微调（SFT），未能充分利用合成数据。

Result: 仅使用合成语音进行GRPO适应，相比SFT将WER相对降低了40%（从36.71%降至22.09%）；结合SFT后GRPO进一步将WER相对降低45%。这表明GRPO在合成语音上显著优于SFT。

Insight: 论文的创新点在于将强化学习（特别是GRPO方法）引入合成语音驱动的ASR适应，以奖励低WER假设来优化策略。客观分析表明，GRPO的增益源于行为层面的改进（如减少插入错误、改善停止校准和语音-文本对齐的注意力锚定），而非底层表示变化，这为资源受限下的ASR优化提供了新思路。

Abstract: LLM-based ASR adapted to regulated domains such as banking is bottlenecked by privacy: real speech is costly and legally constrained to collect, making synthetic text-to-speech (TTS) an attractive substitute. Yet synthetic speech stays acoustically mismatched with real recordings, and work on this gap has stayed within supervised fine-tuning (SFT). We instead turn to reinforcement learning, and show that Group Relative Policy Optimization (GRPO) extracts far more from the same synthetic speech than SFT. Synthetic-only adaptation of the model with GRPO, a critic-free method rewarding low-WER hypotheses, reduces WER by 40% relative to SFT (36.71%$\to$22.09%), and an SFT-then-GRPO combination pushes this further to 45%. We trace the gain to behavior rather than representation: GRPO reduces insertion errors by improving stopping calibration and speech-to-text alignment by better anchoring attention to audio, leaving early-layer representations intact. When synthetic speech is the main resource, reinforcement learning should be preferred over supervised fine-tuning.

[10] UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing cs.CL | cs.AIPDF

Xinlong Zhao, Dongsheng Liu, Hengyu Zhao, Zixuan Fu, Zheng Wang

TL;DR: 本文提出了UltraX框架，旨在通过自适应程序化编辑来精炼大规模预训练数据。该框架通过引入插入操作完善了编辑功能空间，并构建了可靠的程序监督生成流程，包括数据集自适应提示优化、行对齐映射和动态上下文替换，以生成结构化程序监督。同时，通过低置信度示例过滤和比例控制采样提升监督质量，并在推理时通过滑动窗口预测和全局操作聚合确保执行的稳定性。

Details

Motivation: 随着训练数据接近物理极限，缩放定律带来的收益开始减少，提升大型语言模型（LLM）性能更依赖于高质量数据而非数据扩展。然而，现有的大规模语料库精炼方法在质量、效率和可靠性方面存在局限：基于规则的方法受限于固定启发式且难以处理实例级变化，而基于LLM的方法虽提升质量但无法满足大规模数据处理的效率和可靠性要求。

Result: 实验表明，UltraX在所有语料库上实现了最高的平均性能，并且在使用更少训练标记的情况下达到或超越了基线方法，证明了更强的数据效率和精炼可靠性。

Insight: 创新点在于通过引入插入操作扩展了编辑功能空间，实现了细粒度的实例级编辑，并设计了程序监督生成流程和推理时的稳定化机制，从而在大规模数据处理中平衡了质量、效率和可靠性。

Abstract: As available training data approaches its physical limit, gains from Scaling Laws have begun to diminish. Consequently, improving Large Language Models (LLMs) now depends less on data expansion and more on higher-quality data utilization. However, in the context of large-scale corpora, existing refinement methodologies face significant limitations in quality, efficiency, and reliability: Rule-based approaches are constrained by fixed heuristics and struggle with instance-level variations; LLM-based approaches improve quality but fail to meet the efficiency and reliability requirements of large-scale data processing. To address these challenges, we propose UltraX, a function-calling refinement framework for large-scale pre-training data that completes the editing function space by introducing insertion in addition to deletion and modification, enabling fine-grained instance-level editing. Specifically, UltraX builds a reliable program-supervision generation pipeline. In this pipeline, dataset-adaptive prompt optimization first guides an expert LLM to produce high-quality end-to-end refined texts, and Line Alignment Mapping and Dynamic Context Replacement then convert original-refined text pairs into structured program supervision. Meanwhile, UltraX improves supervision quality and stabilizes the training distribution with low-confidence example filtering and ratio-controlled sampling by operation combination. During inference and execution, it normalizes and validates model outputs through sliding-window prediction, global operation aggregation, and systematic post-processing, improving the stability and reliability of large-scale execution. Experiments show that UltraX achieves the highest average performance across all corpora and also matches or surpasses baselines with fewer training tokens, demonstrating stronger data efficiency and refinement reliability.

[11] UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks cs.CLPDF

Zhekai Chen, Chengqi Duan, Kaiyue Sun, Bohao Li, Yuqing Wang

TL;DR: 本文提出了UniClawBench，一个用于评估现实世界任务中主动智能体的能力驱动型通用基准。该基准围绕五大核心能力构建，设计了400个双语任务，并在动态Docker环境中采用细粒度检查点和闭环评估策略进行测试。

Details

Motivation: 现有基准难以有效评估能在现实环境中操作日常工具的主动智能体，因为它们通常依赖沙盒环境、单轮评估范式，且任务分类混杂多种能力，导致失败根因难以定位。

Result: 在多个最先进的模型和智能体框架上进行了综合比较，揭示了基础模型能力和框架设计如何共同影响在现实环境中的性能。

Insight: 创新点在于提出了首个能力驱动的基准，通过五大核心能力分解任务，并设计了在动态容器中基于步骤检查点的闭环评估策略，以更真实地模拟多轮人机交互并隔离框架与模型能力的影响。

Abstract: The rapid development of large language models and multimodal large language models has accelerated the emergence of proactive agents capable of operating everyday tools and assisting users in real-world environments. However, existing benchmarks struggle to evaluate such agents effectively, as they often rely on sandboxed environments and single-turn evaluation paradigms. Moreover, their scenario-based task taxonomies mix multiple model capabilities within the same task category, making it difficult to identify the root causes of agent failures. To address these limitations, we introduce UniClawBench, the first capability-driven benchmark designed to evaluate proactive agents in dynamic, real-world settings. UniClawBench is built around five foundational model capabilities: Skill Usage, Exploration, Long-Context Reasoning, Multimodal Understanding, and Cross-Platform Coordination. Based on these capabilities, we design 400 bilingual real-world tasks. Unlike previous benchmarks that rely on static, pre-recorded answers, our benchmark evaluates agents in live Docker containers using fine-grained, step-by-step completion checkpoints. Furthermore, we design a closed-loop evaluation strategy comprising an executor agent, a hidden supervisor agent, and a user agent to simulate realistic multi-turn human feedback without leaking grading criteria. To disentangle base model capabilities from framework-level design choices, we evaluate state-of-the-art models under multiple agent frameworks. Through comprehensive comparisons across both models and frameworks, we show how base model capabilities and agent framework designs jointly shape performance in real-world environments. To facilitate future research, we make our benchmark and code publicly available at https://github.com/HKU-MMLab/UniClawBench.

[12] Do You Need a Frontier Model as a Citation Verifier? Benchmarking Rubric LLMs for Deep-Research Source Attribution cs.CLPDF

Ethan Leung, Elias Lumer, Corey Feld, Austin Huber, Vamse Kumar Subbiah

TL;DR: 本文研究了在深度研究系统中，使用不同规模的LLM作为引文质量评判员（judge）的校准问题。论文通过一个对抗性长文本基准测试，评估了8个现成LLM模型在判断引文相关性和事实支持性两个维度的表现，发现更便宜的模型在性能上依然具有竞争力，且校准评判员是使用引文评分作为强化学习奖励信号的前提。

Details

Motivation: 在强化学习中，依赖LLM作为评判员来对每个评分标准进行打分，并以此作为训练时的奖励模型。为了信任这种信号，需要了解评判员所需的能力水平及其偏见程度，因此本文针对深度研究系统中的引文质量校准问题展开研究。

Result: 在包含1,248个评分决策（全部经过人工审核，其中378个为来自评判员分歧的困难案例）的对抗性长文本基准测试中，GPT-5-mini在引文相关性维度取得了最强的通过类F1分数0.908（κ=0.636），而在事实支持性维度，各评判员的置信区间重叠，统计上无法区分优劣。

Insight: 研究发现，标量F1分数掩盖了评判员在通过率漂移、假阳性率和假阴性率上的显著差异，而这种方向性偏见恰恰是下游强化学习循环会放大的。关键创新在于指出校准评判员是使用引文评分作为奖励信号的必要前提，并且这种校准不一定需要最昂贵的模型。

Abstract: Reinforcement learning increasingly relies on an LLM judge to score each rubric criterion, and that judge acts as the reward model during training. Before such a signal can be trusted, we need to know how capable the judge must be and how biased it is. We study this calibration question for citation quality in deep-research systems, where a search-grounded LLM must support each claim it writes with a cited source. Citation quality is a structured rubric task in which each attribution-citation pair is judged along two dimensions that require an LLM, source relevance and factual support. On an adversarial long-form benchmark, we score 8 off-the-shelf LLM judges from 3 model families against gold labels over 1,248 rubric decisions, all of which were human-reviewed and 378 of which were hard cases adjudicated from judge disagreements. Cheaper judges remain competitive across both dimensions, with GPT-5-mini attaining the strongest source-relevance pass-class F1 at 0.908 ($κ$=0.636), while on factual support the judges are statistically indistinguishable (overlapping confidence intervals), so no single model dominates. At comparable F1, the judges still differ substantially in pass-rate drift, false positive rate, and false negative rate. Scalar F1 obscures this directional bias, yet it is exactly what a downstream reinforcement learning loop would reinforce. Calibrating the judge is therefore a prerequisite for using citation rubrics as reward signals, and our results show that this calibration does not require the most expensive available model.

cs.CV [Back]

[13] Adversarial Decoys: Misdirecting Attention-Based Defenses in ViT cs.CV | cs.AIPDF

Giulia Marchiori Pietrosanti, Giulio Rossolini, Giorgio Buttazzo

TL;DR: 这篇论文提出了‘对抗性诱饵’方法，旨在误导基于注意力机制的ViT防御系统。该方法通过独立优化的图像补丁，将防御系统的注意力从真正的对抗性区域重定向到选定的目标令牌上，从而在保持攻击有效性的同时规避防御。

Details

Motivation: 动机在于Vision Transformers (ViTs) 对局部对抗攻击（如对抗性补丁）仍然脆弱，而现有的测试时防御通过抑制具有异常高注意力分数的图像令牌来缓解这些攻击。这些防御利用了注意力与对抗有效性之间的强耦合关系，但论文认为这种基于注意力大小的指标存在根本性限制。

Result: 在ImageNet数据集上，针对多种ViT架构和攻击的实验表明，诱饵能够成功将高注意力分数从真正的对抗性区域重定向出去，同时大部分攻击有效性得以保留，从而揭示了基于注意力大小的防御方法的局限性。

Insight: 创新点在于将诱导错误分类和防御规避这两个目标解耦：原始对抗区域负责导致错误预测，而独立的诱饵则专门操纵防御所使用的注意力排名。这种攻击无关的方法可以与任何现有的对抗性补丁攻击轻松集成，并采用分层目标来提升目标令牌的注意力。

Abstract: Vision Transformers (ViTs) remain vulnerable to localized adversarial attacks, e.g., adversarial patches, while recent test-time defenses mitigate them by suppressing image tokens with abnormally high attention scores. These defenses exploit a strong coupling between attention and adversarial effectiveness: adversarial tokens often need to attract substantial attention to influence the prediction. We introduce adversarial decoys, independently optimized image patches that redirect the attention, and therefore related defenses, toward selected target tokens. Rather than jointly optimizing misclassifications and defense evasion, our approach decouples the two objectives: the original adversarial region induces the incorrect prediction, while a separate decoy manipulates the attention ranking used by the defense. A layer-wise objective increases target-token attention and promotes these tokens above competing non-target ones. Since the decoy is optimized independently of the underlying attack, the method is attack-agnostic and can be easily integrated with any existing adversarial patch attack. Experiments on ImageNet across multiple ViT architectures and attacks show that decoys can redirect high attention scores away from the true adversarial region while preserving much of the attack effectiveness. These results reveal a fundamental limitation of using attention magnitude as an indicator of adversarial relevance.

[14] GIRAF: Towards Generalizable Human Interactions with Articulated Objects cs.CVPDF

Xiaohan Zhang, Sebastian Starke, Alexander Winkler, Federica Bogo, Samir Aroudj

TL;DR: 本文提出GIRAF模型，一种基于文本条件的扩散模型，用于生成逼真的全身人体与铰接物体的交互动作。该模型通过物体中心表示、混合域训练策略和基于接触的数据增强方案，解决了现有方法在协调全身运动、精细接触和物体铰接推理方面的局限性。

Details

Motivation: 现有模型在生成全身人体与铰接物体的交互动作方面存在不足，要么局限于静态物体的简单活动，要么仅关注手部操作。本文旨在解决生成协调、逼真且可泛化的全身运动（包括接近、操作和移动铰接物体）这一开放问题，核心难点在于对运动、精细接触和物体铰接的联合推理，以及大规模配对运动-场景数据的稀缺。

Result: 实验表明，该方法在未见过的物体配置上表现出强大的泛化能力，超越了当前最先进的方法。

Insight: 创新点包括：采用物体中心表示统一手-物体接触与物体表面；设计混合域训练策略平衡运动与交互；提出基于接触的数据增强方案以扩展训练多样性。这些设计有助于模型学习跨物体几何形状的细微手-物对应关系，并实现从导航到操作的无缝过渡。

Abstract: Synthesizing realistic full-body human interactions with articulated objects is a fundamental challenge for embodied AI and graphics, with applications in robotics training and virtual agents. Existing models remain limited: some focus on simple activities with static objects, while others restrict attention to hand-only manipulation. This leaves open the problem of generating coordinated full-body motion that approaches, manipulates, and moves articulated objects in a realistic and generalizable way. The key difficulty lies in reasoning jointly about locomotion, fine-grained contact, and object articulation. Models must capture subtle hand-object correspondences that transfer across object geometries, while also producing seamless transitions from navigation to manipulation. At the same time, the scarcity of large-scale paired motion-scene data makes it difficult to generalize across diverse object positions and shapes. We introduce a text-conditioned diffusion model that addresses these challenges through three core ideas: an object-centric representation that unifies hand-object contact with object surfaces, a mixed-domain training strategy that balances locomotion and interaction, and a contact-based augmentation scheme that expands training diversity. Through experiments, our method demonstrated strong generalization to unseen object configurations, surpassing current state-of-the-art methods.

Zixin Guo, Yehonathan Litman, Yifeng He, John Miller, Chuhan Chen

TL;DR: LightCrafter提出了一种混合视频重光照方法，将视频重光照任务重新定义为对代理视频的翻译。该方法首先生成输入视频在目标光照下的PBR渲染作为代理，然后利用视频扩散模型将其精炼为最终结果，从而在保持长时间一致性的同时实现更精细的光照控制。

Details

Motivation: 现有视频重光照方法存在两大局限：基于逆向渲染的方法重建噪声大且难以处理全局光照等复杂效果；基于生成式视频到视频翻译的方法则受限于光照控制能力和长时间稳定性。本文旨在结合两者优势，解决视频重光照中物理准确性与时序一致性的平衡问题。

Result: 该方法在现有真实世界重光照基准测试中超越了最先进方法，并贡献了一个用于进一步分析的合成基准。PBR渲染代理本身已优于部分先前方法，而结合精炼后模型能更好地捕捉全局光照等效果。

Insight: 核心创新在于将重光照任务解耦为PBR代理生成和视频精炼两个阶段，将光照条件直接烘焙到代理中，从而简化了扩散模型的学习任务。通过在后训练中结合合成视频对和真实无配对视频，有效利用了视频生成模型的光度先验知识。

Abstract: Video relighting requires balancing long-form temporal consistency with a physically grounded understanding of light transport, which depends on accurate estimation of intrinsic scene properties such as materials, geometry, and illumination. Existing methods follow two paradigms: (1) reconstruct a video’s photometric properties via inverse rendering and relight them to a target illumination via forward rendering, using physically-based rendering (PBR) or a neural renderer; these suffer from noisy reconstructions and struggle with hard-to-model effects such as global illumination. (2) Frame the task as generative video-to-video translation conditioned on relighting targets (a target environment map or text); this limits relighting control and temporal stability, since diffusion models struggle to translate long-form videos, and is constrained by the availability of input/relit training pairs. We propose LightCrafter, a hybrid pipeline that reformulates video relighting as video translation of a proxy video: rather than translating the input video directly to the target, we translate a PBR rendering of the input under the target illumination to the final target. This bakes illumination targets into the PBR proxy, removing the need to teach the diffusion model illumination concepts like environment maps, and enables more intricate lighting control while naturally providing long-form temporal consistency. We show PBR renders alone already outperform some prior art but struggle with effects like global illumination; to capture these, we leverage photometric priors in video generation models by post-training CogVideoX on synthetic video pairs and real-world unpaired videos. We outperform prior state-of-the-art on existing real-world relighting benchmarks and contribute a synthetic benchmark for further analysis. We will release our dataset, benchmark, metrics, and code.

[16] SAGA: Stable Acceleration Guidance for Autoregressive Video Generation cs.CVPDF

Thanh-Nhan Vo, Trong-Thuan Nguyen, Trung-Hoang Le, Tam V. Nguyen, Minh-Triet Tran

TL;DR: 论文提出了一种名为SAGA的训练免费稳定加速引导方法，用于解决自回归视频扩散模型中因重复使用生成潜在表示作为因果上下文而导致的时间错误放大问题，如闪烁、运动抖动和结构漂移。该方法通过加速度域谱引导目标和结构化自回归噪声初始化策略，在不重新训练或修改骨干网络的情况下，直接应用于现有的分块自回归扩散模型，以提升时间质量。

Details

Motivation: 自回归视频扩散模型在高效流式处理和生成长视频方面具有优势，但重复使用生成的潜在表示作为因果上下文会放大时间误差，导致闪烁、运动抖动和结构漂移等不稳定问题。论文旨在从谱运动学角度分析这一失败模式，并解决自回归视频生成中的时间不稳定性。

Result: 在多个自回归扩散模型上的广泛实验表明，SAGA能一致地提升时间质量。例如，在Self-Forcing模型上，SAGA将时间质量从97.30提升到97.91，图像质量从69.60提升到70.51。谱分析和人类偏好研究也证实SAGA减少了时间不稳定性，同时保持了视觉保真度。

Insight: 创新点在于从谱运动学视角识别离散潜在加速度作为揭示不稳定高频时间扰动的有效信号，并提出了基于有限窗口Slepian投影的加速度域谱引导目标和结构化自回归噪声初始化策略。这些方法无需重新训练，可直接集成到现有模型中，为提升自回归视频生成的时间稳定性提供了可借鉴的思路。

Abstract: Autoregressive video diffusion enables efficient streaming and long-horizon video generation, but repeatedly reusing generated latents as causal context can amplify temporal errors, resulting in flickering, motion jitter, and structural drift. In this paper, we investigate this failure mode from a spectral kinematic perspective and identify discrete latent acceleration as an effective signal for revealing unstable high-frequency temporal perturbations. To this end, we propose SAGA, a training-free \textbf{\textit{s}}table \textbf{\textit{a}}cceleration \textbf{\textit{g}}uidance approach for \textbf{\textit{a}}utoregressive video generation. SAGA integrates an acceleration domain spectral guidance objective based on finite-window Slepian projections with a structured autoregressive noise initialization strategy that suppresses short-range temporal correlations while preserving long-range motion structure. Without retraining or modifying the backbone, SAGA can be directly applied to existing chunk-wise autoregressive diffusion models, which is the prevalent setting for high-quality generation. Extensive experiments show that SAGA consistently improves temporal quality across multiple autoregressive diffusion models. On Self-Forcing, SAGA improves Temporal Quality from 97.30 to 97.91 and Image Quality from 69.60 to 70.51. Moreover, spectral analysis and human preference studies demonstrate that SAGA reduces temporal instability while maintaining visual fidelity.

[17] APIVOT: Adaptive Planning with Interleaved Vision-Language Thoughts cs.CV | cs.AI | cs.LG | cs.ROPDF

Emily Jin, Joy Hsu, Yiqing Xu, Weiyu Liu, Nick Haber

TL;DR: APIVOT是一种基于视觉语言模型（VLM）的机器人规划器，通过自适应地交织语言和视觉思维来解决长时程规划问题。它利用语言进行语义推理，同时使用视觉思维作为对未来状态的想象，以内部验证几何可行性。在长时程厨房任务中，APIVOT在空间受限场景下表现优于通用VLM和现有规划框架。

Details

Motivation: 长时程机器人规划需要同时推理语义任务结构和几何可行性，机器人必须分解目标、选择任务相关对象并序列化动作，同时确保计划满足空间约束（如有限自由空间和物体碰撞）。

Result: 在长时程厨房任务中，APIVOT超越了通用VLM和先前的规划框架，在空间受限设置中取得了最大增益。

Insight: APIVOT的创新点在于自适应地交织语言和视觉思维，让模型学习有意义的模态选择行为，从而提升规划成功率和推理效率。这为结合语义与几何推理的机器人规划提供了新思路。

Abstract: Long-horizon robot planning requires jointly reasoning over semantic task structure and geometric feasibility. To successfully execute a task, a robot must decompose goals, select task-relevant objects, and sequence actions, while ensuring that plans satisfy spatial constraints such as limited free space and object collisions. In this work, we propose APIVOT, a VLM-based planner that adaptively interleaves language and visual thoughts for long-horizon planning. APIVOT learns to leverage language for semantic reasoning, while using visual thoughts as imagined future states for internal verification of geometric feasibility. On long-horizon kitchen tasks, APIVOT outperforms general-purpose VLMs and prior planning frameworks, achieving the largest gains in spatially constrained settings. We find that APIVOT learns meaningful modality selection behavior, demonstrating that adaptive interleaving of vision-language thoughts improves both planning success and reasoning efficiency.

[18] UAV-OVVIS: Unmanned Aerial Vehicles Also Need Open-Vocabulary Video Instance Segmentation cs.CVPDF

Mingyu Dou, Shi Qiu, Ming Hu, Yifan Chen, Zhe Sun

TL;DR: 本文提出了无人机开放词汇视频实例分割（UAV-OVVIS）新任务，旨在根据开放词汇查询发现无人机视频中的目标，并输出具有全局一致身份的实例级分割轨迹。为解决无人机场景实例级标注稀缺问题，作者提出了一个无需训练的通用框架AeroTrack，该框架基于周期性开放词汇检测、短片段掩码传播和跨片段身份统一，并复用了现有的视觉基础模型。

Details

Motivation: 现有无人机视频感知主要依赖于预定义类别下的框级定位和轨迹关联，难以在开放场景中同时支持灵活查询和细粒度的实例级动态理解。

Result: 实验表明，AeroTrack在无人机场景下显著优于现有的通用视频实例分割方法，并在构建的AeroVIS基准（包含9个类别和8,279条轨迹）上表现出强大的开放词汇鲁棒性和泛化能力。

Insight: 创新点在于将开放词汇能力引入无人机视频实例分割，并提出了一个无需训练、复用现有基础模型的统一框架，通过解耦检测、分割和跟踪模块来应对标注稀缺问题，为相关研究提供了新的任务定义和基准。

Abstract: Unmanned Aerial Vehicle (UAV) videos are widely used in traffic monitoring, urban management, and emergency rescue. However, existing UAV video perception mainly relies on box-level localization and trajectory association under predefined categories, making it difficult to simultaneously support flexible queries and fine-grained instance-level dynamic understanding in open scenarios. To this end, we introduce a new task, UAV Open-Vocabulary Video Instance Segmentation (UAV-OVVIS), which discovers targets in UAV videos according to open-vocabulary queries and outputs instance-level segmentation trajectories with globally consistent identities. Considering the scarcity of instance-level annotations in UAV scenarios, we propose AeroTrack, a training-free unified framework. AeroTrack centers on periodic open-vocabulary detection, short-segment mask propagation, and cross-segment identity unification, reusing existing visual foundation models to enable UAV-OVVIS. Based on this framework, we instantiate five AeroTrack variants and construct AeroVIS, an evaluation benchmark for UAV-OVVIS containing 9 UAV object categories and 8,279 trajectories. Experiments show that AeroTrack substantially outperforms existing general video instance segmentation methods in UAV scenarios and demonstrates strong open-vocabulary robustness and generalization. To support future research, we release AeroTrack and AeroVIS as a unified framework and benchmark for UAV-OVVIS.

[19] Post-Training in End-to-End Autonomous Driving cs.CV | cs.ROPDF

Ruining Yang, Muxing Wang, Yixiao Chen, Tongfei Guo, Yi Xu

TL;DR: 本文是一篇关于端到端自动驾驶中后训练技术的综述论文。作者首先指出端到端模型在自动驾驶领域日益重要，但传统开环模仿专家演示的方法存在局限性，如小误差累积、训练数据中恢复行为稀缺，以及无法捕获安全和驾驶舒适性等长期目标。因此，研究转向后训练技术以改进驾驶策略。论文通过定义后训练的范围，将现有文献根据监督形式分为四大类，并讨论了每类的能力、局限性和开放挑战，旨在促进对该新兴领域的系统理解并激发未来研究。

Details

Motivation: 自动驾驶车辆在安全关键和交互密集的环境中运行，传统开环模仿专家演示的方法不足以确保可靠性，因为小误差会随时间累积，且训练数据中缺乏恢复行为，同时点状标签无法捕获安全和驾驶舒适性等长期目标。这些局限性促使研究转向后训练技术，以超越纯模仿进一步优化驾驶策略。

Result: 本文是一篇综述，未提供具体实验数据或基准测试结果，但通过组织现有文献，系统分析了后训练技术在自动驾驶中的应用，旨在为未来研究提供指导。

Insight: 论文的创新点在于首次提出对自动驾驶端到端模型后训练技术的统一视图，将其定义为超越纯模仿的进一步优化，并根据监督形式将现有方法分类为四大类，从而系统梳理了该领域的研究现状、能力和挑战。从客观角度看，这种分类框架有助于研究人员更好地理解和比较不同后训练方法，推动更可靠和高效的自动驾驶系统发展。

Abstract: End-to-end models that map multimodal inputs directly to future trajectories/maneuvers have emerged as an increasingly prominent research paradigm in autonomous driving. This class of models includes both Vision-Language-Action models and trajectory-generative planners. Unlike classic machine learning applications, autonomous vehicles operate in safety-critical and interaction-intensive environments where traditional open-loop imitation of expert demonstrations is not sufficient to ensure reliability. In particular, small execution errors can accumulate over time, while recovery behaviors are scarce in training data. In addition, long-horizon objectives such as safety and driving comfort are not captured by pointwise labels either. These limitations have motivated a shift toward post-training techniques, which further refine driving policies beyond pure imitation. This survey presents a unified view of post-training for autonomous driving by defining its scope and organizing the existing literature into four major families based on the form of supervision they use. For each family, we discuss its capabilities, limitations, and open challenges. We aim to facilitate a systematic understanding of this emerging area and stimulate future research on reliable and efficient post-training for autonomous driving.

[20] Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratio cs.CV | cs.ROPDF

Utkarsh A. Mishra, Yongxin Chen, Danfei Xu, Yang Liu, Xi Chen

TL;DR: 本文研究了生成式视频基础模型在微调为机器人动作模型后出现的视频-动作泛化差距，并提出了一种基于注意力机制的时间比率（TR）度量方法。研究发现TR能预测模型的组合泛化能力，并随任务阶段动态变化，进而提出了一种推理时自适应引导方法，在LIBERO基准和真实任务上有效缓解了分布外组合泛化差距。

Details

Motivation: 生成式视频基础模型虽具有强组合先验，但在微调为机器人动作模型后常丢失这些先验，导致视频-动作泛化差距，本文旨在理解和缓解这一差距。

Result: 在LIBERO基准和真实世界任务上的评估表明，所提出的推理时自适应引导方法缓解了分布内与分布外的组合泛化差距。

Insight: 创新点在于引入了时间比率（TR）这一注意力度量来量化模型对未来潜在状态的依赖，并利用其动态特性设计自适应引导，以在规划阶段增强组合视频条件信号，这为理解模型泛化行为提供了新视角。

Abstract: Generative video foundation models exhibit strong compositional priors, yet world-action models (WAMs) and video-action models (VAMs) often lose these priors after finetuning on robotic action data. We refer to this discrepancy as the video-action generalization gap. In this paper, we systematically investigate this gap by evaluating a comprehensive design space of VAMs, demonstrating that standard design choices yield no emergent explanation pattern. To explain this behavior, we introduce the Temporal Ratio (TR), an attention-based measure of how strongly the action head relies on future latent rollouts relative to the anchored current frame. TR has two key properties: first, a model’s structural reliance on future-predictive latents, measured via TR, acts as a predictor of its compositional generalization capacity; second, it natively fluctuates based on task phase, shifting attention to future frames during planning and reverting to the present frame for precise manipulation. Finally, based on these findings, we propose an inference-time adaptive guidance method, which exploits this intrinsic feature attention pattern to dynamically amplify compositional video conditioning signals precisely when the policy relies on future rollouts. Evaluated on the LIBERO benchmark and real-world tasks, our approach mitigates the OOD-ID compositional generalization gap. More details: https://umishra.me/temporal-ratio/

[21] VSRo-200: A Romanian Visual Speech Recognition Dataset for Studying Supervision and Multimodal Robustness cs.CVPDF

Iulia-Maria Udrea, Alexandra Diaconu, Bogdan Alexe

TL;DR: 本文介绍了VSRo-200，这是首个用于罗马尼亚语视觉语音识别（唇读）的大规模数据集，包含200小时的真实播客视频。所有样本都通过微调的罗马尼亚语ASR模型生成的伪标签进行标注，其中100小时的子集还进行了人工转录，从而能够在统一框架下对监督质量进行可控分析。基于此数据集，作者为低资源环境下的视觉语音识别建立了基准，系统研究了监督质量的影响，并评估了领域偏移下的鲁棒性以及噪声条件下的视听语音识别。

Details

Motivation: 动机是解决罗马尼亚语等低资源语言缺乏大规模视觉语音识别数据集的问题，以便研究监督质量、领域泛化和多模态融合等关键课题。

Result: 实验表明，在固定数据规模下，人工标注能提供更好的性能，但伪标签通过可扩展性实现了持续改进。在噪声条件下，多模态融合相比纯音频模型显著提高了鲁棒性。此外，在VSRo-200上学习到的表征能有效迁移到孤立词识别基准LRRo上，大幅超越了先前报告的结果。

Insight: 创新点在于创建了首个罗马尼亚语大规模视觉语音识别数据集，并提供了统一的框架来系统比较伪标签与人工标注的监督质量。研究还深入分析了多模态融合在噪声条件下的鲁棒性优势，以及表征在跨任务迁移中的有效性，为低资源视觉语音识别研究提供了新的测试平台。

Abstract: We introduce VSRo-200, the first large-scale dataset for visual speech recognition (lip reading) in Romanian, comprising 200 hours of real-world podcast videos. All samples are annotated with pseudo-labels generated by a fine-tuned Romanian ASR model, while a subset of 100 hours is additionally transcribed by humans, enabling controlled analysis of supervision quality under a unified framework. Building on this dataset, we establish a benchmark for visual speech recognition in low-resource settings. We systematically study the impact of supervision quality, showing that while human annotations provide better performance at fixed data scales, pseudo-labels enable continued improvements through scalability. We further evaluate robustness under domain shift using curated out-of-distribution (OOD) test sets, and analyze audio-visual speech recognition (AVSR) under noisy conditions, where multimodal fusion significantly improves robustness compared to audio-only models. Finally, we demonstrate that representations learned on VSRo-200 transfer effectively to the LRRo benchmark for isolated word recognition, substantially outperforming previously reported results. Overall, VSRo-200 provides a new testbed for studying supervision, domain generalization, and multimodal fusion in low-resource visual speech recognition.

[22] Unified Face Attack Detection via Fine-Grained Semantic Guidance cs.CVPDF

Ning Jiang, Shijie Yu, Dingheng Zeng, Haiyang Yi, Yanhong Liu

TL;DR: 本文提出了一种统一的面部攻击检测方法，通过细粒度语义引导来增强检测性能。研究基于包含超过800万张攻击图像的大规模MS-UFAD数据集，为每张图像添加了详细的伪造线索文本描述，并设计了双对齐伪造网络（DAF-Net）来有效利用这些文本信息。实验表明，该方法能从攻击图像中提取更具泛化性和语义意义的伪造表示，优于仅基于视觉的方法和粗粒度描述的方法。

Details

Motivation: 随着人脸识别系统的广泛应用，安全威胁日益多样化，现有数据集缺乏伪造线索的详细文本描述，导致先前方法主要将面部攻击检测视为视觉识别任务，限制了检测性能的提升。

Result: 在MS-UFAD数据集上的大量实验证明，所提出的DAF-Net方法提取的伪造表示更具泛化性和语义意义，性能超越了仅基于视觉的方法和基于粗粒度描述的方法，达到了先进水平。

Insight: 创新点在于首次为大规模攻击图像数据集添加细粒度文本描述，并设计双对齐网络实现视觉与文本信息的深度融合，这为多模态伪造检测提供了新思路，可借鉴其利用语义引导增强模型泛化能力的方法。

Abstract: The growing applications of facial recognition systems are accompanied by increasingly diverse security threats. Existing datasets lack detailed textual descriptions of forgery cues, leading most prior methods to treat face attack detection primarily as a visual recognition task. In this paper, building upon the large-scale MS-UFAD dataset which contains over 8 million attack images, we enrich each image with a fine-grained textual description of forgery cues. Furthermore, we propose a Dual Alignment Forgery Network(DAF-Net) to better leverage these textual information. Extensive experiments demonstrate that our approach extracts more generalizable and semantically meaningful forgery representations from attack images, outperforming both vision-only methods and approaches based on coarse-grained descriptions.

[23] Leveraging Color Naming for Image Enhancement cs.CV | cs.AIPDF

David Serrano-Lozano, Luis Herranz, Michael S. Brown, Javier Vazquez-Corral

TL;DR: 本文提出了一种名为NamedCurves+的图像增强方法，该方法将颜色命名概念引入基于学习的框架，通过色调曲线对每个命名颜色进行全局调整，并结合Transformer块捕获空间依赖性以实现上下文感知的局部编辑。该方法旨在提升图像增强过程的可解释性和用户交互性。

Details

Motivation: 解决现有深度学习方法在图像增强中存在的两个关键问题：可解释性不足，以及缺乏适合用户调整的参数化方式。

Result: 在图像润饰、色调映射和曝光校正等任务上的大量实验表明，NamedCurves+的性能优于最先进的方法。

Insight: 创新点在于将广泛用于软件工具的直观颜色命名概念与基于学习的色调曲线调整相结合，并通过Transformer实现空间上下文建模，从而同时实现了方法的可解释性（色调曲线明确显示每种颜色对增强的贡献）和交互性（允许用户自定义调整）。

Abstract: Enhancing images to make them visually appealing is a persistent challenge in computer vision. Many deep-learning methods train models on paired datasets to replicate expert editing styles. However, these approaches struggle with two key issues: (1) interpretability and (2) a parametrization suitable for user adjustments. To address these challenges, we present NamedCurves+, an approach inspired by the concept of Color Naming, a universal set of familiar colors widely used in software tools for intuitive editing. Our method integrates color names into a learning-based framework, enabling global adjustments for each named color through tone curves. To address local image variations, we incorporate a transformer block that captures spatial dependencies, enabling context-aware edits across the image. NamedCurves+ enhances the retouching process’s interpretability and supports user interaction, allowing flexible modifications of individual tone curves to refine the retouched image according to personal preferences. Extensive experiments on tasks such as image retouching, tone mapping, and exposure correction demonstrate that NamedCurves+ outperforms state-of-the-art methods. Notably, our approach is both explainable, as the tone curves explicitly represent how each color name contributes to the enhancement, and interactive, allowing users to customize the retouching process and achieve results tailored to their liking.

[24] LEEVLA: Seeing What Matters in Latent Environment Evolution for Vision-Language-Action cs.CV | cs.AIPDF

Qi Lyu, Baicheng Liu, Xudong Wang, Jiahua Dong, Lianqing Liu

TL;DR: 本文提出了LEEVLA，一种用于视觉-语言-动作（VLA）模型的新架构，旨在通过显式引导模型关注任务关键证据并保持潜在世界表征的结构化演化，来解决现有方法在复杂动态场景中表现不佳的问题。其核心是名为DGDP和SFFG的组件构成的‘where-how’训练框架，前者指导模型关注何处，后者建模特征应如何演化。

Details

Motivation: 现有VLA模型在处理复杂动态场景时存在不足，因为它们对所有视觉标记进行统一处理，并依赖人工选择的因素进行推理，缺乏强调任务关键证据的机制，且忽略了潜在的环境演化因素。

Result: 在多个VLA基准测试上的广泛实验表明，LEEVLA始终优于现有方法，证实了显式的任务证据引导和结构化的潜在推理对于可扩展的VLA模型至关重要。

Insight: 创新点在于提出了一个集成的‘where-how’训练框架：通过结合动态位置优先级和语义漂移指导的DGDP来识别关键区域，以及通过原型到外围预测和互邻域对比损失来结构化建模特征演化的SFFG。这为VLA模型提供了更精细的感知和推理机制。

Abstract: Vision-language-action (VLA) models aim to map multimodal inputs to robot actions. However, most existing approaches struggle to cover complex dynamic scenarios due to treating all visual tokens uniformly and reasoning with human-selected factors, which lack mechanisms to emphasize task-critical evidence and ignore underlying factors. To address this issue, we propose LEEVLA, a VLA architecture for seeing what matters in Latent Environment Evolution that explicitly guides the model toward informative regions while preserving the structured evolution of latent world representations. To identify salient and instruction-relevant regions, we introduce drift-guided dynamic prioritization (DGDP), which combines dynamic position prioritization (DPP) with semantic drift guidance (SDG) to guide the VLA agent where to attend during training. On top of this, we introduce structured feature flow generation (SFFG), which models how these prioritized features should evolve in latent space via prototype-to-periphery (P2P) prediction, and a mutual-neighborhood contrastive (MC) loss to maintain topological consistency among neighborhoods. Together, DGDP and SFFG form a task-aware “where-how” training framework. Extensive experiments on VLA benchmarks show that LEEVLA consistently outperforms prior methods, confirming that explicit task-evidence guidance and structured latent reasoning are both crucial for scalable VLA. Our code is available at https://github.com/LyuQi127/LEEVLA.

[25] Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions cs.CVPDF

Sarthak Kumar Maharana, Shambhavi Mishra, Yunbei Zhang, Shuaicheng Niu, Taki Hasan Rafi

TL;DR: 这篇论文是一篇关于持续测试时适应（CTTA）的全面综述。它正式定义了CTTA问题，分析了不同的持续域偏移模式，并提出了一个将现有方法分为三大类（基于优化的策略、参数高效方法、基于架构的方法）的层次化分类法。文章系统回顾了各类方法，提供了比较基准和实验结果，并讨论了当前方法的局限性和未来的研究方向。

Details

Motivation: 解决深度神经网络在现实世界部署中，面对持续分布漂移时性能下降的问题。CTTA旨在无需源数据或目标域标签的情况下，动态适应预训练模型，同时缓解灾难性遗忘和长期噪声伪标签导致的错误累积这两个关键失效模式。

Result: 论文提供了跨标准评估设置的比较基准和实验结果，系统性地回顾和比较了各类代表性方法。

Insight: 创新点在于对CTTA领域进行了系统性的梳理，提出了一个清晰的分类法，并指出了未来研究方向，如基础模型和黑盒系统的适应。这为构建鲁棒的持续在线适应系统提供了路线图。

Abstract: Deep neural nets achieve remarkable performance when training and test data share the same distribution, but this assumption frequently breaks in real-world deployment, where data undergoes continual distributional shifts. Continual Test-Time Adaptation (CTTA) addresses this challenge by adapting pretrained models to non-stationary target distributions on-the-fly, without access to source data or labeled targets, while mitigating two critical failure modes: catastrophic forgetting of source knowledge and error accumulation from noisy pseudo-labels over extended time horizons. In this comprehensive survey, we formally define the CTTA problem, analyze the diverse continual domain shift patterns that characterize different evaluation protocols, and propose a hierarchical taxonomy that categorizes existing methods into three families: optimization-based strategies (entropy minimization, pseudo-labeling, parameter restoration), parameter-efficient methods (normalization layer adaptation, adaptive parameter selection), and architecture-based approaches (teacher-student frameworks, adapters, visual prompting, masked modeling). We systematically review representative methods within each category and present comparative benchmarks and experimental results across standard evaluation settings. Finally, we discuss limitations of current approaches and highlight emerging research directions, including adaptation of foundation models and black-box systems, providing a roadmap for future research in robust continual test-time adaptation. We encourage visiting our repository at https://github.com/sarthaxxxxx/Awesome-Continual-Test-Time-Adaptation

[26] Dual-Correlation Hypergraph Network for Unaligned RGBT Video Object Detection and A Large-scale Benchmark cs.CVPDF

Qishun Wang, Yapeng Li, Bin Luo, Zhengzheng Tu, Chenglong Li

TL;DR: 本文提出了一种双相关超图网络（DHNet）来解决RGB-热成像（RGBT）视频目标检测中的空间未对齐问题，并构建了一个大规模、场景多样的基准数据集DVT-VOD1000。DHNet通过显式建模时间相关性和跨模态空间相关性来捕获高维互补信息，在现有和新建的数据集上均达到了最先进的检测精度。

Details

Motivation: RGB-热成像视频目标检测在恶劣条件下能克服传统RGB检测的局限，但RGB与热成像图像对之间普遍存在空间未对齐问题，且领域缺乏大规模、场景多样的基准数据集进行全面评估。

Result: 在VT-VOD50和新建的DVT-VOD1000数据集上的综合实验表明，DHNet实现了最先进的检测精度。

Insight: 创新点包括：提出基于局部区域的顺序空间对齐模块（PSAM）来对齐多模态特征；引入双超图融合模块（DHFM），通过构建独立的时间和跨模态超图进行双相关学习以增强目标判别性；贡献了一个包含1000个视频序列、涵盖多种场景的大规模RGBT VOD数据集DVT-VOD1000。

Abstract: RGB-Thermal (RGBT) Video Object Detection (VOD) has gained significant traction due to its ability to overcome the limitations of conventional RGB-based VOD under challenging conditions. However, spatial misalignment commonly exists between RGBT image pairs. To address this, we propose a Dual-Correlation Hypergraph Network (DHNet) that captures high-dimensional complementary information by explicitly modeling two types of correlations: temporal correlation across consecutive frames and spatial correlation from cross-modal features. Specifically, we first design a Patch-based Spatial Alignment Module (PSAM) to sequentially align the multimodal features at the local region level. Subsequently, we introduce a Dual Hypergraph Fusion Module (DHFM), which constructs separate temporal and multimodal hypergraphs to enhance object discriminability through dual-correlation learning. Furthermore, the field currently lacks a large-scale, scene-diverse benchmark dataset for comprehensive evaluation. To address this gap, we construct DVT-VOD1000, a large-scale RGBT VOD dataset containing 1,000 video sequences with 103,464 RGBT image pairs. The dataset covers diverse scenarios, including campuses, parks, transportation, rural areas, night scenes, rain, and snow. Comprehensive experiments on VT-VOD50 and our DVT-VOD1000 demonstrate that DHNet achieves state-of-the-art detection accuracy. The dataset and source code will be made publicly available on https://github.com/tzz-ahu/ to support academic research.

[27] Dive Into the Implicit Biases of Low-rank Vision-language Alignment cs.CVPDF

Mingjia Shi, Shuo Wang, Xiaobo Wang, Sifan Zhou, Kai Wang

TL;DR: 本文挑战了视觉-语言对齐阶段需全参数更新的传统观点，探究了在该阶段对LLM应用低秩适配（LoRA）的效果。研究发现低秩对齐不仅降低计算成本，且在多数基准上优于全参数对齐，并通过实证、几何和理论分析揭示了其引入的结构保持等隐式偏置。

Details

Motivation: 动机是质疑视觉-语言对齐阶段必须进行全参数预训练这一普遍做法，探索使用低秩适配能否更高效且有效地完成对齐，并深入理解其背后的机制。

Result: 在多个基准测试中，低秩对齐超越了全参数对齐的性能；实验覆盖了超过100种对齐配置、三类低秩算子及不同秩、编码器等设置，验证了其鲁棒性和优越性。

Insight: 创新点在于系统揭示了低秩对齐的隐式偏置：它使模型行为从幻觉转向保守，保持视觉特征的逐令牌线性可分性（避免LS-curse），几何上表征更均匀稳定，理论上偏好梯度平坦和扰动鲁棒的子空间，为结构保持行为提供了原理性解释。

Abstract: Vision-language alignment, the stage that bridges pretrained vision encoders and large language models, is widely treated as a form of pretraining requiring full-parameter updates. We challenge this view and investigate what happens when low-rank adaptation is applied to the LLM during this stage instead. We find that low-rank alignment not only reduces computational costs but also outperforms full-parameter alignment on most benchmarks. To understand this phenomenon, we systematically characterize the implicit biases introduced by low-rank adaptation during alignment. Empirically, we find that low-rank alignment shifts model behavior from hallucinatory to conservative and preserves per-token linear separability of visual features that full-parameter alignment disrupts, a phenomenon we term LS-curse. Geometrically, low rank aligned models exhibit more homogeneous and structurally stable visual representations, maintaining modality-specific knowledge rather than prematurely fusing entity-level semantics. Theoretically, we establish two theorems showing that low-rank alignment induces preferences for parameter subspaces with flat gradients and feature subspaces robust to perturbations, providing a principled explanation for the observed structure-preserving behavior. Extensive experiments cover ablation over 100 alignment configurations, three families of low-rank operators, and various rank, encoder, and other settings.

[28] TMI: Text-to-Image Meets Image-to-Image for Complementary Data Synthesis to Boost Long-Tailed Instance Segmentation cs.CV | cs.AIPDF

Hyeonseop Song, Seokhun Choi, Hoseok Do

TL;DR: 本文提出了一种结合文本到图像（T2I）生成与上下文感知图像到图像（I2I）编辑的混合数据合成方法TMI，旨在解决大规模词汇实例分割中长尾分布和类间模糊性的问题。该方法通过T2I分支提供类别和场景多样性，并引入名为VRAIN的I2I编辑器，将高置信度的稀有类别实例插入到野外场景的语义合适位置，以生成语义连贯、视觉自然的合成数据。

Details

Motivation: 大规模词汇实例分割面临长尾类别分布和细粒度类间模糊性的挑战。现有数据合成方法存在互补的局限性：T2I方法继承了噪声伪标签且在稀有类别上表现不佳，而复制粘贴方法则损害了上下文真实性。本文旨在结合两者的优势，提出一种混合管道来解决这些问题。

Result: 在LVIS基准测试上，该方法超越了现有基线，将整体AP提升了高达+4.0点，并将稀有类别的AP提升了高达+9.5点。同时，该方法能有效随骨干网络容量扩展，展现出优越的性能。

Insight: 主要创新点在于将T2I的广泛多样性与I2I的上下文编辑能力相结合，并引入了VRAIN这一新颖的I2I编辑器，专门针对稀有类别进行高置信度的、语义恰当的增强。从客观角度看，这种混合策略巧妙地利用了不同合成范式的互补优势，通过教师-学生方案确保标签可靠性，并专注于减少领域差距，为长尾实例分割提供了一种有效的数据增强解决方案。

Abstract: Large-vocabulary instance segmentation is constrained by long-tailed category distributions and fine-grained inter-class ambiguity. While data synthesis offers a promising alternative, current paradigms have complementary limitations: text-to-image (T2I) methods inherit noisy pseudo-labels and struggle on rare classes, whereas copy-paste methods compromise contextual realism. To address these issues, we propose a hybrid pipeline coupling T2I generation with context-aware image-to-image (I2I) editing. The T2I branch provides broad category and scene diversity, while a teacher-student scheme ensures label reliability by selectively retaining only prompt-specified categories. To strengthen supervision for rare classes, we introduce VRAIN (Verified Rare-class Augmentation via INstructed editing), a novel I2I editor. VRAIN inserts high-confidence instances at semantically appropriate locations within in-the-wild scenes, yielding semantically coherent and visually natural edits that reduce domain gaps and enable targeted augmentation. On the LVIS benchmark, our method surpasses existing baselines, improving overall AP by up to +4.0 points and rare-class AP by up to +9.5 points, while scaling effectively with backbone capacity. Our project page is available at https://seokhunchoi.github.io/TMI

[29] TVTA: Trajectory-Aware Viseme-Guided Temporal Aggregation for Event-Based Lip Reading cs.CVPDF

Jingrong Zheng, Hongwei Ren, Xiangqian Wu

TL;DR: 本文提出了一种用于基于事件的唇读任务的时序增强框架TVTA，通过轨迹感知差分聚合（TDA）在空间聚合前进行局部时序建模，以及通过音素引导聚合（VGA）模块引入音素感知的序列监督，并结合EMA师生训练策略增强鲁棒性。

Details

Motivation: 现有基于事件的唇读方法通常在充分时序建模前进行空间压缩，可能抑制对区分相似唇部动作至关重要的稀疏局部运动轨迹，且其时序表征优化主要在词分类层面，对底层发音结构约束较弱。

Result: 在DVS-Lip基准上的实验验证了所提设计的有效性，广泛的消融研究进一步证实了TDA、VGA和师生一致性策略的贡献。

Insight: 创新点在于提出先进行局部时序建模再进行自适应空间聚合的TDA模块，以及通过CTC解码器和音素引导门控聚合分支构成的VGA模块来注入音素感知的序列监督，从而更好地建模唇部运动的时序轨迹和发音结构。

Abstract: Event-based lip reading has recently emerged as a promising direction for visual speech recognition, benefiting from the high temporal resolution and motion sensitivity of event cameras. However, existing methods typically perform spatial compression before sufficient temporal modeling, which may suppress sparse and localized motion trajectories that are crucial for distinguishing similar lip movements. Moreover, most current approaches optimize temporal representations mainly at the word-classification level, leaving the underlying articulatory structure weakly constrained. To address these limitations, we propose a temporally enhanced framework for event-based lip reading. First, we introduce Trajectory-Aware Differential Aggregation (TDA), which performs local temporal modeling at each spatial location before adaptive spatial aggregation. Second, we propose Viseme-Guided Aggregation (VGA), a unified temporal module composed of a CTC decoder and a viseme-guided gated aggregation branch, which injects viseme-aware sequence supervision and improves final temporal aggregation for word recognition. Third, we incorporate an EMA teacher–student training strategy to enhance robustness under strong event perturbations. Experiments on the DVS-Lip benchmark verify the effectiveness of the proposed design, and extensive ablation studies further validate the contributions of TDA, VGA, and teacher–student consistency. Qualitative decoding results also demonstrate that the proposed CTC-based temporal modeling learns meaningful viseme-aware structure from event streams.

[30] Multimodal 3D LUT Generation via StatLUT with Statistical Features for Photorealistic Style Transfer cs.CVPDF

Yifan Wang, Zhixiang Hao, Yu Wang, Congchao Zhu

TL;DR: 本文提出StatLUT，一个用于生成3D查找表（LUT）的多模态框架，旨在解决真实感风格迁移（PST）中的语义纠缠、颜色条带和缺乏文本驱动控制的问题。该方法通过Lab-Extractor提取与空间无关的统计特征，使用基于Transformer的序列到序列模型预测拓扑平滑的3D LUT，并引入轻量级扩散Transformer（H-Diffuser）实现从文本提示直接合成统计特征，从而实现灵活的多模态风格迁移。

Details

Motivation: 现有基于深度学习的真实感风格迁移方法存在预训练图像编码器导致的语义纠缠和空间失真问题，同时像素级映射范式忽略了色域拓扑结构，导致颜色条带，并且缺乏直观的文本驱动控制能力。

Result: 在标准基准测试上的大量实验表明，StatLUT在视觉质量和定量指标上均显著优于最先进（SOTA）的方法。

Insight: 创新点包括：1）绕过传统编码器，引入Lab-Extractor提取空间无关的统计特征，从根本上解耦颜色分布与结构语义；2）将LUT生成建模为基于Transformer的Seq2Seq翻译任务，使用多维残差映射器（MR-Mapper）预测拓扑平滑的3D LUT；3）提出轻量级扩散Transformer（H-Diffuser），实现从自然语言提示直接合成统计特征，从而突破单模态限制，实现文本驱动的灵活调色。

Abstract: Photorealistic Style Transfer (PST) aims to transfer the color and tonal style of a reference to a content image while strictly preserving its structural integrity. However, existing deep learning-based methods inherently suffer from semantic entanglement caused by pre-trained image encoders, leading to unnatural spatial distortions. Moreover, current pixel-level mapping paradigms often ignore color gamut topology, resulting in color banding, while also lacking the multimodal capability for intuitive text-driven control. To address these bottlenecks, we propose StatLUT, an innovative multimodal framework for 3D LUT generation. First, we bypass traditional encoders and introduce a Lab-Extractor to derive spatially-agnostic statistical features, fundamentally decoupling color distributions from structural semantics to ensure artifact-free rendering. Second, we formulate LUT generation as a Transformer-based Seq2Seq translation task, utilizing a Multi-dimensional Residual Mapper (MR-Mapper) to predict topologically smooth 3D LUTs. Finally, to break the single-modal barrier, we propose the H-Diffuser, a lightweight Diffusion Transformer that directly synthesizes statistical features from natural language prompts, enabling flexible text-driven color grading. Extensive experiments on standard benchmarks demonstrate that StatLUT significantly outperforms state-of-the-art methods in both visual quality and quantitative metrics, pioneering a highly robust and flexible paradigm for multimodal photorealistic style transfer.

[31] SkelGen4D: Weakly-Supervised Skeleton-Based 4D Generation for Text-Driven Mesh Animation cs.CVPDF

Hao Feng, Zhi Zuo, Jia-Hui Pan, Ka-Hei Hui, Zhengzhe Liu

TL;DR: 本文提出SkelGen4D，一个弱监督的前馈框架，用于从文本生成基于骨架的4D网格动画。该方法通过可微分拟合从动画网格中恢复时间一致的伪骨架，并以前馈方式生成文本条件化的骨架运动序列，再通过Motion-GRPO进行细化，最终合成高质量、可控的动画。

Details

Motivation: 研究4D生成以合成时间连贯的3D几何序列用于动画和内容创作，旨在克服现有基于SDS优化和视频驱动方法的局限，采用与标准工业流程对齐的骨架驱动动画框架，以实现显式控制和编辑。

Result: 在Truebones Zoo和Diffusion4D两个大规模基准测试上评估，结果表明，弱监督的骨架建模在扩展到多样对象类别时，其性能匹配或超越了全监督基线，实现了高质量的文本驱动网格动画。

Insight: 创新点在于提出弱监督的骨架运动生成框架，无需逐帧骨架标注，通过可微分拟合和Motion-GRPO细化确保时间连贯性、物理合理性和关节动画；该方法支持灵活的运动编辑，并与标准动画生产流程对齐，具有实际应用价值。

Abstract: We study 4D generation to synthesize temporally coherent sequences of 3D geometry for animation and content creation. In contrast to existing SDS-based optimization methods and video-driven animation approaches, we adopt a skeleton-driven animation framework aligned with standard industrial pipelines, which enables explicit control and editing. To this end, we propose SkelGen4D, a weakly supervised feed-forward framework for text-driven mesh animation that generates explicit skeleton motions without requiring per-frame skeleton annotations. SkelGen4D first recovers temporally consistent pseudo-skeletons from animated meshes via differentiable fitting, and then generates text-conditioned skeleton motion sequences in a feed-forward manner, further refined with Motion-GRPO to ensure temporally coherent, physically plausible, and articulated animation. We evaluate our method on two large-scale benchmarks, Truebones Zoo and Diffusion4D. Our results show that our weakly supervised skeleton modeling matches or surpasses fully supervised baselines while scaling to diverse object categories for high-quality text-driven mesh animation. Further, our method supports flexible motion editing and is aligned with standard animation production pipelines.

[32] HSA: Hierarchical Slot Attention for Multi-granularity Scene-Decomposition cs.CVPDF

Neelu Madan, Rongzhen Zhao, Andreas Mogelmose, Juho Kannala, Joni Pajarinen

TL;DR: 本文提出了分层槽注意力（HSA），一种用于多粒度语义场景分解的对象中心学习框架。它通过单一模型联合学习整体（前景/背景）、语义（对象类别）和全景（个体实例）三个层次的分解，仅需10%的标注数据并结合分层对齐损失。

Details

Motivation: 现有槽注意力方法将场景分解为单一粒度的扁平槽集，且基于外观而非语义，而人类通过语义层次理解场景。由于类别名称是人类构造而非视觉模式，这种语义层次无法在无监督下自然涌现。

Result: 在COCO和PASCAL VOC数据集上的实验表明，HSA在整体、语义和全景层次上分别比最强的扁平基线高出最多41.5、14.6和10.4 ARI（调整兰德指数），在Pascal VOC上增益更大，且仅需单一模型而非三个。

Insight: 创新点在于引入了多粒度语义层次分解，通过分层对齐损失和少量监督联合学习层次表示；客观分析认为其贡献在于提出了衡量层次是否编码于表示空间（而非仅输出掩码）的度量（分组纯度和包含度），实现了从外观到语义的分解跃升。

Abstract: Slot attention is a powerful framework for object-centric learning, decomposing visual scenes into latent slots through iterative competitive attention. However, existing methods share two critical limitations: they decompose scenes into a flat set of slots at a single granularity, and this decomposition is based on appearance rather than semantics. Yet humans understand scenes through semantic hierarchies: separating foreground from background, recognizing object categories, and identifying individual instances. Crucially, such semantic hierarchies cannot emerge without supervision, because category names are human constructs, not visual patterns. We propose Hierarchical Slot Attention (HSA), which learns multi-granularity semantic scene decomposition from a single model. HSA decomposes scenes at three levels: holistic (foreground/background), semantic (object categories), and panoptic (individual instances). Using only 10% labeled data, combined with hierarchical alignment loss, HSA learns all three levels jointly. We further introduce grouping purity and containment to measure whether the hierarchy is encoded in representation space, not just output masks. Experiments on COCO and PASCAL VOC demonstrate that HSA outperforms the strongest flat baseline by up to \textbf{$+$41.5} ARI at holistic, \textbf{$+$14.6} at semantic, and \textbf{$+$10.4} at panoptic level on COCO, with even larger gains on Pascal VOC, while requiring a single model instead of three. Code will be made available upon acceptance.

[33] UniRef-UAV: A Multimodal Benchmark for Universal Referring in UAV Imagery cs.CVPDF

Haibin Tian, Huichao Xie, Xuelin Qian, Ruitao Lu, Junwei Han

TL;DR: 本文提出了一个名为UniRef-UAV的多模态基准测试，用于无人机图像中的通用指代任务，该任务扩展了查询模态（支持纯文本、纯图像和图文结合）和输出基数（支持无目标、单目标和多目标定位）。同时，论文还提出了一个名为UAV-URNet的检测式基线模型，用于处理异构查询并预测可变大小的目标集合。

Details

Motivation: 现有指代表达理解基准和方法主要围绕纯文本查询和单目标输出构建，限制了其在涉及参考图像、多模态指令、目标缺失和多个有效目标实例等实际无人机场景中的适用性。

Result: 实验表明，UAV-URNet提供了一个稳定且可复现的基线，与大型通用MLLM相比，具有更一致的无目标判别能力和更轻量、可复现的实现。额外的分析表明，多模态查询有助于减少视觉-查询歧义，并促进更统一的查询-目标对齐空间。

Insight: 创新点在于提出了一个广义的无人机指代任务和相应的多模态基准，将查询模态和输出基数进行联合扩展。从客观角度看，其提出的检测式基线模型通过将异构查询映射到共享查询空间并进行集合预测，为处理此类通用任务提供了一个新颖且实用的框架。

Abstract: Unmanned aerial vehicles (UAVs) increasingly rely on visual grounding capabilities to localize task-relevant targets from diverse instructions in complex aerial scenes. Existing referring expression comprehension (REC) benchmarks and methods, however, are largely built around text-only queries and single-object outputs, which limits their applicability to practical UAV scenarios involving reference images, multimodal instructions, absent targets, and multiple valid target instances. To address this gap, we introduce \emph{Universal Referring}, a generalized UAV referring task that jointly expands the query modality and the output cardinality. We construct \emph{UniRef-UAV}, a multimodal benchmark that supports text-only, image-only, and text+image queries with modality-dependent target cardinality, where text-only and text+image queries admit no-target, single-target, and multi-target grounding while image-only queries focus on existence-aware single-instance grounding. It also provides in-domain and cross-domain evaluation protocols for visual-query generalization. We further present \emph{UAV-URNet}, a detection-style baseline that maps heterogeneous queries into a shared query space and predicts variable-size target sets through set prediction. Extensive experiments show that UAV-URNet provides a stable and reproducible baseline with more consistent no-target discrimination and a more lightweight, reproducible implementation than large general-purpose MLLMs. Additional domain analysis, query-representation analysis, and ablation studies demonstrate that multimodal queries help reduce visual-query ambiguity and promote a more unified query–target alignment space. The annotations, visual query crops/images, train/validation/test splits, evaluation scripts, and baseline code will be made publicly available to facilitate reproducible research.

[34] Enhancing the KidSat Model: Integrating Geographical Encoding and Data Quality Assessment for Childhood Poverty Prediction cs.CV | stat.APPDF

Hou Hin Ip, Ka Nam Lam, Joshua Man Yu Ng, Makkunda Sharma, Seth Flaxman

TL;DR: 本文在KidSat框架基础上，提出了一种增强的卫星图像贫困预测流程，通过改进数据预处理、引入图像质量筛选以及融合球谐函数地理编码，显著提升了儿童贫困预测的准确性。

Details

Motivation: 解决卫星图像贫困预测中存在的三大挑战：调查数据监督的噪声与稀疏性、图像质量问题（如云层覆盖和图像损坏），以及纯图像模型缺乏明确空间结构信息的问题。

Result: 在集群级别的严重剥夺比例尺度上，将MAE从0.2167降低至0.1759，相对减少了18.83%。将模型扩展到33个非洲国家后，最佳配置实现了0.1658的整体MAE。

Insight: 创新点在于：1) 通过DHS重聚合解决高基数稀疏性问题；2) 引入两阶段图像质量筛选流程；3) 将DINOv2视觉特征与球谐函数地理特征融合。研究发现，球谐函数特征能稳定提升纯图像基线的性能，而梯度提升树头能最有效地利用融合后的视觉-地理表征中的非线性交互。

Abstract: Accurate poverty mapping using satellite imagery is often hindered by (i) noisy and sparse survey-derived supervision, (ii) image quality issues such as cloud cover and image corruption, and (iii) lack of explicit spatial structure in image-only models. Building on the KidSat framework, we develop an enhanced pipeline that improves predictive accuracy via refined data preprocessing, systematic image quality assessment, and mathematically defined geographic encoding. First, we refine the fine-tuning target matrix by resolving high-cardinality sparsity and reducing one-hot dimensionality from 103 to 51 via DHS re-aggregation. Second, we introduce a simple two-stage quality-screening procedure to filter heavily clouded or corrupted observations. Third, we fuse DINOv2 visual embeddings with Spherical Harmonics (SH) location features. Across extensive experiments, these changes reduce MAE from 0.2167 to 0.1759, corresponding to an 18.83% relative reduction on the cluster-level severe-deprivation proportion scale. When extended from 16 to 33 African countries, the best-performing configuration achieves an overall MAE of 0.1658. We find that SH features consistently improve performance over the image-only backbone, whereas higher-capacity coordinate Multi Layer Perception augmentation (SH+SIREN) can underperform without carefully designed objectives. Finally, gradient-boosted tree heads (XGBoost/LightGBM) most effectively exploit nonlinear interactions in the fused visual-geographic representation. These findings provide a scalable and principled recipe for improving satellite-based socioeconomic predictions using only publicly accessible data.

[35] ARGUS: Accelerated, Robust, General, and Unsupervised Cell Tracking Solutions cs.CV | math.OCPDF

Noah Jaitner, Kandice Tanner, Ingolf Sack, Hossein S. Aghamiry

TL;DR: ARGUS是一个用于加速、鲁棒、通用和无监督细胞跟踪的框架，结合自适应细胞检测、密集光流预测、帧间线性分配和序列级轨迹片段重连，在公开的细胞跟踪挑战数据集上实现了高精度检测与跟踪，且运行速度快。

Details

Motivation: 解决延时显微镜中因噪声、形态变化、细胞重叠及分裂融合等动态事件导致的自动化细胞跟踪难题，以支持细胞动力学的定量分析。

Result: 在Cell Tracking Challenge数据集上，检测精度达0.905-0.971，跟踪精度达0.897-0.964，运行时间在1分钟内（3帧仅需5-6秒）。

Insight: 创新点在于无需训练数据或GPU基础设施的模块化无监督框架，结合光流与线性分配实现鲁棒跟踪，可适应不同成像模式，具有高通用性和可解释性。

Abstract: Background and Objective: Quantitative analysis of cell dynamics is central to modern biological research, providing critical insights into immune cell interactions, disease progression, and drug mechanisms. Automated cell tracking in time-lapse microscopy remains challenging due to noise, morphological variations, overlapping cells, and dynamic events such as divisions and fusions. Methods: We present ARGUS, a framework for Accelerated, Robust, General, and Unsupervised Cell Tracking Solutions. ARGUS combines adaptive cell detection, dense Farneback optical-flow prediction, frame-to-frame linear assignment, and a sequence-level tracklet-refinement step that reconnects trajectory fragments across short temporal gaps. Results: On publicly available Cell Tracking Challenge datasets, ARGUS achieved detection accuracy of 0.905-0.971 and tracking accuracy of 0.897-0.964, with runtimes within 1 minute (5-6 seconds for 3 frames). Conclusions: ARGUS is a modular, interpretable framework that can be adapted to different imaging modalities and biological applications without training data or GPU infrastructure. The implementation is publicly available at https://github.com/Gitinc/argus

[36] Texture Representations in Deep Vision Models: Comparing CNNs, Vision Transformers, and Human Perception cs.CVPDF

Ludovica de Paolis, Marco Baroni, Alessandro Laio, Eugenio Piasini

TL;DR: 该研究比较了卷积神经网络（CNNs）和视觉变换器（ViTs）在纹理感知任务中的内部表征，并评估它们与人类心理物理学数据的对齐程度。研究发现，不同ViTs之间的纹理表征具有一致性，但与CNN的表征不同；ViTs对不同复杂度的纹理形成相似表征；且ViTs的表征比CNNs更能预测人类在纹理识别任务中的表现。

Details

Motivation: 研究动机是探索CNN和ViT等深度视觉模型在超越标准物体识别范式的视觉任务（如纹理感知）中，其内部表征与人类感知的对齐程度是否依然存在。

Result: 在基于相同源图像生成的三种不同复杂度纹理上，使用基于排序的统计量量化模型内部表征信息。结果表明，ViTs的表征在模型间具有一致性，且能更好地预测人类纹理识别性能，而CNN的表征则与ViTs及人类数据存在差异。

Insight: 论文的创新点在于将模型比较从物体中心视角转向纹理感知，并系统量化了模型内部表征与人类感知的相似性。客观分析认为，其核心发现是网络架构（如Transformer vs. CNN）可能主导了纹理刺激的表征方式，且ViTs在模拟人类纹理处理方面可能比CNNs更忠实。

Abstract: In computational vision science, Convolutional Neural Networks (CNNs) have emerged as a popular model of biological vision because of the alignment they can exhibit with neural and behavioral data in humans and animals. However, it remains unclear to what extent this alignment persists for visual tasks that extend beyond the canonical object recognition paradigm based on well defined semantic content. In this study, we diverge from the common object-centric view by focusing on another aspect of vision: texture perception. We consider textures of different complexity generated with three different algorithms from the same source images. Using a rank-based statistic, we quantify the information encoded in the internal representations of a CNN and three Vision Transformers (ViTs), and we compare the similarity of these representations to those inferred from human psychophysics data. We find that the representation of textures is aligned in different ViTs, but not between the ViTs and the CNN; that ViTs form similar representations for textures of different complexity; that human performance in recognizing textures can be better predicted from ViTs representations rather than CNN representations. Taken together, these results suggest that ViTs may capture more faithfully than CNNs how texture patterns are visually processed by humans, and that the representations of texture stimuli in computational models may be driven by the network architecture.

[37] WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving cs.CV | cs.AIPDF

Xuerun Yan, Zhexi Lian, Nuoheng Zhang, Shiyu Fang, Haoran Wang

TL;DR: 论文提出WCog-VLA，一种新颖的双层世界认知视觉-语言-动作模型，用于端到端自动驾驶。该模型通过语义层面的世界认知与推理（结合3D空间感知、智能体令牌和博弈论思维链）以及生成层面的对齐解耦扩散变换器（ADDT）来合成物理上合理的多智能体轨迹，从而将语义世界预测与生成性世界演化相结合，实现主动驾驶。

Details

Motivation: 现有视觉-语言-动作模型要么缺乏全面的世界认知，要么存在碎片化的世界预见性，本质上局限于反应式驾驶。为了解决这一限制，论文旨在通过桥接语义世界预测与生成性世界演化来实现主动自动驾驶。

Result: 在NAVSIM基准测试上的大量实验表明，WCog-VLA实现了最先进的PDMS分数92.9。

Insight: 创新点包括：1) 双层世界认知框架，统一了语义推理与生成性世界演化；2) 引入博弈论思维链（Game-CoT）进行战略推理，并构建了包含85k个Game-CoT标注的大规模数据集；3) 提出对齐解耦扩散变换器（ADDT）作为生成世界模型，通过场景表示对齐减少去噪步骤，显著加速推理。

Abstract: Vision-Language-Action (VLA) models have advanced end-to-end autonomous driving. However, existing methods either lack comprehensive world cognition or suffer from fragmented world foresight, inherently confining these models to reactive driving. To address this limitation, we propose WCog-VLA, a novel dual-level World-Cognitive VLA framework that successfully bridges semantic world forecasting with generative world evolution to achieve proactive autonomous driving. At the semantic level, WCog-VLA unifies world cognition and reasoning by incorporating 3D spatial perception and injecting agent tokens to capture the world dynamics, while concurrently enabling Game-theoretic Chain-of-Thought (Game-CoT) reasoning. At the generative level, we introduce the Aligned Decoupled Diffusion Transformer (ADDT) as a powerful generative world model that synthesizes physically-plausible joint multi-agent trajectories. Through scene representation alignment, ADDT reduces the number of denoising steps required and thus significantly accelerates inference. To facilitate strategic reasoning, we further construct a large-scale dataset featuring 85k Game-CoT annotations. Extensive experiments on the NAVSIM benchmark demonstrate that WCog-VLA achieves a State-Of-The-Art (SOTA) PDMS score of 92.9.

[38] Attribute Retrieving for Open-Vocabulary Endoscopic Compositional Referring Segmentation cs.CVPDF

Shun Liu, Nan Xi, Yang Liu, Tianyu Luan, Xuan Gong

TL;DR: 该论文提出了一个用于内窥镜领域的指代图像分割（RIS）的大规模基准数据集ReferEndoscopy，并在此基础上开发了基于属性检索的开放词汇内窥镜组合指代分割框架AR-ERIS。该框架通过属性检索机制处理复杂的领域特定图像-文本关系，并在模拟和真实内窥镜数据上实现了最先进的性能和强大的泛化能力。

Details

Motivation: 将指代图像分割扩展到内窥镜图像领域面临高质量标注稀缺、图像-文本关系复杂且领域特定等独特挑战，现有视觉-语言模型难以捕捉内窥镜场景中的细粒度文本线索，导致性能不佳和泛化能力有限。

Result: 在提出的ReferEndoscopy基准数据集上进行预训练后，AR-ERIS框架在模拟和真实世界的内窥镜数据上均达到了最先进的性能水平，表现出强大的泛化能力。

Insight: 核心创新点在于构建了首个内窥镜RIS大规模基准数据集，并提出了一个基于属性检索的开放词汇组合指代分割框架，该框架通过检索细粒度属性来增强对复杂文本描述的理解，从而有效应对领域特定挑战。

Abstract: Referring Image Segmentation (RIS) aims to segment image regions specified by natural language, enabling fine-grained and controllable visual understanding. Extending RIS to endoscopic imagery, however, presents unique challenges, including scarce high-quality annotations and complex, domain-specific image-text relationships. Although recent vision-language models demonstrate strong cross-domain alignment, they often fail to capture fine-grained textual cues in endoscopic settings, resulting in suboptimal performance and limited generalization. To address these challenges, we introduce ReferEndoscopy, a large-scale benchmark for RIS in the endoscopy field. Building on this dataset, we propose the Attribute Retrieval-based Endoscopic-RIS (AR-ERIS) framework for open-vocabulary endoscopic compositional referring segmentation. AR-ERIS leverages attribute retrieval for open-vocabulary endoscopic compositional referring segmentation and is pretrained on the curated ReferEndoscopy dataset, achieving state-of-the-art performance with strong generalization across both simulated and real-world endoscopic data. The dataset and code will be publicly released upon completion of the review process.

[39] VEGAS: Human-Aligned Video Caption Evaluation via Gaze cs.CV | cs.AI | cs.HCPDF

Shenghui Chen, Po-han Li, Ximeng Sun, Shijia Yang, Emad Barsoum

TL;DR: 本文提出了一种名为VEGAS（基于注视评分的视频描述评估）的无训练度量方法，用于评估视频描述是否与个体观看者的视觉注意力对齐。该方法利用测试时的注视数据，通过信息论量化候选描述与观看者关注点的匹配程度。实验表明，基于VEGAS选择的描述能更好地与人类注意力对齐，并提升下游任务（如描述到视频检索）的性能。

Details

Motivation: 现有视觉语言模型生成的视频描述通常无法捕捉个体观看者的注意力焦点，因此需要一种能够评估描述与个人视觉注意力对齐程度的指标。

Result: 在包含自我中心活动和教学幻灯片的数据集（配有同步注视和参考标注）上进行实验，结果显示VEGAS选择的描述在人类注意力对齐方面显著优于基线，并提升了描述到视频检索的性能。

Insight: 创新点在于提出了一种无需训练、利用测试时注视数据的跨模态信息论度量方法，将个性化观看者注意力直接整合到推理过程中，以提升描述的相关性和实用性。

Abstract: Vision-language models excel at video captioning, yet typically generate descriptions that fail to capture individual viewers’ attention. We propose VEGAS (Video caption Evaluation via GAze Score), a training-free metric that leverages test-time gaze to sample personalized, attention-aligned text. It is a cross-modal, information-theoretic metric that quantifies how well a candidate caption matches a viewer’s focus. To evaluate VEGAS, we curate a dataset of egocentric activities and instructional slides paired with synchronized gaze and reference annotations. We then select captions based on VEGAS via rejection sampling without model retraining. Experiments show that VEGAS-selected captions align significantly better with human focus and improve downstream caption-to-video retrieval, demonstrating the practical utility of incorporating viewer attention during inference.

[40] Cognitive-structured Multimodal Agent for Multimodal Understanding, Generation, and Editing cs.CV | cs.AI | cs.CL | cs.LGPDF

Feng Wang, Canmiao Fu, Zhipeng Huang, Chen Li, Jing Lyu

TL;DR: 本文提出了一种认知结构化的多模态智能体（CMA），通过将视觉信息外化到情景视觉记忆中并选择性激活相关片段，来解决现有统一多模态模型在长序列多模态对话中因视觉令牌爆炸和跨轮次引用不可靠而受限的问题。该智能体包含感知抽象引擎、认知检索引擎和多模态执行控制器，并利用程序化生成的结构化多轮对话数据集进行强化学习优化。在长序列视觉对话基准测试中，8B参数的CMA实现了91.4%的检索准确率，超越了32B基线模型，同时推理时间减半。

Details

Motivation: 现有统一多模态模型在处理长序列多模态对话时，由于将所有历史视觉和文本输入重复馈入共享上下文窗口，导致视觉令牌爆炸和跨轮次引用不可靠，限制了其长期对话能力。

Result: 在构建的长序列视觉对话基准测试中，8B参数的CMA在20轮对话中实现了91.4%的检索准确率，比32B基线模型提升了8.2%，同时每轮推理时间从23.1秒减少到12.7秒。

Insight: 创新点在于将情景记忆机制引入多模态智能体，通过结构化的视觉抽象和跨轮次记忆检索，实现了更高效、可扩展的长序列多模态交互范式，而非单纯依赖模型参数扩展。此外，提出的程序化数据生成方法解决了现有数据集中缺乏轮次级检索监督的问题。

Abstract: Recent unified multimodal models show a single architecture can jointly perform vision/language understanding and image generation/editing. However, they repeatedly feed all historical visual and textual inputs into a shared context window, limiting long-horizon multimodal dialogue due to visual token explosion and unreliable cross-turn referencing. We propose a Cognitive-structured Multimodal Agent that externalizes visual information into an Episodic Visual Memory and selectively reactivates relevant episodes during reasoning. The agent consists of a Perceptual Abstraction Engine for structured visual abstraction, a Cognitive Retrieval Engine for cross-turn memory retrieval, and a Multimodal Executive Controller for autonomous task inference and action planning. To address the lack of turn-level retrieval supervision in existing datasets, we develop a Unified Scenario Engine that programmatically generates structured multi-turn conversations with fine-grained retrieval annotations, enabling reinforcement learning to optimize abstraction and retrieval policies. We also construct a long-horizon visual-dialogue benchmark stratified by difficulty to evaluate episodic visual recall. Our 8B agent achieves 91.4% retrieval accuracy over 20-turn sessions, surpassing 32B baselines by +8.2% while nearly halving per-turn inference time (23.1s -> 12.7s). We further present the Cognitive-structured Multimodal Agent Harness (CMA-Harness), a tool-augmented deployment of the same cognitive structure integrating persistent multimodal memory, web access, image generation/editing/composition tools, and OpenAI-compatible serving. Structured memory and modular decision-making offer a more scalable, efficient paradigm for long-horizon multimodal agents than monolithic parameter scaling. Code: https://github.com/caseclose/cma-harness ; Project page: https://caseclose.github.io/cma-harness/

[41] Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery cs.CV | cs.AIPDF

Tianyi Song, Sierra Bonilla, Xinwei Ju, Evangelos Mazomenos, Danail Stoyanov

TL;DR: Track2Map是一个用于机器人辅助微创手术的在线可变形SLAM系统，它基于3D高斯泼溅技术，能够直接从手术视频中联合优化相机轨迹和3D可变形场景表示，无需依赖精确的相机轨迹先验。

Details

Motivation: 现有基于高斯泼溅的密集可变形3D解剖重建方法大多是离线的，且严重依赖准确的相机轨迹先验（如机器人运动学数据），这在先验缺失或存在噪声时限制了其应用。

Result: 在StereoMIS数据集上的实验表明，该方法在重建质量和相机轨迹估计上优于其他SLAM方法，甚至优于那些使用了相机轨迹先验的非SLAM方法。

Insight: 核心创新点在于提出了一个在线联合优化框架，并引入了基于密集2D点轨迹的轨迹锚定变形初始化方法，以及利用轨迹统计来解耦相机运动与场景变形，从而在组织运动和视觉线索模糊的情况下稳定优化过程。

Abstract: Gaussian splatting is the current state-of-the-art for dense, deformable 3D anatomy reconstruction in robot-assisted minimally invasive surgery (RAMIS); however, most pipelines are offline and depend on accurate camera trajectory priors (often from robotic kinematics), limiting applicability when priors are missing or noisy. To address these limitations, we propose Track2Map, an online 3D Gaussian Splatting pipeline that jointly optimizes camera trajectory and 3D deformable scene representation directly from surgical video. Track2Map is therefore capable of robust 3D reconstructions when camera trajectory priors are either absent or noisy, and due to its online nature it effectively works as a Simultaneous Localisation and Mapping (SLAM) method. To stabilize optimization in the presence of tissue motion and ambiguous visual cues, we introduce a track-anchored deformation initialization using dense 2D point tracks. Track statistics are further utilized to disentangle camera motion from scene deformation by detecting static camera periods and reducing drift during incremental mapping. Experiments on StereoMIS show improved reconstruction quality and camera trajectory against competing SLAM methods, as well as compared to non-SLAM methods that utilize camera trajectory priors. The code is available at https://track2map.github.io/.

[42] DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models cs.CVPDF

Pengjie Wang, Linger Deng, Zujia Zhang, Shaojie Zhang, Zhenbo Luo

TL;DR: 本文提出了DeltaV，一种统一大型多模态模型，它用视觉状态更新取代了传统的全图像生成范式。DeltaV通过预测紧凑的更新令牌来增量式地捕捉推理步骤间的视觉变化，并引入了时间相似性路由器来动态分配令牌预算。实验表明，该方法在保持重建保真度的同时显著减少了视觉令牌数量，并在多个多模态推理基准上取得了性能提升。

Details

Motivation: 当前统一大型多模态模型在交错多模态推理中通常生成完整的中间图像，这导致了大量的视觉令牌冗余，并稀释了对推理关键但稀疏的状态转换的监督。

Result: 视觉更新范式平均减少了55.6%的新生成视觉令牌，且不影响重建保真度，多模态推理性能提升了3.3%。在StructCoT和大规模多模态数据上训练的DeltaV-2B模型，在领域内多模态推理评估中显著优于更大的开源模型8.4%，在外部多模态推理和理解基准上超越可比规模的Qwen3-VL-2B模型5.9%。

Insight: 核心创新在于用增量式视觉更新取代全图像生成，避免了未变化内容的重复建模。时间相似性路由器的引入实现了令牌预算与视觉变化幅度的自适应对齐。此外，构建的大规模StructCoT数据集为模型训练提供了多样化的交错多模态推理样本。

Abstract: Current Unified Large Multimodal Models (ULMMs) support interleaved multimodal reasoning through textual reasoning and intermediate visual states, but typically generate each visual state as a full image. This full-image generation paradigm introduces substantial visual-token redundancy and dilutes supervision on sparse yet reasoning-critical state transitions. We propose DeltaV, a ULMM that replaces full-image generation with visual updates. Conditioned on historical visual states, DeltaV incrementally predicts compact update tokens that capture the visual changes across reasoning steps, avoiding repeated modeling of unchanged content. To align the token budget of each update with the magnitude of visual change, DeltaV introduces a temporal similarity (TSIM) Router, which stops allocating tokens once the marginal reconstruction gain falls below a threshold. To support more diverse and generalizable reasoning, we further construct StructCoT, a large-scale interleaved multimodal reasoning dataset with 1.05M samples spanning 44 task domains. Experiments show that the visual-update paradigm reduces newly generated visual tokens by 55.6% on average without compromising reconstruction fidelity, and improves multimodal reasoning by 3.3% over full-image generation. Trained with StructCoT and large-scale multimodal data, DeltaV-2B further outperforms substantially larger open-source models by 8.4% on in-domain multimodal reasoning evaluations and surpasses the comparable-scale Qwen3-VL-2B by 5.9% on external multimodal reasoning and understanding benchmarks. Code, models, and StructCoT will be released at https://github.com/Pengjie-W/DeltaV.

[43] CT-CLIP Representations for Multimodal Lung Cancer Survival Prediction cs.CVPDF

Sofie Allgöwer, Mikael Johansson, Andreas Hallqvist, Jonas Andersson, Åse Johnsson

TL;DR: 本研究评估了领域特定基础模型CT-CLIP在数据受限的临床环境中用于多模态肺癌生存预测的潜力。通过将CT-CLIP作为特征提取器处理CT图像和临床变量，结合轻量级生存预测头，在242名患者数据集上验证了其性能。

Details

Motivation: 解决深度学习生存模型因带可靠结局数据的影像队列稀缺而受限的问题，探索在数据约束下利用领域基础模型进行多模态生存预测的可行性。

Result: 冻结的CT-CLIP模型结合可训练的轻量级生存头在性能上超越了临床基线，与其他多模态方法相当或更优，并能将患者分为临床意义明确的高危和低危组。

Insight: 创新点在于利用预训练的领域特定基础模型（CT-CLIP）作为冻结特征提取器，在数据有限场景下实现有效的多模态生存预测，避免了大规模数据标注需求，同时通过轻量级适配策略保持了模型效率。

Abstract: Accurate prognosis prediction is important for treatment planning in lung cancer, but deep learning-driven survival modelling is often limited by the scarcity of curated imaging cohorts with reliable outcome data. This study evaluates whether representations from a domain-specific foundation model can be used for multimodal survival prediction in data-constrained clinical settings. We assess the foundation model CT-CLIP as a feature extractor for pretreatment computed tomography images and clinical variables from 242 diagnosed lung cancer patients. The evaluation includes adaptation strategies based on frozen encoders, full fine-tuning, and low-rank adaptation, together with modality ablations and comparisons with clinical and multimodal baselines. The results show that a frozen CT-CLIP model combined with a trainable lightweight survival head outperforms the clinical baseline and achieves comparable or improved performance relative to other multimodal approaches, and separates patients into clinically meaningful high- and low-risk groups.

[44] Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues? cs.CVPDF

Masatoshi Tateno, Alexandros Stergiou, Risa Shinoda, Yoichi Sato, Dima Damen

TL;DR: 本文提出了一种新的学习范式，旨在提升以自我为中心的视频语言模型在手-物交互识别中对‘手部’和‘物体’中心化线索的捕捉能力。该方法结合了手-物掩码训练和一个HOI动态感知解码器，并通过新构建的DEHOI测试平台和CI-HOI评估协议，系统性地验证了模型对两类线索的独立推理能力。

Details

Motivation: 现有视频语言模型在手-物交互识别中，往往依赖手、物或环境之间的虚假相关性，而非从手和物体本身的外观与动态进行推理，这导致了模型理解的脆弱性。

Result: 在提出的DEHOI测试平台上，该方法在CI-HOI评估中定量和定性地优于现有模型。同时，在标准动作识别、物体状态识别乃至机器人操作动作识别任务上，该方法也取得了性能提升，实现了更鲁棒的HOI理解。

Insight: 核心创新点在于提出了一个结合掩码训练与动态感知解码器的学习范式，以及一个用于解耦评估的CI-HOI协议和DEHOI测试平台，这为评估和提升模型对特定线索的推理能力提供了系统性的框架。

Abstract: Hand-object interaction (HOI) recognition requires capturing both hand manipulations and object transformations. However, existing video-language models often fall into shortcuts by relying on spurious correlations among hands, objects, or environmental context, rather than reasoning from the appearance and dynamics of hands and objects themselves. To address this limitation, we propose a new learning paradigm that combines (i) hand-object masked training, which enables robust reasoning from partial hand or object observations, and (ii) an HOI-dynamics-aware decoder that explicitly learns hand- and object-centric embeddings through auxiliary predictions of their locations and semantics, enhancing sensitivity to both cues. To systematically evaluate such cue-specific reasoning, we introduce Cue-Isolated HOI (CI-HOI), a new evaluation that assesses models’ ability to predict actions from hand- and object-related cues independently. To enable CI-HOI, we curate the DEHOI testbed, which separates hand- and object-related observations for disentangled HOI evaluation through inpainting. Using DEHOI, we demonstrate both quantitatively and qualitatively that our training strategy exploits hand- and object-centric information more effectively than existing models. Our approach improves over existing models on DEHOI, standard action recognition, object state recognition, and even robot manipulation action recognition, leading to more robust HOI understanding.

[45] Whareformer: Learning to Track What is Where in Long Egocentric Videos cs.CVPDF

Jacob Chalk, Saptarshi Sinha, Dima Damen, Yannis Kalantidis, Diane Larlus

TL;DR: 本文提出了Whareformer，这是首个基于学习的解决方案，用于解决以自我为中心视频中的’Out of Sight, Not out of Mind’任务。该模型基于Transformer架构，包含一个可更新的轨迹记忆模块和一个前馈式的轨迹分配模块，能够联合推理演化的物体外观和更新的3D位置，并引入专门的’新轨迹’令牌来处理新物体。

Details

Motivation: 解决在以自我为中心的长视频中，在线跟踪被相机佩戴者移动的物体，即使物体离开视野或被严重遮挡，也能持续维护其实例位置信息的问题。

Result: 模型仅使用56个视频的小型数据集进行训练，但在来自EPIC-KITCHENS-100、IT3DEgo和HD-EPIC三个数据集的260个长测试视频上取得了SOTA性能，相对于先前工作有显著的绝对提升。

Insight: 创新点在于将Transformer架构应用于OSNOM任务，通过可更新的记忆和联合推理’什么’与’哪里’来维持长期物体跟踪；其设计选择（如使用相对距离和演化的轨迹表示）实现了小数据训练下的强泛化能力。

Abstract: The recently established ‘Out of Sight, Not out of Mind’ (OSNOM) task for egocentric videos focuses on tracking objects that are moved by the camera wearer, online, maintaining knowledge of instance locations throughout the video even when they leave the field of view or become heavily occluded. In this paper, we propose the first learning-based solution to the OSNOM task: Whareformer, a transformer-based model with two components: an updatable memory of established tracks and a track assignment module that associates observations with existing tracks in a feed-forward manner. Whareformer jointly reasons over evolving object appearance (what) and updated 3D location (where), and employs a dedicated New Track token to reason about novel objects. Thanks to its design choices of using relative distances and evolving track representations, Whareformer is trained on a small set of 56 videos but achieves SOTA performance on 260 long test videos from three datasets: EPIC-KITCHENS-100 (unseen videos), IT3DEgo, and HD-EPIC, with significant absolute improvements over prior work.

[46] VocaDet: Sample-Driven Open-Vocabulary Object Detection and Segmentation via Visual Tokenization and Vector Database Retrieval cs.CV | cs.AIPDF

ZhiXin Sun

TL;DR: VocaDet是一个样本驱动的开放词汇目标检测与分割框架，它通过视觉分词化和向量数据库检索，直接从用户提供的正负样本集合中学习物体概念，无需模型重新训练。该方法将连续视觉表示转换为离散视觉词汇，并利用可扩展的向量数据库进行高效检索式识别，支持识别能力的持续扩展。

Details

Motivation: 解决现有开放词汇目标检测方法依赖文本提示、有限视觉示例或昂贵特征匹配的问题，使其难以扩展到大规模且持续增长的物体库。

Result: 在UA-DETRAC数据集上的实验表明，VocaDet无需传统检测器训练即可实现有效的开放词汇检测性能，并随着正负样本的积累支持持续可扩展的识别能力。

Insight: 创新点在于将连续视觉特征离散化为多粒度视觉词汇，结合位置去偏表示和空间拓扑信息构建可扩展的物体记忆向量数据库，并引入背景过滤机制以减少冗余检索，实现了高效、可扩展的样本驱动开放词汇识别。

Abstract: Open-vocabulary object detection and segmentation aim to recognize arbitrary objects beyond predefined categories. Although recent vision-language and reference-based approaches have significantly advanced this field, they often rely on text prompts, limited visual examples, or expensive feature matching procedures, making them difficult to scale to large and continuously expanding object repositories. In this work, we propose VocaDet, a sample-driven open-vocabulary object detection and segmentation framework that learns object concepts directly from user-provided positive and negative sample collections without model retraining. The key idea is to transform continuous visual representations into discrete visual vocabularies and perform efficient retrieval-based recognition through a scalable vector database. Specifically, we employ DINOv3 as the visual feature extractor and apply agglomerative clustering with adaptive clustering sensitivity to generate multi-granularity visual tokens. These visual tokens, together with position-debiased representations and spatial topology information, are stored as expandable object memories in a vector database. During inference, query images are converted into visual tokens and efficiently matched against the stored object memories for object localization and segmentation. Furthermore, a background filtering mechanism is introduced to remove frequently occurring background patterns and reduce redundant retrieval operations in practical fixed-camera scenarios. Experiments on the UA-DETRAC dataset demonstrate that VocaDet achieves effective open-vocabulary detection performance without conventional detector training, while supporting continuously expandable recognition capability as additional positive and negative samples are accumulated.

[47] Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning cs.CVPDF

Yiyang Fang, Pei Fu, Jinjie Li, Jian Liang, Wenke Huang

TL;DR: 本文提出Switch-Reasoner框架，通过强化学习让多模态大语言模型（MLLMs）自适应地选择推理模式，在异构多任务场景中优化效率。它允许模型直接回答或先进行显式推理，并通过双级调节机制稳定决策过程，在11个多模态任务上实现了准确性与效率的更好权衡。

Details

Motivation: 解决MLLMs固定’先思考后回答’范式在异构多任务中的效率问题，因为简单输入无需显式推理而困难输入则能从中受益，且后训练中学习何时思考存在不稳定风险。

Result: 在11个多模态任务上的实验表明，Switch-Reasoner减少了不必要的推理，同时保持了强劲性能，实现了更好的准确率-效率权衡。

Insight: 创新点在于将思考视为虚拟工具调用，并引入基于GRPO的双级调节机制来平衡推理模式选择；客观分析其核心贡献是动态推理决策的稳定化方法，为MLLMs的轻量化部署提供了新思路。

Abstract: Multimodal Large Language Models (MLLMs) often follow a fixed Think-then-Answer paradigm, which is inefficient in heterogeneous multitask settings because simple inputs may not require explicit reasoning while difficult ones can benefit substantially from it. Learning when to think is also unstable during post-training, where imbalanced rollouts can drive the model toward always-thinking or always-direct behavior. We propose Switch-Reasoner, a GRPO-based framework that learns to adaptively select reasoning modes for MLLMs. It treats thinking as a virtual tool invocation and allows the model to either answer directly or invoke explicit reasoning before answering. To stabilize this decision, we introduce a dual-level regulation mechanism that balances the overall use of Thinking Mode and Direct Mode while providing sample-level supervision based on the relative benefit of the two choices. Experiments on 11 multimodal tasks show that Switch-Reasoner reduces unnecessary reasoning while maintaining strong performance, achieving a better accuracy-efficiency trade-off.

[48] Multi-Resolution Feature Stem for Diabetic Retinopathy lesion segmentation cs.CVPDF

Indranil Dutta, Taehee Jeong

TL;DR: 该论文研究了输入分辨率对糖尿病视网膜病变（DR）病灶分割的影响，发现提高分辨率对不同病灶类型有相反效果：高分辨率对微小动脉瘤分割至关重要，但会损害大出血病灶的分割性能。为此，作者提出了一种新颖的多分辨率特征主干（Multi-Resolution Feature Stem），与UNet++结合，以并行处理多尺度信息，有效解决了这一权衡问题。

Details

Motivation: 糖尿病视网膜病变病灶尺寸差异巨大（从微小动脉瘤到大出血和渗出物），导致模型架构和输入分辨率设计存在冲突，传统认为更高分辨率总是有益的观点可能不成立。

Result: 通过在512×512和1024×1024分辨率下对多种架构（U-Net、UNet++、Vision Transformers、DeepLabV3+）进行系统实验，发现提高分辨率对不同病灶类型有相反影响，并验证了所提多分辨率特征主干能有效解决这一权衡，实现了参数高效的性能提升。

Insight: 创新点在于揭示了输入分辨率对不同病灶分割性能的复杂、反直觉影响，并提出了一个输入级金字塔结构，通过并行多尺度处理同时捕获细粒度细节和上下文信息，为医学图像分割中的分辨率选择提供了新视角和实用架构。

Abstract: Diabetic Retinopathy (DR) is a leading cause of preventable blindness worldwide, requiring automated lesion segmentation using deep learning models for early detection and monitoring. However, DR lesions vary dramatically in size from tiny microaneurysms to large hemorrhages and exudates. This variability creates conflicting demands on the model architecture and input resolution, posing a challenge for effective design. This work investigates the impact of input resolution on different lesion types. Through systematic experimentation with multiple architectures (U-Net, UNet++, Vision Transformers, DeepLabV3+) at $512 \times 512$ and $1024 \times 1024$ resolutions, we identify a critical, counter-intuitive phenomenon where increasing input resolution has opposing effects on different lesion types. We demonstrate that while higher resolution is essential for resolving fine-grained microaneurysms, it can unexpectedly degrade performance on larger hemorrhages. This finding challenges the common assumption that higher resolution is uniformly beneficial. To address this, we propose a novel Multi-Resolution Feature Stem, an input-level pyramid integrated with a UNet++ backbone. This architecture processes multiple scales in parallel, capturing fine-grained details without sacrificing contextual information. This work contributes crucial empirical evidence of this complex, resolution-dependent behavior and a practical, parameter-efficient architecture that successfully resolves this trade-off.

[49] SAM-MT: Real-Time Interactive Multi-Target Video Segmentation cs.CVPDF

Ruiqi Shen, Chang Liu, Henghui Ding

TL;DR: 本文提出SAM-MT，一个基于Segment Anything 2 (SAM2)的实时交互式多目标视频分割框架。它通过显式查询表示不同目标，结合解耦掩码注意力和稀疏记忆机制，实现了目标数量与处理延迟的解耦，在保持SAM2鲁棒性能的同时，达到与单目标基准相当的实时速度（例如10个目标时>36 FPS）。

Details

Motivation: 现有视频对象分割（VOS）方法在单目标场景表现优异，但扩展到多目标时通常需要为每个目标重复单目标处理流程，导致帧率随目标数量增加而下降，延迟无界。

Result: SAM-MT在保持SAM2鲁棒视频分割性能的同时，成功将延迟与目标数量解耦，在10个目标下仍能实现>36 FPS的实时速度，与单目标基线相当。

Insight: 核心创新在于将模型转变为交互式多目标分割框架，通过显式查询与全局共享表征并行、解耦掩码注意力防止目标间干扰、以及稀疏记忆稳定时序演化，并辅以专门的遮挡处理和重叠预防策略，实现了高效的多目标实时分割。

Abstract: Modern Video Object Segmentation (VOS) involves tracking and segmenting user-specified targets. While recent approaches have achieved remarkable performance in single-target scenarios, extending them to multi-target settings typically involves replicating the single-target processing for each individual object, resulting in reduced frame rates (FPS) with unbounded latency as target count increases. Built upon Segment Anything 2 (SAM2), we propose SAM-MT, which addresses this by transforming the model into an interactive framework for real-time Multi-Target video segmentation. SAM-MT uses explicit queries to represent different individual targets, in parallel with a shared representation for global context. It employs decoupled masked attention to keep individual identities distinct from cross-target interference, and sparse memory for stable temporal evolution, along with specialized strategies for occlusion handling and overlap prevention. SAM-MT successfully decouples latency from the number of targets, achieving real-time speed on par with single-target baselines (>36 FPS for 10 targets) while maintaining SAM2’s robust video segmentation performance.

[50] When Structured Sparse Autoencoders Learn Consistent Concepts Across Modalities cs.CV | cs.AI | cs.LGPDF

Weiduo Liao, Yunqiao Yang, Ying Wei

TL;DR: 本文针对视觉语言模型（VLMs）中传统稀疏自编码器（SAEs）难以学习跨模态一致概念的问题，提出了一种结构化稀疏自编码器（S²AE）。该方法通过基于Transformer注意力相似性和空间邻近性对图像块进行分组，并引入包含组间排他性稀疏和组内一致性稀疏的结构化稀疏正则化，以从语义和空间角度强制视觉模态中的概念一致性。在Qwen2.5-VL-7B-Instruct模型上的评估表明，该方法在保持高重构保真度的同时，显著提升了语义对齐和表示效率，并增强了跨模态特征的语义一致性和单义性。

Details

Motivation: 传统稀疏自编码器在视觉语言模型中学习到的概念在视觉模态上往往呈现碎片化覆盖，难以形成跨模态一致的概念表示。本文旨在解决这一挑战，使SAEs能够学习到在视觉模态上语义和空间一致的概念。

Result: 在Qwen2.5-VL-7B-Instruct模型上评估，S²AE在语义对齐（mIoU）上平均提升6.06%，在表示效率（l0范数）上提升60.81，同时保持高于99%的解释方差（重构保真度）。跨模态分析显示，该方法使多模态特征的语义一致性和单义性得分分别平均提升3.08%和2.37%。

Insight: 核心创新在于将视觉结构先验（图像块分组）与结构化稀疏正则化（组间排他性稀疏与组内一致性稀疏）相结合，引导SAE的潜在神经元专注于不同且语义基础一致的概念。这为在复杂多模态模型中学习更连贯、更解耦的表示提供了一种新思路。

Abstract: Sparse autoencoders (SAEs) have emerged as a promising technique for mechanistic interpretability by learning a set of sparse latent features in large models, each of which encodes a distinct concept. However, in vision-language models (VLMs), vanilla SAEs struggle to learn modality-consistent concepts, with concepts often exhibiting fragmented coverage (i.e., disjoint regions) in the visual modality. To address this challenge, we propose a Structured Sparse AutoEncoder ($S^2AE$) that enforces concept consistency from both semantic and spatial perspectives in the visual modality. Specifically, we group image patches based on Transformer attention similarity and spatial proximity, and introduce a structured sparsity regularization when training the vanilla SAE. The regularization consists of exclusive sparsity for inter-group concept disentanglement and group sparsity for intra-group concept consistency, which drives the latent neurons by SAEs to specialize in distinct, semantically grounded concepts. Evaluated on the \texttt{Qwen2.5-VL-7B-Instruct} model, the method achieves 6.06% average improvement in semantic alignment (mIoU) and 60.81 in representational efficiency (lower l0 norm) while maintaining near-perfect reconstruction fidelity with an Explained Variance above 99%. Cross-modal analysis further demonstrates that $S^2AE$ enhances neuronal monosemanticity by this visual structural prior, achieving a 3.08% average gain in semantic consistency and a 2.37% average gain in monosemanticity scores for both modalities of multimodal features, thereby fostering more coherent and disentangled representations.

[51] HumanForge: A Human-Centric Deepfake Video Benchmark with Multi-Agent Forgery Rationales cs.CVPDF

Wenbo Xu, Zhimin Chen, Xiaojie Liang, Hengrui Liu, Wei Lu

TL;DR: 本文提出了HumanForge，一个以人为中心的深度伪造视频基准数据集，用于评估伪造检测方法。为了解决现有基准在人物交互和多模态对齐方面的不足，作者构建了一个大规模、多范式的视频伪造数据集。同时，论文介绍了Gen2Anno，一个基于LangGraph的模块化多智能体流水线，用于自动生成高质量视频片段和结构化、对比性的全方位标注。

Details

Motivation: 现有视频伪造基准主要关注人脸替换或全局文本到视频合成，忽略了人物-物体或人物-人物交互以及多模态对齐等关键维度，这给数字内容取证带来了前所未有的挑战。

Result: 在HumanForge数据集上，使用最先进的传统检测器和大型多模态模型（LMMs）进行的广泛基准测试表明，模型在零样本泛化和细粒度推理方面面临重大挑战。

Insight: 论文的核心创新点在于构建了一个统一、大规模、多范式的以人为中心的视频伪造数据集（HumanForge），并提出了一个创新的模块化主动多智能体流水线（Gen2Anno）来自动生成数据及标注，避免了劳动密集型的手动标注或单一的幻觉提示，从而提供了包含二元决策、细粒度伪影类别和时空定位的结构化对比标注。

Abstract: Rapid advancements in video diffusion models and temporal editing tools have enabled the generation of highly realistic human-centric videos, posing unprecedented challenges to digital content forensics. Existing benchmarks primarily focus on either face-swapping or global text-to-video synthesis, overlooking the crucial dimensions of human-object or human-human interactions and multi-modal alignment. To address these limitations, we introduce HumanForge, a unified, large-scale, and multi-paradigm human-centric video forgery dataset. To construct and annotate this dataset without labor-intensive manual labeling or hallucinated monolithic prompts, we propose Gen2Anno, a modular active multi-agent pipeline built on LangGraph. Gen2Anno coordinates six specialized agents-ranging from source profiling to MoE-based reference analysis and closed-loop forensic verification-to generate over 18K high-fidelity video segments and produce structured, contrastive omni-annotations containing binary decisions, fine-grained artifact categories, and spatio-temporal localization. Extensive benchmarks using state-of-the-art traditional detectors and Large Multimodal Models (LMMs) demonstrate the significant challenges of zero-shot generalization and fine-grained reasoning on HumanForge. Code and dataset will be publicly released.

[52] LTM: Large-scale Terrain Model for Wildfire-prone Landscapes cs.CV | cs.LGPDF

Xiao Fu, Yue Hu, Meida Chen, Peter Anthony Beerel, Barath Raghavan

TL;DR: 本文提出了一种名为LTM的大规模地形模型，用于野火易发区域的三维地形重建。该方法利用过时的数字高程模型（DEM）作为几何先验，结合图像数据进行多模态重建，通过基于物理的像素级对齐技术显著降低了计算复杂度。

Details

Motivation: 野火易发区域通常面积广阔，传统重建方法效果不佳；机载激光雷达成本高且更新频率低，而基于图像的方法因视觉特征稀疏和图像重叠有限而面临挑战。

Result: 在基于真实野火易发区域构建的大型地形模拟器上进行评估，该方法在重建精度和计算效率方面相比现有技术有显著提升，能够生成高保真深度图并保持实时性能。

Insight: 创新点在于利用过时DEM作为几何先验进行多模态重建，以及基于物理的像素级对齐技术替代昂贵的特征匹配过程，从而实现了可扩展的野火应急响应解决方案。

Abstract: Accurate 3D terrain maps are essential for emergency response when assessing wildfire hazards. However, wildfire-prone regions often span vast areas where conventional reconstruction methods underperform. Airborne LiDAR systems provide high-resolution terrain data, but they are expensive and infrequently updated. Image-based methods offer a lower-cost alternative, but struggle due to sparse visual features and limited image overlap. We propose a multi-modal reconstruction framework leveraging outdated Digital Elevation Models (DEMs) as geometric priors for image-based 3D reconstruction. Our key innovation is physics-based pixel-pixel alignment between images and DEM data, dramatically reducing computational complexity by eliminating expensive feature matching procedures. To validate our approach, we developed a large-terrain simulator based on a real wildfire-prone area, generating realistic images enabling a comprehensive evaluation. Given posed images and legacy DEMs, our method produces high-fidelity depth maps while maintaining real-time performance. We find significant improvements in reconstruction accuracy and computational efficiency over existing techniques, offering a scalable solution for wildfire response.

[53] Pose-to-Biomechanics: Bridging 3D Human Pose Estimation and Biomechanical Attribute Prediction cs.CV | cs.AI | cs.LGPDF

Ayda Eghbalian, Kevin Desai

TL;DR: 本文提出BioModule，一个轻量级的时序Transformer插件，可附加在任何3D人体姿态估计器下游，从标准的17关节3D骨架预测生物力学属性。该方法无需修改上游姿态模型，实现了从视觉姿态估计到具有物理可解释性的运动分析的扩展。

Details

Motivation: 现有3D人体姿态估计器主要优化几何关键点精度，而康复、运动科学、人体工程学等实际应用需要描述身体运动、负荷和激活的生物力学量，因此需要桥接姿态估计与生物力学预测。

Result: 在Human3.6M视频和3D关键点与Human3.6Mplus生物力学标签空间对齐的大规模数据集上训练和评估BioModule，并在七个SOTA 3D姿态估计器上进行了基准测试，首次系统分析了上游姿态估计质量对下游生物力学预测保真度的影响。

Insight: 创新点在于提出一个与估计器无关的轻量级插件模块，通过时序Transformer实现姿态到生物力学的映射，并建立了跨模态对齐数据集以支持监督学习，为现有姿态估计器提供了可扩展的生物力学分析能力。

Abstract: Recent progress in 3D human pose estimation has made markerless recovery of skeletal motion increasingly accurate and scalable. However, most pose estimators remain optimized for geometric keypoint accuracy, while many real-world applications in rehabilitation, sports science, ergonomics, and clinical movement analysis require biomechanical quantities that describe how the body moves, loads, and activates. In this work, we propose BioModule, a lightweight plug-in temporal transformer that attaches downstream of any 3D pose estimator and predicts biomechanical attributes from standard 17-joint 3D skeletons. BioModule is estimator-agnostic and requires no modification of the upstream pose model, enabling existing pose estimators to be extended toward physically interpretable motion analysis. To train and evaluate BioModule, we construct a large-scale aligned dataset pairing Human3.6M video and 3D keypoints with the biomechanical label space of Human3.6Mplus. We establish and verify anatomical correspondence between coordinate systems of the two datasets, enabling frame-accurate cross-modal supervision. Using this aligned supervision, BioModule predicts biomechanical quantities. We further benchmark BioModule across seven state-of-the-art 3D pose estimators, providing the first systematic analysis of how upstream pose estimation quality propagates to downstream biomechanical prediction fidelity. The results position BioModule as a compact, modular bridge between vision-based pose estimation and biomechanically meaningful human motion analysis.

[54] OpenCoF: Learning to Reason Through Video Generation cs.CV | cs.AIPDF

Xinyan Chen, Ziyu Guo, Renrui Zhang, Dongzhi Jiang, Hongsheng Li

TL;DR: 本文提出了OpenCoF框架，旨在通过视频生成来增强模型的推理能力。该框架包含一个覆盖11类任务的OpenCoF-17K推理视频数据集，以及一个名为Wan-CoF的微调视频模型，用于研究多样化的时序监督是否能改进链式帧（CoF）推理行为。

Details

Motivation: 现有视频生成模型主要在通用视频语料上训练，缺乏针对链式帧（CoF）推理的多样化监督和专门设计，无法有效支持通过时序连贯帧进行推理。

Result: 在四个视频推理基准测试中，Wan-CoF模型相比Wan2.2-I2V-A14B基线取得了显著提升。

Insight: 创新点在于引入了视觉和文本推理令牌机制，分别捕捉低层视觉线索和高层语义先验，以支持空间和时序推理；研究表明，更强的视频推理需要广泛的时序监督和用于组织中间推理状态的显式机制。

Abstract: Reasoning has become a core capability for large models, especially when reliable decisions require understanding logical consequences. Recent video generation models offer a reasoning path distinct from previous Chain-of-Thought (CoT): reasoning can unfold through temporally connected frames, known as Chain-of-Frame (CoF) reasoning. However, existing video generators are primarily trained on general video corpora, still lacking diverse supervision and dedicated designs for CoF reasoning. To address this gap, we introduce OpenCoF, a framework comprising the OpenCoF-17K dataset, a reasoning video dataset spanning 11 task families, and Wan-CoF, a fine-tuned video model for studying whether diverse temporal supervision improves CoF behavior. Across four video reasoning benchmarks, Wan-CoF achieves considerable gains over the Wan2.2-I2V-A14B baseline. Building on this, we empirically explore more advanced designs for CoF capabilities, i.e., equipping the model with visual and textual reasoning tokens. This mechanism respectively captures low-level visual cues and high-level semantic priors for spatial and temporal reasoning. Through performance comparisons and attention analysis, we examine how these tokens contribute across model depth, denoising steps, space, and time. Our results suggest that stronger video reasoning requires both broad temporal supervision and explicit mechanisms for organizing intermediate reasoning state. We open-source the dataset, model, and code to facilitate future research on reasoning-oriented video generation.

[55] WaspMOT: A Benchmark for Long-Term Multi-Object Tracking of Trichogramma Wasps cs.CVPDF

Tomasz Stanczyk, Yuan Gao, Hardik Agarwal, Seongroo Yoon, Tiantao Zhang

TL;DR: 本文提出了WaspMOT基准测试，旨在评估多目标跟踪（MOT）方法在长期身份保持方面的能力。该基准基于在受控生态实验中拍摄的赤眼蜂长时视频序列，包含10段约1.2万帧的视频，并提供了密集标注和完美检测框以专注于关联性能评估。实验表明，现有主流跟踪方法在长期身份保持上均存在显著轨迹断裂问题，而一个简单的空间轨迹拼接基线方法能有效提升性能。

Details

Motivation: 现有MOT基准测试多基于短时视频序列，无法充分评估算法在长期跟踪中保持目标身份一致性的能力。本文旨在填补这一空白，通过构建一个长时、封闭场景的跟踪数据集来专门研究长期关联问题。

Result: 在WaspMOT基准上评估了ByteTrack、BoT-SORT、C-BIoU、OC-SORT和McByte五种主流检测跟踪方法。结果表明，即使使用完美检测框，所有方法都出现了显著的轨迹断裂，凸显了长期身份保持的挑战。一个简单的空间轨迹拼接基线方法能持续提升性能，表明仍有较大的改进空间。

Insight: 论文的创新点在于构建了一个专门针对长期身份保持挑战的封闭场景跟踪基准，揭示了现有方法在长时跟踪中的固有局限。从客观角度看，其核心洞察是：在目标持续存在、外观高度相似且存在遮挡和突变运动的长时场景中，关联算法本身（而非检测性能）是当前性能瓶颈，这为未来研究指明了方向。

Abstract: Multi-object tracking (MOT) has achieved strong performance on benchmarks dominated by short video sequences. However, such datasets do not adequately evaluate long-term identity preservation, where objects must be tracked consistently over extended durations. We introduce WaspMOT, a benchmark designed to address this gap through long-duration tracking of Trichogramma wasps in controlled ecological experiments. The dataset contains 10 sequences of approximately 12,000 frames each (over 8 minutes at 25 FPS), with dense MOTChallenge annotations and oracle detections to isolate association performance. Unlike existing benchmarks, WaspMOT forms a closed-set tracking scenario where all individuals remain present throughout the sequence, requiring consistent identity assignment across thousands of frames despite abrupt jumps, occlusions, and highly similar appearance. We establish a benchmark by evaluating five tracking-by-detection methods, including ByteTrack, BoT-SORT, C-BIoU, OC-SORT, and McByte, under a unified protocol. Results show that all methods suffer from significant trajectory fragmentation, highlighting the difficulty of long-term identity preservation even with perfect detections. A simple spatial tracklet stitching baseline consistently improves performance, indicating that substantial gains remain possible. WaspMOT provides a new benchmark for studying long-term association and reveals limitations of current tracking approaches that are not observable on conventional datasets. The benchmark will be made publicly available at the project repository: https://github.com/tstanczyk95/WaspMOT/ .

[56] Enhancing In-context Panoramic Generation via Geometric-aware Pretraining cs.CVPDF

Haoran Feng, Ruiyang Zhang, Longyi Zhang, Dizhe Zhang, Lu Qi

TL;DR: 本文提出了Canvas360，一个两阶段框架，用于上下文全景图生成，结合了几何感知预训练和下游任务特定微调。为了解决缺乏针对上下文全景任务的大规模高质量训练数据的问题，提出了Canvas360Dataset，包含100万个用于风格迁移、修复、外绘和编辑的高质量配对全景样本。在建模方面，通过并行深度生成、速度循环填充和相似性损失正则化等技术，增强了文本到全景图的生成能力。

Details

Motivation: 解决上下文全景图生成任务中缺乏大规模高质量训练数据的问题，并提升模型对全景图几何结构和一致性的理解与生成能力。

Result: 大量实验表明，Canvas360提升了全景图像的保真度，在全景图特定的FAED指标上表现尤为出色，并在报告的定量评估中取得了具有竞争力或领先的结果。

Insight: 创新点包括构建大规模高质量全景数据集Canvas360Dataset，以及引入并行深度生成、速度循环填充和相似性损失正则化等几何感知建模技术，从而实现了支持多种下游任务的统一上下文全景生成框架，在任务覆盖范围和建模灵活性上超越了先前方法。

Abstract: In this work, we present Canvas360, a two-stage framework for in-context panoramic generation that combines geometry-aware pretraining with downstream task-specific fine-tuning. To address the lack of large-scale, high-quality training data tailored to in-context panoramic tasks, we propose Canvas360Dataset, a collection of 1M high-quality paired panoramic samples for style transfer, inpainting, outpainting, and editing, enabling effective supervision across diverse in-context generation scenarios. On the modeling side, Canvas360 enhances text-to-panorama generation through parallel depth generation, velocity circular padding, and similarity loss regularization, enabling the model to learn geometry-aware representations, capture object distortion details, and improve geometric consistency and global coherence. Furthermore, empowered by strong panoramic priors, Canvas360 enables a unified in-context panoramic generation framework that supports diverse downstream tasks via token-level concatenation, surpassing prior methods in both task coverage and modeling flexibility. Extensive experiments show that Canvas360 improves panoramic image fidelity, achieving particularly strong performance on the panorama-specific FAED metric and competitive or leading results across the reported quantitative evaluations. More information can be found on our project page: https://zry000.github.io/Canvas360/

[57] Geometry and Gradient-based Partitioning for Panoramic Outdoor Reconstruction cs.CVPDF

Weijian Chen, Weibo Yao, Yuhang Zhang, Xiaolin Tang, Guo Wang

TL;DR: 本文提出PanoLOG，一个用于大规模全景户外场景重建的两阶段从粗到精框架。它通过几何和梯度驱动的分区策略（G²PS）解决了全景图像在3D高斯泼溅（3DGS）重建中因全局可见性导致的优化难题，并构建了首个大规模全景数据集Pano360作为基准。

Details

Motivation: 将3D高斯泼溅（3DGS）扩展到大规模户外场景在数据采集和计算上成本高昂。全景图像虽能减少采集工作量，但其全向可见性使得依赖局部相机视锥的现有分区策略失效，导致分块优化退化为全局训练。

Result: 在构建的Pano360基准数据集上进行的大量实验表明，G²PS在保持可扩展、分块并行训练的同时，实现了最先进的渲染质量。

Insight: 创新点在于针对全景3DGS重建提出了两阶段框架和G²PS策略，该策略通过视差驱动的不确定性构建自适应边界体积，并利用基于梯度的重要性评分分配相机，有效解决了全景场景的优化分区问题；同时贡献了首个大规模全景户外重建数据集。

Abstract: Scaling 3D Gaussian Splatting (3DGS) to large outdoor scenes is costly in both data acquisition and computation. Adopting panoramic images with equirectangular projection (ERP) can reduce capture effort via their full $360^{\circ}$ field of view, yet the resulting omnipresent visibility invalidates existing partitioning strategies that rely on local camera frustums, causing block-wise optimization to degenerate into global training. Thus, we propose PanoLOG, a two-stage coarse-to-fine framework equipped with a Geometry and Gradient-based Partitioning Strategy tailored for large-scale panoramic 3DGS reconstruction. In the global coarse stage, PanoLOG leverages sky-sphere modeling and panoramic monocular depth supervision for reliable geometry, while in the refinement stage, G$^2$PS builds adaptive bounding volumes via parallax-driven uncertainty and assigns cameras via gradient-based importance scoring. Furthermore, we construct Pano360, the first benchmark on large-scale panoramic dataset for outdoor scene reconstruction. Extensive experiments demonstrate that G$^2$PS achieves state-of-the-art rendering quality while maintaining scalable, block-parallel training. Our models, training code, and dataset are publicly available.

[58] OPSD-V: On-Policy Self-Distillation for Post-Training Few-Step Autoregressive Video Generators cs.CVPDF

Hongyu Liu, Chun Wang, Feng Gao, Xuanhua He, Yue Ma

TL;DR: 本文提出了OPSD-V，一种用于后训练少步自回归视频扩散模型的在线策略自蒸馏范式。该方法旨在减少长序列生成中的性能退化，同时保持原有的少步推理路径，通过引入真实长视频数据作为时间上下文并提供密集的轨迹级监督来实现。

Details

Motivation: 现有的少步自回归视频生成器虽然能以低延迟生成长视频，但在长序列自回归展开过程中仍存在误差累积和运动动态减弱的问题，OPSD-V旨在解决这些长时域退化问题。

Result: 实验表明，OPSD-V在视觉质量、运动动态和VBenchLong分数上均带来了一致的提升。一项包含10名参与者的用户研究显示，在排除平局的情况下，OPSD-V在82.5%的整体偏好判断中优于基线模型。

Insight: 核心创新在于引入了在线策略自蒸馏，利用真实视频上下文为教师模型提供更干净的AR一致缓存，从而在不改变推理时采样器、去噪步数或缓存机制的情况下，为学生的去噪过程提供密集的纠正目标。这是一种新颖的后训练优化方法。

Abstract: We propose OPSD-V, an on-policy self-distillation paradigm for post-training few-step autoregressive (AR) video diffusion models. Existing few-step AR video generators can produce long videos with low latency, but still suffer from error accumulation and weakened motion dynamics during long autoregressive rollout. OPSD-V reduces long-horizon degradation while preserving the original few-step inference path. The key idea is to introduce real long-video data as temporal context during training and use it to provide dense trajectory-level supervision. Specifically, the student follows the exact inference-time rollout, generating each chunk conditioned on its own previously generated KV cache. In parallel, the teacher is evaluated at the same student-visited denoising states, but uses a cleaner AR-consistent temporal cache in which older history can be replaced by real-video context. This provides dense denoising-level corrective targets under on-policy AR cache dynamics, without changing the sampler, number of denoising steps, or inference-time cache mechanism. We apply OPSD-V to representative few-step AR video models, including Self-Forcing and LongLive. Experiments show consistent improvements in visual quality, motion dynamics, and VBenchLong scores. A user study with 10 participants comparing 20 video pairs shows that OPSD-V is preferred over the base models in 66.0% of overall-preference judgments (82.5% excluding ties).

[59] LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation with Video Diffusion Models cs.CVPDF

Cheng-De Fan, Chun-Wei Tuan Mu, Chen-Wei Chang, Chin-Yang Lin, Kun-Ru Wu

TL;DR: LongE2V是一种利用预训练视频扩散模型先验的新方法，用于联合处理基于事件的视频重建、预测和帧插值任务。该方法通过微调基础视频模型实现了高数据效率和卓越的感知质量，并通过引入自回归展开、自适应上下文切换、重编码对齐与交叉残差校正等技术，解决了长序列中的时间漂移和双向一致性问题。

Details

Motivation: 解决从稀疏事件流中恢复高质量视频的挑战，现有回归方法常导致纹理模糊，而现有生成模型则难以保证长期稳定性。

Result: 在真实世界基准测试上的大量实验表明，LongE2V在所有三项任务（重建、预测、帧插值）上均优于最先进方法，展现出卓越的时间一致性和零样本泛化能力。

Insight: 核心创新在于利用预训练视频扩散先验进行联合任务处理，并提出了针对长序列稳定性的自回归展开与自适应上下文切换机制，以及确保帧插值精确双向一致性的重编码对齐与交叉残差校正方法。事件体素密度增强也提升了模型对不同传感器分辨率的鲁棒性。

Abstract: Recovering high-quality video from sparse event streams is a challenging task. Regression methods often blur textures, while existing generative models struggle with long-term stability. We propose LongE2V, a novel approach that leverages pre-trained video diffusion priors to jointly handle event-based video reconstruction, prediction, and frame interpolation. By fine-tuning a foundational video model, our approach achieves high data efficiency and superior perceptual quality. We introduce Autoregressive Unrolling and Adaptive Context Switching to mitigate temporal drift in extremely long sequences. We also propose Reencoding Alignment with Cross Residual Correction to ensure precise bidirectional consistency during frame interpolation. Furthermore, Event Voxel Density Augmentation ensures robustness across varying sensor resolutions. Extensive experiments on real-world benchmarks demonstrate that LongE2V outperforms state-of-the-art methods across all three tasks, exhibiting exceptional temporal coherence and zero-shot generalization. Project page: https://cdfan0627.github.io/LongE2V-page/

[60] Wat3R: Underwater 3D Geometry Learning without Annotations cs.CVPDF

Jiangwei Ren, Xingyu Jiang, Zijie Song, Wei Xu, Hongkai Lin

TL;DR: 本文提出Wat3R，一种无需水下3D标注的跨域半监督学习框架，用于将前馈式3D重建模型从空气域适应到水下场景。该方法基于师生架构，仅利用大量无标注的真实水下视频学习鲁棒的几何表示，并设计了跨视图一致性损失来补偿水下衰减和散射造成的信息退化。此外，作者构建了Water3D数据集用于评估。

Details

Motivation: 水下环境的光衰减、散射以及缺乏大规模高质量3D标注，使得水下3D几何估计面临独特挑战，现有依赖密集标注的方法不切实际。

Result: 实验结果表明，Wat3R在水下多视角深度估计和点云重建任务上超越了当前最先进（SOTA）方法。

Insight: 主要创新点在于无需任何标注水下数据的跨域适应框架，以及利用多视图几何线索补偿水下退化的损失函数；客观来看，构建专门的水下几何评估数据集Water3D也是一个重要贡献。

Abstract: Estimating 3D geometry in underwater environments presents unique challenges due to light attenuation, scattering, and the absence of large-scale, high-quality 3D annotations. Pioneering methods rely on massive dense annotations that are impractical in underwater settings. In this paper, we propose Wat3R, a cross-domain semi-supervised learning framework designed to adapt feed-forward 3D reconstruction models from air to underwater scenes. Uniquely, our method eliminates the need for any annotated underwater data following a teacher-student architecture, that learns robust geometry representations merely on abundant unlabeled real underwater video footage. We also design a cross-view consistency loss that leverages geometric cues from other views to compensate for the information degradation in the current view caused by water attenuation and scattering. Furthermore, considering the lack of comprehensive evaluation benchmarks, we construct Water3D, a diverse dataset covering various water bodies and underwater scenarios, designed for geometric task evaluation. Experimental results demonstrate that Wat3R outperforms current state-of-the-art methods in underwater multi-view depth estimation and point cloud reconstruction. The dataset and code are available at https://github.com/LSXI7/Wat3R .

cs.RO [Back]

[61] Time-to-Collision Based Dynamic Obstacle Avoidance Using Pretrained Vision Models for Robots in Unstructured Environments cs.RO | cs.AI | cs.CV | eess.IVPDF

Erik Jagnandan, Mulugeta Haile, Gregory Barber, Pratik Chaudhari

TL;DR: 本文提出了一种用于非结构化户外环境中机器人动态避障的数据高效且可解释的视觉方法。该方法利用预训练的UniDepth模型从RGB视频生成密集深度图，并通过扩展SuperPoint和SuperGlue特征匹配流程来跟踪关键点、计算其碰撞时间，最终选择地面运动基元使机器人远离最紧急的碰撞点。整个流程完全基于真实世界数据，无需机器人特定的大规模训练或仿真策略。

Details

Motivation: 解决非结构化户外环境中自主移动机器人动态避障的挑战，特别是在大规模机器人特定训练数据和基于仿真的策略不切实际的情况下。旨在避免仿真训练策略固有的sim-to-real迁移问题，并实现数据高效和可解释的避障。

Result: 在M3ED数据集的真实世界数据上评估，该方法在识别碰撞时间小于1秒的帧时，精确度为0.49，召回率为0.38。在真阳性检测中，84%的情况下能正确生成规避运动方向。在测试序列的22个独特物理障碍物中，成功检测出20个至少有一帧碰撞时间小于1秒。

Insight: 创新点在于将预训练的单目深度估计模型（UniDepth）与基于特征点（SuperPoint/SuperGlue）的长序列跟踪、光束法平差及碰撞时间计算相结合，构建了一个完全无需训练、数据效率极高（仅需74秒数据进行超参数调优）且可解释的避障流程。这为依赖大规模仿真或机器人特定数据的学习方法提供了替代方案，具有良好的泛化性。

Abstract: Dynamic obstacle avoidance in unstructured outdoor environments remains a critical challenge for autonomous mobile robots, particularly when large-scale robot-specific training data and simulation-based policies are impractical. We present a data-efficient, interpretable method for vision-based dynamic obstacle avoidance that operates entirely on real-world data, avoiding the sim-to-real transfer problem inherent in simulation-trained policies. Our approach leverages UniDepth, a large pretrained monocular depth estimation model, to produce dense depth maps from RGB video without requiring stereo cameras or LiDAR at inference time. Dynamic obstacle avoidance is achieved by extending the SuperPoint and SuperGlue feature correspondence pipeline to track keypoints across long frame sequences, projecting their 2D pixel-space positions into 3D using camera intrinsics and predicted depth, running bundle adjustment initialized from these 3D keypoints, and computing per-keypoint time-to-collision (TTC). A 2D motion primitive in the ground plane is then selected to move the robot away from the closest point of approach of the minimum-TTC keypoint. Evaluated on real-world data from the M3ED dataset, our pipeline achieves a precision of 0.49 and a recall of 0.38 in identifying frames with a ground truth TTC below 1 second, and correctly generates the evasive motion direction in 84% of true positive detections. Crucially, it detects at least one frame with TTC less than 1 second for 20 out of 22 unique physical obstacles present in our test sequences. Unlike end-to-end learned methods that demand thousands of hours of robot-specific training data, our approach eliminates model training entirely, requiring only 74 seconds of data for hyperparameter tuning. This demonstrates exceptional data efficiency while preserving interpretable and generalizable behavior across diverse obstacle types.

[62] Native Video-Action Pretraining for Generalizable Robot Control cs.RO | cs.CVPDF

Qihang Zhang, Lin Li, Luyao Zhang, Shuai Yang, Yiming Luo

TL;DR: 本文提出了LingBot-VA 2.0，一个专门为具身智能体控制从头构建的视频-动作基础模型。它通过四个核心设计原则改进了前代模型：引入语义视觉-动作分词器以对齐视觉表征与语义和动作；采用因果预训练范式以适应时序动态；使用稀疏混合专家（MoE）骨干网络以平衡容量与效率；并通过增强的异步推理方案实现实时闭环控制。真实世界部署验证了其在复杂操作任务上具有强大的少样本泛化能力。

Details

Motivation: 作者认为，为数字内容创作设计的视频生成模型在重新用于物理环境机器人控制时存在固有不足，因此需要构建一个专门为具身智能体设计的视频-动作基础模型来弥合这一差距。

Result: 真实世界部署验证了LingBot-VA 2.0作为一个鲁棒的基础模型，在复杂的操作任务上表现出强大的少样本泛化能力。

Insight: 创新点包括：1）语义视觉-动作分词器，将视觉表征与语义和动作对齐，以提升后续策略学习的指令跟随和动作精度；2）因果预训练范式，避免双向架构适应时常见的灾难性遗忘；3）稀疏MoE骨干网络，在不牺牲效率的情况下扩展模型容量；4）增强的异步推理方案，通过并行预测未来潜在状态与执行动作，并结合学习的前向动力学基于最新观测进行重定位，实现实时闭环控制。

Abstract: The advent of video-action models offers a promising path for robot control. Nevertheless, we argue that repurposing video generative models designed for digital content creation is inherently inadequate for physical environments. To bridge this gap, we present LingBot-VA 2.0, a video-action foundation model built from the ground up for embodiment. Four core design principles showcase its evolution from LingBot-VA. (1) Departing from traditional reconstruction-focused VAEs, we introduce a semantic visual-action tokenizer, which aligns visual representations with both semantics and actions, improving instruction following and action precision in subsequent policy learning. (2) Given the strictly causal nature of temporal dynamics, we adopt a causal pretraining paradigm, training from scratch to circumvent the catastrophic forgetting that frequently occurs when adapting bidirectional architectures. (3) To meet the demands of high-frequency inference, our model employs a sparse MoE backbone, expanding model capacity without compromising efficiency. (4) Real-time closed-loop control is realized through an enhanced asynchronous inference scheme, which predicts future latents in parallel with action execution while re-grounding each rollout on the latest observation via learned forward dynamics. Real-world deployment validates LingBot-VA 2.0 as a robust foundation model, as evidenced by its few-shot generalization across complex manipulation tasks.

cs.MA [Back]

[63] Collective Intelligence with Foundation Models cs.MA | cs.AI | cs.CLPDF

J. de Curtò, I. de Zarzà

TL;DR: 本章提出了一个多智能体框架，旨在通过协调多个基础模型进行协作推理，以构建更安全、更可靠的AI系统。该框架包含求解器模型生成独立草稿、评论者智能体进行结构化批判与修订，以及聚合者智能体合成最终共识解决方案，并通过评分模块进行语义、数值和程序评估。

Details

Motivation: 随着基础模型规模和多样性的增长，如何将多个模型协调成协作推理系统，以实现更安全、可靠的AI，是研究的核心动机。

Result: 在涵盖微积分、物理、化学、生物、经济、优化、统计和数学的基准测试中，异质框架配置（使用多样化专业模型）实现了卓越的逐步准确性（0.64 vs. 个体模型的0.54；比同质配置提升2.3倍），并降低了跨类别和难度级别的方差，显著提升了逐步推理质量。

Insight: 创新点在于提出了一个结构化多智能体协作框架，强调模型异质性（而非框架结构或冗余采样）是驱动性能大幅提升的关键因素，异质智能体提供了互补的错误检测和推理细化，这对于可解释性和可审计性至关重要。

Abstract: As foundation models grow in scale and diversity, coordinating multiple models into cooperative reasoning systems offers a path toward safer, more reliable AI. This chapter presents a multi-agent framework where solver models generate independent drafts, each undergoes structured critique and revision by a critic agent, and an aggregator agent synthesizes a final consensus solution. A scoring module provides semantic, numerical, and procedural evaluation across all agents. Through ablation studies on a benchmark spanning calculus, physics, chemistry, biology, economics, optimization, statistics, and mathematics, we isolate the contributions of framework architecture versus model diversity. We compare four configurations: (1) Individual Baseline, (2) Homogeneous Framework using one shared model, (3) Redundant Homogeneous Solvers using multiple instances of the same model, and (4) Heterogeneous Framework with diverse specialized models. Results show that while framework structure and redundant sampling yield modest gains, model heterogeneity is the critical factor driving substantial performance improvements. The heterogeneous configuration achieves superior step-wise accuracy (0.64 vs. 0.54 for individual models; 2.3x improvement over homogeneous configurations) with reduced variance across categories and difficulty levels. Step-wise reasoning quality (correctness of intermediate steps, not just final answers) improves dramatically only with model diversity, showing that heterogeneous agents provide complementary error detection and reasoning refinement essential for explainability and auditability. We discuss architectural principles, evaluation methodology, and implications for Global Applied AI, showing how heterogeneous multi-agent coordination supports transparent, auditable, high-confidence decision-making across scientific and industrial domains.

cs.GR [Back]

[64] ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation cs.GR | cs.CV | cs.LG | cs.ROPDF

Kaifeng Zhao, Mathis Petrovich, Haotian Zhang, Tingwu Wang, Siyu Tang

TL;DR: 本文提出了ARDY，一种用于交互式人体运动生成的流式生成框架，它结合了自回归和扩散模型，并采用混合表示（显式根特征与潜在身体嵌入），实现了高保真、可控的实时运动生成。

Details

Motivation: 现有离线运动生成方法虽控制精确但推理速度慢，而在线方法虽实时但可控性差且难以处理复杂文本语义和长期目标，ARDY旨在弥合这一差距。

Result: 在HumanML3D基准和Bones Rigplay数据集上的广泛评估表明，ARDY实现了高运动质量和约束遵循度，验证了其架构设计的有效性。

Insight: 创新点在于提出了结合自回归与扩散的两阶段去噪器、支持可变历史上下文和长期运动学约束的混合表示，实现了实时、高保真且高度可控的运动生成。

Abstract: Generating realistic 3D human motions in real-time within interactive applications is key for animation, simulation, and humanoid robotics. While recent offline motion generation approaches offer precise control via text and kinematic constraints, they lack the inference speed required for interactive settings. Conversely, existing online methods enable real-time synthesis but often sacrifice controllability or struggle with complex text semantics and long-horizon goals due to limited context windows. In this work, we introduce ARDY, a streaming generation framework that bridges this gap by enabling high-fidelity motion generation controllable via online text prompts and flexible kinematic constraints. ARDY employs a hybrid representation that combines explicit root features with a latent body embedding, balancing precise trajectory control with efficient generative learning. We propose a two-stage autoregressive transformer denoiser that features variable history context and supports conditioning on flexible, long-horizon kinematic constraints. By training on a large-scale motion capture dataset and being directly conditioned on text labels and kinematic constraints sampled from ground truth poses, ARDY natively learns controllable generation that supports online prompting and flexible long-horizon goals. Extensive evaluations on the HumanML3D benchmark and the large-scale, high-fidelity Bones Rigplay dataset demonstrate ARDY’s high motion quality and constraint adherence, validating the efficacy of our key architectural decisions. Finally, we demonstrate the method’s practical versatility through an interactive demo featuring dynamic text control, diverse keyframe pose constraints, path following, and interactive locomotion control via mouse and keyboard. Supplementary video results, code, and model releases can be found at https://research.nvidia.com/labs/sil/projects/ardy/.

cs.LG [Back]

[65] Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks cs.LG | cs.AI | cs.CL | cs.CR | cs.MMPDF

Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu, Vaidehi Patil

TL;DR: 这篇论文是一篇关于多模态遗忘的综述，系统性地回顾了在视觉、语言、音频和视频等模态中，如何从基础模型中选择性移除敏感、有偏见或不安全知识的方法、数据集和基准测试。

Details

Motivation: 动机在于解决多模态基础模型（如VLMs、LLMs、DMs、AFMs）在训练数据中可能编码有害跨模态关联的问题，由于完全重新训练不切实际且知识分布在共享表示中难以定位，因此需要有针对性的遗忘技术。

Result: 作为一篇综述，本文未提出具体模型或实验，但通过分类法系统比较了不同模型架构和模态下的方法，并分析了在删除强度、保留能力、效率、可逆性和鲁棒性之间的权衡。

Insight: 创新点在于提供了一个统一、系统导向的多模态遗忘研究框架，明确了该领域的开放问题和实际考量，并发布了整理好的资源库以支持未来研究。

Abstract: With the growing adoption of VLMs, DMs, LLMs, and AFMs, these multimodal foundation models can inadvertently encode sensitive, copyrighted, biased, or unsafe cross-modal associations that originate from their training data. Retraining after deletion requests or policy updates is often impractical, and targeted forgetting remains difficult because knowledge is distributed across shared representations. Multimodal unlearning addresses this challenge by enabling selective removal across modalities while retaining overall utility. This survey offers a unified, system-oriented view of multimodal unlearning across vision, language, audio, and video, grounded in recent advances, emerging applications, and open problems. Our taxonomy enables systematic comparison across model architectures and modalities, clarifying trade-offs among deletion strength, retention, efficiency, reversibility, and robustness. This survey highlights open problems and practical considerations to support future research and deployment of multimodal unlearning. We release a curated repository: https://smsnobin77.github.io/Awesome-Multimodal-Unlearning/

[66] Ensemble Diversity Optimization for Subjective Supervision cs.LG | cs.CLPDF

Xia Cui, Ziyi Huang, N. R. Abeynayake

TL;DR: 本文提出了一种名为集成多样性优化（EDO）的预测空间框架，用于处理主观NLP任务中存在的系统性标注者分歧问题。该框架通过一个统一的可微分目标，联合优化集成权重、有效基数（规模）和校准，并引入一个有符号的多样性正则化器来控制分歧的保留或抑制。

Details

Motivation: 动机在于主观NLP任务中普遍存在标注者之间的系统性分歧，需要模型能够表征这种不确定性，而不是将其消除或忽略。

Result: 在四个主观文本分类基准（ArMIS, ConvAbuse, HS-Brexit, MD-Agreement）上的实验表明，EDO显著改善了概率校准，相对于Soft-CE、Soft-MD、Top-5 Voting和WEL等基线方法，交叉熵降低了40-78%，Brier分数也更低，同时保持了有竞争力的F1分数以及与标注者分布更好的对齐。

Insight: 创新点在于提出了一个端到端联合优化集成结构（包括组成和规模）和校准的框架，其核心是通过一个有符号的多样性正则化器，提供了一种模型无关的、可控的方式来权衡模型效用与校准，从而有效地建模监督学习中的主观性。

Abstract: Subjective NLP tasks often exhibit systematic annotator disagreement, requiring models that represent uncertainty rather than collapse it. We introduce Ensemble Diversity Optimization (EDO), a prediction-space framework that jointly optimizes ensemble weights, effective cardinality, and calibration through a unified differentiable objective. EDO learns ensemble composition and size end-to-end via Gumbel-Softmax relaxation and incorporates a signed diversity regularizer, tuned on validation data, to steer optimization toward either preserving or suppressing disagreement. This regularization prevents ensemble collapse and enables controlled navigation of the utility-calibration trade-off. The framework integrates a soft F1 surrogate, class-weighted cross-entropy to address imbalance, and reliability-weighted diversity to regulate intra-ensemble variability. Experiments on four subjective text-classification benchmarks (ArMIS, ConvAbuse, HS-Brexit, MD-Agreement) show that EDO substantially improves probabilistic calibration, reducing cross-entropy (40-78% depending on baseline) and lowering Brier scores relative to Soft-CE, Soft-MD, Top-5 Voting, and WEL, while maintaining competitive F1 and better alignment with annotator distributions. These results demonstrate that jointly optimizing ensemble structure with a signed diversity regularizer provides an efficient, model-agnostic approach for modeling human subjectivity in supervised learning.

cs.AI [Back]

[67] CausalDS: Benchmarking Causal Reasoning in Data-Science Agents cs.AI | cs.CL | cs.LGPDF

Andrej Leban, Yuekai Sun

TL;DR: CausalDS是一个用于评估数据科学智能体中因果推理能力的基准测试。它通过生成包含结构因果模型、观测数据和自然语言场景的合成实例，全面评估智能体在数据科学工作流中的因果推理、工具使用和不确定性量化等综合能力。

Details

Motivation: 当前LLM作为数据科学智能体的评估基准存在割裂：符号因果推理基准缺乏真实数据分析，而数据分析基准又缺乏严谨的因果数据生成结构。现有因果数据集多基于有限模板变体，缺乏系统生成的新颖合成因果结构。

Result: 该论文提出了CausalDS基准框架，但摘要中未提及具体的定量实验结果或与其他模型的对比数据。

Insight: 创新点在于将完全合成的因果结构生成与真实世界经验分布相结合，既保留了经验结构，又通过合成生成降低了’因果鹦鹉’风险。基准设计覆盖了Pearl因果阶梯的所有三个层级，并将’无法回答时弃权’作为首要评分结果，实现了对因果推理、数据科学、不确定性量化和工具使用的联合评估。

Abstract: Large language models (LLMs) increasingly act as integrated data-science agents, combining abstract reasoning with advanced tool use. Yet the relevant benchmark landscape largely divides into symbolic causal reasoning benchmarks without realistic data analysis or data analysis benchmarks without a principled causal data-generating structure. Furthermore, existing causal evaluation datasets are often restricted to curated examples from existing sources, with diversity coming from limited templatized variations rather than from systematic generation of novel synthetic causal structures. We introduce CausalDS, a benchmark for evaluating causal reasoning in agentic data-science workflows. Each benchmark instance is a scene consisting of a sampled structural causal model (SCM) with generated observational data and an accompanying synthetic natural-language story grounded in a realistic domain. We optionally ground the composition of the benchmark components in empirical distributions obtained from real-world datasets, thus retaining empirical structure while reducing the “causal parrot” risk through completely synthetic generation. From each scene, we then derive tasks spanning all three of Pearl’s rungs, with typical data-science prediction tasks appearing as Rung 1. Most tasks include a data science coding component, where the model typically needs to use several tools to arrive at the final answer due to the frequent presence of imperfect observations, which are generated by an observation model. Additionally, recognizing when a question admits no warranted answer and abstaining is treated as a first-class scored outcome. The benchmark thus jointly evaluates symbolic causal reasoning, data science, uncertainty quantification, abstention, and tool use/coding.

[68] Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning cs.AI | cs.CLPDF

Lu Dai, Ziyang Rao, Yili Wang, Hanqing Wang, Hao Liu

TL;DR: 本文研究了大型语言模型在微调过程中存在的“知识记忆-使用鸿沟”问题，即模型能快速记忆新事实，却无法将其用于下游推理任务。作者通过一种名为“自修补”的新干预技术，内部监测知识在模型中的空间渗透动态，发现记忆的表征可能因知识电路错位而未被路由到计算有效的层。基于此诊断，作者设计了一种简单的启发式策略，在泛化失败案例中恢复了58-75%的理想性能提升空间。

Details

Motivation: 解决大型语言模型在微调注入新知识时面临的核心挑战：模型能快速记忆新事实，但在下游推理任务中却无法有效利用这些知识，即存在“知识记忆-使用鸿沟”。

Result: 实验表明，通过提出的自修补技术识别出的关键激活位置，其表征重定位能显著改善泛化失败案例。基于此设计的简单启发式策略，在跨领域实验中恢复了58-75%的理想性能提升空间。

Insight: 论文的创新点在于将泛化失败形式化为“知识记忆-使用鸿沟”，并提出“自修补”这一新颖的干预技术来内部监测知识渗透动态，从而验证了“知识电路错位”的假设。这为理解LLM内部知识表示与计算路径的关联提供了机制性视角，并启发了基于诊断的实用改进策略。

Abstract: Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream reasoning tasks. We formalize this failure as the \textit{\textbf{Knowing–Using Gap}}, characterized by an accuracy gap and a temporal lag between memorization and generalization. To understand this phenomenon, we fine-tune LLMs with unseen knowledge and monitor the spatial permeation dynamics of the knowledge internally using a novel intervention technique called self-patching. Self-patching identifies activation locations where relocating representations substantially improves failed generalization cases. These results are consistent with a knowledge-circuit misalignment hypothesis: memorized representations can exist internally but may not be routed to computation-effective layers. To demonstrate the practicality of this diagnostic finding, we design a simple heuristic strategy which recovers 58–75% of the oracle headroom in generalization failure. Experiments are done cross-domain for the robustness of this finding.

[69] Different Teachers, Different Capabilities: Sub-1B On-Device Distillation for Structured Text Enrichment cs.AI | cs.CLPDF

Vinay Kumar Chaganti

TL;DR: 本文研究了将结构化文本富集任务从大型教师模型（8B推理模型）蒸馏到小型设备端学生模型（0.6B）的实际效果。通过对比不同教师（推理型、非推理型、管理流水线）的蒸馏结果，发现不同教师传递了不同的能力：推理教师提升了摘要质量，管理流水线提升了标签多样性。最终提出了一种基于任务子领域的设备端路由方案。

Details

Motivation: 解决大规模结构化提取任务中，大型模型延迟高、成本高的问题，旨在通过知识蒸馏将任务迁移到小型设备端模型，以实现可比的输出性能，同时大幅降低时间和成本。

Result: 在93条新闻文章测试集上，学生模型（0.6B）推理速度约为0.8秒/篇，远快于教师模型（8B）的39秒/篇。在摘要质量上，学生模型恢复了基础模型与教师模型之间差距的58%，比约束解码基线高出16.8个百分点，比少样本提示基线高出4.9个百分点。

Insight: 核心创新在于系统性地评估了不同能力类型教师（推理型 vs. 非推理型 vs. 管理流水线）对蒸馏结果的影响，揭示了教师模型的能力特性（如推理能力、标签多样性）会选择性传递给学生。这为设备端模型部署提供了基于任务子领域的能力路由策略，而非追求单一最优模型。

Abstract: High-volume structured extraction pays a large model’s latency on every item, so distilling the task into a small on-device model is attractive: comparable output at a fraction of the time and cost. We measure what that distillation actually delivers, per sub-task. Each news article is mapped to one JSON object with a short summary and five categorical labels. We distill an 8B reasoning teacher (deepseek-r1:8b) into a 0.6B student (Qwen3-0.6B; QLoRA, three seeds), and add two teacher controls: a same-size non-reasoning teacher and a larger managed pipeline. A blinded, reference-free, three-judge panel scores every arm against the full article, alongside two non-distillation baselines, few-shot prompting and constrained decoding. The student runs at about 0.8 s per article against the teacher’s 39 s, and recovers 58% of the base-to-teacher gap on summary quality, beating its primary baseline (constrained decoding) by +16.8 points and few-shot prompting by a secondary +4.9. A same-size non-reasoning teacher trains a student no better than the untuned base, so the summary gain follows from the teacher’s reasoning nature rather than its scale. Capabilities then split by teacher: the reasoning teacher transfers writing quality and the managed pipeline transfers label diversity, while a same-size instruction teacher’s students stay more grounded on the 22 short, thin-source articles in the 93-item test set (74 versus 55 faithful), where the reasoning-lineage student fabricates. That grounding difference is a consistent ordering rather than a significant aggregate effect, and the subgroup is small, so we report it as a direction. Because no single engine wins every field, the deliverable is a per-field routing map for on-device enrichment.

[70] Playing ZendoWorld: Challenging AI Agents on Active Visual Concept Induction cs.AI | cs.CVPDF

Sophia Koehler, Antonia Wüst, Inga Ibs, Wasu Top Piriyakulkij, Wolfgang Stammer

TL;DR: 该论文提出了ZendoWorld交互式环境，用于评估AI智能体在视觉概念归纳任务中的表现，包括感知、假设形成和主动实验设计能力。研究发现现有智能体（如VLM、贝叶斯滤波等）在规则归纳和主动信息获取方面存在显著缺陷，与人类表现存在差距。

Details

Motivation: 旨在解决智能系统在复杂感知、假设形成和主动实验设计方面的联合能力评估问题，为科学发现等领域的智能体开发提供测试平台。

Result: 在ZendoWorld基准测试中，现有智能体虽能高准确率预测观察样本标签，但无法恢复底层逻辑规则；VLM智能体设计的实验信息量接近零，人类数据则显示在复杂规则归纳上存在明显性能差距。

Insight: 创新点在于构建了可量化评估感知-归纳-实验设计闭环能力的视觉交互环境；关键发现揭示了感知与归纳能力解耦、主动信息获取不足等智能体核心瓶颈，为改进神经符号方法提供了方向。

Abstract: A central challenge in building intelligent systems is enabling agents to jointly perceive complex inputs, form hypotheses about hidden patterns, and design informative experiments to test them. To study this problem, we propose ZendoWorld, a controlled interactive environment in which agents must infer a logical rule about visual game observations, acquire information by proposing new scenes, and refine their hypotheses based on feedback from the game environment. We evaluate several agents spanning pure VLM reasoning, Bayesian particle filtering, dynamic concept discovery, and neuro-symbolic methods. Our main findings are: (1) high accuracy in predicting labels for observed examples does not imply recovery of the underlying rule; (2) perception and induction are distinct bottlenecks for different agent classes; and (3) VLM-based agents propose near-uninformative experiments, failing to actively reduce hypothesis uncertainty. To compare these results, we collect human data on the task, which reveals a gap in inductive reasoning, particularly for more complex rules. Overall, ZENDOWORLD takes an important step toward evaluating intelligent agents and identifies concrete avenues for improvement, particularly in domains like scientific discovery.

Raunak Mondal, Peter Washington

TL;DR: 本研究旨在优化基于视频的自闭症谱系障碍（ASD）相关自我刺激手部刻板行为的自动检测方法。研究重点包括：确定最佳的序列神经网络架构（LSTM和GRU）与时间采样率（帧间隔），以及评估在小规模行为数据集上训练时的数据增强策略。

Details

Motivation: 自闭症谱系障碍影响广泛，但目前可扩展的远程行为筛查计算方法有限。本研究致力于解决从视频中自动检测自闭症相关自我刺激行为的两大挑战：寻找最优的序列模型架构与采样率，以及为小数据集训练设计有效的数据增强策略。

Result: 在SSBD数据集上，LSTM和GRU模型在每15帧采样间隔下达到峰值准确率，分别为97.5%和98.75%，显著超越先前CNN基线（62-76%准确率）。在数据增强消融实验中，水平翻转单独使用效果最佳（48.78%准确率），而上采样被证明对复杂行为视频增强至关重要。个性化机器学习方法也产生了稳定的预测结果。

Insight: 论文的创新点在于系统性地评估了帧采样率对序列模型性能的影响，并量化了不同数据增强策略在稀缺临床视频数据上的边际贡献。其核心洞察是，在行为视频分类中，适中的时间下采样（如每15帧）结合循环神经网络（如GRU）能取得优异性能，且针对小数据集的增强策略需要精心设计，上采样是关键组件。

Abstract: Autism spectrum disorder (ASD) affects over 75 million individuals worldwide, yet scalable computational methods for remote behavioral screening remain limited. This study addresses two complementary challenges in automated detection of autism-related self-stimulatory behaviors from video: (1) identifying the optimal sequence-based neural network architecture and temporal sampling rate, and (2) characterizing data augmentation strategies for training on small behavioral datasets. For the first objective, long short-term memory (LSTM) and gated recurrent unit (GRU) models were trained on pose-derived features from the Self-Stimulatory Behavior Diagnosis (SSBD) dataset at frame sampling intervals of 1, 5, 15, 30, 45, and 90 frames. Both architectures exceeded prior convolutional neural network (CNN) baselines (62-76% accuracy), with peak accuracies of 97.5% (LSTM) and 98.75% (GRU) at a sampling interval of every 15 frames. For the second objective, ten data augmentation strategies were applied to an I3D transfer learning pipeline, with an ablation study quantifying the marginal contribution of each technique. Horizontal flip achieved the highest standalone accuracy (48.78%), while exclusion of upsampling from the augmentation pipeline produced the largest performance degradation, indicating its necessity for complex behavioral video augmentation. A personalized machine learning approach, in which per-subject models were trained and tested on temporally split segments of each video, produced consistent predictions (mean loss 1.84, SD 0.79). These results provide practitioners with concrete guidance on architecture selection, sampling rate, and augmentation strategy for video-based behavioral classification in data-scarce clinical domains.

[72] AUTOPILOT VQA: Benchmarking Vision-Language Models for Incident-Centric Dashcam Understanding cs.AI | cs.CVPDF

Siddharth Damodharan, Radhika Gupta, Ali Alshami, Ryan Rabinowitz, Jugal Kalita

TL;DR: 本文提出了AUTOPILOT-VQA，一个以事故为中心的视觉问答基准，用于评估视觉语言模型在行车记录仪视频中对安全关键事件的理解和推理能力。该基准包含围绕真实世界驾驶事故和险情设计的结构化问题，覆盖天气、交通环境、道路状况、事故细节等多个安全相关类别，旨在推动模型从物体识别向时序性、安全感知的推理发展。

Details

Motivation: 现有视觉语言模型在自动驾驶任务（如场景理解、决策）上虽有进步，但评估它们能否对安全关键事件进行可靠推理仍具挑战。为填补这一空白，作者创建了AUTOPILOT-VQA基准。

Result: 该基准作为AUTOPILOT CVPR 2026竞赛的一部分发布，为评估自动驾驶系统在不同场景下的可靠性提供了标准化测试平台。摘要未提及具体模型在该基准上的定量性能结果。

Insight: 主要创新在于构建了一个以事故为中心、强调安全感知推理的视觉问答基准，要求模型回答关于场景上下文和事件级事故细节的接地问题，从而推动模型向更可解释、鲁棒且安全意识的系统发展。

Abstract: Recent advances in Vision-Language Models, Large Language Models, and Multimodal Large Language Models have improved autonomous driving tasks such as scene understanding, decision making, trajectory prediction, and visual question answering. However, evaluating whether these models can reliably reason about safety-critical incidents remains challenging. To address this gap, we present AUTOPILOT-VQA, an incident-centric visual question answering benchmark for dashcam video understanding. The dataset evaluates different systems through structured questions designed around real-world driving incidents and near-incidents. The benchmark covers diverse safety-relevant categories, including weather and lighting conditions, traffic environment, road layout, road surface state, signage, involved entities, accident occurrence, impact location, and avoidability-related reasoning. By requiring models to answer grounded questions about both contextual scene properties and event-level incident details, AUTOPILOT-VQA moves beyond object recognition toward temporally grounded, safety-aware reasoning. The dataset is released as part of the AUTOPILOT CVPR 2026 competition and provides a standardized benchmark for assessing the reliability of autonomous driving systems in different scenarios. Our benchmark support developments for more interpretable, robust, and safety-conscious vision-language systems for real-world autonomous driving.

Table of Contents

cs.CL [Back]

[1] From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontier cs.CL | cs.AIPDF

[2] DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment cs.CLPDF

[3] Hallucination Self-Play: Bootstrapping Reinforced Detector via Evolved Generator cs.CL | cs.LGPDF

[4] When Implausible Tokens Get Reinforced: Tail-Aware Credit Calibration for LLM Reinforcement Learning cs.CL | cs.AI | cs.LGPDF

[5] Can We Trust LLM’s Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Framework cs.CL | cs.AIPDF

[6] What LLM Forecasters Know but Don’t Say: Probing Internal Representations for Calibration and Faithfulness cs.CL | cs.AIPDF

[7] MASTE: A Multi-Agent Pipeline for Zero-Shot Aspect Sentiment Triplet Extraction cs.CLPDF

[8] Diarization-Guided Qwen-ASR Adaptation for Multilingual Two-Speaker Conversational Speech cs.CLPDF

[9] When Synthetic Speech Is All You Have: Better Call GRPO cs.CL | cs.AIPDF

[10] UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing cs.CL | cs.AIPDF

[11] UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks cs.CLPDF

[12] Do You Need a Frontier Model as a Citation Verifier? Benchmarking Rubric LLMs for Deep-Research Source Attribution cs.CLPDF

cs.CV [Back]

[13] Adversarial Decoys: Misdirecting Attention-Based Defenses in ViT cs.CV | cs.AIPDF

[14] GIRAF: Towards Generalizable Human Interactions with Articulated Objects cs.CVPDF

[15] LightCrafter: PBR-Conditioned Video Diffusion Refinement for Controllable and Consistent Relighting cs.CV | cs.GRPDF

[16] SAGA: Stable Acceleration Guidance for Autoregressive Video Generation cs.CVPDF

[17] APIVOT: Adaptive Planning with Interleaved Vision-Language Thoughts cs.CV | cs.AI | cs.LG | cs.ROPDF

[18] UAV-OVVIS: Unmanned Aerial Vehicles Also Need Open-Vocabulary Video Instance Segmentation cs.CVPDF

[19] Post-Training in End-to-End Autonomous Driving cs.CV | cs.ROPDF

[20] Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratio cs.CV | cs.ROPDF

[21] VSRo-200: A Romanian Visual Speech Recognition Dataset for Studying Supervision and Multimodal Robustness cs.CVPDF

[22] Unified Face Attack Detection via Fine-Grained Semantic Guidance cs.CVPDF

[23] Leveraging Color Naming for Image Enhancement cs.CV | cs.AIPDF

[24] LEEVLA: Seeing What Matters in Latent Environment Evolution for Vision-Language-Action cs.CV | cs.AIPDF

[25] Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions cs.CVPDF

[26] Dual-Correlation Hypergraph Network for Unaligned RGBT Video Object Detection and A Large-scale Benchmark cs.CVPDF

[27] Dive Into the Implicit Biases of Low-rank Vision-language Alignment cs.CVPDF

[28] TMI: Text-to-Image Meets Image-to-Image for Complementary Data Synthesis to Boost Long-Tailed Instance Segmentation cs.CV | cs.AIPDF

[29] TVTA: Trajectory-Aware Viseme-Guided Temporal Aggregation for Event-Based Lip Reading cs.CVPDF

[30] Multimodal 3D LUT Generation via StatLUT with Statistical Features for Photorealistic Style Transfer cs.CVPDF

[31] SkelGen4D: Weakly-Supervised Skeleton-Based 4D Generation for Text-Driven Mesh Animation cs.CVPDF

[32] HSA: Hierarchical Slot Attention for Multi-granularity Scene-Decomposition cs.CVPDF

[33] UniRef-UAV: A Multimodal Benchmark for Universal Referring in UAV Imagery cs.CVPDF

[34] Enhancing the KidSat Model: Integrating Geographical Encoding and Data Quality Assessment for Childhood Poverty Prediction cs.CV | stat.APPDF

[35] ARGUS: Accelerated, Robust, General, and Unsupervised Cell Tracking Solutions cs.CV | math.OCPDF

[36] Texture Representations in Deep Vision Models: Comparing CNNs, Vision Transformers, and Human Perception cs.CVPDF

[37] WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving cs.CV | cs.AIPDF

[38] Attribute Retrieving for Open-Vocabulary Endoscopic Compositional Referring Segmentation cs.CVPDF

[39] VEGAS: Human-Aligned Video Caption Evaluation via Gaze cs.CV | cs.AI | cs.HCPDF

[40] Cognitive-structured Multimodal Agent for Multimodal Understanding, Generation, and Editing cs.CV | cs.AI | cs.CL | cs.LGPDF

[41] Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery cs.CV | cs.AIPDF

[42] DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models cs.CVPDF

[43] CT-CLIP Representations for Multimodal Lung Cancer Survival Prediction cs.CVPDF

[44] Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues? cs.CVPDF

[45] Whareformer: Learning to Track What is Where in Long Egocentric Videos cs.CVPDF

[46] VocaDet: Sample-Driven Open-Vocabulary Object Detection and Segmentation via Visual Tokenization and Vector Database Retrieval cs.CV | cs.AIPDF

[47] Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning cs.CVPDF

[48] Multi-Resolution Feature Stem for Diabetic Retinopathy lesion segmentation cs.CVPDF

[49] SAM-MT: Real-Time Interactive Multi-Target Video Segmentation cs.CVPDF

[50] When Structured Sparse Autoencoders Learn Consistent Concepts Across Modalities cs.CV | cs.AI | cs.LGPDF

[51] HumanForge: A Human-Centric Deepfake Video Benchmark with Multi-Agent Forgery Rationales cs.CVPDF

[52] LTM: Large-scale Terrain Model for Wildfire-prone Landscapes cs.CV | cs.LGPDF

[53] Pose-to-Biomechanics: Bridging 3D Human Pose Estimation and Biomechanical Attribute Prediction cs.CV | cs.AI | cs.LGPDF

[54] OpenCoF: Learning to Reason Through Video Generation cs.CV | cs.AIPDF

[55] WaspMOT: A Benchmark for Long-Term Multi-Object Tracking of Trichogramma Wasps cs.CVPDF

[56] Enhancing In-context Panoramic Generation via Geometric-aware Pretraining cs.CVPDF

[57] Geometry and Gradient-based Partitioning for Panoramic Outdoor Reconstruction cs.CVPDF

[58] OPSD-V: On-Policy Self-Distillation for Post-Training Few-Step Autoregressive Video Generators cs.CVPDF

[59] LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation with Video Diffusion Models cs.CVPDF

[60] Wat3R: Underwater 3D Geometry Learning without Annotations cs.CVPDF

cs.RO [Back]

[61] Time-to-Collision Based Dynamic Obstacle Avoidance Using Pretrained Vision Models for Robots in Unstructured Environments cs.RO | cs.AI | cs.CV | eess.IVPDF

[62] Native Video-Action Pretraining for Generalizable Robot Control cs.RO | cs.CVPDF

cs.MA [Back]

[63] Collective Intelligence with Foundation Models cs.MA | cs.AI | cs.CLPDF

cs.GR [Back]

[64] ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation cs.GR | cs.CV | cs.LG | cs.ROPDF

cs.LG [Back]

[65] Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks cs.LG | cs.AI | cs.CL | cs.CR | cs.MMPDF

[66] Ensemble Diversity Optimization for Subjective Supervision cs.LG | cs.CLPDF

cs.AI [Back]

[67] CausalDS: Benchmarking Causal Reasoning in Data-Science Agents cs.AI | cs.CL | cs.LGPDF

[68] Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning cs.AI | cs.CLPDF

[69] Different Teachers, Different Capabilities: Sub-1B On-Device Distillation for Structured Text Enrichment cs.AI | cs.CLPDF

[70] Playing ZendoWorld: Challenging AI Agents on Active Visual Concept Induction cs.AI | cs.CVPDF

[71] Evaluating the Effect of Frame Rate in Sequence-Based Classification of Autism-Related Self-Stimulatory Hand Idiosyncrasies cs.AI | cs.CV | cs.LGPDF

[72] AUTOPILOT VQA: Benchmarking Vision-Language Models for Incident-Centric Dashcam Understanding cs.AI | cs.CVPDF