cs.CL [Total: 17]
cs.CV [Total: 98]
cs.AI [Total: 7]
cs.LG [Total: 11]
cs.RO [Total: 4]
eess.IV [Total: 3]

cs.CL [Back]

[1] Reflective Prompt Tuning through Language Model Function-Calling cs.CLPDF

Farima Fatahi Bayat, Moin Aminnaseri, Pouya Pezeshkpour, Estevam Hruschka

TL;DR: 本文提出了一种名为Reflective Prompt Tuning（RPT）的框架，利用大语言模型（LLM）的函数调用能力来模拟人类提示工程师的迭代工作流程，以自动化地优化提示（prompt）。该方法通过诊断函数在整个优化集上评估目标模型、总结重复出现的失败模式，并基于结构化的诊断报告和历史记忆来迭代修订提示，从而提升任务性能和置信度校准。

Details

Motivation: 现有提示设计方法劳动密集且对格式、措辞和指令顺序高度敏感，而自动化的提示优化方法通常基于单个示例或小批量数据进行搜索或固定流程的批判-精炼，难以捕捉系统性的错误模式并基于失败历史进行有针对性的编辑。

Result: 在三个推理任务上，RPT相比初始提示提升了高达12.9个百分点，与现有最优方法（SOTA）保持竞争力，并改善了置信度校准。分析表明，RPT在多跳推理和数学推理任务上尤其有效。

Insight: 创新点在于利用LLM的函数调用能力构建了一个模拟人类反思迭代的自动化提示优化框架，通过在整个优化集上进行结构化诊断和累积历史报告来指导有针对性的提示修订，并整合了置信度感知的优化机制。

Abstract: Large language models (LLMs) have become increasingly capable of following instructions and complex reasoning, making prompting a flexible interface for adapting models without parameter updates. Yet prompt design remains labor-intensive and highly sensitive to formatting, phrasing, and instruction order, motivating automated prompt optimization methods that reduce manual effort while preserving inference-time flexibility. However, existing methods often search over prompt candidates or use fixed critique-refine pipelines driven by individual examples or small batches, limiting their ability to capture systematic error patterns and make targeted edits grounded in failure history. We propose Reflective Prompt Tuning (RPT), a framework that uses LLM function calling to simulate the iterative workflow of human prompt engineers. An LLM optimizer calls a diagnostic function that evaluates the target model over an entire optimization set, summarizes recurring failure modes, and returns a structured diagnostic report. The optimizer uses this report, together with an accumulated memory of prior reports, to revise the prompt for the next iteration. RPT further supports confidence-aware optimization by using calibration signals in diagnostic feedback and final prompt selection. Across three reasoning tasks, RPT improves over initial prompts by up to 12.9 points, remains competitive with state of the art, and improves confidence calibration. Our analyses show that RPT is especially effective on multi-hop and mathematical reasoning, producing targeted prompt revisions that align with diagnosed failure patterns and lead to gains in task performance and calibration.

[2] When Cases Get Rare: A Retrieval Benchmark for Off-Guideline Clinical Question Answering cs.CLPDF

Doeun Lee, Muge Zhang, Yi Yu, Ashish Manne, Stephen Koesters

TL;DR: 本文介绍了OGCaReBench，一个专注于自由形式检索的基准测试，旨在评估大语言模型（LLMs）在回答需要超越典型临床指南的医学问题上的能力。该基准从已发表的医学病例报告中提取，并由医学专家验证，包含需要自由文本答案的长篇临床问题。实验表明，即使性能最佳的基线模型（GPT-5.2）也仅能正确回答56%的问题，而通过检索医学文章进行增强可将性能提升至82%。

Details

Motivation: 解决当前医学大语言模型主要依赖记忆常见、指南导向的知识，而无法有效处理现实中大量未被指南覆盖的罕见、长尾临床病例的问题。

Result: 在OGCaReBench基准上，最佳基线模型GPT-5.2的正确率为56%，专业医学模型仅为42%。通过检索增强后，GPT-5.2的性能提升至82%。

Insight: 创新点在于构建了一个专注于罕见、超越指南的临床问答的检索基准，强调了证据检索（evidence-grounding）对于现实世界医学推理任务的重要性，为评估和推进通用及医学专用LLMs在复杂临床场景中的可靠性奠定了基础。

Abstract: Across medical specialties, clinical practice is anchored in evidence-based guidelines that codify best studied diagnostic and treatment pathways. These pathways routinely fall short for the long tail of real-world care not covered by guidelines. Most medical large language models (LLMs), however, are trained to encode common, guideline-focused medical knowledge in their parameters. Current evaluations test models primarily on recalling and reasoning with this memorized content, often in multiple-choice settings. Given the fundamental importance of evidence-based reasoning in medicine, it is neither feasible nor reliable to depend on memorization in practice. To address this gap, we introduce OGCaReBench, a free-form retrieval-focused benchmark aimed at evaluating LLMs at answering clinical questions that require going beyond typical guidelines. Extracted from published medical case reports and validated by medical experts, OGCaReBench contains long-form clinical questions requiring free-text answers, providing a systematic framework for assessing open-ended medical reasoning in rare, case-based scenarios. Our experiments reveal that even the best-performing baseline (GPT-5.2) correctly answers only 56% of our benchmark with specialized models only reaching 42%. Augmenting models with retrieved medical articles improves this performance to up to 82% (using GPT-5.2) highlighting the importance of evidence-grounding for real-world medical reasoning tasks. This work thus establishes a foundation for benchmarking and advancing both general-purpose and medical LLMs to produce reliable answers in challenging clinical contexts.

[3] ACC: Compiling Agent Trajectories for Long-Context Training cs.CL | cs.AIPDF

Qisheng Su, Zhen Fang, Shiting Huang, Yu Zeng, Yiming Zhao

TL;DR: 本文提出了一种名为Agent Context Compilation (ACC)的方法，用于将智能体在解决问题过程中产生的多轮交互轨迹（包括工具调用和环境观察）编译成包含长上下文信息的问答对，从而为LLMs提供可扩展的长上下文推理监督训练数据。该方法无需额外标注，并能与现有长上下文扩展技术结合，有效提升模型在需要跨轮次指代消解和图遍历等长程依赖任务上的性能。

Details

Motivation: 当前智能体的发展对LLMs的长上下文推理能力提出了新需求，但训练这种能力通常需要成本高昂的长文档整理或启发式的上下文合成。作者观察到，智能体轨迹中分散在多轮交互中的证据未被标准监督微调有效利用，形成了监督盲区。

Result: 在MRCR和GraphWalks这两个需要长程依赖建模的基准测试上，使用ACC训练的Qwen3-30B-A3B模型分别取得了68.3分（提升18.1）和77.5分（提升7.6）的成绩，其性能可与更大的Qwen3-235B-A22B模型相媲美，同时在GPQA、MMLU-Pro等通用能力基准上保持了原有水平。

Insight: 核心创新点在于将智能体的多轮交互轨迹编译成显式包含问题与分散证据依赖关系的长上下文QA对，从而为长上下文推理提供了直接的监督信号。这提供了一种简单、可扩展且无需额外标注的数据生成方法，其机制分析还揭示了模型在训练后出现了任务自适应的注意力重组和专家特化现象。

Abstract: Recent development of agents has renewed demand for long-context reasoning capacity of LLMs. However, training LLMs for this capacity requires costly long-document curation or heuristic context synthesis. We observe that agents produce massive trajectories when solving problems, invoking tools and receiving environment observations across many turns. The evidence needed to answer the original question is thus scattered throughout these turns, requiring integration of distant context segments. Nevertheless, standard agent SFT masks tool responses and only trains turn-level tool selection, creating a supervision blind spot where these scattered signals go unused. We propose Agent Context Compilation (ACC), which converts trajectories from search, software engineering, and database querying agents into long-context QA pairs that combine the original question with tool responses and environment observations gathered across multiple turns, training the model to answer directly without tool use. This makes the dependencies between the question and the evidence explicit, enabling direct supervision of long-context reasoning over distant segments without additional annotation. ACC is a simple but effective approach that can be combined with any existing long-context extension or training method, providing scalable supervised fine-tuning data. We validate ACC on long-range dependency modeling tasks through MRCR and GraphWalks, challenging benchmarks requiring cross-turn coreference resolution and graph traversal over extended contexts. Training Qwen3-30B-A3B with ACC achieves 68.3 on MRCR (+18.1) and 77.5 on GraphWalks (+7.6), results comparable to Qwen3-235B-A22B, while preserving general capabilities on GPQA, MMLU-Pro, AIME, and IFEval. Further mechanism analysis reveals that the ACC-trained model exhibits task-adaptive attention restructuring and expert specialization.

Yifan Dai, Zhenhua Wu, Bohan Zeng, Daili Hua, Jialing Liu

TL;DR: 本文提出LatentOmni，一种用于音频-视觉联合推理的跨模态框架，通过将文本推理与音频-视觉潜在状态交织，在统一潜在空间中进行推理，以克服现有多模态大语言模型在细粒度跨模态推理上的局限。

Details

Motivation: 当前多模态大语言模型在需要音频和视觉细粒度证据的联合推理任务上表现不佳，因为基于文本的思维链将连续信号压缩为离散标记，削弱了时序对齐并偏向语言先验。

Result: 在多个音频-视觉推理基准测试中，LatentOmni在开源模型中达到最佳性能，并持续优于显式文本思维链基线。

Insight: 创新点在于提出使用统一潜在空间作为推理媒介以保留密集感官信息，并引入特征级监督和Omni-Sync位置嵌入来对齐潜在状态与任务相关特征及保持时序一致性；同时构建了包含35K条音频-视觉交织推理轨迹的数据集用于监督潜在空间推理。

Abstract: Joint audio-visual reasoning is essential for omnimodal understanding, yet current multimodal large language models (MLLMs) still struggle when reasoning requires fine-grained evidence from both modalities. A central limitation is that explicit text-based chain-of-thought (CoT) compresses continuous audio-visual signals into discrete tokens, weakening temporal grounding and shifting intermediate reasoning toward language priors. We argue that a unified latent space is a better medium for such reasoning because it preserves dense sensory information while remaining compatible with autoregressive generation. Based on this insight, we propose \textbf{LatentOmni}, a cross-modal reasoning framework that interleaves textual reasoning with audio-visual latent states. LatentOmni introduces feature-level supervision to align latent reasoning states with task-relevant sensory features and uses Omni-Sync Position Embedding (OSPE) to maintain temporal consistency between latent audio and visual states. We further construct \textbf{LatentOmni-Instruct-35K}, a dataset of audio-visual interleaved reasoning trajectories for supervising latent-space reasoning. Comprehensive evaluation across multiple audio-visual reasoning benchmarks demonstrates that LatentOmni achieves the best performance among the evaluated open-source models and consistently outperforms the Explicit Text CoT baseline, supporting latent-space joint reasoning as a promising path toward stronger omnimodal understanding.

[5] Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention cs.CL | cs.CVPDF

Changyuan Tian, Zhicong Lu, Huaxing Liu, Xiang Wang, Shuai Li

TL;DR: 本文提出Faithful-MR1框架，通过锚定和强化视觉注意力来解决多模态大语言模型在推理中的忠实性问题。该框架包含两个阶段：锚定阶段将感知作为显式子任务，直接监督模型关注图像区域；强化阶段通过反事实图像干预来奖励正确利用视觉证据的推理轨迹。实验表明，该方法在Qwen2.5-VL模型上优于现有基线，且所需训练数据更少。

Details

Motivation: 现有基于可验证奖励的强化学习方法扩展到多模态大语言模型时，面临忠实性挑战：模型难以忠实地感知任务相关的视觉证据，并在推理中忠实地使用这些证据，导致多模态基准测试性能提升有限。

Result: 在Qwen2.5-VL-Instruct 3B和7B骨干网络上进行的广泛实验表明，Faithful-MR1优于近期的多模态推理基线方法，同时使用的训练数据显著减少。

Insight: 创新点在于将忠实性分解为感知和使用两个部分，并分别通过锚定（直接监督图像区域注意力）和强化（反事实干预奖励因果性视觉关注）来解决。这提供了一种显式连接多模态感知与推理、确保视觉证据被忠实利用的新训练范式。

Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a promising paradigm for advancing complex reasoning in large language models, and recent work extends RLVR to multimodal large language models (MLLMs). This transfer, however, surfaces a faithfulness challenge: faithful perception of task-relevant visual evidence and faithful use of that evidence during reasoning, leading to unsatisfactory gains on multimodal benchmarks. Specifically, existing perception supervision often operates on textual descriptions rather than natively on image regions, and faithful use is largely overlooked, exposing the perception-reasoning disconnect where correctly perceived evidence is dropped or contradicted during reasoning. To close these gaps, we propose Faithful-MR1, a training framework that anchors and reinforces visual attention to address both halves of faithful multimodal reasoning. The Anchoring stage turns perception into an explicit pre-reasoning subtask, supervising a dedicated token’s attention directly against image regions rather than through textual descriptions. The Reinforcing stage exposes faithful use through counterfactual image intervention, rewarding answer-correct trajectories that concentrate visual attention where vision causally matters. Extensive experiments demonstrate that Faithful-MR1 outperforms recent multimodal reasoning baselines on both Qwen2.5-VL-Instruct 3B and 7B backbones while using substantially less training data.

[6] ArabDiscrim: A Decade-Long Arabic Facebook Corpus on Racism and Discrimination cs.CLPDF

Wajdi Zaghouani, Shimaa Amer Ibrahim, Mabrouka Bessghaier, Houda Bouamor

TL;DR: 本文介绍了ArabDiscrim，这是一个长达十年的阿拉伯语Facebook语料库，包含29.3万条公开帖子（2014-2024年），专注于讨论种族主义和歧视话题。与现有以Twitter为中心的语料库不同，它整合了平台原生的互动信号（如反应、分享、评论）和页面元数据，支持语言与受众反应的联合分析。该资源包含200个精选术语、形态学正则表达式家族以及20个歧视维度，旨在为面向公平、平台感知的阿拉伯语NLP研究奠定基础。

Details

Motivation: 现有关于种族主义和歧视的阿拉伯语数据集主要基于Twitter，缺乏对Facebook等主流社交媒体平台生态及用户互动信号的整合研究。

Result: 该研究构建并发布了包含29.3万条帖子的语料库ArabDiscrim，提供了术语集、形态学变体模式和歧视维度标注，支持弱监督、维度感知抽样和平台生态研究。

Insight: 创新点在于将平台原生互动信号（如反应、分享）与文本内容整合分析，增强了生态效度；同时通过形态学正则表达式家族处理阿拉伯语复杂的词形变化，并明确标注歧视的具体维度，为跨平台、细粒度的公平性NLP研究提供了新资源和方法思路。

Abstract: We present ArabDiscrim, a decade-long lexical resource and corpus of 293K public Arabic Facebook posts (2014–2024) discussing racism and discrimination. Unlike existing Twitter-centric datasets, ArabDiscrim integrates platform-native engagement signals, including reactions, shares, comments, and page metadata, enabling joint analysis of language and audience response. The resource includes 200 curated terms (100 racism-related and 100 discrimination-related) with morphological regex families (13+ inflections per lemma), and 20 discrimination axes capturing identity-based grounds for unequal treatment. It also provides explicit attribution patterns. Released under a restricted research-use license for ethical compliance with platform terms, ArabDiscrim supports weak supervision, axis-aware sampling, and platform ecology research. By bridging lexical depth and ecological validity, it establishes a foundation for fairness-oriented, platform-aware Arabic NLP.

[7] Psy-Chronicle:A Structured Pipeline for Synthesizing Long-Horizon Campus Psychological Counseling Dialogues cs.CLPDF

Chaogui Gou, Jiarui Liang

TL;DR: 本文提出了Psy-Chronicle，一个用于合成长期校园心理咨询对话的结构化数据生成框架。该框架通过生成学期跨度的时间压力事件图来建模校园压力事件的时序和演化依赖关系，并利用学生与咨询师智能体间的交互模拟以及结构化记忆整合机制，生成了具有跨会话连续性的长程对话。基于此框架，作者构建并开源了中文长程校园心理咨询数据集CPCD及其评估基准CPCD-Bench。

Details

Motivation: 现有心理咨询数据多为单轮问答或短多轮对话，难以刻画大学生心理困扰在校园生活事件中长期累积、交互和演化的过程。

Result: 实验表明，CPCD数据集能有效提升同基座模型在会话级响应生成和长程记忆召回方面的能力。然而，在时序因果推理方面的提升有限，表明事件链组织和因果解释是长程心理咨询建模的关键挑战。

Insight: 创新点在于提出了一个结合时间压力事件图、智能体交互模拟和结构化记忆的框架来生成具有长期连续性的心理咨询对话，并构建了相应的数据集和评估基准，揭示了长程建模中事件链与因果推理的难点。

Abstract: In recent years, large language models have shown substantial potential in psychological support tasks. However, existing psychological counseling data mostly rely on single-turn question answering or short multi-turn dialogues, making it difficult to characterize how college students’ psychological distress accumulates, interacts, and gradually evolves over long periods within campus life events. To address this issue, this paper proposes Psy-Chronicle, a structured data-generation framework for synthesizing long-horizon campus psychological counseling dialogues. We generate a semester-spanning temporal stress event graph to model the chronological order and evolutionary dependencies among campus stress events. Through interactive simulation between a student agent and a counselor agent, together with a structured memory integration mechanism, Psy-Chronicle generates long-horizon dialogues with continuity across counseling sessions. Based on Psy-Chronicle, we construct and open-source CPCD, a Chinese long-horizon dialogue dataset for college psychological counseling, containing 100 student profiles, 90,000 counseling dialogues. We further build CPCD-Bench to evaluate models’ long-horizon campus counseling capabilities from three dimensions: session-level response, long-horizon memory recall, and temporal-causal reasoning. Experimental results show that CPCD effectively improves session-level response generation and long-horizon memory recall for models with the same base architecture. Meanwhile, improvements in temporal-causal reasoning remain limited, indicating that event-chain organization and causal explanation are key challenges in long-horizon psychological counseling modeling. The related code and data are available at: https://github.com/EdwinUSTB/Psy-Chronicle

[8] Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models? cs.CLPDF

Luca Modica, Filip Landin, Mehrdad Farahani, Livia Qian, Gabriel Skantze

TL;DR: 本文研究了多模态语言模型中事实回忆机制在文本与语音模态间的迁移性，通过因果中介分析发现，尽管模型整合了离散语音标记，但文本到文本与语音到文本的结果存在差异，表明事实回忆机制仅部分从文本迁移到语音。

Details

Motivation: 探究多模态语言模型在文本和语音两种模态下，内部机制（特别是事实知识的编码、存储和检索）的相似性与差异性，以深化对语音语言模型的理解。

Result: 使用整合离散语音标记的多模态模型SpiritLM进行实验，结果显示文本到文本与语音到文本的结果存在不一致，表明事实回忆机制在模态间仅部分迁移。

Insight: 创新点在于将因果中介分析应用于多模态语言模型，以揭示事实关联的存储和回忆机制；客观分析表明，语音模态的事实处理机制可能与文本模态不同，这为改进语音AI系统提供了重要见解。

Abstract: In recent years, several Speech Language Models (SLMs) that represent speech and written text jointly have been presented. The question then emerges about how model-internal mechanisms are similar and different when operating in the two modalities. We focus on how these systems encode, store, and retrieve factual knowledge, which has previously been investigated for text-only models. To investigate mechanisms behind the storage and recall of factual association in SLMs, we leverage Causal Mediation Analysis, a technique previously applied to text-based models. Initial results using SpiritLM, a multimodal model integrating discrete speech tokens reveal discrepancies between text-to-text and speech-to-text results, suggesting that the emergent mechanisms for factual recall are only partially carried over from the text to the speech modality. These results advance our understanding of how internal mechanisms encode factual associations in SLMs while contributing insights for improving speech-enabled AI systems.

[9] Unified Data Selection for LLM Reasoning cs.CLPDF

Xiaoyuan Li, Yubo Ma, Chengpeng Li, Fengbin Zhu, Yiyao Yu

TL;DR: 本文提出了一种名为高熵和（HES）的无训练度量方法，用于量化大语言模型（LLM）推理数据的质量。该方法通过仅计算每个推理样本中最高熵（如前0.5%）令牌的熵值之和，来高效地区分高质量与低质量推理数据。论文在监督微调（SFT）、拒绝微调（RFT）和强化学习（RL）三种主流训练范式上验证了HES的有效性，证明了其能统一、高效地提升LLM的复杂推理能力。

Details

Motivation: 当前训练LLM进行复杂、长链思维链（CoT）推理的瓶颈在于需要海量高质量推理数据，而现有方法要么计算成本高，要么无法可靠地区分数据质量。

Result: 在SFT中，仅使用HES排名前20%的数据进行训练，其性能与使用全部数据集相当，而使用低HES数据则导致性能下降。在RFT中，基于HES的训练方法显著优于基线方法。在RL中，使用HES选择的成功轨迹使模型能学习到强推理模式，显著超越其他对比方法。

Insight: 论文的核心创新点是提出了一种无训练、计算开销低的度量标准HES，它通过聚焦于推理过程中最不确定（高熵）的令牌来评估数据质量，为不同训练范式提供了一个统一、高效的数据选择方案，从而有效缓解高质量推理数据稀缺的瓶颈问题。

Abstract: Effectively training Large Language Models (LLMs) for complex, long-CoT reasoning is often bottlenecked by the need for massive high-quality reasoning data. Existing methods are either computationally expensive or fail to reliably distinguish high- from low-quality reasoning samples. To address this, we propose High-Entropy Sum (HES), a training-free metric that quantifies reasoning quality by summing only the entropy of the top (e.g., 0.5%) highest-entropy tokens in each reasoning sample. We validate HES across three mainstream training paradigms: Supervised Fine-tuning (SFT), Rejection Fine-tuning (RFT), and Reinforcement Learning (RL), with extensive results demonstrating its consistent effectiveness and significantly reduced computational overhead. In SFT, training on the top 20% HES-ranked data matches full-dataset performance, while using the lowest-HES data degrades it. In RFT, our HES-based training approach significantly outperforms baseline methods. In RL, HES-selected successful trajectories enable the model to learn strong reasoning patterns, significantly surpassing other compared methods. Our findings establish HES as a robust, training-free metric that enables a unified, effective, and efficient method for developing advanced reasoning in LLMs.

[10] DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Memory QA cs.CL | cs.AI | cs.LGPDF

Jianing Yin, Tan Tang

TL;DR: DeferMem是一个用于长时记忆问答的框架，通过强化学习在查询时进行证据蒸馏。它采用轻量级的分段链接结构组织原始历史记录并检索候选证据，然后使用DistillPO算法将高召回但噪声大的候选证据蒸馏为忠实、自包含且与查询相关的证据。

Details

Motivation: 解决大型语言模型在长时记忆问答中面临的挑战，即答案支持证据分散在长对话历史中且被大量无关内容淹没，现有记忆系统通常在查询未知时处理记忆，基于相似性而非实用性检索，导致下游答案生成器需要去噪和重构证据。

Result: 在LoCoMo和LongMemEval-S基准测试中，DeferMem在问答准确性和记忆系统效率方面超越了强基线，实现了最高的QA准确率、最快的运行速度，且记忆操作零商业API令牌成本。

Insight: 创新点包括将长时记忆问题解耦为高召回候选检索和查询条件证据蒸馏，并引入DistillPO强化学习算法，通过结构化动作（消息选择和证据重写）和分解门控奖励管道优化证据蒸馏过程，提高证据的忠实性和自包含性。

Abstract: Large language model (LLM) agents still struggle with long-term memory question answering, where answer-supporting evidence is often scattered across long conversational histories and buried in substantial irrelevant content. Existing memory systems typically process memory before future queries are known, then retrieve the resulting units based on similarity rather than their utility for answering the query. This workflow leaves downstream answerers to denoise retrieved candidates and reconstruct query-specific evidence. We present DeferMem, a long-term memory framework that decouples this problem into high-recall candidate retrieval and query-conditioned evidence distillation. DeferMem uses a lightweight segment-link structure to organize raw history and retrieve broad candidates at query time. It then applies a memory distiller trained with DistillPO, our reinforcement learning algorithm for distilling the high-recall but highly noisy candidates into a set of faithful, self-contained, and query-conditioned evidence. DistillPO formulates post-retrieval evidence distillation as a structured action comprising message selection and evidence rewriting. It optimizes this action with a decomposed-and-gated reward pipeline and structure-aligned advantage assignment, gating reward components from validity to quality checks while exposing task-level correctness feedback early and assigning each reward to its responsible output span. On LoCoMo and LongMemEval-S, DeferMem surpasses strong baselines in QA accuracy and memory-system efficiency, achieving the highest QA accuracy with the fastest runtime and zero commercial-API token cost for memory operations.

[11] GHI: Graphormer over Conditioned Hypergraph Incidence for Aspect-Based Sentiment Analysis cs.CLPDF

Yu Du, Wenlong Zhu, Xingze Li, Chenglong Cao, Jing Wang

TL;DR: GHI是一个基于条件超图关联的Graphormer框架，用于方面级情感分析（ABSA）。该框架通过二分拓扑构建基于关联的结构推理层，将多样化的语言和语义证据表示为令牌-超边关联关系，实现了不同结构信号的统一整合。

Details

Motivation: 方面级情感分析需要模型将情感证据与正确的方面绑定，这使其成为细粒度结构推理的自然测试平台。GHI旨在通过统一的结构推理层解决传统模型在复杂语义关联和鲁棒性方面的不足。

Result: 在六个标准ABSA基准测试中，GHI在SemEval领域优于所有基线模型，多种子评估显示其对强基线DeBERTa有稳定提升。仅用2.47亿参数的GHI在ISE基准上接近110亿参数Flan-T5方法的性能，并在挑战性ARTS数据集上表现出强鲁棒性。

Insight: 创新点在于将条件超图关联与Graphormer结合，通过二分拓扑统一处理多种结构信号。客观来看，该研究证明了紧凑的结构推理模型可作为规模驱动方法的有效替代方案，尤其在细粒度任务中保持高性能和鲁棒性。

Abstract: Aspect-based sentiment analysis (ABSA) requires models to bind sentiment evidence to the correct aspect, making it a natural testbed for fine-grained structural reasoning. We introduce GHI, a Graphormer-over-Conditioned-Hypergraph-Incidence framework that is designed as an incidence-based structural reasoning layer built on a bipartite topology. GHI represents diverse linguistic and semantic evidence as token–hyperedge incidence relations, allowing different structural signals to be incorporated through a unified interface. Extensive experiments on six standard ABSA benchmarks show that GHI outperforms all baselines on the SemEval domains, and multi-seed evaluations show stable improvements over strong DeBERTa. Further experiments show that with only 247M parameters, GHI approaches the performance of 11B Flan-T5 based methods on the ISE benchmark. Moreover, it demonstrates strong robustness on the challenging ARTS datasets, maintaining highly competitive performance where traditional models degrade. These results demonstrate that compact structural reasoning remains a valuable alternative to scale-driven approaches for fine-grained tasks.

[12] Polite on the Surface, Wrong in Practice: A Curated Dataset for Fixing Honorific Failures in Multilingual Bangla Generation cs.CLPDF

Md. Asaduzzaman Shuvo, Mahedi Hasan, Md. Tashin Parvez, Azizul Haque Noman, Md. Shafayet Hossain Ovi

TL;DR: 本文针对多语言大语言模型在孟加拉语生成中存在的敬语使用不当问题，引入了一个名为BLADE的、包含4196个精心策划的交互对的文化对齐指令微调数据集和基准测试框架。作者使用LoRA适配器在4位NF4量化框架下对DeepSeek-8B和LLaMA-3.2-3B等模型进行微调，结果表明，经过该数据集微调的模型在结构保真度和敬语对齐方面取得了显著改进。

Details

Motivation: 尽管多语言大语言模型增强了跨语言对话能力，但在处理低资源语言（如孟加拉语）的结构变体、区域习语和敬语一致性方面，现有最先进的模型仍存在严重的语用鸿沟，无法实现符合文化背景和语境的交流。

Result: 基于BLADE数据集对领先的开源模型（如DeepSeek-8B和LLaMA-3.2-3B）进行微调后，模型在结构保真度和敬语对齐方面取得了实质性提升，为弥合低资源多语言文本生成中的语用差异提供了一个严格的基准。

Insight: 论文的创新点在于创建了一个专门针对孟加拉语文化细微差别（特别是敬语使用）的指令微调数据集，并通过参数高效的LoRA微调方法，有效提升了模型在特定文化语境下的语用能力。这为改善低资源语言生成模型的实用性和文化适应性提供了可借鉴的数据集构建和微调策略。

Abstract: Recent advances in Multilingual Large Language Models (MLLMs) have significantly enhanced cross-lingual conversational capabilities, yet modeling culturally nuanced and context-dependent communication remains a critical bottleneck. Specifically, existing state-of-the-art models exhibit a severe pragmatic gap when handling structural variations, regional idioms, and honorific consistencies in low-resource contexts like Bangla. To address this limitation, we introduce a novel, culturally aligned instruction-tuning dataset for \textbf{BangLa Application and DialoguE generation - BLADE} and benchmarking framework comprising $4,196$ meticulously curated interaction pairs. We leverage this resource to systematically fine-tune and evaluate leading open-weight architectures, including DeepSeek-8B and LLaMA-3.2-3B, utilizing parameter-efficient fine-tuning via LoRA adapters in a 4-bit NormalFloat (NF4) quantization framework. Our empirical evaluations demonstrate that models fine-tuned on our dataset yield substantial improvements in structural fidelity and honorific alignment, providing a rigorous benchmark for bridging pragmatic disparities in low-resource multilingual text generation. Code and dataset: https://github.com/ashuvo25/Bangla_Application_LLM/tree/main

[13] LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance cs.CLPDF

Yuchun Fan, Bei Li, Peiguang Li, Yilin Wang, Yongyu Mu

TL;DR: 本文提出了LANG框架，一种利用语言条件提示来指导非英语推理任务探索的强化学习方法。该方法通过渐进衰减调度和语言自适应切换机制，在保持语言一致性的同时显著提升了多语言推理性能，并展示了在数学以外领域的泛化能力。

Details

Motivation: 现有强化学习方法在多语言环境中面临权衡：优先输入语言一致性会严重损害推理质量，而优先推理则常导致语言向英语漂移。本文旨在解决这一挑战，实现推理性能与语言一致性的平衡。

Result: 在具有挑战性的多语言数学基准测试中，LANG显著提升了推理性能，且未损害语言一致性。结果表明，该框架能推广到数学以外的领域，促进跨模型层更一致的语言对齐。

Insight: 创新点在于引入语言条件提示来引导探索，并设计了渐进衰减和语言自适应切换机制以防止对提示的依赖。这为多语言环境下的强化学习推理提供了一种平衡性能与一致性的新思路。

Abstract: Reinforcement learning has proven effective for enhancing multi-step reasoning in large language models (LLMs), yet its benefits have not fully translated to multilingual contexts. Existing methods struggle with a fundamental trade-off: prioritizing input-language consistency severely hampers reasoning quality, while prioritizing reasoning often leads to unintended language drift toward English. We address this challenge with LANG, a novel framework that leverages language-conditioned hints to guide exploration in non-English reasoning tasks. Our method incorporates two key mechanisms to prevent dependency on these hints: a progressive decay schedule that gradually withdraws scaffolding, and a language-adaptive switch that tailors learning horizons to specific language difficulties. Empirical results on challenging multilingual mathematical benchmarks reveal that LANG substantially enhances reasoning performance without compromising language consistency. Moreover, we show that our framework generalizes beyond mathematics, fostering more consistent language alignment across model layers

[14] Seeing the Poem: Image-Semantic Detection of AI-Generated Modern Chinese Poetry with MLLMs cs.CL | cs.CVPDF

Shanshan Wang, Fengying Ye, Hanjia Lyu, Caiwen Gou, Junchao Wu

TL;DR: 本文提出了一种基于图像语义引导的现代汉语诗歌检测方法，利用多模态大语言模型（MLLMs）结合诗歌文本和反映其内容的图像信息，以提升AI生成诗歌的检测性能。

Details

Motivation: 现有检测研究表明LLMs作为检测器效果不佳，且缺乏针对现代汉语诗歌的检测研究，本文旨在评估并提升LLMs在此特定领域的检测能力。

Result: 实验表明，基于该方法的LLM检测器在多个LLM生成数据上均优于纯文本基线检测器，甚至超越了最佳传统检测器RoBERTa；其中Gemini检测器实现了85.65%的Macro-F1分数，达到SOTA水平。

Insight: 创新性地引入与诗歌内容相关的图像信息，通过示例驱动的方式整合图像的意境、意象和情感，与文本形成互补判断，从而有效提升了检测性能。

Abstract: Previous detection studies have shown that LLMs cannot be effectively used as detectors, but these studies have not addressed modern Chinese poetry. Moreover, no relevant research has explored the performance of LLMs in detecting modern Chinese poetry. This paper evaluates and enhances the performance of LLMs as detectors for modern Chinese poetry, and proposes an image-semantic guided poetry detection method. Compared with traditional detection approaches, our method innovatively incorporates images that reflect the content of the poetry. Through example-driven approaches, our method effectively integrates information such as meaning, imagery, and feeling from the image, then forms a complementary judgment with the poem text. Experimental results demonstrate that the LLM detectors based on our method outperform baseline detectors based on plain text, and even surpass the best-performing traditional detector, RoBERTa. The Gemini detector using our method achieves a Macro-F1 score of 85.65%, reaching the state-of-the-art level. The performance improvements of different LLM detectors on multiple LLMs-generated data prove the effectiveness of our method.

[15] Self-Policy Distillation via Capability-Selective Subspace Projection cs.CLPDF

Guangya Hao, Yitong Shang, Yunbo Long, Zhuokai Zhao, Hanxue Liang

TL;DR: 本文提出了一种名为自策略蒸馏（SPD）的新方法，用于提升大语言模型的性能。该方法通过从模型自身梯度中提取一个低秩能力子空间，在自生成过程中将关键值（KV）激活投影到该子空间，并利用标准的下一个词预测损失对原始输出进行微调，从而实现了无需外部信号、可泛化的能力选择性蒸馏。

Details

Motivation: 现有自蒸馏方法要么依赖昂贵且对前沿模型不可用的外部信号来筛选自生成输出，要么直接使用所有原始输出进行训练，后者通常领域特定且难以泛化。两者都存在一个更深层次的弱点：自生成输出将任务相关能力与风格模式、格式伪影和模型特定错误等其他因素纠缠在一起，稀释了旨在改进的特定能力的信号。

Result: 在代码生成、数学推理和多项选择问答等任务上的大量实验表明，SPD在无需外部信号的情况下，比最先进的自蒸馏方法提升了高达13%，比预训练基线提升了高达16%。值得注意的是，SPD表现出卓越的泛化能力，在领域外泛化设置下实现了15%的性能提升。

Insight: 核心创新在于通过模型自身梯度定义的低秩能力子空间进行投影，实现了对特定能力的针对性蒸馏，而无需依赖外部筛选信号。这种方法解耦了任务相关能力与其他混杂因素，从而更有效地利用了自生成数据，并展现出强大的跨领域泛化能力。

Abstract: Self-distillation bootstraps large language models (LLMs) by training on their own generations. However, existing methods either rely on external signals to curate self-generated outputs (e.g., correctness filtering, execution feedback, and reward search), which are costly and unavailable for the best-performing frontier models, or skip curation entirely and train on all raw outputs, an approach that is often domain-specific and hard to generalize. Both also share a deeper weakness that self-generated outputs entangle task-relevant capability with others, such as stylistic patterns, formatting artifacts, and model-specific errors, diluting the signal for the specific capability one aims to improve. In this paper, we propose Self-Policy Distillation (SPD), which achieves generalizable, capability selective without any external signal. Specifically, SPD extracts a low-rank capability subspace from the model’s own gradients on correctness-defining tokens, projects key-value (KV) activations into this subspace during self-generation, and fine-tunes on the resulting raw outputs with standard next-token prediction loss. Through extensive experiments across code generation, mathematical reasoning, and multiple-choice QA, we show that SPD achieves up to 13% improvement over state-of-the-art self-distillation methods without external signals and up to 16% improvement over pre-trained baselines. Notably, SPD demonstrates superior generalizability, achieving 15% better performance under out-of-domain generalization settings.

[16] ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoning cs.CLPDF

Md Shamim Ahmed, Farzaneh Firoozbakht, Lukas Galke Poech, Jan Baumbach, Richard Röttger

TL;DR: 本文提出了ChronoMedKG，一个包含时间信息的生物医学知识图谱，覆盖13,431种疾病，包含460,497个证据链接的三元组，每个关联都带有发病窗口或进展阶段等时间成分。此外，作者还引入了ChronoTQA基准测试，包含3,341个问题，用于评估模型在临床推理中的时间理解能力。

Details

Motivation: 现有生物医学知识图谱（如PrimeKG、Hetionet）将疾病关联视为静态事实，缺乏时间信息，这限制了它们在纵向临床推理和检索增强中的应用。

Result: ChronoMedKG在Orphadata上达到92.7%的一致性，并为6,250种疾病（包括1,657种罕见病）提供了时间基础。在ChronoTQA基准上，前沿大语言模型从静态问题转向时间问题时性能下降约30分，而ChronoMedKG检索能挽回其47-65%的长尾失败，优于HPOA-RAG的17-29%。

Insight: 创新点在于构建了首个时间基础的生物医学知识图谱，通过多智能体流程利用前沿大语言模型从文献中提取知识，并基于多模型共识、可信度过滤和本体对齐进行筛选；同时，提出的ChronoTQA基准为评估临床时间推理提供了新工具。

Abstract: Biomedical knowledge graphs (KGs) treat disease associations as static facts, but temporal information is crucial for clinical reasoning, e.g., a symptom diagnostic of one disease at age 3 may imply a different disease at age 13. Existing KGs such as PrimeKG, Hetionet, and iKraph do not encode when a finding becomes clinically relevant over the course of a disease. This limits their usefulness for longitudinal clinical reasoning and retrieval augmentation. We introduce ChronoMedKG, a temporal biomedical knowledge graph that contains 460,497 evidence-linked triples (filtered from 13M raw extractions) covering 13,431 diseases. Each association is tied to temporal components like onset window or progression stage, which are backed by PMID-traceable evidence and a multi-signal credibility score. The graph is constructed through a disease-autonomous multi-agent pipeline in which multiple frontier LLMs independently extract knowledge from PubMed and PMC literature. Only those relations are kept that are supported by multi-model consensus, survive credibility filtering, as well as ontology alignment. ChronoMedKG scored 92.7% agreement against Orphadata and adds temporal grounding for 6,250 diseases absent from HPOA, Orphadata, and Phenopackets, including 1,657 Orphanet-coded rare diseases. We further introduce ChronoTQA, a benchmark of 3,341 questions across eight task types (six temporal plus two static controls), with a 12-question supplementary probe. Frontier LLMs lose roughly 30 points moving from static to temporal questions; ChronoMedKG retrieval rescues 47-65% of their long-tail failures, against 17-29% for HPOA-RAG. As such, ChronoMedKG provides a crucial temporal axis for retrieval-augmented clinical systems that was previously absent.

[17] Evaluating Commercial AI Chatbots as News Intermediaries cs.CLPDF

Mirac Suzgun, Emily Shen, Federico Bianchi, Alexander Spangher, Thomas Icard

TL;DR: 该论文系统评估了六款商业AI聊天机器人作为新闻中介的准确性，通过为期14天对2100个基于当日BBC新闻的事实性问题进行测试，发现最佳系统在多项选择题上准确率超过90%，但在自由回答中下降11-13%。研究揭示了区域性偏差、检索依赖性和对错误前提的脆弱性等关键问题。

Details

Motivation: 随着AI聊天机器人日益成为人们获取新闻的渠道，缺乏对其在多语言和跨区域环境下处理新兴事实准确性的系统性评估，本研究旨在填补这一空白。

Result: 在基于BBC新闻的多项选择题测试中，最佳模型准确率超过90%；但在自由回答评估中，相同模型准确率下降11-13%，整体下降16-17%。模型在印地语问题上表现最差（79% vs. 其他语言89-91%），且对含错误前提的问题准确率骤降至19-70%。

Insight: 研究揭示了AI新闻中介存在系统性区域不平等（如检索偏向英语来源）、错误主要由检索而非推理失败导致（>70%），以及高准确率模型对错误前提的脆弱性，同时发现了前提检测与答案恢复能力部分独立的悖论现象。

Abstract: AI chatbots are rapidly shaping how people encounter the news, yet no prior study has systematically measured how accurately these systems, with their proprietary search integrations and retrieval-synthesis pipelines, handle emerging facts across languages and regions. We present a 14-day (February 9-22, 2026) evaluation of six AI chatbots (Gemini 3 Flash and Pro, Grok 4, Claude 4.5 Sonnet, GPT-5 and GPT-4o mini) on 2,100 factual questions derived from same-day BBC News reporting across six regional services (US & Canada, Arabic, Afrique, Hindi, Russian, Turkish). The best systems achieve over 90% multiple-choice accuracy on questions about events reported hours earlier. The same systems, however, lose 11-13% under free-response evaluation, and 16-17% across the cohort. We further characterize three failure patterns. First, every model achieves its lowest accuracy on Hindi (79% vs. 89-91% elsewhere) and citations indicate an Anglophone retrieval bias (e.g., models answering Hindi queries cite English Wikipedia more than any Hindi outlet). Second, retrieval, not reasoning, failures drive over 70% of all errors. When models retrieve a correct source, they often extract the correct answer; the problem is to land on the right source in the first place. Third, models achieving 88-96% accuracy on well-formed questions drop to 19-70% when questions contain subtle false premises, with the most vulnerable model accepting fabricated facts 64% of the time. We also identify a detection-accuracy paradox: the best false-premise detector ranks second in adversarial accuracy (abstention rate), while a weaker detector ranks first, showing that premise detection and answer recovery are partially independent capabilities. Overall, these suggest that high accuracy can mask systematic regional inequity, near-total dependence on retrieval infrastructure, and vulnerability to imperfect queries real users pose.

cs.CV [Back]

[18] GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillation cs.CVPDF

Sixiang Chen, Zhaohu Xing, Tian Ye, Xinyu Geng, Yunlong Lin

TL;DR: GenEvolve是一个自我演进的图像生成框架，通过工具编排的视觉经验蒸馏，将每次生成尝试建模为一个工具编排的轨迹，并利用轨迹间的比较来提炼结构化视觉经验，以密集的token级监督指导学生模型内化更好的搜索、知识激活、参考选择和提示构建能力。

Details

Motivation: 随着图像生成请求变得日益多样化和复杂化，高质量生成需要智能体结合模型内部生成能力与外部资源，因此旨在开发一个能够通过轨迹自我演进、并更有效利用工具应对各种生成挑战的通用图像生成智能体。

Result: 在公开基准测试和自建的GenEvolve-Bench上的实验表明，该方法相比强基线取得了显著提升，在当前图像生成框架中达到了最先进的（SOTA）性能水平。

Insight: 核心创新点在于将生成过程建模为工具编排的轨迹，并通过比较同一请求下的多个轨迹，将最佳-最差差异抽象为结构化视觉经验，仅提供给特权教师分支，再通过视觉经验蒸馏实现密集的token级监督，这不同于主要依赖图像级标量奖励的现有方法。

Abstract: Open-ended image generation is no longer a simple prompt-to-image problem. High-quality generation often requires an agent to combine a model’s internal generative ability with external resources. As requests become more diverse and demanding, we aim to develop a general image-generation agent that can self-evolve through trajectories and use tools more effectively across varied generation challenges. To this end, we propose GenEvolve, a self-evolving framework based on Tool-Orchestrated Visual Experience Distillation. In GenEvolve, each generation attempt is modeled as a tool-orchestrated trajectory, where the agent gathers evidence, selects references, invokes generation skills, and composes them into a prompt-reference program. Unlike existing agentic generation methods that mainly rely on image-level scalar rewards, GenEvolve compares multiple trajectories for the same request and abstracts best-worst differences into structured visual experience, provided only to a privileged teacher branch. Inspired by on-policy self-distillation, Visual Experience Distillation provides dense token-level supervision, helping the student internalize better search, knowledge activation, reference selection, and prompt construction. We further construct GenEvolve-Data and GenEvolve-Bench. Experiments on public benchmarks and GenEvolve-Bench show substantial gains over strong baselines, achieving state-of-the-art performance among current image-generation frameworks. Our website is as follows: https://ephemeral182.github.io/GenEvolve/

[19] Lens: Rethinking Training Efficiency for Foundational Text-to-Image Models cs.CVPDF

Dong Chen, Fangyun Wei, Ziyu Wan, Dongdong Chen, Jiawei Zhang

TL;DR: Lens是一个3.8B参数的文本到图像生成模型，通过高效训练策略，在多项基准测试中性能媲美甚至超越超过6B参数的SOTA模型，同时显著减少训练计算量。其效率源于高信息密度数据集（Lens-800M）和优化的架构设计（如语义VAE和强语言编码器），并支持多分辨率、多宽高比及多语言生成。

Details

Motivation: 解决大规模文本到图像模型训练计算成本高昂的问题，旨在以更小的模型尺寸和更少的计算资源实现与更大模型相当甚至更优的性能。

Result: 在多个基准测试中，性能与超过6B参数的SOTA模型竞争甚至超越，例如训练计算量仅为Z-Image的约19.3%；支持1:2至2:1的任意宽高比和高达1440^2的分辨率，在单张H100 GPU上生成1024^2图像仅需3.15秒。

Insight: 创新点包括：使用GPT-4.1生成的密集标注数据集（Lens-800M）提升数据信息密度；采用多分辨率/宽高比批处理扩大视觉覆盖；结合语义VAE和强语言编码器加速收敛；通过RL微调、推理器模块和蒸馏技术系统优化生成质量与速度。

Abstract: We introduce Lens, a 3.8B-parameter T2I model that achieves performance competitive with, and in several cases surpassing, state-of-the-art models with more than 6B parameters across various benchmarks, while requiring significantly less training compute. For example, Lens requires only about 19.3% of the training compute used by Z-Image. The training efficiency of Lens stems from two key strategies beyond its compact model size. First, we maximize data information density per training batch by (i) training on Lens-800M, a dataset of 800M densely captioned image-text pairs whose captions are generated by GPT-4.1 and contain approximately 109 words on average, providing richer semantic supervision than conventional short captions, and (ii) constructing each batch from images with multiple resolutions and diverse aspect ratios, thereby enlarging the effective visual coverage of each optimization step. Second, we improve convergence speed through careful architectural choices, including adopting a semantic VAE that provides better latent representations and employing a strong language encoder that accelerates optimization while enabling multilingual generalization from English-only training data. After pre-training, we apply RL with taxonomy-driven prompts (Lens-RL-8K) and structured reward rubrics to suppress artifacts and improve visual quality, a reasoner module with training-free system prompt search to better align user requests with the model, and distillation-based acceleration for 4-step inference. Through efficient training and systematic optimization, Lens generalizes to arbitrary aspect ratios from 1:2 to 2:1 and resolutions up to 1440^2, and supports prompts in several commonly used languages. Thanks to its compact size, Lens generates a 1024^2 image in 3.15 seconds on a single NVIDIA H100 GPU, while its distilled turbo version performs 4-step generation in 0.84 seconds.

[20] Flat-Pack Bench: Evaluating Spatio-Temporal Understanding in Large Vision-Language Models through Furniture Assembly cs.CV | cs.AI | cs.CLPDF

Aditya Chetan, Eric Cai, Peeyush Kushwaha, Bharath Raj Nagoor Kani, Utkarsh Mall

TL;DR: 本文介绍了Flat-Pack Bench，一个专注于家具组装任务的新型基准测试，用于评估大型视觉语言模型在细粒度时空理解方面的能力。该基准通过多项选择题和视觉提示来测试模型对组装动作的时间排序、状态定位、部件配合和跟踪等任务。实验表明，当前最先进的LVLMs在细粒度时空推理方面存在显著困难。

Details

Motivation: 现有视频理解基准主要关注粗粒度任务（如动作分割、分类），且依赖易于语言描述的实体，无法充分评估模型在复杂真实场景（如家具组装）中所需的逐步细粒度时空理解能力。

Result: 在Flat-Pack Bench上的实验结果显示，最先进的LVLMs在细粒度时空推理任务上表现不佳，突显了它们在有效利用视频时间信息、跟踪能力以及对空间交互（如物理接触）理解方面的局限性。

Insight: 创新点在于提出了一个专注于家具组装这一需要精细时空推理的领域的新基准，通过设计强调部件参考的视觉提示和多项选择题，系统地评估模型在动作排序、状态定位等细粒度任务上的能力，揭示了当前LVLMs在时空理解上的关键短板。

Abstract: The emergence of Large Vision-Language Models (LVLMs) has significantly advanced video understanding capabilities. However, existing benchmarks focus predominantly on coarse-grained tasks such as action segmentation, classification, captioning, and retrieval. Furthermore, these benchmarks often rely on entities that can be easily identified verbally, like household objects, animals, human subjects, etc., limiting their applicability to complex, in-the-wild video scenarios. But, many applications such as furniture assembly, cooking, etc., require step-by-step fine-grained spatio-temporal understanding of the video, which is not sufficiently evaluated in current benchmarks. To address this gap, we introduce Flat-Pack Bench, a novel benchmark centered on furniture assembly tasks. Our benchmark evaluates LVLMs on nuanced tasks, including temporal ordering of assembly actions, temporal localization of assembly state, understanding part mating, and tracking, using multiple-choice questions paired with visual prompts highlighting relevant parts as references for fine-grained questions. Our experiments reveal that state-of-the-art LVLMs struggle significantly with fine-grained spatio-temporal reasoning, highlighting their limitations in effectively leveraging temporal information from videos, limited tracking ability, and understanding of spatial interactions like physical contact.

[21] PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects cs.CV | cs.ROPDF

Ziang Cao, Yinghao Liu, Haitian Li, Runmao Yao, Fangzhou Hong

TL;DR: 本文提出了PhysX-Omni，一个用于生成多种类型（刚性、可变形、铰接式）仿真就绪物理3D资产的统一框架。该方法引入了一种专为视觉语言模型设计的新型高效几何表示，并构建了首个通用的仿真就绪3D数据集PhysXVerse以及一个全面的评估基准PhysX-Bench。实验表明，该框架在生成和理解任务上均表现优异，并具有应用于场景生成和机器人策略学习的潜力。

Details

Motivation: 现有3D生成方法大多忽视物理属性或仅限于单一资产类别（如刚性、可变形或铰接式物体），限制了其在需要物理仿真的下游任务中的应用。本文旨在解决这些限制，实现跨多种资产类型的统一、仿真就绪的物理3D生成。

Result: 广泛的实验表明，PhysX-Omni在传统指标和提出的PhysX-Bench基准上，在生成和理解能力方面均表现强劲。PhysX-Bench从几何、绝对尺度、材料、功能、运动学和功能描述六个关键属性进行综合评估。

Insight: 主要创新点包括：1）提出了一种专为视觉语言模型设计的新型高效几何表示，能直接编码高分辨率3D结构而无压缩，提升了生成性能；2）构建了首个通用的仿真就绪3D数据集PhysXVerse；3）提出了一个全面且灵活的评估基准PhysX-Bench，用于在真实场景中评估生成和理解能力。这为面向具身AI和物理仿真的统一3D资产生成提供了新思路。

Abstract: Simulation-ready physical 3D assets have emerged as a promising direction owing to their broad applicability in downstream tasks. However, most existing 3D generation methods either neglect physical properties or are limited to a single asset category, e.g., rigid, deformable, or articulated objects. To address these limitations, we introduce PhysX-Omni, a unified framework for simulation-ready physical 3D generation across diverse asset types. Specifically, we develop a novel and efficient geometry representation tailored for Vision-Language Models, which directly encodes high-resolution 3D structures without compression, significantly improving generation performance. In addition, we construct the first general simulation-ready 3D dataset, PhysXVerse, covering diverse indoor and outdoor categories. Furthermore, to comprehensively and flexibly evaluate both generative and understanding capabilities in the wild, we propose PhysX-Bench, which encompasses six key attributes: geometry, absolute scale, material, affordance, kinematics, and function description. Extensive experiments with conventional metrics and PhysX-Bench show that PhysX-Omni performs strongly in both generation and understanding. Moreover, additional studies further validate the potential of PhysX-Omni for applications in simulation-ready scene generation and robotic policy learning. We believe PhysX-Omni can significantly advance a wide range of downstream applications, particularly in embodied AI and physics-based simulation.

[22] UniVL: Unified Vision-Language Embedding for Spatially Grounded Contextual Image Generation cs.CV | cs.LGPDF

Jiayun Wang, Yu Wang, Weijie Gan, Zhenting Wang, Wei Wei

TL;DR: 本文提出了UniVL模型，用于解决空间接地上下文图像生成任务。该模型通过单一的统一视觉输入（将文本指令渲染到空间掩码上）来绑定语义与空间位置，从而在推理时无需独立的文本编码器。UniVL采用两阶段流程，首先将UniVL嵌入与VAE嵌入空间对齐，然后完全基于UniVL嵌入条件化预训练的扩散主干网络，显著降低了计算开销。

Details

Motivation: 动机在于重构条件化范式，解决传统可控图像生成中需要分别使用视觉和语言编码器处理参考图像和全局文本提示的问题，旨在通过统一视觉输入直接绑定语义与空间位置，实现更高效、空间接地的图像生成。

Result: 在构建的UniVL-ImgGen基准（包含477K掩码标注图像）上，UniVL相比基于文本提示的基线方法提升了图像质量，将FID从14降低到11，PSNR从16提高到20；同时完全消除了文本编码器，推理TFLOPs减少高达52%，运行时间减少高达44%。

Insight: 创新点在于提出空间接地上下文图像生成任务和UniVL框架，通过统一视觉输入融合视觉语义意图与空间位置，实现无需独立文本编码器的条件化；从客观角度看，这种范式重构在保持性能的同时显著提升了效率，为高效、可控的图像生成提供了新思路。

Abstract: We introduce spatially grounded contextual image generation, a controllable image generation task that reframes the conditioning paradigm. Instead of supplying a reference image and a global text prompt through two separate encoders, one for vision and one for language, UniVL is trained to bind semantics to spatial locations directly from a single unified visual input, where the textual instruction is rendered onto the spatial mask. This removes the need for a standalone text encoder at inference time. The resulting model supports contextual image generation by following user-specified instructions about what should appear where, while substantially reducing computation. To address this task, we propose a framework in which the UniVL encoder, adapted from an optical-character-recognition-pretrained backbone, reads the unified condition optically and produces a UniVL embedding, fVIL, that fuses visual and semantic intent with spatial locations in a single token sequence. A two-stage pipeline first aligns UniVL with the VAE embedding space and then conditions a pretrained diffusion backbone entirely on UniVL embeddings, eliminating the standalone text encoder, such as T5. Although this reframing uses a deliberately minimal text interface, it yields strong empirical gains. On UniVL-ImgGen, a benchmark of 477K mask-annotated images that we construct for training and evaluation, UniVL improves image quality over text-prompted baselines, reducing FID from 14 to 11 and increasing PSNR from 16 to 20. It also eliminates the text encoder entirely, reducing inference TFLOPs by up to 52% and runtime by up to 44%. Additional ablation studies validate the contributions of the proposed components, paving the way for efficient, spatially grounded image generation with a unified conditioning paradigm.

[23] AVI-HT: Adaptive Vision-IMU Fusion for 3D Hand Tracking cs.CV | cs.ROPDF

Ziyi Kou, Ankit Kumar, Mia Huang, Taylor Niehues, Vatsal Mehta

TL;DR: AVI-HT是一种自适应视觉-IMU融合方法，通过联合建模第一人称视角图像与手套上的6自由度IMU信号来跟踪3D手部姿态。该方法在视觉遮挡严重的手-物体交互场景中显著提高了准确性和可用性，其核心包括同步多模态训练数据和跨传感器深度注意力机制。

Details

Motivation: 解决在视觉遮挡严重的手-物体交互场景中，现有方法难以准确跟踪3D手部姿态的问题，通过融合视觉和IMU传感器数据来提升鲁棒性。

Result: 在DexGloveHOI数据集上进行了广泛实验，与多种单模态和多模态跟踪方法相比，AVI-HT在UmeTrack和MANO两种手部模型下，平均关键点误差分别降低了16.1%和24.2%，达到了SOTA水平。

Insight: 创新点包括同步多模态训练数据配对和跨传感器深度注意力机制，后者能自适应调整对视觉和单个IMU传感器的信任度；从客观角度看，该方法有效解决了多传感器融合中的噪声和时间错位问题，提升了在遮挡场景下的性能。

Abstract: We present AVI-HT, an adaptive visual-IMU fusion approach for tracking 3D hand poses by jointly modeling the egocentric image with on-glove 6-DoF IMU signals. AVI-HT achieves significantly improved accuracy and availability, particularly in hand-object interaction (HOI) scenarios involving heavy visual occlusion. Two complementary ingredients underpin its success: (1) synchronized multi-modal training data pairing on-body vision-IMU sensor streams with ground-truth 3D hand poses from a motion-capture system, and (2) a cross-sensor deep attention mechanism that adaptively modulates the trust assigned to the vision and individual IMU sensors. To evaluate AVI-HT in real-world settings, we conduct extensive experiments on our DexGloveHOI dataset that consists of 100K+ pairwise vision-IMU samples with synchronized 3D annotated poses, in which users manipulate a variety of objects during daily tasks. We compare against multiple single- and multi-modal tracking approaches under two hand models (UmeTrack, MANO). The results show that AVI-HT reduces mean keypoint error by 16.1% and its wrist-aligned variant by 24.2% over the baselines. Ablation studies further reveal the per-finger contribution of IMU sensors across activity types, and the model’s sensitivity to IMU noise and temporal misalignment in vision-IMU fusion.

[24] BEiTScore: Reference-free Image Captioning Evaluation with an Efficient Cross-Encoder Model cs.CV | cs.CL | cs.LGPDF

Gonçalo Gomes, Bruno Martins, Chrysoula Zerva

TL;DR: 本文提出了一种名为BEiTScore的新型图像描述评估指标，旨在解决现有评估方法在计算成本、细粒度敏感性和组合泛化能力方面的不足。该方法基于一个轻量级的交叉编码器，该编码器从视觉问答模型初始化，并通过精心设计的监督学习数据混合（包括基于对抗性LLM的数据增强）进行训练。实验表明，该指标在保持高效的同时，达到了最先进的性能水平。

Details

Motivation: 随着视觉语言模型向生成长篇、上下文丰富的描述等更具挑战性的能力发展，图像描述评估面临重大挑战。现有最先进的评估方法要么涉及使用大型语言模型作为评判者带来的巨大计算成本，要么受限于基于标准CLIP编码器的缺陷，如严格的令牌限制、缺乏细粒度敏感性或将描述视为“词袋”而缺乏组合泛化能力。

Result: 实验结果表明，所提出的指标在保持大规模基准测试、质量感知解码或奖励引导所需效率的同时，实现了最先进的性能。论文还引入了一个新的基准，旨在评估不同场景下的详细描述评估。

Insight: 主要创新点包括：1) 提出一个从视觉问答模型初始化的轻量级交叉编码器架构，平衡了强大的权重初始化与计算效率；2) 采用精心组装的监督学习数据混合，并利用基于对抗性LLM的数据增强来增强模型对细粒度视觉语言错误的敏感性；3) 引入了一个新的基准来全面评估详细描述评估能力。

Abstract: Image captioning evaluation remains a significant challenge, as vision-language models evolve toward more challenging capabilities such as generating long-form and context-rich descriptions. State-of-the-art evaluation metrics involve extensive computational costs associated with the use of Large Language Models (LLMs) as judges, or instead suffer from the limitations of standard CLIP-based encoders, such as strict token limits, lack of fine-grained sensitivity, or lack of compositional generalization by treating captions as ``bags-of-words.’’ We propose a new learned metric that tackles the aforementioned challenges, based on a lightweight cross-encoder that is initialized from a visual question-answering model checkpoint, balancing a strong weight initialization with computational efficiency. Our training scheme uses a carefully assembled data mixture for supervised learning, featuring adversarial LLM-based data augmentations to enhance model sensitivity to fine-grained visual-linguistic errors. We also introduce a new benchmark designed to assess detailed captioning evaluation across diverse scenarios. Experimental results demonstrate that the proposed metric achieves state-of-the-art performance while maintaining the efficiency required for large-scale benchmarking, quality-aware decoding, or reward guidance.

[25] Improving 3D Labeling in Self-Driving by Inferring Vehicle Information using Vision Language Models cs.CV | cs.ROPDF

Steven Chen, Shivesh Khaitan, Nemanja Djuric

TL;DR: 本文提出了一种利用视觉语言模型（VLM）进行零样本推断，以改进自动驾驶应用中3D车辆标注的方法。该方法通过VLM从图像中推断车辆的制造商、型号、代际信息，并输出精确的3D边界框尺寸，作为手动标注的初始参考。实验表明，该方法能有效减少标注时间并提升标注质量。

Details

Motivation: 动机是解决自动驾驶数据标注中，手动标注3D车辆边界框耗时且在某些情况下（如严重遮挡）精度不足的问题，旨在通过VLM的零样本推断能力来辅助和优化标注流程。

Result: 在公开和专有数据集上的实验表明，该方法在车辆边界框尺寸推断和制造商/型号/代际识别上均表现出高准确率，优于强基线方法，并在车辆严重遮挡等特定失败场景中，其推断的尺寸甚至优于基于激光雷达的初始人工标注。

Insight: 创新点在于将VLM的零样本推理能力直接应用于3D标注流程，通过迭代提示工程优化VLM输出，实现了对车辆信息的细粒度识别和精确3D尺寸估计，从而在提升自动化程度的同时保证了标注质量，为数据标注提供了新的VLM集成范式。

Abstract: We present an approach to improve 3D vehicle labeling in self-driving applications through zero-shot inference of vehicle information, leveraging Vehicle Make and Model Recognition (VMMR) methods. The proposed approach utilizes a Vision Language Model (VLM) to both infer a vehicle’s make, model, and generation from image crops, and output accurate 3D bounding box dimensions to seed manual labeling. We evaluate the impact of iterative prompt engineering and the choice of different VLMs on both vehicle bounding box inference and make/model/generation recognition. When compared to strong baselines, the proposed approach not only shows high accuracy, but also excels in mitigating specific failure modes where VLMs provide better dimensions than initial lidar-aided human annotated labels (e.g., in cases of significant vehicle occlusion). Experiments on both public and proprietary data strongly suggest that our conclusions are generalizable across different labelers and datasets. The results demonstrate that integrating VLMs into the labeling process can reduce manual labeling time while increasing label quality.

[26] Look-Closer-Then-Diagnose: Confidence-Aware Ultrasound VQA via Active Zooming cs.CV | cs.AIPDF

Yue Zhou, Erxuan Wu, Yikang Sun, Hongjoo Lee, Yuan Bi

TL;DR: 本文提出了一种名为’Look-Closer-Then-Diagnose’的置信度感知超声视觉问答框架，旨在模拟超声医师的认知工作流程。该框架包含一个’Zoom-then-Diagnose’范式，用于交互式聚焦病灶区域进行推理，并结合一个基于组相对策略优化的不确定性感知奖励机制，以估计预测一致性作为模型置信度的代理。

Details

Motivation: 现有视觉语言模型在超声图像问答任务中表现不佳，且未能模拟医师交互式聚焦病灶的诊断过程，同时通常将标注视为无偏的绝对真值，忽略了其固有的主观性和模糊性。

Result: 在肝脏、乳腺和甲状腺数据集上的实验表明，该框架将病灶定位性能提升了39.3%，证明了模型学会了主动’细看’并诊断的能力。

Insight: 创新点在于将医师的交互式搜索认知流程结构化到模型中，并引入基于随机组间推演的不确定性感知奖励来量化预测一致性，从而在清晰病例上强化准确预测，在模糊情况下保持谨慎。

Abstract: Vision-Language Models (VLMs) have significantly advanced medical visual question answering, yet their performance in ultrasound remains suboptimal. In clinical practice, sonographers explicitly focus on lesion regions to formulate reports, though diagnostic interpretations sometimes vary due to inherent subjectivity. However, existing VLMs are not explicitly structured to interactively zoom into lesions prior to diagnosis; moreover, they typically treat annotations as unbiased ground truths, failing to account for their inherent subjectivity and ambiguity. In this paper, we propose a framework specifically designed to consider the sonographer’s cognitive workflow. We first introduce a structured Zoom-then-Diagnose paradigm, which replicates the interactive search process to enable lesion-focused reasoning. Furthermore, within the Group Relative Policy Optimization (GRPO) framework, we introduce an uncertainty-aware reward derived from stochastic group-wise rollouts to estimate prediction consistency as a proxy for model confidence. Together, these two components encourage the model to reinforce accurate predictions on clear cases while remaining cautious under ambiguity. Experiments across liver, breast, and thyroid datasets show that our framework improves lesion localization by 39.3%, demonstrating that our model has learned the ability to actively look closer and diagnose.

[27] BodyReLux: Temporally Consistent Full-Body Video Relighting cs.CV | cs.GRPDF

Li Ma, Mingming He, Xueming Yu, David M. George, Ahmet Levent Taşel

TL;DR: 本文提出BodyReLux，一个基于视频扩散模型的主体特定框架，用于实现时间一致的全人体视频重光照。该方法通过混合数据集（结合静态OLAT捕捉和新型动态表演捕捉）训练，并利用预训练文本到视频模型的生成先验，引入光源令牌和掩码注意力机制进行光照控制，实现高质量、鲁棒且时间一致的重光照效果。

Details

Motivation: 解决在后期制作和内容创作中，对人体表演进行高质量、时间一致的视频重光照这一基本任务。

Result: 通过精心设计的数据增强流程，实现了照片级真实感、鲁棒且时间一致的主体特定人体表演视频重光照。

Insight: 创新点包括：结合静态OLAT与新型动态表演捕捉（利用人眼闪烁融合阈值以上快速交错变化光照序列）构建混合数据集；从预训练文本到视频模型微调以利用生成先验；提出将每个光源表示为令牌的光照条件方法，并使用掩码注意力支持动态光照控制。

Abstract: Being able to relight human performance is a fundamental task for post production and content creation. We present BodyReLux, a subject-specific video diffusion-based framework for relighting full-body human performances in a temporally consistent way. Our model is trained on a hybrid dataset of pixel-aligned video relighting pairs, covering a diverse combination of lighting conditions, performances and viewpoints. To acquire such dataset, we combine traditional static One-Light-at-a-Time (OLAT) capture and a novel dynamic performance capture in which two smoothly varying lighting sequences are rapidly interleaved. Because the lighting operates above the human flicker-fusion threshold, the interleaving does not appear to strobe. We train our video relighting model from a pretrained text-to-video model to fully leverage the generative priors for producing high quality videos. To achieve accurate lighting control, we introduce a new lighting conditioning method that represents each light source as a token. We further condition on sequences of lighting using masked attention to support dynamic lighting control. Together with a carefully designed data augmentation pipeline, we achieve photorealistic, robust, and temporally consistent video relighting of subject-specific human performances.

[28] Ablate-to-Validate: Are Vision-Language Models Really Using Continuous Thought Tokens? cs.CVPDF

Tianyi Zhang, Mahtab Bigverdi, Ranjay Krishna

TL;DR: 本文提出了一种名为‘消融验证’的诊断原则，旨在检验视觉语言模型是否真正利用了其内部的连续思维令牌进行推理。通过实例化为令牌替换测试，研究发现在多个模型和任务中，性能提升主要依赖于令牌的存在而非其内容，揭示了模型并未有效使用这些潜在通道作为信息瓶颈。

Details

Motivation: 尽管视觉语言模型通过引入连续或潜在的非文本令牌提升了任务准确率，但尚不清楚这种提升是否源于模型真正利用这些令牌进行推理，还是由其他混杂因素（如增加的上下文长度、特殊令牌锚定或训练正则化）导致。

Result: 在LLaVA-13B、Qwen2.5-VL-3B以及Mirage、Mull-Tokens、CoVT等现成系统上，应用令牌替换测试于BLINK、VSP和CV-Bench等基准测试表明，即使令牌内容被破坏或替换，模型仍能保持大部分性能提升，说明准确率增益不能可靠地反映潜在令牌推理。

Insight: 创新点在于提出了一个标准化的诊断框架（令牌替换测试），能够隔离并验证模型性能对令牌内容而非令牌存在的依赖性，为评估任何引入连续思维令牌的方法提供了除准确率外的关键诊断工具。

Abstract: Vision-language models (VLMs) are increasingly augmented with continuous or latent non-textual tokens intended to support “visual thinking.” Despite improved task accuracy, this alone does not show that models actually use these tokens for reasoning – gains may arise from confounds such as added context length, special-token anchoring, or training-time regularization. We formalize a diagnostic principle, Ablate-to-Validate, for testing whether latent-token content is genuinely utilized, and instantiate it as the Token Replacement Test (TRT), a standardized suite of content-replacement ablations. TRT holds the prompt, image, token budget, and decoding fixed while replacing intermediate tokens with zero, random, first-repeat, or oracle alternatives, isolating whether performance depends on token content or merely on token presence. As a controlled testbed, we study relative depth reasoning with LLaVA-13B and Qwen2.5-VL-3B, training models to predict and consume continuous or discrete depth spans across multiple frozen encoders (SigLIP2, CLIP, DINOv2) and token budgets. We additionally apply TRT to three off-the-shelf visual-thinking systems (Mirage, Mull-Tokens, CoVT) on BLINK, VSP, and CV-Bench. Across all settings, accuracy gains are a misleading proxy for latent-token reasoning: VLMs retain most improvement even when token content is corrupted or replaced, revealing a persistent gap between having a latent channel and using it as an information bottleneck. We recommend TRT as a standard diagnostic alongside accuracy for any method introducing continuous thought tokens.

[29] MM-Conv: A Multimodal Dataset and Benchmark for Context-Aware Grounding in 3D Dialogue cs.CV | cs.CLPDF

Anna Deichler, Jim O’Regan, Fethiye Irmak Dogan, Lubos Marcinek, Anna Klezovich

TL;DR: 本文提出了一个名为MM-Conv的多模态数据集和基准测试，用于评估AI系统在动态3D对话环境中对语言指代进行上下文感知的接地能力。该数据集基于6.7小时的第一人称VR交互构建，包含同步的语音、动作、注视和3D场景几何信息，并包含超过4200个手动验证的指代表达式。同时，论文提出了一种两阶段的接地流程，通过先解析对话中的歧义再进行视觉定位，显著提升了性能。

Details

Motivation: 当前视觉语言模型在静态图像任务上表现出色，但在处理自发、多轮对话中出现的模糊指代表达时存在困难。本文旨在填补这一空白，研究如何在动态3D物理世界中实现对话式的语言接地。

Result: 在提出的基准测试上，论文提出的上下文重写方法将接地性能平均提升了11-22个百分点。具体而言，使用纯检测器（GroundingDINO）在代词指代任务上，经过重写后达到了56.7%的准确率，几乎是现有最佳端到端基线方法的两倍。

Insight: 核心创新点在于将对话中的语言推理与视觉感知解耦，提出了一个先进行上下文消歧再进行视觉定位的两阶段流程。这挑战了端到端方法的普遍范式，证明了对于对话式接地任务，显式处理语言上下文比端到端方法更有效。同时，构建的大规模、多模态、包含丰富交互上下文的3D对话数据集也是一个重要的贡献。

Abstract: Grounding language in the physical world requires AI systems to interpret references that emerge dynamically during conversation. While current vision-language models (VLMs) excel at static image tasks, they struggle to resolve ambiguous expressions in spontaneous, multi-turn dialogue. We address this gap by introducing (1) a benchmark for referential communication in dynamic 3D environments, built from 6.7 hours of egocentric VR interaction with synchronized speech, motion, gaze, and 3D scene geometry, and (2) a two-stage grounding pipeline that explicitly resolves conversational ambiguity before visual localization. The benchmark includes over 4,200 manually verified referring expressions spanning full, partitive, and pronominal types. Our contextual rewriting approach improves grounding performance by 11-22 percentage points on average, with a pure detector (GroundingDINO) reaching 56.7% on pronominals after rewriting, nearly double the best end-to-end baseline. Results demonstrate that decoupling linguistic reasoning from visual perception is more effective than end-to-end approaches for conversational grounding.

[30] CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models cs.CV | cs.AIPDF

Zhi Liu

TL;DR: 本文提出CrossVLA，一个针对视觉-语言-动作（VLA）模型的跨范式后训练与推理优化的实证研究。它解决了在连续动作流匹配模型上应用直接偏好优化（DPO）的难题，并比较了LoRA与DoRA等参数高效微调方法在VLA DPO中的效果。研究还分析了推理延迟瓶颈，并提出了一个用于下游任务初始化的预训练投影头。

Details

Motivation: 当前VLA模型主要集中于自回归范式，而连续动作流匹配范式（如pi-0.5）的后训练对齐（尤其是DPO）研究不足，这限制了VLA模型的性能优化和跨范式比较。

Result: 在LIBERO 4-suite基准测试（600次试验，3个随机种子）上，使用DoRA的DPO相比OpenVLA SFT平均提升10.4个百分点，其中Object任务提升20.0个百分点且零种子方差。推理分析显示去噪循环占采样延迟的78.6%，而KV缓存加速上限为21%。

Insight: 创新点包括：1）提出一种无需概率流ODE积分的替代流匹配对数概率估计器，使DPO能应用于连续动作主干；2）实证发现DoRA在VLA DPO中优于LoRA；3）揭示了VLA推理中延迟瓶颈主要在于去噪循环，而非KV缓存。

Abstract: Vision-Language-Action (VLA) models have rapidly converged on a small set of architectural patterns: discrete-token autoregression (e.g. OpenVLA) and continuous-action flow-matching (e.g. pi-0.5). Yet preference alignment via Direct Preference Optimisation (DPO) – the de-facto post-training step in language models – has been studied almost exclusively on autoregressive VLAs. We present CrossVLA, an empirical study of cross-paradigm VLA post-training. Three contributions: (i) a surrogate flow-matching log-probability estimator that lets DPO operate on continuous-action backbones without probability-flow ODE integration; (ii) a head-to-head comparison of LoRA and DoRA as the parameter-efficient layer for VLA DPO, finding DoRA improves over OpenVLA SFT by a mean +10.4 pp across LIBERO 4-suite (600 trials, 3 seeds) – per-suite +20.0 Object, +11.0 Long-horizon, +8.0 Goal, +2.7 Spatial – with zero seed variance on Object (38/50 on each of 3 seeds); (iii) an inference-time anatomy showing the denoise loop dominates 78.6% of sample_actions latency and prefix-K/V caching a la VLA-Cache caps at a 21% acceleration ceiling – both chunk-level and token-level cache strategies degrade success rate to 0-80% in our benchmarks. We further pretrain a multi-view + temporal projection head on 6000 LIBERO frames, achieving 99.5% k-NN recall@1 for same-task retrieval (36x over random), available as a downstream initialisation. All code, ckpts, training logs, and reproduction scripts are open at https://github.com/lz-googlefycy/vla-lab.

[31] Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding cs.CVPDF

Lina Zhang, Tonmoy Monsoor, Peizheng Li, Jiarui Cui, Xinyi Peng

TL;DR: 本文介绍了Seizure-Semiology-Suite (S3)，这是一个用于癫痫发作症状学理解的多模态数据集、基准测试和模型。该数据集包含438个癫痫发作视频，标注了超过35,000个密集标签，涵盖20个ILAE定义的半学特征。基于此，作者提出了一个包含七项任务的层次化基准，系统评估多模态大语言模型从低级视觉感知到时间序列、叙述报告生成及癫痫诊断的能力，并引入了Seizure-RQI指标用于报告质量评估。实验表明现有MLLM存在系统性弱点，而特定微调和神经符号框架能显著提升性能。

Details

Motivation: 尽管多模态大语言模型在通用视频理解上表现出色，但其在解释非自主、时空演变的病理运动行为（如癫痫发作症状学）方面的能力尚未得到充分测试。为了填补这一空白，作者旨在建立一个临床基础的数据集和基准，用于细粒度、结构化的癫痫发作症状学理解。

Result: 在11个开源MLLM上的广泛基线测试揭示了其在偏侧性推理、时间定位、症状排序和临床忠实报告方面的系统性弱点。癫痫特异性微调显著提升了各项任务性能，而一个两阶段神经符号框架在癫痫性与非癫痫性发作分类上达到了0.96的F1分数。

Insight: 论文的创新点在于创建了首个针对癫痫发作症状学的临床多模态数据集和层次化基准测试套件，并引入了Seizure-RQI用于生成报告的临床质量评估。从客观角度看，其提出的神经符号框架展示了在安全关键医疗视频理解任务中结合领域知识进行模型适配的有效路径，为开发临床可靠、领域自适应的多模态智能系统提供了指导。

Abstract: While Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in general video understanding, their capacity to interpret involuntary, and spatio-temporally evolving pathologic motor behaviors such as seizure semiology remains largely untested. To address this gap, we introduce Seizure-Semiology-Suite, a clinically grounded dataset and benchmark for fine-grained, structured seizure semiology understanding. The dataset includes 438 seizure videos annotated with over 35,000 dense labels covering 20 ILAE-defined semiological features. Building on this dataset, we propose a seven-task hierarchical benchmark that systematically evaluates MLLMs from low-level visual perception to temporal sequencing, narrative report generation, and seizure diagnosis. To enable clinically meaningful evaluation of generated reports, we further introduce the Report Quality Index for Seizure Semiology (Seizure-RQI). Extensive baselines across 11 open-weight MLLMs reveal systematic weaknesses in laterality reasoning, temporal localization, symptom sequencing, and clinically faithful reporting. We show that seizure-specific fine-tuning substantially improves performance across tasks, and that a two-stage neuro-symbolic framework achieves an F1 score of 0.96 on epileptic versus non-epileptic seizure classification. Seizure-Semiology-Suite establishes a rigorous benchmark for evaluating multimodal models in safety-critical medical video understanding and guides the development of clinically reliable, domain-adaptive multimodal intelligence.

[32] Two-Stage Multimodal Framework for Emotion Mimicry Intensity Prediction cs.CV | cs.AI | cs.HCPDF

Dinithi Dissanayake, Shaveen Silva, Ovindu Atukorala, Prasanth Sasikumar, Suranga Nanayakkara

TL;DR: 本文提出了一种用于情感模仿强度预测的两阶段多模态框架，旨在从真实场景的多模态视频片段中预测六个连续情感强度维度：钦佩、娱乐、决心、共情痛苦、兴奋和喜悦。该框架结合了文本、声学和视觉表示，并可选地加入运动分支，首先独立训练各模态编码器，然后通过带有模态丢弃和受控编码器适应的轻量级回归器融合学习到的表示。

Details

Motivation: 解决Hume-ABAW10情感模仿强度挑战中从真实视频预测连续情感强度的任务，需要有效整合多模态信息以提升预测性能。

Result: 在验证集上，文本-音频-视觉-运动融合模型在扩展的4:1分割下取得了最佳性能，平均皮尔逊相关系数为0.4722；在测试集上，团队在挑战中排名第三，平均皮尔逊相关系数为0.57。

Insight: 创新点包括两阶段训练策略（先独立训练模态编码器再融合）以及使用模态丢弃和受控编码器适应来增强融合鲁棒性；运动分支虽增益有限，但提供了有趣的研究方向，为情感模仿强度预测提供了实用且可复现的基线。

Abstract: We present our submission to the Hume-ABAW10 Emotional Mimicry Intensity (EMI) Challenge, which aims to predict six continuous emotion intensity dimensions: Admiration, Amusement, Determination, Empathic Pain, Excitement, and Joy, from in-the-wild multimodal video clips. We propose a staged multimodal framework that combines textual, acoustic, and visual representations, with an optional motion branch. Our approach first trains modality-specific encoders independently and then fuses their learned representations through a lightweight regressor with modality dropout and controlled encoder adaptation. Across our submitted systems, the best validation performance is obtained by the text–audio–vision–motion fusion model under the expanded 4:1 split, achieving an average Pearson correlation of 0.4722. Although the motion branch yields only very slight gains, its behavior can be interesting to study. Our team was placed third in the EMI challenge, achieving an average Pearson correlation of 0.57 for the test set. Overall, we provide a practical and reproducible baseline for EMI prediction.

[33] Learning Emergent Modular Representations in Multi-modality Medical Vision Foundation Models cs.CV | cs.AIPDF

Yuting He, Chenyu You, Shuo Li

TL;DR: 本文提出了一种名为Director-Experts (DEX)的模块化网络架构，用于解决多模态医学视觉基础模型训练中因非独立同分布特征统计导致的梯度冲突和表征塌陷问题。该方法通过专家池和导演模块的协同设计，在超过400万张图像、涵盖10种模态的新基准Medical Vision Universe上进行预训练，并在26个下游任务上验证了其优化行为和可迁移性的提升。

Details

Motivation: 多模态医学视觉基础模型面临的核心挑战是异构成像模态间显著的非独立同分布特征统计，这导致传统的单体自监督优化会产生冲突梯度，使表征塌陷于模态主导的捷径。

Result: 在涵盖10种成像模态、超过400万张图像的新基准Medical Vision Universe上进行预训练，并在26个下游任务上进行了广泛评估，结果表明DEX改善了优化行为并提升了可迁移性。

Insight: 创新点在于将多模态表征学习的失败重新定义为涌现模块化中专业化与协调性的失衡，并提出了DEX架构，通过图像级激活策略动态适配专家池以自主专业化，以及通过组指数移动平均更新的导演模块来蒸馏多专家知识到共享空间，从而实现跨模态语义整合。这为通用多模态医学AI提供了一个有原则的步骤。

Abstract: Multi-modality medical vision (MV) foundation models (FM) are fundamentally challenged by pronounced Non-IID feature statistics across heterogeneous imaging modalities. Monolithic self-supervised optimization on such data induces conflicting gradients, driving representations to collapse toward modality-dominant shortcuts. This work reframes this failure as an imbalance between specialization and coordination in emergent modularity, and proposes Director-Experts (DEX), a modular network that explicitly regulates these dynamics in stacked modules. Each DEX module comprises a pool of experts, dynamically adapted by our image-wise activation strategy, autonomously specializing in modality-dominant statistics, together with a director, updated via our group exponential moving average, which distills multi-expert knowledge into a shared space for semantic integration across modalities, thus driving the emergence of modular representations. We curate a new benchmark, Medical Vision Universe, over 4 million images across 10 modalities, which provides a FM-level pre-training with the broadest coverage of distinct imaging modalities to our DEX. Extensive evaluations on 26 downstream tasks demonstrate improved optimization behavior and transferability, indicating DEX as a principled step toward general-purpose multi-modality medical AI. Our code and dataset will be opened at https://github.com/YutingHe-list/DEX.

[34] MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks cs.CV | cs.AIPDF

Han Zhang, Wanting Jiang, Tomasz Kornuta, Tian Zheng, Vidya Murali

TL;DR: 本文提出了MAVEN，一种多阶段智能标注流水线，用于自动将原始视频转化为包含思维链推理轨迹的多任务训练数据，以支持视频事件推理任务。该方法通过合成多尺度时空事件描述作为中间表示，并支持智能体驱动的领域自适应和分层细化循环来迭代提升数据质量。

Details

Motivation: 为视频语言模型训练提供高质量、结构化的事件推理标注数据，解决人工标注无法满足大规模需求的问题，并实现跨领域的自动适配。

Result: 在超过5300个交通视频上应用MAVEN生成数据并微调Cosmos-Reason2-8B模型，在私有CCTV评估集上超越了Gemini 2.5 Pro和3.1 Flash，多项选择题准确率比零样本提升38.8分；在AccidentBench上，仅使用CCTV数据微调使模型提升10.7分，匹配Gemini 2.5 Pro，结合智能体适配的行车记录仪数据和强化学习后训练后，整体性能超越了Gemini基线。

Insight: 创新点在于提出了一个以指定焦点事件为中心、生成多尺度时空事件描述作为核心中间表示的多阶段智能体流水线，并引入了支持领域自适应的智能体驱动提示重设计和基于错误分类学的分层细化循环，实现了从视频到高质量推理训练数据的全自动、可迭代优化的生成流程。

Abstract: Training Vision Language Models (VLMs) for video event reasoning requires high-quality structured annotations capturing not only what happened, but when, where, why, and with what consequence, at a scale manual labelling cannot support. We present MAVEN (Multi-stage Agentic Video Event aNnotation), a multi-stage agentic pipeline that turns raw videos into multi-task training data with Chain-of-Thought (CoT) reasoning traces, organized around a designated Event of Focus. At its core, MAVEN synthesizes a Multi-Scale Spatio-Temporal Event Description (MSTED) from three complementary caption levels; this explicit intermediate serves as the sole input to downstream Q&A generation across multiple task formats. Crucially, MAVEN supports agent-driven domain adaptation: given a new video dataset and target question examples, the agent redesigns all prompts top-down without manual re-engineering. A hierarchical refinement loop further classifies annotation errors against a taxonomy, traces root causes to the originating pipeline stage, and applies targeted edits that rewrite prompts or modify the pipeline structure itself, iteratively improving data quality. We apply MAVEN to label over 5,300 traffic videos and fine-tune Cosmos-Reason2-8B on the resulting data. On a private CCTV evaluation set, fine-tuning surpasses both Gemini 2.5 Pro and 3.1 Flash, including a $+38.8$-point gain in MCQ accuracy over zero-shot. On AccidentBench, CCTV-only training lifts Cosmos-Reason2 by $+10.7$ MCQ points and matches Gemini 2.5 Pro despite seeing no dashcam videos; adding agent-adapted dashcam annotations narrows the gap to Gemini 3.1 Flash, and RL post-training pushes overall performance past both Gemini baselines. Qualitative results on warehouse surveillance and public safety videos further show the agentic workflow readily adapts the pipeline to new domains.

[35] SDGBiasBench: Benchmarking and Mitigating Vision–Language Models’ Biases in Sustainable Development Goals cs.CV | cs.AIPDF

Zihang Lin, Huaiyuan Qin, Muli Yang, Hongyuan Zhu

TL;DR: 该论文提出了SDGBiasBench，一个用于评估和缓解视觉-语言模型在可持续发展目标相关任务中偏见的大规模基准套件。它包含50万个专家参与的多选题和5万个回归任务，旨在全面评估模型在决策和估计层面的偏见。论文还提出了CADE方法，一种无需训练、即插即用的对比自适应去偏集成方法，以缓解模型偏见。

Details

Motivation: 现有基准通常孤立地评估可持续发展目标监测中的定性判断和定量估计，掩盖了模型用先验知识替代证据时产生的系统性偏见。论文旨在填补这一空白，全面评估视觉-语言模型在可持续发展目标导向的多模态推理中的偏见。

Result: 在SDGBiasBench上的评估揭示了当前视觉-语言模型存在内在的可持续发展目标偏见，其预测常由特定先验驱动而非可靠的多模态线索。提出的CADE方法显著提升了性能，在多个视觉-语言模型上将多选题准确率最高提升25%，并将回归任务的MAE最高降低12个点。

Insight: 论文的创新点在于构建了首个全面评估视觉-语言模型在可持续发展目标领域偏见的基准，并提出了无需训练、利用模态特定答案先验的去偏方法CADE。这为开发更公平可靠的可持续发展AI系统提供了评估工具和缓解策略。

Abstract: Assessing progress toward the Sustainable Development Goals (SDGs) requires multi-step reasoning over visual cues, contextual knowledge, and development indicators, where incomplete evidence use and imperfect evidence integration can introduce hidden prediction biases. Real-world SDG monitoring further spans both qualitative judgments and quantitative estimation. However, existing benchmarks typically evaluate these aspects in isolation, obscuring systematic biases that emerge when models substitute priors for evidence. To address this gap, we propose SDGBiasBench, a large-scale benchmark suite for SDG-oriented vision-language reasoning. Spanning 500k expert-involved multiple-choice questions and 50k regression tasks, the benchmark enables comprehensive assessment of both decision-level and estimation-level bias in Vision–Language Models (VLMs). Evaluations on SDGBiasBench reveal an intrinsic SDG bias in current VLMs, where predictions are frequently driven by SDG specific priors rather than reliable multi-modal cues. To mitigate such bias, we propose CADE (Contrastive Adaptive Debias Ensemble), a training-free, plug-and-play method that leverages modality-specific answer priors. CADE yields significant gains on the proposed benchmark, improving multiple-choice accuracy by up to 25% and reducing regression MAE by up to 12 points across multiple VLMs. We hope our work can foster the development of more fair and reliable AI systems for sustainable development.

[36] SceneGraphGrounder: Zero-Shot 3D Visual Grounding via Structured Scene Graph Matching cs.CV | cs.ROPDF

Xuefei Sun, Xujia Zhang, Brendan Crowe, Doncey Albin, Christoffer Heckman

TL;DR: 论文提出SceneGraphGrounder框架，将零样本3D视觉定位任务重新定义为在重建的3D场景图上的结构化图匹配问题。该方法通过视觉标记提示策略，利用视觉语言模型从2D视图推断物体间关系，并构建包含空间和语义关系的持久3D场景图。给定查询时，构建查询图并与场景图进行约束对齐，确保多视图一致性和可解释推理。

Details

Motivation: 解决现有基于视觉语言模型的零样本3D视觉定位方法依赖视图相关推理或隐式表示的问题，这些方法在空间一致性和组合查询的可解释性方面存在局限。

Result: 在ScanRefer基准测试中，该方法在仅使用RGB-D输入的情况下，在零样本方法中取得了有竞争力的性能，并通过在移动机器人上的真实世界部署验证了其在长期物理环境中具有鲁棒的空间推理能力。

Insight: 创新点在于将3D视觉定位重新定义为结构化图匹配问题，并提出了视觉标记提示策略来构建持久3D场景图，这增强了空间一致性和推理的可解释性，为组合查询提供了更可靠的解决方案。

Abstract: Zero-shot 3D visual grounding requires localizing objects in unstructured environments from free-form natural language. Recent vision-language model (VLM) approaches achieve promising results but rely on view-dependent reasoning or implicit representations, limiting spatial consistency and interpretability for compositional queries. We propose SceneGraphGrounder, a framework that reformulates 3D grounding as structured graph matching over a reconstructed 3D scene graph. To enable this formulation, we introduce a visual marker prompting strategy that enables a VLM to infer object-object relationships from 2D views, which are subsequently lifted into a persistent 3D scene graph encoding both spatial and semantic relations. Given a query, we construct a query graph and perform constrained alignment with the scene graph, ensuring multi-view consistency and interpretable reasoning. Experiments on the ScanRefer benchmark demonstrate that our method achieves competitive performance among zero-shot approaches, using only RGB-D inputs. We further validate our framework through real-world deployment on a mobile robot, demonstrating robust spatial reasoning in long-horizon physical environments. We will make our code publicly available upon acceptance.

[37] Thermo-VL: Extending Vision-Language Models to Thermal Infrared Perception cs.CVPDF

Rusiru Thushara, Yasiru Ranasinghe, Jay Paranjape, Vishal M. Patel

TL;DR: 本文提出了Thermo-VL，一种波长感知的视觉语言模型，通过引入可训练的热红外编码器和文本引导的双注意力融合模块，扩展了冻结的Molmo-7B主干网络，以增强模型在低光照条件下的感知能力。

Details

Motivation: 现有视觉语言模型（VLMs）在低光照条件下表现不佳，因为其视觉基础主要从RGB图像学习，而热红外在可见线索退化时能保留互补的场景结构，因此需要融合热红外信息来提升鲁棒性。

Result: 在Thermo-VL-Bench（一个手动筛选的RGB-热红外VQA基准）上的实验表明，该方法在热红外单独以及RGB+热红外推理任务上取得了显著提升，验证了提示条件多光谱融合的价值。

Insight: 创新点包括：文本引导的双注意力融合模块，能根据语言和RGB上下文条件化热特征；通过门控残差注入将热证据融入冻结的RGB流，而不破坏预训练的RGB-语言接口；以及辅助对齐和正则化损失，改善了跨模态基础并减少对RGB的过度依赖。

Abstract: Vision-language models (VLMs) often fail under low illumination because their visual grounding is learned predominantly from RGB imagery, whereas thermal infrared preserves complementary scene structure when visible cues degrade. We present Thermo-VL, a wavelength-aware VLM that augments a frozen Molmo-7B backbone with a trainable thermal encoder and a text-guided dual-attention fusion module. Given aligned RGB tokens, thermal tokens, and prompt embeddings, the fusion module conditions thermal features on both language and RGB context, then injects a gated residual into the frozen RGB stream so thermal evidence can be incorporated without disrupting Molmo’s pretrained RGB-language interface. We train the model with the standard language-modeling objective together with auxiliary alignment and regularization losses that improve cross-modal grounding and reduce over-reliance on RGB. We also introduce a pixel-aligned RGB-thermal instruction-tuning dataset and Thermo-VL-Bench, a manually screened RGB-thermal VQA benchmark for low-light and cross-spectrum reasoning. Experiments show strong gains on challenging thermal-only and RGB+thermal reasoning tasks, highlighting the value of prompt-conditioned multispectral fusion. Our dataset and code are publicly available at: https://thusharakart.github.io/Thermo-VL

[38] Universal CT Representations from Anatomy to Disease Phenotype through Agglomerative Pretraining cs.CVPDF

Yuheng Li, Yuan Gao, Haoyu Dong, Yuxiang Lai, Shansong Wang

TL;DR: 本文提出了FlexiCT，一个通过聚合式持续预训练在266,227个CT体素上构建的CT基础模型家族。该模型采用二维轴向预训练、三维解剖预训练和报告引导语义对齐的三阶段训练策略，支持切片级、体素级和视觉语言分析。在分割、分类、配准、视觉语言理解和临床检索等五个下游任务家族中，FlexiCT在多个基准测试上达到或超越了先前任务特定方法的性能。

Details

Motivation: 当前基于CT的人工智能研究分散在分割、分类、配准和报告分析等任务特定模型中，缺乏统一的表示学习框架。本文旨在通过大规模CT数据预训练，构建一个通用的CT基础模型，以整合这些任务并提升性能。

Result: FlexiCT在五个下游任务家族（分割、分类、配准、视觉语言理解和临床检索）的多个基准测试中，匹配或超越了先前的任务特定方法。其嵌入表示还能沿肿瘤分期梯度组织CT扫描，表明模型能捕捉与疾病表型表征相关的成像特征。

Insight: 创新点包括聚合式持续预训练策略，结合二维、三维和视觉语言对齐阶段，实现了从解剖到疾病表型的通用CT表示学习。客观分析认为，该方法通过大规模多数据集整合和跨任务统一建模，为医学影像基础模型提供了可扩展的框架。

Abstract: Computed tomography (CT) is a central to three-dimensional medical imaging, yet CT-based artificial intelligence remains fragmented across task-specific models for segmentation, classification, registration, and report analysis. Here we present FlexiCT, a family of CT foundation models trained by agglomerative continual pretraining on 266,227 CT volumes from 56 publicly available datasets, forming a large-scale public resource for CT representation learning. FlexiCT uses agglomerative pretraining across three stages: two-dimensional axial pretraining, three-dimensional anatomical pretraining and report-guided semantic alignment. This training strategy supports slice-level, volume-level and vision-language analysis. Across five downstream task families (segmentation, classification, registration, vision-language understanding and clinical retrieval), FlexiCT matches or exceeds prior task-specific approaches on multiple benchmarks. Its embeddings further organize CT scans along gradients associated with various tumor stages, suggesting that CT foundation models can capture imaging features relevant to disease phenotype characterization. Code is available at https://github.com/ricklisz/FlexiCT

[39] EvoVid: Temporal-Centric Self-Evolution for Video Large Language Models cs.CVPDF

Shiqi Huang, Ziyue Wang, Zhongrong Zuo, Han Qiu, Qi She

TL;DR: 本文提出了EvoVid，一种以时间为中心的自进化框架，旨在让视频大语言模型能够直接从原始、未标注的视频中学习并提升推理能力。该方法通过引入两个互补的时间中心化奖励机制，解决了现有自进化方法在捕捉视频时序动态方面的不足。

Details

Motivation: 现有视频大语言模型的强化学习训练严重依赖人工标注的任务和解决方案，成本高且受限于人类专业知识。而现有的自进化框架主要针对文本和图像等静态模态设计，无法有效捕捉视频推理核心的时序动态。

Result: 在四个基础模型和六个基准测试上的广泛实验表明，该方法相比基础模型和现有自进化基线均取得了一致的性能提升，并与有监督方法达到了具有竞争力的性能水平。

Insight: 核心创新点在于提出了时间感知的提问者奖励和时间基础的求解者奖励，前者通过时序扰动敏感性鼓励生成时序依赖的问题，后者通过视频片段的内在定位提供自动的时序监督，从而实现了对视频时序动态的自主学习和进化。

Abstract: Recent Video Large Language Models (Video-LLMs) have demonstrated strong capabilities in video reasoning through reinforcement learning (RL). However, existing RL pipelines rely heavily on human-annotated tasks and solutions, making them costly to scale and fundamentally constrained by human expertise. Self-evolving frameworks have recently emerged as a promising alternative through autonomous Questioner-Solver self-play. Unfortunately, these approaches are primarily designed for static modalities such as text and images, fundamentally failing to capture the temporal dynamics that are central to video reasoning. In this work, we propose $\textbf{EvoVid}$, a temporal-centric self-evolving framework that enables Video-LLMs to improve directly from raw, unannotated videos. Specifically, we introduce two complementary temporal-centric rewards: a temporal-aware Questioner reward that encourages temporally dependent question generation through temporal perturbation sensitivity, and a temporal-grounded Solver reward that provides automatic temporal supervision via inherent video segment localization. Extensive experiments across four base models and six benchmarks demonstrate consistent improvements over both base models and existing self-evolving baselines, achieving competitive performance with supervised methods. These results highlight temporal-centric self-evolution as an effective and scalable paradigm for video understanding and reasoning.

[40] MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cues cs.CV | cs.AIPDF

Dazhao Du, Liao Duan, Jian Liu, Tao Han, Yujia Zhang

TL;DR: 本文研究了多模态大语言模型（MLLMs）在视频时序定位任务中的表现，发现模型在预填充阶段已能通过特定注意力头感知目标时间区间，但在自回归解码生成答案时却会丢失该信号。为此，作者提出了一种无需训练或修改模型的推理时‘先读后再生’框架，通过利用并重新聚焦这些‘时序定位头’的注意力来提升定位精度。

Details

Motivation: 视频时序定位是检验MLLMs是否理解事件‘何时’发生的关键任务，但现有MLLMs的时序预测不可靠，而现有改进方法要么需要昂贵的标注数据后训练，要么依赖粗糙的启发式方法。本文旨在探究MLLMs的跨模态注意力机制，以揭示并弥补其感知与生成之间的差距。

Result: 在无需参数更新或架构改动的情况下，该框架在MiMo-VL-7B、Qwen3-VL-8B和TimeLens-8B三个模型上，于三个VTG基准测试上取得了一致的性能提升，最高提升了+3.5 mIoU。

Insight: 核心创新点在于揭示了MLLMs内部存在的‘感知-生成差距’，并识别出负责时序定位的特定注意力头（TG-Heads）。基于此观察，提出的推理时框架通过提取并重新聚焦这些头的注意力信号来抑制视觉干扰，有效且低成本地提升了时序定位性能。

Abstract: Video temporal grounding (VTG), which localizes the start and end times of a queried event in an untrimmed video, is a key test of whether multimodal large language models (MLLMs) understand not only what happens but also when it happens. Although modern MLLMs describe video content fluently, their timestamp predictions remain unreliable, while existing remedies either require costly post-training on temporal annotations or rely on coarse training-free heuristics. In this work, we probe the cross-modal attention of MLLMs and uncover a perception-generation gap. Our key finding is that MLLMs often know the target interval during prefill, but lose this signal when generating the final answer. In the prefill stage, a sparse set of attention heads, which we call \emph{Temporal Grounding Heads} (TG-Heads), concentrates query-to-video attention on the ground-truth interval. During autoregressive decoding, however, the answer tokens shift attention away from this interval toward visually salient but query-irrelevant segments. This observation motivates an inference-time read-then-regenerate framework. We first convert TG-Head prefill attention into a debiased frame-level relevance signal and extract the high-attention interval it highlights. We then re-invoke the MLLM with visual context restricted to this interval, using video cropping or attention masking to suppress distractors. Without parameter updates and architectural changes, our framework consistently improves MiMo-VL-7B, Qwen3-VL-8B, and TimeLens-8B on three VTG benchmarks, with gains of up to +3.5 mIoU. The project website can be found at https://ddz16.github.io/mllmsknowwhen.github.io/.

[41] Bounding-Box Trajectories Matter for Video Anomaly Detection cs.CVPDF

Inpyo Song, Jangwon Lee

TL;DR: 本文提出TrajVAD框架，强调边界框轨迹对视频异常检测的重要性。该框架利用归一化流建模多类边界框轨迹以学习正常运动模式，其仅使用轨迹的变体TrajVAD-T在ShanghaiTech数据集上超越了所有基于姿态的方法，而融合姿态信息的TrajVAD-P版本进一步提升了性能。

Details

Motivation: 现有基于人体姿态的异常检测方法虽然性能强劲，但往往忽略了边界框轨迹这一在姿态处理流程中本就可用的信息。本文旨在探索并利用边界框轨迹作为主要的异常检测线索。

Result: 在ShanghaiTech数据集上，仅使用轨迹的TrajVAD-T在AP指标上达到87.7%，超越了所有对比的基于姿态的方法；融合姿态的TrajVAD-P版本达到88.6% AUROC和90.9% AP，在MSAD数据集上也取得了最佳结果。

Insight: 核心创新点在于将边界框轨迹确立为一种有效但未被充分探索的模态，并系统地通过归一化流对其进行建模。客观来看，该方法揭示了在姿态之外，目标检测框的简单运动轨迹本身就蕴含了丰富的、对异常敏感的动力学信息。

Abstract: Video anomaly detection is critical for public safety and security, yet remains highly challenging despite extensive research due to large variations in appearance, viewpoint, and scene dynamics. Among existing approaches, human pose-based methods have emerged as a major line of research, showing strong performance since many anomalies in public datasets involve humans and pose representations are robust to appearance changes while providing compact motion descriptions. However, these methods often overlook bounding-box trajectories, although such information is inherently available in pose-based pipelines. In this paper, we explicitly leverage these trajectories as a primary anomaly cue. We present TrajVAD, a framework that models multi-class bounding-box trajectories using normalizing flows to learn normal kinematic patterns. Its trajectory-only variant (TrajVAD-T) eliminates pose estimation and surpasses all compared pose-based methods on ShanghaiTech in AP (87.7%), while achieving the best results on MSAD. An extended version (TrajVAD-P) incorporates pose information and further improves performance to 88.6% AUROC and 90.9% AP on ShanghaiTech, highlighting bounding-box trajectories as an effective yet underexplored modality for video anomaly detection.

[42] Foresee-to-Ground: From Predictive Temporal Perception to Evidence-Driven Reasoning for Video Temporal Grounding cs.CVPDF

Zelin Zheng, Xinyan Liu, Ruixin Li, Antoni B. Chan, Guorong Li

TL;DR: 本文提出Foresee-to-Ground (F2G)框架，将视频时序定位重新定义为可验证的’识别-再测量’问题。该框架通过预测性时序感知学习边界敏感的表征以构建候选事件片段池，并利用证据驱动推理将这些片段作为可引用的证据单元提供给大语言模型，从而将边界预测与明确的事件假设绑定。

Details

Motivation: 解决现有基于视频大语言模型的时序定位方法因直接从非结构化视觉标记流生成时间戳而导致的数值不稳定和边界不一致问题。

Result: 在多个基准测试上一致提升了定位精度，能够鲁棒地迁移到不同的视频大语言模型骨干网络，并保持了通用的视频理解能力。

Insight: 核心创新在于将时序定位解耦为事件识别和精确边界测量两个可验证的步骤，通过构建显式的、可引用的证据池来稳定预测并增强模型的可解释性。

Abstract: Current Video-LLM approaches for Video Temporal Grounding (VTG) typically rely on direct timestamp generation from an unstructured visual-token stream, often leading to brittle numerics and inconsistent boundaries. To address this, we propose Foresee-to-Ground (F2G), a framework that reformulates VTG as a verifiable Identify-then-Measure problem. F2G integrates Predictive Temporal Perception with Evidence-Driven Reasoning: it learns boundary-sensitive temporal representations to build a video-wide evidence pool of candidate event segments, and exposes these segments to the LLM as citable evidence units that bind boundary prediction to explicit event hypotheses. By decoupling event identification from precise boundary measurement, F2G stabilizes grounding and makes predictions verifiable. Extensive experiments demonstrate that F2G consistently improves grounding accuracy across diverse benchmarks, transfers robustly across different Video-LLM backbones, and preserves general video understanding capabilities.

[43] Video as Natural Augmentation: Towards Unified AI-Generated Image and Video Detection cs.CV | cs.AIPDF

Zhengcen Li, Chenyang Jiang, Liangxu Su, Tong Shao, Shiyang Zhou

TL;DR: 本文提出了一种名为VINA的统一AI生成内容检测框架，旨在解决现有AI生成图像检测器在处理视频帧时性能下降的问题。该框架通过联合训练图像和视频数据，利用视频帧作为自然增强，并引入跨模态监督对比学习目标，以提升检测器的泛化能力和鲁棒性。

Details

Motivation: 随着AI生成内容（AIGC）的快速发展，需要能够跨数据源、部署流程和视觉模态泛化的检测器。现有最先进的AI生成图像检测器在处理视频帧时经常失效，这源于视频处理带来的分布偏移和视频生成器引入的特定指纹，因此需要一种统一的方法来弥合跨模态差距。

Result: 在14个图像、视频和真实世界基准测试上的广泛实验表明，VINA实现了双向性能提升，增强了鲁棒性和可迁移性，在几乎所有评估设置中达到了最先进的性能，且无需复杂的增强或数据集特定调优。

Insight: 创新点在于将视频帧作为物理基础的自然增强数据，并设计跨模态监督对比学习目标来对齐图像和视频表示，从而统一检测框架。这为多模态AIGC检测提供了可借鉴的思路，即通过联合训练和表示对齐来提升泛化能力。

Abstract: AI-generated content (AIGC) is rapidly improving, creating an urgent need for detectors that generalize across data sources, deployment pipelines, and visual modalities. A strongly generalizable detector should remain robust under distributional variations. However, we identify a consistent failure mode: SOTA AI-generated image detectors often collapse when applied to frames extracted from videos. Through systematic analysis, we show that this cross-modal gap arises from both entangled synthesis-agnostic video processing shifts, including color conversion, codec compression, resizing, and blur, and model-specific fingerprints introduced by modern video generators. Motivated by these findings, we propose VINA (Video as Natural Augmentation), a unified AIGC detection framework that jointly trains on image and video data. VINA uses video frames as physically grounded natural augmentations and further introduces a cross-modal supervised contrastive objective to align image and video representations under a shared real/fake decision boundary. Extensive experiments on 14 image, video, and in-the-wild benchmarks show that VINA delivers bidirectional gains, improves robustness and transferability, and achieves state-of-the-art performance across nearly all evaluated settings without complex augmentation or dataset-specific tuning.

Chengsheng Zhang, Chenghao Sun, Zhining Xie, Xinmei Tian

TL;DR: 本文提出了一种基于转向向量的因果归因框架，用于解释和增强大型视觉语言模型（LVLM）中的情感电路。通过构建专门的数据集，揭示了LVLM内部‘适应-聚合-执行’三阶段机制中情感信息流的功能解耦现象，并基于此通过干预信息路由来增强情感表达。在MER-UniBench基准上的实验表明，该方法能通过推理时干预显著提升性能，有效缓解情感幻觉。

Details

Motivation: 大型视觉语言模型在情感理解方面展现出强大能力，但其将抽象视觉刺激转化为连贯情感叙述的内部机制尚不明确，主要由于视觉反事实数据的稀缺和情感表达的弥散性。本文旨在填补这一空白。

Result: 在综合基准MER-UniBench上进行的大量实验表明，所提方法通过推理时干预显著提升了性能，有效缓解了情感幻觉，并证实了所发现电路的因果保真度。

Insight: 创新点在于提出了一个专门用于描述性情感推理的因果归因框架，并发现了LVLM情感处理中视觉情感线索在中间层通过特定情感注意力头聚合，随后在深层通过通用情感通路转化为叙述生成的功能解耦现象。这为理解和可控地增强模型的情感能力提供了新视角。

Abstract: Large Vision-Language Models (LVLMs) represent a significant leap towards empathetic agents, demonstrating remarkable capabilities in emotion understanding. However, the internal mechanisms governing how LVLMs translate abstract visual stimuli into coherent emotional narratives remain largely unexplored, primarily due to the scarcity of visual counterfactuals and the diffuse nature of emotional expression. In this paper, we bridge this gap by introducing a steering-vector-based causal attribution framework tailored for descriptive emotional reasoning. To this end, we construct a specialized dataset to demystify the emotional circuits underlying the three-stage ``Adapt-Aggregate-Execute’’ mechanism. Crucially, we discover a functional decoupling: visual emotional cues are aggregated in middle layers via sentiment-specific attention heads, but are subsequently translated into narrative generation in deep layers through emotion-general pathways. Guided by these insights, we regulate the emotional information routing to strengthen attention flow and amplify the semantic activation to consolidate expression. Extensive experiments on the comprehensive MER-UniBench demonstrate that our methods significantly improve performance via inference-time intervention, effectively mitigating emotional hallucinations and corroborating the causal fidelity of the discovered circuits.

[45] Learning Spatiotemporal Sensitivity in Video LLMs via Counterfactual Reinforcement Learning cs.CV | cs.AIPDF

Dazhao Du, Jian Liu, Jialong Qin, Tao Han, Bohai Gu

TL;DR: 本文针对视频大语言模型（Video LLMs）在回答问题时过度依赖单帧线索和语言先验等捷径，而非真正理解时空动态的问题，提出了一种名为反事实关系策略优化（CRPO）的双分支强化学习框架。该方法通过构建水平翻转和时间反转的反事实视频，并引入反事实关系奖励（CRR）来鼓励模型对动态问题敏感、对静态问题稳定，从而提升时空敏感性。作者还提出了DyBench评估基准来严格衡量模型性能。实验表明，CRPO在时空敏感任务上优于现有RL方法，同时保持了通用的视频理解能力。

Details

Motivation: 现有视频大语言模型虽然在基准测试上表现良好，但往往通过单帧线索或语言先验等捷径回答问题，而非真正追踪视频的时空动态。尤其是在仅以正确性为奖励的强化学习后训练中，这种捷径策略会被进一步强化，导致模型缺乏对视频动态的敏感性。

Result: 在提出的DyBench基准（包含3,014个视频，涵盖可逆动态、移动方向和事件序列）上，CRPO方法在Qwen3-VL-8B模型上将DyBench的配对准确率（P-Acc）提升了+7.7，将TimeBlind的I-Acc提升了+8.2。这表明模型显著提升了时空敏感性，而非更依赖静态捷径。CRPO在时空敏感评估上优于先前的RL方法，同时保持了有竞争力的通用视频性能。

Insight: 核心创新点在于通过反事实推理（构建视觉世界改变而问题不变的反事实视频）来定义和提升模型的“时空敏感性”。具体技术贡献包括：1）CRPO双分支RL框架，通过同时训练原始和反事实分支并引入跨分支的CRR奖励，迫使模型超越捷径策略；2）DyBench评估基准及其严格的配对准确率指标，有效防止固定答案捷径对分数的虚高，为评估时空敏感性提供了更可靠的基准。

Abstract: Video large language models (Video LLMs) achieve strong benchmark accuracy, yet often answer video questions through shortcuts such as single-frame cues and language priors rather than by tracking spatiotemporal dynamics. This issue is exacerbated in RL post-training, where correctness-only rewards can further reinforce shortcut policies that obtain high reward without tracking video dynamics. We address this by asking a controlled counterfactual question: if the visual world changed while the question remained fixed, should the answer change or stay the same? Based on this view, we propose \textbf{Counterfactual Relational Policy Optimization (CRPO)}, a dual-branch RL framework for improving \emph{spatiotemporal sensitivity}. CRPO constructs counterfactual videos through horizontal flips and temporal reversals, trains on both original and counterfactual branches, and introduces a \textbf{Counterfactual Relation Reward (CRR)} between their answers. CRR encourages answers to change for dynamic questions and remain unchanged for static questions. This cross-branch constraint makes it difficult for shortcut policies to be consistently rewarded across both branches. To evaluate this property, we introduce \textbf{DyBench}, a paired counterfactual video benchmark with 3,014 videos covering reversible dynamics, moving direction, and event sequence, together with a strict pair-accuracy metric that prevents fixed-answer shortcuts from inflating scores. Experiments show that CRPO outperforms prior RL methods on spatiotemporal-sensitive evaluations while maintaining competitive general video performance. On Qwen3-VL-8B, CRPO improves DyBench P-Acc by +7.7 and TimeBlind I-Acc by +8.2 over the base model, indicating improved spatiotemporal sensitivity rather than stronger reliance on static shortcuts. The project website can be found at https://ddz16.github.io/crpo.github.io/ .

[46] Visual-Advantage On-Policy Distillation for Vision-Language Models cs.CVPDF

Ruiqi Liu, Xiaolei Lv, Gengsheng Li, Ximo Zhu, Zhiheng Wang

TL;DR: 本文提出了一种针对视觉语言模型（VLMs）的视觉优势在线策略蒸馏方法（VA-OPD），旨在解决标准在线策略蒸馏在提升学生模型输出质量的同时，未能有效增强其对视觉输入依赖性的问题。该方法通过引入视觉优势（VA）这一指标来量化教师模型在有无细粒度视觉细节时对学生生成序列的评分差异，并基于此对高VA和低VA的token进行分组处理，从而强化视觉监督信号。

Details

Motivation: 标准在线策略蒸馏在应用于视觉语言模型时，虽然能提升学生模型的输出质量，但未能有效增强其对视觉输入的依赖，尤其是在视觉关键token上，学生模型的预测对细粒度视觉细节的存在与否不敏感，而教师模型则高度依赖于此。

Result: 在两个数学数据集（Geometry3K和ViRL39K）上训练，并在涵盖数学推理和视觉理解的八个基准测试上评估，使用Qwen3-VL系列的三种教师模型规模（4B、8B和32B）。VA-OPD在所有基准测试上均优于标准在线策略蒸馏，且增益随着教师模型规模和数据集规模的增加而单调增长。

Insight: 创新点在于引入了视觉优势（VA）来识别和量化视觉关键token，并据此设计了分组的KL散度蒸馏目标，从而更有效地传递视觉监督信号，避免了其被大量语言token稀释。这为视觉语言模型的蒸馏提供了一种新的、更精细的监督信号利用方式。

Abstract: On-policy knowledge distillation has proven effective for language models, yet its application to vision-language models (VLMs) remains underexplored. We observe that standard on-policy distillation can improve a student’s output quality while failing to strengthen its reliance on visual input: on vision-critical tokens, the student’s predictions remain largely unchanged whether or not fine-grained visual detail is present, even though the teacher’s predictions depend heavily on it.To make this difference observable, we introduce visual advantage (VA), the token-level log-probability difference when the teacher scores a student-generated rollout with versus without access to fine-grained visual detail. VA is concentrated in a small minority of tokens, and these high-VA tokens are the ones that actually carry the visual supervision signal. This motivates a distillation objective that treats them differently from language scaffolding, so their contribution is not diluted by the abundant surrounding language tokens.We propose Visual-Advantage On-Policy Distillation (VA-OPD), which uses VA at two granularities: rollout-level reweighting by trajectory-averaged VA, and token-level KL averaged within high-VA and low-VA groups separately. We train on two math datasets (Geometry3K and ViRL39K) and evaluate on eight benchmarks covering both mathematical reasoning and visual understanding, across three teacher sizes (4B, 8B, and 32B) on the Qwen3-VL family. VA-OPD improves over standard on-policy distillation on every benchmark, with the gain growing monotonically along both the teacher-size and data-scale axes, suggesting that these factors compound consistently.

[47] PointLLM-R: Enhancing 3D Point Cloud Reasoning via Chain-of-Thought cs.CV | cs.GR | cs.LGPDF

Chaoqi Chen, Qile Xu, Wenjun Zhou, Hui Huang

TL;DR: 本文提出了一种数据驱动的框架，用于构建针对3D点云理解的大规模思维链监督数据，并基于此训练了具备推理能力的3D多模态语言模型PointLLM-R。该框架通过两阶段流程（基于视觉语言模型的质量评估与参考引导精炼、人机交互提示优化）构建了包含5.5万样本的PoCoTI数据集，并在生成式3D分类和描述任务上实现了最先进的性能。

Details

Motivation: 解决3D点云数据因结构不规则及现有3D多模态模型缺乏显式推理能力而导致的语言理解难题，并探索将思维链推理从语言模型和图像多模态模型有效扩展到3D理解领域。

Result: 在生成式3D分类和描述任务上，PointLLM-R取得了最先进的性能，并能稳健地泛化到真实世界扫描点云和多轮对话场景。

Insight: 创新点在于提出了一个专门为3D点云定制的大规模思维链监督数据构建框架（包含数据质量评估与精炼、人机交互优化），并首次将思维链推理系统性地引入3D多模态语言模型，显著提升了模型的推理能力与泛化性能。

Abstract: Understanding 3D point clouds through language remains a fundamental challenge in computer graphics and visual computing, due to the irregular structure of point cloud data and the lack of explicit reasoning in existing 3D multimodal models. While Chain-of-Thought (CoT) reasoning has shown strong effectiveness in LLMs and image-based MLLMs, its extension to 3D understanding remains largely underexplored. In this paper, we propose a data-centric framework for constructing large-scale CoT supervision tailored to 3D point cloud understanding. Our framework consists of a two-stage pipeline that first refines point-text instruction data via vision-language-model-based quality evaluation and reference-guided refinement, and then synthesizes high-quality reasoning paths through Human-in-the-Loop Prompt Optimization (HiLPO). Using this approach, we build PoCoTI, a CoT-enhanced point-text instruction-following dataset containing 55K samples with explicit reasoning paths. Fine-tuning PointLLM on PoCoTI yields PointLLM-R, a reasoning-capable 3D multimodal language model. Extensive experiments on generative 3D classification and captioning demonstrate that PointLLM-R achieves state-of-the-art performance and generalizes robustly to real-world scanned point clouds and multi-turn dialogue scenarios.

[48] ORBIS: Output-Guided Token Reduction with Distribution-Aware Matching for Video Diffusion Acceleration cs.CV | cs.ARPDF

Hangyeol Lee, Joo-Young Kim

TL;DR: 本文提出了ORBIS，一个软硬件协同设计的加速器，用于加速视频扩散Transformer模型。该方法通过利用前一个时间步的输出激活来更准确地估计token相似性，并引入分布感知token匹配算法，从而显著提高token缩减率，实现更高的加速比和能效。

Details

Motivation: 视频DiT模型中的3D时空注意力机制导致token长度随帧数线性增长，计算成本急剧增加。现有token缩减方法依赖不准确的相似性估计和轻量级匹配算法，导致匹配质量差，加速效果有限。

Result: 在广泛的实验中，ORBIS相比最先进的AsymRnR方法，实现了约2倍的token缩减率提升。与NVIDIA A100 GPU相比，ORBIS实现了高达4.5倍的加速和79.3%的能耗降低。

Insight: 核心创新点在于利用前一时间步的输出激活（输出引导）来指导token缩减，这比仅基于当前输入特征的相似性估计更准确。同时，提出的分布感知token匹配算法能捕获全局token分布并显式最小化token对损失，进一步提升了匹配质量。硬件设计上，通过量化和深度流水线设计，以极小的面积开销（占总面积的2.4%）隐藏了算法延迟。

Abstract: Diffusion Transformer (DiT) has emerged as a powerful model architecture for generating high-quality images and videos. In the case of video DiT, 3D Spatio-Temporal Attention increases token length in proportion to the number of frames, sharply increasing computational cost. Token reduction methods mitigate this cost by exploiting spatial redundancy, but existing approaches rely on inaccurate similarity estimates and lightweight matching algorithms, resulting in poor matching quality and only marginal acceleration. To overcome these limitations, we propose ORBIS, an SW-HW co-designed accelerator for video DiT. ORBIS leverages the output activation from the previous timestep to obtain more accurate inter-token similarity, substantially improving matching quality and enabling a higher token reduction ratio. We further introduce a Distribution-Aware Token Matching (DATM) algorithm that captures global token distribution and explicitly minimizes token-pair loss for additional gains. To fully hide DATM latency, we design specialized, deeply pipelined hardware and minimize its hardware cost through quantization, occupying only 2.4% of total area with negligible accuracy loss. Extensive experiments show that ORBIS achieves about 2x higher token reduction ratio than the state-of-the-art approach, AsymRnR, while delivering up to 4.5x speedup and 79.3% energy reduction compared to an NVIDIA A100 GPU.

[49] Diverse Yet Consistent: Context-Guided Diffusion with Energy-Based Joint Refinement for Multi-Agent Motion Prediction cs.CVPDF

Lei Chu, Yuhuan Zhao

TL;DR: 本文提出了一种基于扩散模型的多智能体运动预测框架，通过历史轨迹的上下文引导机制增强预测的多样性，并引入基于能量的联合优化方法确保交互一致性。

Details

Motivation: 现有深度生成模型在捕捉多模态分布方面有优势，但难以同时保证多智能体预测的多样性和交互一致性，且评估多依赖单智能体指标，未能充分反映联合动态。

Result: 在四个基准数据集上的实验表明，该方法显著优于现有方法，在ETH/UCY数据集上单智能体指标（ADE/FDE）和联合指标（JADE/JFDE）均有大幅提升，相比先前联合预测方法在单智能体指标上取得显著增益的同时保持竞争力的联合性能。

Insight: 创新点在于结合上下文引导扩散模型与基于能量的联合优化，既利用历史信息增强多样性，又通过能量函数约束联合轨迹分布以保持交互一致性，为多智能体运动预测提供了兼顾多样性与一致性的新思路。

Abstract: Deepgenerative models havebecomeapromisingapproach for human motion prediction due to their ability to capture multimodal distributions and represent diverse human be haviors. However, generating predictions that are both di verse and jointly consistent among interacting agents re mains challenging. In addition, most existing approaches are primarily evaluated using single-agent (marginal) met rics, which fail to fully reflect the joint dynamics of multi agent interactions. We propose a diffusion-based frame work that improves multi-agent motion prediction by lever aging rich contextual information from historical trajecto ries. This information is incorporated through a guidance mechanism to enhance the diversity and expressiveness of predicted motions. To further enforce interaction consis tency, we introduce an energy-based formulation that re fines the joint trajectory distribution while preserving the plausibility of individual trajectories. Extensive experi ments on four benchmark datasets demonstrate that our approach consistently outperforms existing methods. No tably, our approach substantially improves both marginal (ADE/FDE) and joint (JADE/JFDE) metrics on ETH/UCY over strong marginal baselines. Compared with prior joint prediction methods, it delivers significant gains in marginal metrics while maintaining competitive joint performance.

Connor Malone, Sebastien Demmel, Sebastien Glaser

TL;DR: FRED是首个专注于道路积水场景的多模态自动驾驶数据集，包含来自五个不同地点的图像、LiDAR点云和IMU/GNSS数据，数据格式兼容KITTI和RTMaps，并提供语义标注以支持水害检测、定位和SLAM等任务的训练与评估。

Details

Motivation: 现有自动驾驶数据集缺乏专门针对道路积水等水害场景的数据，限制了相关感知和决策算法的开发与评估，因此需要构建一个多模态、真实场景的水害数据集。

Result: 论文构建并发布了FRED数据集，提供了多传感器数据及语义标注，但摘要中未提及基于该数据集的定量实验结果或基准测试性能。

Insight: 创新点在于首次构建了专注于道路积水危害的多模态自动驾驶数据集，并提供了兼容现有工具链的数据格式和语义标注，有助于推动恶劣天气条件下自动驾驶感知技术的研究。

Abstract: The Flooded Road Environments Dataset (FRED) is, to our knowledge, the first multi-modal autonomous driving dataset specifically targeting the collection of data from scenarios involving water hazards on the road. The dataset contains images from a 2.3 MP FLIR Blackfly USB3 camera, 64-beam 360$^\circ$ point clouds from an Ouster OS1-64 LiDAR, and data from an iXblue ATLANS-C IMU corrected by a Geoflex RTK GNSS, from five separate locations captured both during and after flooding events. The data has been released in two formats: a KITTI-style format for easy integration with existing data tools, and the RTMaps format for direct replay of the vehicle’s data capture. We provide semantic labels to enable the training and evaluation of both single-sensor and sensor-fusion methods for water hazard detection. Position and velocity, as well as data captured under dry conditions, are provided to enable the development of location-based detection methods that may incorporate maps, and to evaluate other tasks such as localisation and SLAM.

[51] AgroVG: A Large-Scale Multi-Source Benchmark for Agricultural Visual Grounding cs.CV | cs.AIPDF

Haocheng Li, Juepeng Zheng, Zenghao Yang, Kaiqi Du, Guilong Xiao

TL;DR: 本文提出了AgroVG，一个用于农业视觉定位任务的大规模多源基准测试。该基准将农业视觉定位定义为广义集合预测问题，要求模型根据图像和自然语言查询返回所有匹配的目标实例或在无目标时主动放弃。AgroVG包含来自十个数据集的10,071个标注图像-查询对，涵盖六类农业目标，并支持边界框和实例掩码两种定位任务。

Details

Motivation: 农业视觉定位是农业AI系统的基础能力，但可靠的评估面临挑战，因为农业目标通常具有小、重复、遮挡或不规则形状等特点，且查询可能涉及单个、多个或无目标。现有评估方法难以全面测试定位精度、目标集完整性和存在感知的放弃能力。

Result: 在AgroVG基准上对26种模型配置（包括闭源MLLM、开源VLM和专用定位系统）进行零样本评估，结果显示存在显著差距：最佳模型的多目标Set-F1分数仅为0.35，在IoU@0.75阈值下的阳性查询掩码成功率低于0.17。

Insight: 论文的创新点在于将农业视觉定位形式化为广义集合预测问题，并构建了一个大规模、多源、多任务的基准测试集，专门针对农业场景的独特挑战（如小目标、多目标、目标缺失）设计了评估协议。这为评估和推动农业AI的视觉理解能力提供了重要的基准工具。

Abstract: Visual grounding, the task of localizing objects described by natural-language expressions, is a foundational capability for agricultural AI systems, enabling applications such as selective weeding, disease monitoring, and targeted harvesting. Reliable evaluation of agricultural visual grounding remains challenging because agricultural targets are often small, repetitive, occluded, or irregularly shaped, and instructions may refer to one, many, or no objects in an image. Evaluating this capability therefore requires jointly testing localization accuracy, target-set completeness, and existence-aware abstention. To address these challenges, we introduce \textbf{AgroVG}, a multi-source benchmark that formulates agricultural grounding as generalized set prediction: given an image and a referring expression, a model must return all matching target instances or abstain when no target is present. AgroVG contains 10{,}071 annotation-grounded image-query pairs from ten source datasets across six target families: crop/weed, fruit, wheat head, pest, plant disease, and tree canopy. It supports bounding-box grounding (T1) across all six families and instance-mask grounding (T2) on sources with reliable instance-level pixel annotations, with queries covering single-target, multi-target, and target-absent regimes. AgroVG further provides task-specific protocols for box-set matching and query-level mask coverage. Zero-shot evaluation of 26 model configurations spanning closed-source MLLMs, open-source VLMs, and specialized grounding systems reveals persistent gaps: the best multi-target Set-$F_1$ reaches only 0.35, and the best positive-query mask success rate at IoU@0.75 remains below 0.17. Data and code are available at https://anonymous.4open.science/r/AgroVG-5172/ .

[52] Distributed Image Compression with Multimodal Side Information at Extremely Low Bitrates cs.CVPDF

Guojun Xu, Mingyang Zhang, Jianwen Xiang, Cheng Tan, Yanchao Yang

TL;DR: 本文提出了一种多模态分布式图像压缩框架（MDIC），旨在解决极低码率（<0.1 bpp）下多视图传输中，现有方法难以充分利用边信息全局上下文和物体级细节，导致重建图像局部模糊和细节丢失的问题。该框架首次将边信息以多模态方式引入，通过基于文本到图像扩散的解码器捕获全局语义，并设计特征掩码生成器增强视觉边信息利用，从而在保持细节语义一致性的同时提升全局感知质量。

Details

Motivation: 现有分布式图像压缩方法在极低码率下难以有效利用边信息的全局上下文和物体级细节，导致重建质量下降，存在局部模糊和精细细节丢失的问题。

Result: 在KITTI Stereo和Cityscapes数据集上的大量实验表明，MDIC在极低码率下达到了最先进的感知质量。

Insight: 创新点在于首次将多模态边信息引入分布式图像压缩范式，结合文本引导的扩散解码器捕获全局语义，以及通过多模态细粒度对齐任务监督的特征掩码生成器来增强视觉细节利用和补偿类别信息损失，有效协调了全局感知与局部细节保持。

Abstract: Distributed Image Compression (DIC) is crucial for multi-view transmission, especially when operating at extremely low bitrates (< 0.1 bpp). Its core challenge is effectively utilizing side information to achieve high-quality reconstruction under strict bitrate budgets. However, existing DIC approaches struggle to exploit global context and object-level details from side information, leading to local blurring and the loss of fine details in the reconstruction. To address these limitations, we propose a Multimodal DIC framework (MDIC), which, for the first time, leverages side information in a multimodal manner into the DIC paradigm, effectively preserving fine-grained local details and enhancing global perceptual quality in reconstructed images. Specifically, we introduce a text-to-image diffusion-based decoder conditioned on textual side information extracted from correlated images to capture shared global semantics. Moreover, we design a feature-mask generator, supervised by a multimodal fine-grained alignment task, to strengthen the exploitation of visual side information. The generated mask serves two purposes: first, it guides the extraction of fine-grained details from losslessly transmitted side information to preserve the semantic consistency of reconstructed details; second, it regulates the extraction of clustered feature representations from the quantized VQ-VAE embeddings, compensating for category information lost under the extreme compression of the primary image. Extensive experiments on the widely used KITTI Stereo and Cityscapes datasets demonstrate that MDIC achieves state-of-the-art perceptual quality at extremely low bitrates.

Jiahao Yang, Zihan Wang, Xiangyang Li, Xing Zhu, Yujun Shen

TL;DR: 本文提出了一种用于视觉语言导航（VLN）的几何感知鸟瞰图表示方法（GA-BEV），以解决现有方法依赖密集RGB视频导致计算开销大和空间推理能力有限的问题。该方法通过将RGB-D输入投影到3D空间构建紧凑的、基于智能体中心的BEV空间地图，并结合预训练3D基础模型的隐式几何先验，从而显著提高了导航效率和性能。

Details

Motivation: 现有VLN方法依赖密集RGB视频，会产生过多的图像块标记并缺乏明确的空间结构，导致计算开销巨大且空间推理能力受限。

Result: 实验表明，该方法仅使用导航数据（无需DAgger增强或混合VQA训练）就取得了最先进（SOTA）的结果，证明了其鲁棒性和数据效率。

Insight: 主要创新点在于将显式（基于深度投影）和隐式（从大规模3D重建任务中学习的先验）几何线索集成到基于MLLM的导航系统中，形成紧凑且空间表达能力强的BEV表示，从而有效减少标记冗余并增强几何理解。

Abstract: Despite significant progress in Vision-Language Navigation (VLN), existing approaches still rely on dense RGB videos that produce excessive patch tokens and lack explicit spatial structure, resulting in substantial computational overhead and limited spatial reasoning. To address these issues, we introduce the Geometry-Aware BEV (GA-BEV) - a compact, 3D-grounded feature representation that integrates both explicit and implicit geometric cues into multimodal large language model (MLLM) - based navigation systems. We construct BEV spatial maps from RGB-D inputs by projecting visual features into 3D space and aggregating them into an agent-centric layout that preserves geometric consistency while reducing token redundancy. To further enrich geometric understanding, we incorporate features from a pretrained 3D foundation model into the BEV space, injecting structural priors learned from large-scale 3D reconstruction tasks. Together, these complementary cues - explicit depth-based projection and implicit learned priors - yield compact yet spatially expressive representations that substantially improve navigation efficiency and performance. Experiments show that our method achieves state-of-the-art results using only navigation data, without DAgger augmentation or mixed VQA training, demonstrating the robustness and data efficiency of the proposed GA-VLN framework.

[54] Echo4DIR: 4D Implicit Heart Reconstruction from 2D Echocardiography Videos cs.CV | cs.AIPDF

Yanan Liu, Qinya Li, Hao Zhang, Kangjian He, Xuan Yang

TL;DR: 本文提出了一种名为Echo4DIR的新型测试时4D隐式重建框架，用于从稀疏的2D超声心动图视频中重建4D（3D+时间）心脏几何结构。该方法通过心脏条件SDF从统计形状模型中学习稳健的3D形状先验，并利用极线掩码编码器融合多视图特征。通过自监督的SDF定制可微分渲染策略进行患者特定的3D形状适应，并引入径向SDF对齐策略实现物理连续的4D扩展，从而克服几何模糊性和时间不连续性的挑战。

Details

Motivation: 从稀疏的2D超声心动图重建4D心脏几何结构具有很高的临床需求，但面临着几何模糊性和时间不连续性的根本性挑战。现有方法难以在缺乏3D真实标注的临床环境中，从稀疏的2D观测中生成解剖学可靠且时间连续的4D几何。

Result: 在合成基准和真实临床数据集上的大量实验表明，Echo4DIR在4D心脏网格重建方面达到了最先进的性能，在临床重叠度指标上取得了高达98.35%的Dice系数和96.75%的IoU的优异结果。

Insight: 创新点包括：1）通过心脏条件SDF和极线掩码编码器学习并融合稳健的3D形状先验；2）提出自监督的SDF定制可微分渲染策略，仅使用未校准的临床掩码进行患者特定适应，无需3D真实标注，有效弥合合成到真实的领域差距；3）利用隐式表示的固有连续性克服稀疏观测限制；4）引入径向SDF对齐策略，将形状演化严格锁定到预测的速度场，从根本上消除了网格漂移，实现了物理连续的4D扩展。

Abstract: Reconstructing 4D (3D+t) cardiac geometry from sparse 2D echocardiography is highly desirable yet fundamentally challenged by geometric ambiguity and temporal discontinuity. To tackle these issues, we propose Echo4DIR, a novel test-time 4D implicit reconstruction framework. Specifically, we learn robust 3D shape priors from statistical shape models (SSMs) via a cardiac conditional SDF, constructing an Epipolar Mask Encoder module with epipolar cross attention to effectively fuse multi-view features. To bridge the synthetic-to-real domain gap, we introduce a self-supervised SDF-tailored differentiable rendering strategy for patient-specific 3D shape adaptation using uncalibrated clinical masks without requiring 3D ground truth. Crucially, the inherent continuity of implicit representation overcomes sparse observations, enabling anatomically reliable geometry at arbitrary resolutions. Furthermore, to empower our framework with physically continuous 4D extension, we introduce a Radial SDF Alignment strategy that strictly locks shape evolution to the predicted velocity field, fundamentally eliminating mesh drift. Extensive experiments on synthetic benchmarks and real clinical datasets demonstrate that Echo4DIR achieves state-of-the-art 4D cardiac mesh reconstruction, notably yielding an impressive clinical overlap of up to 98.35% Dice and 96.75% IoU.

[55] COCOTree: A Dataset and Benchmark for Open Tree-Structured Visual Decomposition cs.CVPDF

Junhyub Lee, Seunghun Chae, Hyosu Kim

TL;DR: 论文提出了开放树分解任务，旨在将图像分割为具有无约束粒度和灵活性的层次化视觉组件树。作者通过开发自动化生成流水线构建了大规模基准数据集COCOTree，包含超过21K图像和1.8M结构节点，并提出了Open Tree Quality（OTQ）评估指标来标准化评估协议。

Details

Motivation: 解决图像层次化视觉分解中手动标注认知和物理成本过高的问题，并建立开放词汇、灵活粒度的结构化视觉理解新范式。

Result: 构建的COCOTree数据集包含21K图像和1.8M节点，覆盖3.5K独特标签的长尾分布；人工评估证实生成标注与人类结构判断高度一致，并提出了OTQ评估指标。

Insight: 创新点在于结合大型视觉语言模型（LVLM）的语义推理与SAM的几何定位实现全自动层次化标注生成，以及提出开放树分解任务与OTQ多维度评估指标，为结构化视觉理解提供了新基准。

Abstract: We formalize and enable the task of open tree decomposition, which segments an image into hierarchical trees of visual components with unconstrained granularity and flexibility. Specifically, we provide the foundation benchmark for this new paradigm with the following three key contributions. First, we overcome the prohibitively high cognitive and physical bottlenecks of manual annotation by developing a fully automated generation pipeline that synergizes the semantic reasoning of Large Vision-Language Models (LVLMs) with the precise geometric grounding of SAM 3. Second, leveraging this pipeline, we construct COCOTree, a massive-scale benchmark featuring over 21K images and 1.8M structural nodes. By embracing an open-vocabulary space of over 3.5K unique labels, it successfully captures the long-tail distribution of complex physical assemblies. Notably, rigorous human evaluation confirms our generated annotations demonstrate strong alignment with human structural judgment. Third, we establish a standardized evaluation protocol by proposing the Open Tree Quality (OTQ) metric, which jointly assesses mask precision, label accuracy, and structural consistency. We release our dataset and benchmark code at https://github.com/melonkick3090/COCOTree.

[56] TWINGS: Thin Plate Splines Warp-aligned Initialization for Sparse-View Gaussian Splatting cs.CV | cs.LGPDF

Hyeseong Kim, Geonhui Son, Deukhee Lee, Dosik Hwang

TL;DR: 论文提出TWINGS框架，通过薄板样条（TPS）变形模型对齐反投影点与三角化控制点，为稀疏视图下的3D高斯溅射（3DGS）提供快速且几何精确的初始化，从而提升场景重建的结构细节和颜色保真度。

Details

Motivation: 解决稀疏视图输入下新颖视图合成的挑战，特别是针对3D高斯溅射中因点云稀疏性导致的重建质量下降问题。

Result: 在DTU、LLFF和Mip-NeRF360基准测试中，TWINGS均优于现有方法，在稀疏视图场景下实现了详细且准确的重建。

Insight: 创新点在于利用TPS的平滑非刚性变形模型进行全局一致的对齐，为3DGS提供几何精确的初始化，这直接针对点云稀疏性，提升了重建的鲁棒性和效率。

Abstract: Novel view synthesis from sparse-view inputs poses a significant challenge in 3D computer vision, particularly for achieving high-quality scene reconstructions with limited viewpoints. We introduce TWINGS, a framework that enhances 3D Gaussian Splatting (3DGS) by directly addressing point sparsity. We employ Thin Plate Splines (TPS), a smooth non-rigid deformation model that minimizes bending energy to estimate a globally coherent warp from control-point correspondences, to align backprojected points from estimated depth with triangulated 3D control points, yielding calibrated backprojected points. By sampling these calibrated points near the control points, TWINGS provides a fast and geometrically accurate initialization for 3DGS, ultimately improving structural detail preservation and color fidelity in reconstructed scenes. Extensive experiments on DTU, LLFF, and Mip-NeRF360 demonstrate that TWINGS consistently outperforms existing methods, delivering detailed and accurate reconstructions under sparse-view scenarios.

[57] JMed48k: A Multi-Profession Japanese Medical Licensing Benchmark for Vision-Language Model Evaluation cs.CV | cs.AIPDF

Yue Xun, Junyu Liu, Qian Niu, Xinyi Wang, Zheng Yuan

TL;DR: 本文提出了JMed48k，一个用于评估视觉语言模型的多专业日本医疗执照考试基准。该基准基于日本厚生劳动省发布的官方PDF材料构建，包含2005年至2025年间11项国家执照考试的48,862个考题和20,142张图像，视觉内容按8种类型分类。作者从中提取了JMed48k-Eval评估子集，并对21个专有、开源和医疗专用模型进行了评估，分别报告了纯文本和带图像问题的性能，并引入了一种成对图像移除审计方法来分析模型对视觉证据的利用情况。

Details

Motivation: 为了解决在医疗执照考试场景中，缺乏一个大规模、多专业、包含丰富视觉内容的日语基准来系统评估视觉语言模型性能的问题。

Result: 在JMed48k-Eval子集（包含12,484个计分问题）上评估了21个模型。审计结果显示，专有和开源模型从图像中获益显著，而医疗专用系统对视觉证据的利用有限。不同专业间，图像移除的净效应差异巨大，从医师问题的+5.7分到公共卫生护士问题的+39.8分不等。

Insight: 创新点在于构建了一个大规模、多专业、时间跨度长的日语医疗视觉问答基准，并引入了成对图像移除审计方法，能够量化分析模型对图像信息的依赖程度，揭示了不同模型类型和不同专业问题对视觉信息的利用存在显著差异，为医疗领域的VLM评估提供了新工具和洞见。

Abstract: We introduce JMed48k, a multi-profession Japanese healthcare licensing benchmark for evaluating vision-language models. Built from official PDF materials released by the Japanese Ministry of Health, Labour and Welfare, JMed48k contains 48,862 exam questions and 20,142 images from 11 national licensing examinations between 2005 and 2025, with visual content annotated under an 8-type taxonomy. From this corpus, we derive JMed48k-Eval, a recent five-year evaluation subset with 12,484 scored questions, including 9,905 text-only questions and 2,579 questions with images. We evaluate 21 proprietary, open-source, and medical-specific models, reporting text-only and with-image performance separately. Because these subsets contain different questions, we further introduce a paired image-removal audit that evaluates questions with images before and after removing visual content to explore four answer-transition states. The audit shows that proprietary and open source models gain substantially from images, whereas medical-specific systems show limited observable use of visual evidence, with many correct answers persisting after image removal. Even among proprietary models, the net image-removal effect varies sevenfold across professions, from +5.7 points on Physician questions to +39.8 points on Public Health Nurse questions. We release JMed48k to support reproducible, profession-stratified evaluation of vision-language models in medical licensing settings.

[58] LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model cs.CV | cs.AIPDF

Xiaodong Mei, Diankun Zhang, Hongwei Xie, Guang Chen, Hangjun Ye

TL;DR: LVDrive提出了一种基于潜在视觉表示增强的视觉-语言-动作自动驾驶模型，通过引入未来场景预测任务，在高级潜在空间中学习未来表示，并联合建模未来场景与运动预测，以单次前向传播实现未来感知推理，最终通过两阶段轨迹解码策略利用学习到的潜在未来表示来优化轨迹生成。

Details

Motivation: 现有视觉-语言-动作模型通常依赖稀疏的动作监督，未能充分利用其强大的场景理解与推理能力；而近期通过世界建模引入密集视觉监督的方法又往往过度强调像素级图像重建，忽视了语义上有意义的场景表示学习。

Result: 在具有挑战性的Bench2Drive基准测试中进行的广泛实验表明，LVDrive在闭环驾驶性能上取得了显著提升，超越了仅使用动作监督的方法和基于图像重建的世界模型方法。

Insight: 创新点在于将未来场景预测任务整合到VLA范式中，在预训练视觉骨干网络的辅助监督下于高级潜在空间学习未来表示，并采用非自回归的联合建模与单次前向推理，以及显式利用潜在未来表示的两阶段轨迹解码策略，从而更有效地进行语义场景表示学习与未来感知推理。

Abstract: Vision-Language-Action (VLA) models have emerged as a promising framework for end-to-end autonomous driving. However, existing VLAs typically rely on sparse action supervision, which underutilizes their powerful scene understanding and reasoning capabilities. Recent attempts to incorporate dense visual supervision via world modeling often overemphasize pixel-level image reconstruction, neglecting semantically meaningful scene representation learning. In this work, we propose LVDrive, a Latent Visual representation enhanced VLA framework for autonomous driving. LVDrive introduces a future scene prediction task into the VLA paradigm, where future representations are learned entirely in a high-level latent space under auxiliary supervision from a pretrained vision backbone. Departing from inefficient autoregressive generation, we jointly model future scene and motion prediction within a unified embedding space, processed in a single forward pass to conduct the future-aware reasoning. We further design a two-stage trajectory decoding strategy that explicitly leverages the learned latent future representations to refine trajectory generation. Extensive experiments on the challenging Bench2Drive benchmark demonstrate that LVDrive achieves significant improvements in closed-loop driving performance, outperforming both action supervised methods and image-reconstruction-based world model approaches.

[59] TextTeacher: What Can Language Teach About Images? cs.CV | cs.AI | cs.LGPDF

Tobias Christian Nauen, Stanislav Frolov, Brian Bernhard Moser, Federico Raue, Ahmed Anwar

TL;DR: 本文提出TextTeacher方法，利用语言模型的语义知识来提升视觉模型的性能。该方法通过将文本嵌入作为辅助信息注入图像分类训练，无需改变推理时的模型结构，即可显著提升ImageNet上的分类准确率，并实现高效的迁移学习。

Details

Motivation: 受柏拉图表示假说启发，研究语言模型的语义知识能否高效改进视觉模型，旨在通过跨模态信息注入提升图像分类性能。

Result: 在ImageNet上使用标准ViT骨干网络，TextTeacher将准确率最高提升2.7个百分点，平均迁移增益达1.0个百分点；相比视觉知识蒸馏方法，在相同计算预算下准确率更高，或达到相似准确率但训练速度快33%。

Insight: 创新点在于利用预训练冻结文本编码器生成语义锚点，作为特征空间预处理器，在训练早期塑造深层表示并提供互补语义线索；该方法计算开销极小，无需昂贵多模态训练，保持了纯视觉模型的简洁性和低延迟。

Abstract: The platonic representation hypothesis suggests that sufficiently large models converge to a shared representation geometry, even across modalities. Motivated by this, we ask: Can the semantic knowledge of a language model efficiently improve a vision model? As an answer, we introduce TextTeacher, a simple auxiliary objective that injects text embeddings as additional information into image classification training. TextTeacher uses readily available image captions, a pre-trained and frozen text encoder, and a lightweight projection to produce semantic anchors that efficiently guide representations during training while leaving the inference-time model unchanged. On ImageNet with standard ViT backbones, TextTeacher improves accuracy by up to +2.7 percentage points (p.p.) and yields consistent transfer gains (on average +1.0 p.p.) under the same recipe and compute. It outperforms vision knowledge distillation, yielding more accuracy at a constant compute budget or similar accuracy, but 33% faster. Our analysis indicates that TextTeacher acts as a feature-space preconditioner, shaping deeper layers in the first stages of training, and aiding generalization by supplying complementary semantic cues. TextTeacher adds negligible overhead, requires no costly multimodal training of the target model and preserves the simplicity and latency of pure vision models. Project page with code and captions: https://nauen-it.de/publications/text-teacher

[60] VISTA: Validation-Guided Integration of Spatial and Temporal Foundation Models with Anatomical Decoding for Rare-Pathology VCE Event Detection – after competition results cs.CVPDF

Bo-Cheng Qiu, Fang-Ying Lin, Ming-Han Sun, Yu-Fan Lin, Chia-Ming Lee

TL;DR: 本文提出了VISTA框架，用于解决胶囊内窥镜中罕见病理事件检测的挑战。该框架通过集成EndoFM-LV（用于时序上下文）和DINOv3 ViTL/16（用于帧级视觉语义）两个基础模型，并结合多样头集成、验证引导加权融合以及解剖感知时序事件解码等技术，以任务指标为导向进行优化。在RAREVISION任务中，该方法在赛后评估中取得了优异的性能。

Details

Motivation: 胶囊内窥镜事件检测面临临床相关发现稀疏、视觉异质性高且需在事件级别而非帧级别进行评估的挑战，现有方法难以有效应对。

Result: 在RAREVISION任务的隐藏测试集上，原始官方提交的时序mAP@0.5为0.3530，mAP@0.95为0.3235；赛后通过全局粗搜索扩展局部阈值优化，性能提升至mAP@0.5 0.3726和mAP@0.95 0.3431，使团队在赛后评估中排名第二。

Insight: 创新点在于提出了一个与评估指标对齐的多主干集成框架，通过验证引导的加权融合和解剖感知的解码策略，有效结合了时空基础模型的优势，并展示了通过后处理（全局阈值搜索）进一步提升模型性能的潜力。

Abstract: Capsule endoscopy event detection is challenging because clinically relevant findings are sparse, visually heterogeneous, and evaluated at the event level rather than by frame accuracy. We propose VISTA, a metric-aligned multi-backbone framework for the RAREVISION task. VISTA combines EndoFM-LV for temporal context and DINOv3 ViTL/16 for frame-level visual semantics, followed by a Diverse Head Ensemble (DHE), Validation-Guided Weighted Fusion (VGWF), and Anatomy-Aware Temporal Event Decoding (ATED). The original official submission achieved hidden-test temporal mAP@0.5 of 0.3530 and mAP@0.95 of 0.3235. After the competition, extending local threshold refinement with a global coarse search improved performance to 0.3726 mAP@0.5 and 0.3431 mAP@0.95, ranking Team ACVLab second in the post-competition evaluation.

[61] AesFormer: Transform Everyday Photos into Beautiful Memories cs.CVPDF

Tianxiang Du, Hulingxiao He, Yuxin Peng

TL;DR: 该论文提出了AesFormer框架，旨在通过结构重建来提升日常照片的美学质量，同时保持主体身份和场景语义。该框架采用两阶段方法：第一阶段使用AesThinker模型分析照片并生成可执行的编辑动作；第二阶段由AesEditor模型根据这些动作进行结构编辑。论文还构建了AesRecon基准数据集来支持美学照片重建任务。

Details

Motivation: 现有修图和肖像增强方法无法修复日常照片中的结构缺陷（如构图、视角或姿态），而当前图像编辑模型缺乏美学理解，导致编辑结果语义合理但美学效果不佳。

Result: 实验表明，AesFormer显著提升了美学照片重建性能，并与Nano Banana Pro模型表现相当。

Insight: 创新点在于将美学规划与图像编辑解耦的两阶段框架，以及使用GRPO-A鼓励多样化动作规划的强化学习方法；客观来看，构建严格对齐的AesRecon基准数据集也为该领域提供了重要评估资源。

Abstract: In everyday photography, aesthetically appealing moments are often captured with structural flaws (e.g., composition, camera viewpoint, or pose) that existing retouching and portrait enhancement methods cannot fix. We formulate Aesthetic Photo Reconstruction (APR) as improving a photo’s aesthetic quality via structural reconstruction while preserving subject identity and scene semantics. Although recent advances in image editing models make APR feasible, they often lack aesthetic understanding, yielding edits that are semantically plausible yet aesthetically weak. To address this, we propose AesFormer, a two-stage framework that decouples aesthetic planning from image editing. In Stage 1, an aesthetic action model (AesThinker) analyzes the input along seven progressive photographic dimensions and outputs executable editing actions; we further apply GRPO-A to encourage broad exploration over diverse action plans beyond SFT. In Stage 2, an action-conditioned editor (AesEditor) performs structural edits guided by these actions. To support APR, we build a video-based corpus-mining pipeline (VCMP) and construct AesRecon, a benchmark of 9,071 strictly aligned (poor, good) image pairs. Experiments show that AesFormer substantially improves APR performance and is competitive with Nano Banana Pro. Code is available at https://github.com/PKU-ICST-MIPL/AesFormer_ICML2026.

[62] OPERA: An Agent for Image Restoration with End-to-End Joint Planning-Execution Optimization cs.CVPDF

Feng Zhu, Shuyang Xie, Yihan Zeng, Ming Liu, Wangmeng Zuo

TL;DR: 本文提出了OPERA框架，通过端到端联合优化规划与执行来解决复杂混合退化下的图像复原问题。该框架采用强化学习优化工具组合规划，并引入智能体引导的协同训练使工具在序列组合中学习协作行为。

Details

Motivation: 现有基于智能体的图像复原方法受限于隐式约束的规划空间以及独立预训练工具间缺乏协调，导致性能受限。

Result: 在多种混合退化基准测试和真实数据集上的实验表明，OPERA在多样复杂退化场景下持续优于一体化复原模型和现有基于智能体的方法。

Insight: 创新点在于将规划与执行端到端联合优化，通过强化学习直接优化工具组合，并设计协同训练机制促进工具间的协作，突破了传统方法中规划与执行分离的局限。

Abstract: Real-world image restoration is challenging due to complex and interacting mixed degradations. Recent agent-based approaches address this problem by composing multiple task-specific restoration tools. However, empirical analysis reveals that their performance is fundamentally limited by implicitly constrained planning spaces and the lack of coordination among independently pretrained tools. To address these issues, we propose OPERA (Optimized Planning-Execution Restoration Agent), a framework that jointly optimizes restoration planning and tool execution in an end-to-end manner. On the planning side, OPERA uses reinforcement learning to directly optimize tool composition over a combinatorial plan space, with the final restoration quality as the reward. On the execution side, OPERA introduces agent-guided co-training of restoration tools, enabling them to learn cooperative behaviors under sequential composition. Extensive experiments on multi-degradation benchmarks and real-world datasets demonstrate that OPERA consistently outperforms both all-in-one restoration models and existing agent-based methods across diverse and complex degradation scenarios.

[63] Accelerating Vision Foundation Models with Drop-in Depthwise Convolution cs.CVPDF

Carmelo Scribano, Mohammad Mahdi, Nedyalko Prisadnikov, Yuqian Fu, Giorgia Franchini

TL;DR: 本文提出了一种加速预训练视觉基础模型推理的方法，通过利用ViT中某些注意力头固有的类卷积行为，用高效的深度可分离卷积层作为即插即用的替代品，并结合识别可替换头的策略和微调程序，在图像分类和分割任务上实现了17-20%的推理加速，且性能损失极小。

Details

Motivation: 预训练的视觉基础模型（如ViT）虽然性能强大，但其Transformer架构推理成本高，难以部署在资源受限的设备上，因此需要在不显著损害特征提取能力的前提下加速推理。

Result: 在图像分类和分割任务上，该方法实现了17-20%的推理速度提升，同时性能下降最小，并通过详细的推导、广泛的实验和效率基准测试验证了有效性。

Insight: 创新点在于识别并利用ViT注意力头中的类卷积行为，设计即插即用的深度可分离卷积替换层，并结合头选择策略和微调来保持性能，为加速视觉Transformer提供了一种轻量级、可部署的解决方案。

Abstract: Pretrained vision foundation models deliver strong performance across tasks with limited fine-tuning. However, their Vision Transformer (ViT) backbones impose high inference costs, limiting deployment on resource-constrained devices. In this work, we accelerate large-scale pretrained ViTs while preserving their feature extraction capabilities by exploiting the intrinsic convolution-like behavior of some attention heads. Specifically, we introduce an efficient depthwise convolution-based layer that serves as a drop-in replacement for these heads. Additionally, we propose simple strategies to identify which heads can be replaced and introduce a fine-tuning procedure that recovers downstream task performance. Across both image classification and segmentation tasks, our method achieves 17-20% percent inference speedup with minimal performance degradation. We validate the approach through detailed derivations, extensive experiments, and efficiency benchmarks. The reference implementation is publicly available.

[64] EventGait: Towards Robust Gait Recognition with Event Streams cs.CVPDF

Senyan Xu, Shuai Chen, Chuanfu Shen, Kean Liu, Zhijing Sun

TL;DR: 本文提出EventGait，一种基于事件相机的步态识别方法，通过双流框架分别建模运动和形状信息，以解决传统相机在光照变化和运动模糊下的性能下降问题。该方法利用混合脉冲专家（MoSE）处理动态流，并通过跨模态结构对齐（CroSA）学习静态形状表示。

Details

Motivation: 传统步态识别在非受控环境中易受光照和运动影响，而事件相机具有高时间分辨率和高动态范围，能自然捕获鲁棒的动态线索并抑制静态噪声，但现有方法常将事件流聚合成图像，丢失了细粒度运动动态。

Result: 在合成的SUSTech1K-E和CCGR-Mini-E基准测试以及真实世界事件步态数据集上，EventGait达到了最先进水平（SOTA），在正常光照下与基于相机的步态识别结果相当，在低光场景下显著优于后者。

Insight: 创新点包括：1）端到端双流框架分别处理运动和形状，保留事件优势；2）动态流采用MoSE增强复杂场景下的鲁棒动态感知；3）静态流通过CroSA与大型视觉基础模型对齐学习密集形状表示；4）引入了合成管道和新的事件步态数据集以解决数据稀缺问题。

Abstract: Gait recognition enables non-intrusive, privacy-preserving identification but suffers in uncontrolled environments due to illumination and motion sensitivity of conventional cameras. In this work, we explore gait recognition using event cameras, which offer microsecond temporal resolution and high dynamic range, naturally capturing robust dynamic cues and suppressing static noise. Existing event-based approaches typically aggregate event streams into event images over long time windows, thereby discarding fine-grained motion dynamics critical for gait recognition. Therefore, we propose \textbf{EventGait}, an end-to-end dual-stream framework that separately models motion and shape while preserving the advantages of events. Our dynamic stream leverages a Mixture of Spiking Experts (MoSE) with diverse neuron constants for robust dynamic perception across complex motion and illumination scenes, while the static stream learns dense shape representations via Cross-modal Structure Alignment (CroSA) with large vision foundation models. To address the absence of large-scale event-based gait datasets, we introduce a synthesis pipeline and release two new benchmarks: SUSTech1K-E and CCGR-Mini-E. Extensive experiments have shown that event-based gait recognition not only achieves results comparable to camera-based gait recognition under normal conditions but also significantly outperforms it in low-light scenarios. Our approach sets a new state of the art on both synthesized and real-world event-based gait benchmarks, highlighting the robustness and potential of event-driven gait analysis. The code and datasets are released at https://github.com/QUEAHREN/EventGait.

[65] One Sentence, One Drama: Personalized Short-Form Drama Generation via Multi-Agent Systems cs.CVPDF

Yufei Shi, Weilong Yan, Naixuan Huang, Yucheng Chen, Chenyu Zhang

TL;DR: 本文提出了一种名为’One Sentence, One Drama’的分层多智能体框架，用于将用户的单句想法自动转化为完整的短剧。该方法通过基于多智能体辩论的故事生成、基于3D的初始帧生成机制以及多阶段审查循环，解决了现有方法在叙事节奏、空间一致性和制作级质量控制方面的不足。

Details

Motivation: 解决现有数字短剧生成方法（通常依赖一次性LLM生成脚本和松散耦合的流水线）在叙事节奏（如钩子薄弱、冲突升级不足）、跨片段空间一致性（如场景布局漂移、角色位置不一致）以及制作级质量控制（需要大量人工审核）方面的关键缺陷。

Result: 实验结果表明，该方法在叙事质量、跨片段一致性和整体观看体验上显著优于现有流水线。评估基于作者提出的Short-Drama-Bench基准，该基准扩展了标准视频质量指标，加入了短剧特定标准。

Insight: 主要创新点包括：1）采用分层多智能体框架进行结构化中间模块生成和迭代优化；2）引入基于辩论的故事生成模块来强化叙事节奏；3）提出3D锚定的首帧生成机制以确保跨片段的空间一致性；4）设计多阶段审查循环进行全流程错误检测与针对性修订；5）构建了专门的短剧评估基准。

Abstract: Existing approaches for digital short-drama production typically rely on one-shot LLM generated scripts and loosely coupled pipelines, which fail to satisfy three key requirements of short-drama generation: (1) narrative pacing, resulting in weak hooks, insufficient escalation, and unattractive endings; (2) spatial consistency, leading to drifting scene layouts and inconsistent character positions across clips; and (3) production-level quality control, requiring extensive manual review and correction across script and visual stages. We present One Sentence, One Drama, a hierarchical multi-agent framework that transforms a user’s single-sentence idea into a fully produced short drama through structured intermediate modules and iterative refinement. Our approach is built upon three key components: (1) a multi-agent debate-based story generation module that enforces short-drama pacing and narrative coherence; (2) a 3D-grounded first-frame generation mechanism that establishes a shared spatial reference for consistent character positioning and scene layout across clips; and (3) multi-stage reviewer loops that perform comprehensive error detection and targeted revision across script, visual, and video generation stages. We also introduce scene-level BGM matching and scene transition planning to improve the audience’s immersive experience. To systematically evaluate this task, we introduce Short-Drama-Bench, a benchmark that extends standard video quality metrics with short-drama-specific criteria. Experimental results demonstrate that our method significantly outperforms existing pipelines in narrative quality, cross-clip consistency, and overall viewing experience.

[66] Balancing Uncertainty and Diversity of Samples: Leveraging Diversity of Least, High Confidence Samples for Effective Active Learning cs.CVPDF

Vipul Arya, S. H. Shabbeer Basha, Srikrishna U N, Sunainha Vijay, Snehasis Mukherjee

TL;DR: 本文针对主动学习（AL）中样本选择策略的局限性，提出了一种结合不确定性和多样性的混合采样方法。该方法通过同时选择高置信度（简单）和低置信度（困难）且多样化的样本进行标注，以更有效地提升模型性能。实验表明，所提出的LCD（Least Confident and Diverse）方法在多个基准测试中优于现有方法。

Details

Motivation: 现有主动学习方法通常仅基于样本的多样性或不确定性单一维度进行选择，这可能限制模型性能的提升。本文旨在解决这一问题，通过同时考虑样本的难度（不确定性）和多样性，以更高效地利用标注预算。

Result: 在广泛的实验中，所提出的LCD混合采样方法在多个基准测试上（具体数据集未在摘要中提及）表现优于现有最先进（SOTA）方法，证明了其有效性。

Insight: 核心创新点在于将样本的难度（通过置信度衡量）与多样性相结合进行选择，而非依赖单一标准。这有助于模型学习更丰富的特征，为设计更高效的主动学习策略提供了新思路。

Abstract: Deep learning models, including Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), have achieved state-of-the-art performance on various computer vision tasks such as object classification, detection, segmentation, generation, and many more. However, these models are data-hungry as they require more training data to learn millions or billions of parameters. Especially for supervised learning tasks, curating a large number of labeled samples for model training is an expensive and time-consuming task. Active Learning (AL) has been used to address this problem for many years. Existing active learning methods aim at choosing the samples for annotation from a pool of unlabeled samples that are either diverse or uncertain. Choosing such samples may hinder the model’s performance as we pool based on one dimension, i.e., either diverse or uncertain. In this paper, we propose four novel hybrid sampling methods for pooling both easy and hard samples, which are also diverse. To verify the efficacy of the proposed methods, extensive experiments are conducted using high and low-confidence samples separately. We observe from our experiments that the proposed hybrid sampling method, Least Confident and Diverse (LCD), consistently performs better compared to state-of-the-art methods. It is observed that selecting uncertain and diverse instances helps the model learn more distinct features. The codes related to this study will be available at https://github.com/XXX/LCD.

[67] Enhancing Multimodal Large Language Models for Safety-Critical Driving Video Analysis cs.CV | cs.LGPDF

Tomaso Trinci, Henrique Piñeiro Monteagudo, Leonardo Taccari

TL;DR: 本文提出了一种增强多模态大语言模型在安全关键驾驶视频分析中性能的管道。该管道通过融合下采样视频帧、同步高频遥测数据以及专用计算机视觉模型的语义洞察来提升模型感知能力，并生成高质量伪标签用于训练。通过使用DoRA适配器微调QwenVL-2.5模型，在有限参数量和计算预算下显著提升了模型对安全关键事件的识别与解释能力。

Details

Motivation: 当前多模态大语言模型在通用视觉理解方面表现出色，但在安全关键驾驶场景中，对罕见高风险动态事件的准确感知与推理能力仍显不足，因此需要针对此类场景进行专门增强。

Result: 实验表明，该方法在微调QwenVL-2.5模型后，在识别和解释安全关键事件方面取得了显著改进，且仅使用少于5000万可训练参数和有限计算预算。

Insight: 创新点在于提出了一种融合多源数据（视频、IMU/GPS遥测、CV模型输出）的管道来生成针对安全关键事件的训练伪标签，并通过参数高效的DoRA适配器进行微调，实现了在特定领域以低成本提升模型性能。

Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in general visual understanding. However, their application to safety-critical driving scenarios remains limited by an inability to accurately perceive and reason about rare high-stakes dynamic events, such as collisions or near-collisions. To address this, we introduce a pipeline that enhances MLLM perception by fusing downsampled video frames with synchronized high-frequency telematics data (IMU and GPS) and semantic insights from specialized computer vision models. Our pipeline generates high-quality pseudo-labels, including descriptive captions and question-answer pairs, specifically designed to train MLLMs to identify and describe Safety-Critical Events (SCEs) in real-world driving footage. We show the effectiveness of our approach fine-tuning the open-source QwenVL-2.5 model via DoRA adapters: our experiments demonstrate significant improvements in identifying and explaining safety-critical events, with fewer than 50M trainable parameters and limited computational budget.

[68] No Pose, No Problem in 4D: Feed-Forward Dynamic Gaussians from Unposed Multi-View Videos cs.CVPDF

Matteo Balice, Yanik Kunzi, Chenyangguang Zhang, Matteo Matteucci, Marc Pollefeys

TL;DR: 本文提出了NoPo4D，首个前馈式系统，能够从无姿态标定的多视角视频中一次性重建动态3D场景。该方法基于预训练的几何主干网络和4D高斯框架，通过速度分解将高斯运动分离为图像平面位移和深度变化，并利用伪真实光流进行监督。系统还包含双向运动编码器和视角相关的不透明度机制，以提升特征聚合并缓解高斯错位问题。

Details

Motivation: 现有前馈式3D高斯泼溅方法在动态内容、多视角输入和未知相机姿态这三个方面存在割裂：处理动态的方法要么需要精确相机姿态，要么仅接受单目输入；无姿态的多视角方法仅适用于静态场景；而逐场景优化方法虽能弥补部分差距，但计算成本高昂。本文旨在填补这一空白，实现单次前馈即可从无姿态多视角视频中重建动态场景。

Result: 在四个多视角动态基准测试中，NoPo4D一致优于先前的前馈基线方法；通过可选的后期优化阶段，其性能甚至超越了逐场景优化方法，同时运行速度提升了数个数量级。

Insight: 创新点在于提出了速度分解策略，将3D高斯运动解耦为2D图像平面位移和深度变化，从而能够直接利用伪真实光流进行监督，避免了先前方法对可微分渲染（依赖姿态精度）或3D运动真值的需求。此外，双向运动编码器和视角相关不透明度机制的设计，有效聚合了跨视角和跨帧特征，并缓解了高斯错位问题，提升了重建的鲁棒性和质量。

Abstract: Recent feed-forward 3D gaussian splatting methods have made dramatic progress on individual aspects of 3D scene reconstruction, but no existing method jointly addresses dynamic content, multi-view input, and unknown camera poses in a single feed-forward pass. Methods that handle dynamics either require accurate camera poses or accept only monocular input; pose-free multi-view methods address only static scenes; and per-scene optimization methods bridge some of these gaps but at minutes-to-hours cost per scene. We introduce NoPo4D, the first feed-forward system that addresses this empty quadrant. Building on a pretrained geometry backbone and recent 4D Gaussian frameworks, NoPo4D introduces a velocity decomposition that splits Gaussian motion into per-pixel image-plane shifts and depth changes, allowing direct supervision from pseudo ground-truth optical flow on the 2D component. This sidesteps both the differentiable rendering that couples prior posed methods to pose accuracy and the 3D motion ground truth that prior pose-free methods require. The system is rounded out by a bidirectional motion encoder for cross-view and cross-frame feature aggregation, and view-dependent opacity that mitigates cross-view and cross-timestep Gaussian misalignments. On four multi-view dynamic benchmarks, NoPo4D consistently outperforms prior feed-forward baselines, and with an optional post-optimization stage surpasses per-scene optimization methods, while running orders of magnitude faster.

[69] OSS: Open Suturing Skills Vision-Based Assessment Challenge 2024-2025 cs.CV | cs.AI | cs.LGPDF

Hanna Hoffmann, Setareh Bady, Claas de Boer, Max Kirchner, Jan Egger

TL;DR: 本文介绍了MICCAI挑战赛OSS（Open Suturing Skills）2024-2025的结果，该挑战赛旨在推动基于视觉的开放手术缝合技能自动评估。挑战赛提供了在干式实验室环境下录制的缝合任务视频数据集，包含视频和器械轨迹数据，并设置了技能等级分类、OSATS评分预测以及手部与工具追踪三项任务。

Details

Motivation: 自动化、数据驱动的技能评估对改善外科培训至关重要，但目前基于机器学习的方法主要应用于微创手术，在开放手术领域的应用仍然有限。本挑战赛旨在为开放手术的视觉技能评估建立基准并推动其发展。

Result: 通用时空视频模型在挑战中表现最为稳健，尤其是在技能分类任务上。然而，预测细粒度的OSATS评分仍然具有挑战性，但增加训练数据能带来显著提升。关键点追踪任务则因频繁遮挡和出镜情况而表现不佳，限制了其当前在基于动作的技能分析中的应用。

Insight: 创新点在于通过公开挑战赛的形式，系统性地收集和评估了多种针对开放手术技能评估的解决方案（如深度学习视频模型、追踪驱动方法和混合方法）。客观分析表明，基于通用视频模型的方案目前最具潜力，但细粒度评分预测和鲁棒追踪仍是未来需要攻克的关键方向，以推动自动化评估走向临床实用。

Abstract: Achieving high levels of surgical skill through effective training is essential for optimal patient outcomes. Automated, data-driven skill assessment holds significant potential to improve surgical training. While machine learning-based methods are increasingly popular for assessing skills in minimally invasive surgery, their application to open surgery remains limited. We present the results of a dedicated MICCAI challenge designed to benchmark and advance vision-based skill assessment in open surgery. The challenge dataset comprises videos of an open suturing training task recorded with a static GoPro camera in a dry-lab setting, with instrument trajectories available in addition to the primary video modality. The OSS Challenge was hosted over two consecutive years, comprising two and three independent tasks, respectively: (1) classifying skill level into four classes, (2) predicting the full Objective Structured Assessment of Technical Skills across eight categories, and (3) tracking hands and surgical tools. Participants submitted diverse solutions including deep learning-based video models, tracking-driven methods, and hybrid approaches. General-purpose spatiotemporal video models consistently achieved the strongest performance, though conceptually diverse approaches reached competitive levels when well-executed. Predicting fine-grained OSATS scores remains challenging but benefits substantially from increased training data. Keypoint tracking proves difficult given frequent occlusions and out-of-frame instances, limiting current applicability for motion-based skill analysis. This work benchmarks innovative and diverse solutions for surgical skill assessment, highlighting both the promise and current limitations of video-based evaluation in open surgery and identifying critical directions for advancing automated skill assessment toward clinical impact.

[70] Zero-Shot Temporal Action Localization Through Textual Guidance cs.CVPDF

Benedetta Liberatori, Alessandro Conti, Lorenzo Vaquero, Paolo Rota, Yiming Wang

TL;DR: 本文提出了一种名为TEGU的新方法，用于零样本时序动作定位（ZS-TAL），该方法通过利用大语言模型生成的丰富文本信息和从视频描述中提取的结构化文本，来补偿训练数据中监督信息的缺乏，从而在未修剪视频中对未见过的动作类别进行分类和定位。

Details

Motivation: 现有基于视觉-语言模型（VLMs）的零样本时序动作定位方法在细粒度动作分类上面临挑战，难以直接区分动作的存在与否，而大多数现有方法依赖于大规模带标注视频数据集的训练，这限制了泛化性能。本文旨在探索无需标注数据的替代方案，通过利用文本信息来提升细粒度动作定位能力。

Result: 在THUMOS14和ActivityNet-v1.3数据集上的实验表明，TEGU方法通过利用丰富文本信息改进动作定位，性能优于不涉及训练的当前最先进（SOTA）零样本时序动作定位方法。

Insight: 创新点在于提出了一种无需训练数据监督的零样本时序动作定位框架，通过整合大语言模型生成的文本上下文和视频描述的结构化文本，为细粒度动作差异提供更丰富的线索，从而提升定位精度。从客观角度看，该方法有效利用了多模态信息（视觉与文本）的互补性，为无监督或弱监督动作定位提供了新思路。

Abstract: Zero-shot temporal action localization (ZS-TAL) consists of classifying and localizing actions in untrimmed videos, where action classes are unseen at training time. Existing work uses Vision and Language Models (VLMs), taking advantage of their strong zero-shot transfer capabilities. Yet, these models face evident challenges with fine-grained action classification, making it difficult to directly use them to distinguish between the presence and absence of an action. Most current methods for ZS-TAL address these challenges by training models on large-scale video datasets, which require annotated data and often result in limited generalization performance. Recently, approaches discarding the use of labeled data have emerged as an alternative. Following this direction, we propose a novel approach, ``Textual Guidance for finer localization of actions in videos’’ (TEGU), that compensates for the lack of supervision from training data by exploiting rich textual information derived from large language models and structured text extracted from captions. This additional linguistic context can improve fine-grained discrimination by providing richer cues about fine-grained action differences within videos. We validate the effectiveness of the proposed method by conducting experiments on the THUMOS14 and the ActivityNet-v1.3 datasets. Our results show that, by exploiting rich textual information for improved action localization, TEGU outperforms state-of-the-art ZS-TAL approaches that do not involve training

[71] MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering cs.CV | cs.AI | cs.MMPDF

Junbin Xiao, Jiajun Chen, Tianxiang Sun, Xun Yang, Angela Yao

TL;DR: 本文提出MuKV方法，通过多粒度KV缓存压缩模块和半层次检索机制，解决长流式视频问答中视觉token增长与LLM推理长度限制的挑战，在提升准确率的同时保持内存与在线效率。

Details

Motivation: 针对现有KV缓存方法在长流式视频QA中因逐帧缓存导致内存冗余、丢失细粒度时空细节的问题，旨在设计更高效的缓存压缩与检索方案。

Result: 在长流式视频QA基准测试中，MuKV显著提高了答案准确率，且未牺牲内存和在线QA效率；其压缩机制在准确率、内存和效率上均优于基线方法。

Insight: 创新点包括多粒度（patch、frame、segment）视觉表示提取与双信号token压缩机制，以及半层次检索方法，平衡了局部线索与全局时序上下文的保留，提升了缓存效率与QA精度。

Abstract: Long streaming video QA remains challenging due to growing visual tokens and limited reasoning length of large language models (LLMs). KV-caching stores the Key-Value (KV) of the historical tokens via LLM prefill and enables more efficient streaming QA. However, existing methods cache every one or two frames, causing redundant memory usage and losing fine-grained spatial details within frame or temporal contexts across frames. This paper proposes MuKV, a method that features a multi-grained KV cache compression module and a semi-hierarchical retrieval approach to improve both efficiency and accuracy for long streaming VideoQA. For the offline KV cache, MuKV extracts visual representations at patch-, frame-, and segment-levels. The multiple levels of granularity preserve both local cues and global temporal context, while maintaining efficiency with a dual signal token compression mechanism guided by self-attention and frequency. For online QA, MuKV designs a semi-hierarchical retrieval method to retrieve relevant KV caches for answer generation. Experiments on long-streaming VideoQA benchmarks show that MuKV significantly improves answer accuracy, without sacrificing memory and online QA efficiency. Moreover, our compression mechanism alone brings consistent benefits across answer accuracy, memory, and QA efficiency over baselines, showcasing highly effective contribution.

[72] EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning cs.CVPDF

Kailin Zhuang, Jiawei Wu, Zhi Jin

TL;DR: 本文提出EvoIR-Agent，一个通过经验驱动学习实现自我进化的图像修复智能体系统。该系统通过构建层次化经验池和引入自我进化机制，解决了现有基于训练的方法兼容性差、以及免训练方法因经验不足而试错开销大的困境，在性能和效率之间取得了帕累托最优平衡。

Details

Motivation: 现有MLLM驱动的图像修复智能体在零样本规划时因缺乏经验而经常失败，导致大量试错开销。基于训练的方法推理效率高但缺乏对新工具或退化的兼容性，而免训练方法虽有兼容性但经验朴素，仍存在试错开销。本文旨在解决这一困境。

Result: 大量实验表明，EvoIR-Agent在全参考指标上取得了显著领先，并且在性能和效率之间实现了显著的帕累托最优平衡，超越了最先进的方法。

Insight: 创新点在于系统性地形式化了免训练图像修复智能体的经验组件，并构建了层次化经验池以实现从粗到细的指导，同时引入了从零开始利用累积记录更新经验池的自我进化机制，从而显著提升了性能和效率。

Abstract: Multimodal Large Language Model (MLLM)-driven image restoration agent demonstrates effectiveness in degradation coupling scenarios by flexibly selecting tools and determining removal orders. However, their zero-shot planning often fails without experience, necessitating severe trial-and-error overhead to achieve satisfactory outcomes. Currently, two paradigms are employed to address this issue, yet a dilemma persists: Training-based methods embed intrinsic experience into parameters, achieving high inference efficiency but lacking compatibility with new tools or degradation. In contrast, training-free methods utilize explicit experience storage for compatibility but still incur trial-and-error overhead due to naive experience. To resolve the dilemma, we propose EvoIR-Agent, which first systematically formulates the experience components of a training-free image restoration agent. Subsequently, a hierarchical experience pool is constructed, which enables coarse-to-fine guidance for diverse tools and removal orders. Furthermore, a self-evolving mechanism is introduced to update the pool from scratch using accumulated records, thereby greatly improving performance and efficiency. Extensive experiments reveal that EvoIR-Agent achieves a significant lead in the full reference metrics and yields a remarkable Pareto-optimal balance between performance and efficiency compared to the state-of-the-art methods.

[73] GALAR-TemporalNet v2: Anatomy-Guided Dual-Branch Temporal Classification with Bidirectional Mamba and Dual-Graph GCN for Video Capsule Endoscopy – after competition results cs.CVPDF

Jiye Won, Seangmin Lee, Soon Ki Jung

TL;DR: 本文提出了GALAR-TemporalNet v2，一个用于视频胶囊内窥镜（VCE）多标签时序分类的层次化时序模型。该模型旨在解决类别极度不平衡、长程时序依赖以及病理-解剖结构纠缠三大核心挑战。其架构结合了局部建模的窗口自注意力、用于全局帧关系的双图GCN以及用于选择性边界上下文编码的双向Mamba。

Details

Motivation: 视频胶囊内窥镜分析面临三大挑战：类别极度不平衡、帧间长程时序依赖以及病理表现与正常解剖外观的紧密纠缠，这给同时定位8个解剖区域和检测9种病理发现带来了困难。

Result: 在RARE-VISION测试集上，竞赛版本的GALAR-TemporalNet取得了mAP@0.5为0.2644和mAP@0.95为0.2353的成绩。改进后的v2版本通过重构病理分支、优化损失函数和扩展后处理，将性能提升至mAP@0.5为0.3409和mAP@0.95为0.3333。

Insight: 创新点包括：1）新颖的解剖原型残差通路，用于解耦病理偏差信号与正常器官外观；2）帧级GCN跳跃连接，用于稳定视觉上易混淆的罕见类别的训练；3）将窗口自注意力、双图GCN和双向Mamba结合，分别处理局部、全局和边界上下文信息。

Abstract: Video Capsule Endoscopy (VCE) poses a challenging multi-label temporal classification problem, requiring simultaneous localization of 8 anatomical regions and detection of 9 pathological findings across tens of thousands of frames. We present GALAR-TemporalNet v2, a hierarchical temporal model that addresses three core challenges: extreme class imbalance, long-range temporal dependencies, and pathology–anatomy entanglement. Our architecture combines windowed self-attention for local modeling, a Dual-Graph GCN for global frame relationships, and Bidirectional Mamba for selective boundary context encoding. A novel anatomy prototype residual pathway decouples pathological deviation signals from normal organ appearance, and a frame-level GCN skip connection stabilizes training of visually confusable rare classes. The competition version, GALAR-TemporalNet, achieved an overall mAP@0.5 of 0.2644 and mAP@0.95 of 0.2353 on the RARE-VISION test set. Following the competition, the redesigned GALAR-TemporalNet v2 – incorporating a restructured pathology branch, refined loss functions, and extended post-processing – improved these results to mAP@0.5 of 0.3409 and mAP@0.95 of 0.3333.

[74] Exposing Vulnerabilities in Visible-Infrared VLMs: A Unified Geometric Adversarial Framework with Cross-Task Transferability cs.CVPDF

Xiang Chen, Yuxian Dong, Chao Li, Chengyin Hu, Jiaju Han

TL;DR: 本文提出了一种名为CFGPatch的对抗性补丁框架，专门用于攻击可见光-红外（VIS-IR）视觉语言模型（VLMs）。该框架结合了基于贝塞尔曲线的分形几何结构和特定模态的弗雷泽螺旋渲染机制，通过全局形状干扰和局部纹理扭曲来破坏模型的跨模态感知。实验表明，CFGPatch在攻击效果和鲁棒性上优于基线方法，并且其生成的对抗样本在零样本分类、图像描述和视觉问答等下游任务间展现出良好的跨任务可迁移性。

Details

Motivation: 可见光-红外感知系统在现实世界感知中广泛应用，但当前视觉语言模型在此跨模态场景下的对抗鲁棒性研究不足，存在安全风险。本文旨在探索并暴露VIS-IR VLMs在此类威胁下的脆弱性。

Result: 在VIS-IR VLM攻击任务上的大量实验表明，CFGPatch在攻击有效性和鲁棒性上持续优于标准的对抗补丁基线方法。此外，为分类任务优化的对抗样本能很好地迁移到图像描述和视觉问答任务，证明了其强大的跨任务可迁移性和泛化能力。

Insight: 核心创新点在于将贝塞尔曲线引入分形几何以生成更灵活、平滑的对抗形状，并结合了针对可见光和红外模态的弗雷泽螺旋纹理渲染机制，实现了全局几何与局部外观的协同干扰。从客观角度看，其提出的跨任务攻击可迁移性分析对评估多模态模型的对抗鲁棒性具有重要参考价值。

Abstract: Vision-language models (VLMs) have achieved strong performance across diverse multimodal tasks, but their adversarial robustness in visible-infrared (VIS-IR) scenarios remains underexplored. This gap is critical because VIS-IR sensing is widely used in real-world perception systems to support reliable understanding under challenging imaging conditions. To address this cross-modal threat setting, we propose CFGPatch, a curved-edge fractal geometric adversarial patch framework for attacking VIS-IR VLMs. CFGPatch builds on triangular fractal geometry and replaces rigid straight-edged primitives with Bezier-curved elements, preserving multi-scale fractal self-similarity while introducing smoother contours, richer directional variation, and more flexible shape deformation. In addition, we design a modality-specific Fraser-spiral rendering mechanism to inject fine-grained texture distortions and misleading perceptual cues into visible and infrared images. By coupling global curved-fractal geometry with local spiral-based appearance interference, CFGPatch disrupts both shape perception and texture interpretation. We further adopt expectation over transformation (EOT) to improve robustness against common image-level transformations. Extensive experiments show that CFGPatch effectively fools VIS-IR VLMs and consistently outperforms standard patch baselines in attack effectiveness and robustness. Moreover, adversarial samples optimized for zero-shot classification transfer well to image captioning and visual question answering, demonstrating strong cross-task transferability and generalizability across downstream tasks.

[75] AgroTools: A Benchmark for Tool-Augmented Multimodal Agents in Agriculture cs.CVPDF

Zi Ye, Yibin Wen, Xiaoya Fan, Xinyu Zhang, Jing Wu

TL;DR: 本文提出了AgroTools基准，用于评估农业领域中工具增强的多模态智能体。该基准包含539个问答实例和1097张农业图像，涵盖五个任务族和14个可执行工具，支持对过程级执行质量和结果级任务成功的双视图评估。

Details

Motivation: 现有农业多模态基准主要评估最终答案的正确性，缺乏对模型使用外部工具完成精度敏感工作流程能力的评估，因此需要一个新的基准来填补这一空白。

Result: 在AgroTools上对9个开源和4个闭源多模态大语言模型进行基准测试，结果显示当前模型在农业工具使用场景中可靠性仍远不足，在工具规划、参数生成、执行恢复和最终答案合成方面存在明显瓶颈。

Insight: 创新点在于引入了结构化工具使用轨迹的标注，实现了过程与结果的双重评估；客观分析表明，该基准为高精度农业应用中的多模态智能体研究提供了系统性的评估框架。

Abstract: Agricultural decision-making increasingly requires multimodal systems that can transform visual observations into reliable, executable actions. However, existing agricultural multimodal benchmarks mainly evaluate final-answer correctness and provide limited support for assessing whether models can use external tools to complete precision-sensitive workflows. In this paper, we introduce AgroTools, a benchmark for evaluating tool-augmented multimodal agents in agriculture. AgroTools contains 539 question-answer instances paired with 1,097 heterogeneous agricultural images, spanning five task families and an executable environment of 14 agricultural tools. Each query is annotated with structured tool-use traces, enabling a dual-view evaluation of both process-level execution quality and outcome-level task success. We benchmark 9 open-source and 4 closed-source multimodal large language models on AgroTools. Results show that current models remain far from reliable in agricultural tool-use settings, with clear bottlenecks in tool planning, argument generation, execution recovery, and final-answer synthesis. We hope AgroTools will support future research on multimodal agents for high-precision agricultural applications. The benchmark and evaluation are available at https://huggingface.co/datasets/AgroTools/AgroTools.

[76] Bernini: Latent Semantic Planning for Video Diffusion cs.CV | cs.AI | cs.MMPDF

Bernini Team, Chenchen Liu, Junyi Chen, Lei Li, Lu Chi

TL;DR: 论文提出Bernini框架，将多模态大语言模型（MLLMs）和扩散模型统一起来，用于视频生成和编辑。MLLM负责语义规划，在ViT嵌入空间预测目标语义表示；基于DiT的渲染器则根据该规划、文本特征以及编辑时的源VAE特征合成像素。通过引入SA-3D RoPE和思维链推理，框架在多个视频生成和编辑基准上实现了SOTA性能。

Details

Motivation: MLLMs擅长多模态语义推理，扩散模型擅长逼真图像/视频合成，但两者尚未有效统一。论文旨在通过分工协作，结合两者的优势，构建一个统一的视频生成和编辑框架。

Result: Bernini在广泛的视频生成和编辑基准测试中取得了最先进的（SOTA）性能，特别是在具有挑战性的编辑任务上展现出强大的泛化能力。

Insight: 核心创新在于将MLLM作为语义规划器与扩散模型作为渲染器解耦，通过语义表示作为接口，实现组件独立训练与轻量协同训练，保留了预训练模型的优势。此外，SA-3D RoPE和规划器中的思维链推理增强了多视觉输入处理和语义理解到生成的转化。

Abstract: Multimodal large language models (MLLMs) and diffusion models have each reached remarkable maturity: MLLMs excel at reasoning over heterogeneous multimodal inputs with strong semantic grounding, while diffusion models synthesize images and videos with photorealistic fidelity. We argue that these two families can be unified through a simple division of labor: MLLMs perform semantic planning, while diffusion models render pixels from high-level semantic guidance and low-level visual features. Building on this idea, we propose Bernini, a unified framework for video generation and editing. An MLLM-based planner predicts the target semantic representation directly in the ViT embedding space, and a DiT-based renderer synthesizes pixels conditioned on this plan, augmented by text features and, for editing, source VAE features for detail preservation. Because semantics serve as the interface, the planner and renderer can be trained separately and only lightly co-trained, preserving the pretrained strengths of both components while keeping training efficient. To better handle multiple visual inputs, we introduce Segment-Aware 3D Rotary Positional Embedding (SA-3D RoPE), and further incorporate chain-of-thought reasoning in the planner to better transfer understanding into generation. Bernini achieves state-of-the-art performance across a wide range of video generation and editing benchmarks, with the MLLM’s pretrained understanding translating into strong generalization on challenging editing tasks.

[77] Translating Signals to Languages for sEMG-Based Activity Recognition cs.CVPDF

Ming Wang, Haoxuan Qu, Qiuhong Ke, Wei Zhou, Hossein Rahmani

TL;DR: 本文提出了一种名为LLM-sEMG的新颖框架，该框架利用大语言模型（LLMs）作为表面肌电（sEMG）信号的活动识别器。其核心是设计了一种面向语言的映射机制，将连续的sEMG序列转换为sEMG语言，从而利用LLMs的泛化和推理能力来识别活动意图。实验表明，该框架能够实现高精度的sEMG活动识别。

Details

Motivation: 现有sEMG活动识别方法主要集中于设计更复杂的模型架构或进行大规模预训练以提升性能。受大语言模型（LLMs）在自然语言处理中展现出的卓越泛化与推理能力启发，特别是其从大量动作语言描述中学习到的隐含知识，本文旨在探索利用LLMs来解读sEMG信号并推断活动意图的新可能性。

Result: 广泛的实验证明，所提出的LLM-sEMG框架能够实现高精度的sEMG信号活动识别。

Insight: 主要创新点在于提出了一种将连续sEMG信号序列映射为“sEMG语言”的机制，从而将活动识别任务转化为LLMs能够处理的“语言”问题，巧妙地利用了LLMs的隐含知识。这为跨模态信号理解（从生理信号到语言）提供了一种新颖的思路。

Abstract: Surface electromyography (sEMG) signal-based activity recognition has attracted increasing research attention in recent years. To develop accurate sEMG signal-based activity recognizers, numerous approaches have been proposed. Some studies focus on designing larger and more expressive model architectures to enhance the representational capacity of sEMG signals, while others aim to enrich model priors through large-scale pretraining, thereby improving recognition performance. Recently, large language models (LLMs) have shown remarkable generalization and reasoning capabilities in natural language processing, whose implicit knowledge, learned from extensive linguistic descriptions of actions, opens new possibilities for interpreting sEMG signals and inferring activity intentions. Motivated by this, we propose LLM-sEMG, a novel framework that leverages LLMs as sEMG activity recognizers. Within this framework, we design a language-oriented mapping mechanism that converts continuous sEMG sequences into sEMG language, integrating several strategies to further facilitate the signal-to-language mapping process. Extensive experiments demonstrate that the proposed framework achieves highly accurate sEMG signal-based activity recognition using large language models.

[78] From Recognition to Reasoning: Benchmarking and Enhancing MLLMs on Real-World Receipt Document Understanding cs.CVPDF

Yandi Wang, Libin Zhan, Ziwei Huang, Tiancheng Luo, Yuxuan Jiang

TL;DR: 该论文针对视觉文档信息提取（VIE）任务，指出当前多模态大语言模型（MLLMs）的基准测试在规模、真实性和语义粒度上存在不足。为此，作者提出了一个名为ReceiptBench的大规模、人工标注的收据理解基准，包含1万张多样化收据，并将任务划分为四个层次化的子任务：基础感知、格式规范化、语义推理和结构解析。同时，作者提出了一种包含度量感知分组相对策略优化（GRPO）的两阶段训练框架，以提升模型的结构一致性。实验表明该方法在复杂推理任务上达到了最先进的性能。

Details

Motivation: 现有视觉文档信息提取（VIE）的基准测试在规模、真实性和任务覆盖度上存在局限，无法充分评估多模态大语言模型（MLLMs）在真实世界复杂文档（如收据）上的理解与推理能力。

Result: 在提出的ReceiptBench基准上进行广泛实验，结果表明所提出的方法（包含GRPO的两阶段训练框架）取得了最先进的性能，在复杂的语义推理任务上超越了领先的专有模型。

Insight: 主要创新点在于构建了一个大规模、层次化、贴近真实场景的收据理解基准（ReceiptBench），将信息提取任务分解为从感知到推理的四个渐进子任务；并提出了一种将严格评估约束转化为强化学习信号（GRPO）的训练框架，以增强模型输出的结构一致性，这对于处理具有复杂嵌套结构的文档信息提取任务具有借鉴意义。

Abstract: Extracting structured information from visual documents (Visual Information Extraction, VIE) is a cornerstone of business automation. While recent Multimodal Large Language Models (MLLMs) have shown promising capabilities, existing benchmarks suffer from critical limitations in scale and realism, lack semantic granularity, and fail to cover diverse document types. To bridge this gap, we introduce ReceiptBench, a large-scale, human-annotated benchmark consisting of 10k diverse receipts, organizing information extraction into four hierarchical sub-tasks: (1) Basic Perception for raw text spotting, (2) Format Normalization for strictly following standardization instructions, (3) Semantic Reasoning for inferring implicit attributes from context, and (4) Structure Parsing for handling nested line items. Furthermore, we propose a two-stage training framework incorporating Metric-Aware Group Relative Policy Optimization (GRPO), which translates rigorous evaluation constraints into reinforcement learning signals to enhance structural consistency. Extensive experiments demonstrate that our method yields state-of-the-art performance, surpassing leading proprietary models on complex reasoning tasks. We release our datasets and code at https://github.com/wwwT0ri/ReceiptBench.

[79] Towards Clinically Interpretable Ophthalmic VQA via Spatially-Grounded Lesion Evidence cs.CV | cs.AIPDF

Xingyue Wang, Bo Liu, Meng Wang, Zhixuan Zhang, Chengcheng Zhu

TL;DR: 该论文提出了FundusGround，一个用于临床可解释眼科视觉问答（VQA）的新基准，强调空间定位的病灶证据。它通过一个三阶段流程构建了一个包含10,719张眼底图像和15,595个图像级病灶标注的数据集，所有病灶均使用ETDRS网格进行空间定位，并在此基础上生成了72,706个问题。实验表明，融入病灶级视觉证据能持续提升模型性能和透明度。

Details

Motivation: 当前的眼科VQA基准主要关注答案准确性，而忽视了临床可解释性所必需的明确视觉证据。为了解决这个问题，作者旨在创建一个强调空间定位病灶证据的基准，以提高模型的可靠性和可解释性。

Result: 论文在提出的FundusGround基准上对多个通用和医学大型视觉语言模型进行了基准测试，使用了答案准确性和病灶级推理的双重指标。实验结果表明，融入病灶级视觉证据能持续提升模型性能和透明度。

Insight: 主要创新点在于构建了一个结构化的、基于ETDRS网格进行空间定位的病灶证据数据集，并以此为基础生成多样化问题，为临床可解释的眼科VQA提供了新的评估框架。从客观角度看，将解剖学一致性（ETDRS网格）与VQA任务结合，是提升医学AI模型透明度和可信度的一个有前景的方向。

Abstract: Visual Question Answering (VQA) holds great promise for clinical support, particularly in ophthalmology, where retinal fundus photography is essential for diagnosis. However, ophthalmic VQA benchmarks primarily emphasize answer accuracy, neglecting the explicit visual evidence necessary for clinical interpretability. In this work, we introduce FundusGround, a new benchmark for clinically interpretable ophthalmic VQA with spatially-grounded lesion evidence. Specifically, we propose a three-stage pipeline that collects 10,719 fundus images with 15,595 image-level meticulously annotated lesions. To ensure anatomical consistency and clinical validity, all lesions are spatially localized using the Early Treatment Diabetic Retinopathy Study (ETDRS) grid, enabling standardized mapping to nine clinically meaningful retinal regions. Built upon this structured lesion evidence, 72,706 questions are then generated spanning four formats: open-ended, closed-ended, single-choice, and multiple-choice. We further benchmark multiple general- and medical- large vision-language models using dual metrics for answer accuracy and lesion-level reasoning. The experiments demonstrate that incorporating lesion-level visual evidence consistently improves model performance and transparency, highlighting the necessity of explicit spatial grounding for reliable and explainable ophthalmic VQA.

[80] SpaceDG: Benchmarking Spatial Intelligence under Visual Degradation cs.CV | cs.CLPDF

Xiaolong Zhou, Yifei Liu, Ziyang Gong, Jiarui Li, Qiyue Zhao

TL;DR: 论文提出了SpaceDG数据集和SpaceDG-Bench基准，用于评估多模态大语言模型在视觉退化条件下的空间智能鲁棒性。该数据集通过基于物理的退化合成引擎，将九种退化类型嵌入3D高斯溅射渲染中生成，包含约100万个QA对。评估25个开源和闭源MLLM发现，视觉退化会显著损害空间推理，暴露了鲁棒性差距。微调实验表明，使用SpaceDG训练能显著提升退化鲁棒性，甚至在退化条件下超越人类表现，且不影响干净图像上的性能。

Details

Motivation: 现有空间推理基准大多假设视觉输入是完美的，忽略了现实部署中常见的退化（如运动模糊、低光照、恶劣天气等），因此需要评估当前MLLM在视觉观测不完美时的空间智能鲁棒性。

Result: 在SpaceDG-Bench基准（包含1,102个问题，涵盖11个推理类别和9种视觉退化类型，超过10K个VQA实例）上评估25个MLLM，发现视觉退化会持续且显著地损害空间推理。微调实验显示，使用SpaceDG训练能显著提升退化鲁棒性，甚至在退化条件下超越人类表现，且不影响干净图像上的性能。

Insight: 创新点包括：首次提出大规模退化感知空间理解数据集SpaceDG，以及基于物理的退化合成引擎，将退化形成过程嵌入3DGS渲染以实现真实模拟；客观分析认为，该工作强调了现实世界视觉退化对MLLM评估的重要性，并展示了退化感知训练在提升鲁棒性方面的潜力，为鲁棒空间智能研究提供了新基准和方法。

Abstract: Multimodal Large Language Models (MLLMs) have made rapid progress in spatial intelligence, yet existing spatial reasoning benchmarks largely assume pristine visual inputs and overlook the degradations that commonly occur in real-world deployment, such as motion blur, low light, adverse weather, lens distortion, and compression artifacts. This raises a fundamental question: how robust is the spatial intelligence of current MLLMs when visual observations are imperfect? To answer this question, we introduce SpaceDG, the first large-scale dataset for degradation-aware spatial understanding. It is constructed with a physically grounded degradation synthesis engine that embeds degradation formation process into 3D Gaussian Splatting (3DGS) rendering, enabling realistic simulation of nine degradation types. The resulting dataset contains approximately 1M QA pairs from nearly 1,000 indoor scenes. We further introduce SpaceDG-Bench, an human-verified benchmark with 1,102 questions spanning 11 reasoning categories and 9 visual degradation types, yielding over 10K VQA instances. Evaluating 25 open- and closed-source MLLMs reveals that visual degradations consistently and substantially impair spatial reasoning, exposing a critical robustness gap. Finally, we show that finetuning on SpaceDG markedly improves degradation robustness and can even surpass human performance under degraded conditions without any performance drop on clean images, highlighting the promise of degradation-aware training for robust spatial intelligence.

[81] Pre-VLA: Preemptive Runtime Verification for Reliable Vision-Language-Action and World-Model Rollouts cs.CV | cs.AI | cs.ROPDF

Zhen Sun, Yongjian Guo, Haoran Sun, Luqiao Wang, Wei Lu

TL;DR: 本文提出Pre-VLA，一种用于视觉-语言-动作（VLA）模型和世界模型（WM）的运行时验证架构，旨在通过预执行的动作有效性评估，提升长视野具身智能系统的可靠性和效率。它采用高效的多模态骨干网络和轻量级双分支头来预测动作块的安全置信度和优势分数，并结合多任务目标进行训练，以处理类别不平衡和决策边界不稳定问题。在部署时，通过双模式预执行重采样调度器过滤低质量动作，并在有限计算预算下触发自适应重采样。

Details

Motivation: 解决大型VLA模型和生成式世界模型在实际部署中，由于基于学习的动作生成存在不确定性，可能导致物理执行失败或世界模型推演产生误导性结果并带来冗余渲染成本的问题。

Result: 在LIBERO基准测试中，Pre-VLA将四个任务套件的平均闭环成功率从RynnVLA-002的30.79%提升至37.62%，减少了任务执行步数，实现了每个动作块平均183.9毫秒的前向验证时间，并缓解了世界模型推演中的错误累积。

Insight: 创新点在于提出了一个统一的预执行运行时验证架构，通过模态感知池化和轻量级双分支头进行高效的多模态动作评估，并设计了结合Focal分类、优势回归和软阈值校准的多任务训练目标来处理类别不平衡和决策边界问题。此外，双模式预执行重采样调度器实现了在有限计算资源下的自适应动作优化。

Abstract: While large vision-language-action (VLA) models and generative world models (WM) have advanced long-horizon embodied intelligence, their practical deployment remains challenged by uncertainty in learning-based action generation. Low-quality actions may cause physical failures during execution or lead to misleading world-model rollouts with redundant rendering costs. To address this issue, we propose Pre-VLA, a unified runtime verification architecture that performs preemptive action validity assessment before physical execution or world-model imagination. Pre-VLA leverages an efficient multimodal backbone with modality-aware pooling and a lightweight dual-branch head to predict both safety confidence and critic-derived advantage scores for candidate action chunks. To handle severe class imbalance and unstable boundary decisions, we train Pre-VLA with a multi-task objective combining Focal classification, advantage regression, and soft-threshold calibration. During deployment, a dual-mode preemptive resampling scheduler filters low-quality actions and triggers adaptive resampling under a limited computation budget. Experiments on the LIBERO benchmark show that Pre-VLA improves the average closed-loop success rate across four suites from 30.79% to 37.62% over RynnVLA-002, reduces task execution steps, achieves 183.9 ms average forward verification time per action chunk, and mitigates error accumulation in world-model rollouts.

[82] FastTab: A Fast Table Recognizer with a Tiny Recursive Module and 1D Transformers cs.CV | cs.AIPDF

Laziz Hamdi, Amine Tamasna, Pascal Boisson, Thierry Paquet

TL;DR: FastTab是一种快速表格结构识别模型，通过结合轻量级递归模块和一维Transformer编码器，实现非自回归的网格中心化识别。它预测行列数、表头及分隔符以构建网格，并利用ROI对齐特征推断跨行跨列单元格，在多个基准测试中达到竞争性性能且推理延迟低。

Details

Motivation: 解决表格结构识别中需要兼顾表格整体连贯性（如行列数、表头、跨单元格）与精确分隔符定位的问题，避免自回归HTML解码带来的计算开销。

Result: 在PubTabNet、FinTabNet、PubTables-1M和SciTSR四个基准测试上实现了竞争性的结构恢复性能，同时保持低延迟推理；还展示了在像素级匿名化下的鲁棒性，并可扩展至相机拍摄文档的弯曲分隔符处理。

Insight: 创新点包括使用轻量级递归模块进行全局推理，结合轴向一维Transformer编码器捕获行列长程依赖，以网格为中心的非自回归方法提升效率；客观分析其模块化设计兼顾精度与速度，适用于实际部署场景。

Abstract: Table structure recognition (TSR) requires both table-level coherence (row/column counts, headers, spanning cells) and precise separator localization. We introduce FastTab, a grid-centric TSR model that avoids autoregressive HTML decoding by combining (i) a lightweight Tiny Recursive Module (TRM) for global reasoning and (ii) axial 1D Transformer encoders that capture long-range dependencies along rows and columns. The model predicts row/column counts, header rows, and separators to construct a grid, then infers rowspan/colspan using ROI-aligned cell features. Across four benchmarks (PubTabNet, FinTabNet, PubTables-1M, and SciTSR), FastTab achieves competitive structure recovery performance while operating at low-latency inference. We further study robustness under pixel-level anonymisation and show an extension to curved separators for camera-captured documents. The source code will be made publicly available at https://github.com/hamdilaziz/FastTab .

[83] Making the Discrete Continuous: Synthetic RAW Augmentations for Fine-Grained Evaluation of Person Detection Performance in Low Light cs.CV | cs.AI | cs.LG | physics.opticsPDF

Valeria Pais, Malena Mendilaharzu, Daniele Faccio, Luis Oala, Christoph Clausen

TL;DR: 本文提出了一种合成RAW图像增强技术，用于生成与相机传感器噪声模型匹配的低光照样本，以解决自动驾驶中行人检测在黑暗环境下的评估数据稀疏问题。通过合成数据填补真实数据集的空白，论文展示了如何利用合成低光照样本来更全面地评估最先进目标检测模型在不同光照条件下的性能。

Details

Motivation: 真实世界AI视觉模型的部署受限于训练和测试数据的可用性，尤其是在低密度区域（如低光照条件）样本稀少，导致评估困难。论文聚焦于自动驾驶安全关键场景中的黑暗环境行人检测，旨在通过合成数据改善数据覆盖和性能评估。

Result: 在真实和合成低光照数据上的性能指标相似，表明AI模型难以区分两者，验证了合成数据的有效性。这有助于更连续地采样输入空间，并在低光照条件下对目标检测模型进行细粒度评估。

Insight: 创新点在于使用合成RAW图像增强技术生成与真实相机噪声匹配的低光照样本，从而扩展基准测试的数据覆盖。从客观角度看，该方法为数据稀疏场景下的模型评估提供了可借鉴的合成数据生成方案，特别是在安全关键应用中。

Abstract: Real-world deployment of AI vision models is both fueled and limited by the data available for training and testing. Real datasets are sparse and uneven: long-tailed or unbalanced distributions hinder generalization, and the low number of samples in low density regions makes it hard to run evaluations. Synthetic data can fill these gaps, providing us with a way to sample the input space more continuously and improve data coverage for benchmarks. Focusing on the autonomous driving safety-critical case of pedestrian detection in the dark, we show how synthetic low-light samples can be used to better characterize the performance of a state-of-the-art object detection model as a function of the scene illumination. We use a synthetic RAW image augmentation technique to generate low-light samples that match the noise model of the camera sensor. Performance metrics on real and synthetic low-light data are similar, indicating that the AI model finds it hard to distinguish between them.

[84] Moment-Reenacting: Inverse Motion Degradation with Cross-shutter Guidance cs.CVPDF

Ji Xiang, Lin Guixu, Yin Zhengwei, Zhao Jiancheng, Zheng Yinqiang

TL;DR: 本文提出了一种统一框架来解决运动退化问题，通过联合利用全局快门（GS）模糊和卷帘快门（RS）失真的互补特性，来反转运动退化并重现成像瞬间。作者引入了一种新颖的双快门设置来捕获同步的模糊-RS图像对，并进一步扩展到窄基线的立体Blur-RS配置。此外，还构建了一个三轴成像系统来收集带有对齐GS-RS对和真实高速帧的真实世界数据集。

Details

Motivation: 运动退化（表现为全局快门图像中的模糊或卷帘快门图像中的失真）是计算成像中的一个基本挑战，尤其是在快速运动或低光条件下。先前的工作将模糊分解和RS时间超分辨率视为独立任务，未能利用它们内在的互补性。

Result: 广泛的实验验证了该方法的优越性和泛化性，在复杂运动退化下实现了逼真的高速视频重建，为这一任务建立了新范式。

Insight: 主要创新点包括：提出统一框架联合处理GS模糊和RS失真；引入双快门设置和立体Blur-RS配置以解决时空模糊性；构建真实世界数据集；网络通过双流运动解释模块显式解耦运动表示，并结合自提示帧重建阶段。

Abstract: Motion degradation, manifested as blur in global shutter (GS) images or rolling shutter (RS) distortion in RS counterparts, remains a fundamental challenge in computational imaging, especially under fast motion or low-light conditions. While prior works have treated blur decomposition and RS temporal super-resolution as separate tasks, this separation fails to exploit their intrinsic complementarity. In this paper, we propose a unified framework to invert motion degradation and reenact imaging moment by jointly leveraging the complementary characteristics of GS blur and RS distortion. To this end, we introduce a novel dual-shutter setup that captures synchronized blur-RS image pairs and demonstrate that this combination effectively resolves temporal and spatial ambiguities inherent in both modalities. For allowing flexible performance-cost trade-offs, we further extend this dual-shutter setup to a stereo Blur-RS configuration with a narrow baseline. In addition, we construct a triaxial imaging system to collect a real-world dataset with aligned GS-RS pairs and ground-truth high-speed frames, enabling robust training and evaluation beyond synthetic data. Our proposed network explicitly disentangles motion into context-aware and temporally-sensitive representations via a dual-stream motion interpretation module, followed by a self-prompted frame reconstruction stage. Extensive experiments validate the superiority and generalizability of our approach, establishing a new paradigm for realistic high-speed video reconstruction under complex motion degradations. Codes and more resources are available at https://jixiang2016.github.io/dualBR_site/.

[85] Matching with Deliberation: Test-Time Evolutionary Hierarchical Multi-Agents for Zero-Shot Compositional Image Retrieval cs.CVPDF

Xingtian Pei, Yukun Song, Changwei Wang, Shunpeng Chen, Rongtao Xu

TL;DR: 本文提出了一种名为PDF的分层感知-审议框架，用于解决零样本组合图像检索任务中同时保持参考图像视觉连续性和准确执行修改文本语义变量的核心挑战。该框架采用分层多智能体架构，通过意图路由管理器动态调度多视角感知信号构建候选池，并结合无训练推理策略蒸馏与锦标赛式测试时间缩放策略实现自演化的细粒度推理。

Details

Motivation: 现有方法在单一空间中存在感知短视，或在迭代协作中因底层检索器的感知上限而陷入逻辑漂移，无法有效平衡视觉连续性与文本修改语义。

Result: 在CIRR、CIRCO和FashionIQ三个基准数据集上取得了最先进的性能。

Insight: 首次将经验自演化和测试时间缩放定律引入ZS-CIR任务，通过分层多智能体架构实现动态意图路由与自演化推理，为细粒度多媒体检索提供了一条可扩展的新路径。

Abstract: Zero-Shot Compositional Image Retrieval (ZS-CIR) requires both preserving the visual continuity of the reference image and faithfully executing the semantic variables specified in the modification text, which constitutes the core challenge of the task. Existing methods often suffer from Perception Myopia in a single space, or fall into Logic Drift in iterative collaboration due to the perception ceiling of the underlying retriever. To address this issue, we propose a one-stop hierarchical Perception-to-Deliberation Framework (PDF), which, to the best of our knowledge, is the first to introduce experience self-evolution and Test-Time Scaling Law (TTS) into ZS-CIR. Relying on a hierarchical multi-agent architecture, PDF first utilizes an Intent Routing Manager to dynamically dispatch multi-view Worker perception signals based on modification intents to construct a high-recall candidate pool. Subsequently, the Decision Manager combines a Training-free Reasoning Policy Distillation mechanism with a Tournament-style TTS strategy to achieve self-evolving fine-grained reasoning, yielding the final retrieval results. Experimental results demonstrate that PDF achieves SOTA performance on three benchmark datasets: CIRR, CIRCO, and FashionIQ. This study indicates that experience-driven self-evolution and TTS represent a highly promising and scalable path for achieving zero-shot fine-grained multimedia retrieval. The code will be made publicly available upon acceptance.

[86] Supervised Classification Heads as Semantic Prototypes: Unlocking Vision-Language Alignment via Weight Recycling cs.CVPDF

David Méndez, Roberto Confalonieri, Natalia Díaz Rodríguez

TL;DR: 本文提出一种利用预训练视觉模型的分类头权重作为语义原型的方法，以低成本实现视觉语言对齐。该方法通过回收通常被丢弃的分类头权重，将其用作语义锚点实现零样本对齐，并作为数据增强策略与真实图像-文本对混合。实验表明，该方法与多种先进的后处理对齐技术结合，能持续提升跨模态检索、零样本和少样本分类任务的准确率。

Details

Motivation: 视觉语言模型通常需要大量配对数据和昂贵的端到端训练来实现跨模态对齐，现有后处理方法虽降低了计算成本，但仍需大量配对数据。本文旨在探索利用预训练视觉模型中通常被丢弃的分类头权重作为语义原型，以更低的成本实现有效的视觉语言对齐。

Result: 该方法与多种最先进的后处理对齐技术结合，在跨模态检索、零样本和少样本分类任务上均能持续提升准确率，展示了其有效性和通用性。

Insight: 创新点在于将预训练视觉模型的分类头权重重新利用为语义原型，这既可作为零样本对齐的语义锚点，又可作为数据增强策略。从客观角度看，这种权重回收方法巧妙地利用了现有模型中的语义信息，以极低成本实现了跨模态对齐的增强，为后处理对齐提供了新的思路。

Abstract: Vision-Language Models (VLMs) excel at tasks like zero-shot classification and cross-modal retrieval by mapping images and text to a shared space, but this requires expensive end-to-end training with massive paired datasets. Current post-hoc alignment methods reduce computational costs by connecting pretrained encoders through lightweight mappings, yet still demand substantial paired data. In this work, we investigate the potential of repurposing the classification heads of pretrained vision models as semantic prototypes. The recycling of these weights, typically discarded after pretraining, unlocks two distinct capabilities: it enables zero-shot alignment by using weights as semantic anchors, and serves as a robust data augmentation strategy by mixing these prototypes with real image-text pairs. We demonstrate that integrating our approach with several state-of-the-art post-hoc alignment techniques consistently boosts accuracy in cross-modal retrieval, zero- and few-shot classification tasks.

[87] Segment Anything with Motion, Geometry, and Semantic Adaptation for Complex Nonlinear Visual Object Tracking cs.CVPDF

Deyi Zhu, Yuji Wang, Yong Liu, Yansong Tang, Bingyao Yu

TL;DR: 该论文提出了一种名为SAMOSA的新型视觉目标跟踪框架，旨在解决传统方法在泛化性和处理复杂非线性运动场景方面的局限性。该框架通过引入轻量级非线性运动预测器、语义线索和几何约束，对基础视觉模型SAM 2进行适配，以显式建模目标动态并增强跨帧的一致性，从而提升跟踪的鲁棒性和泛化能力。

Details

Motivation: 传统视觉目标跟踪方法依赖任务特定的监督训练，难以泛化到未见过的物体以及存在干扰、遮挡和非线性运动的复杂场景。尽管SAM 2等视觉基础模型通过大规模预训练学习了强大的视频理解先验，但其直接应用于目标跟踪仍存在不足，因为它未能显式建模目标运动动态或强制跨帧的几何与语义一致性。

Result: 大量实验表明，SAMOSA在通用基准测试上持续优于基于SAM 2的最先进方法，展现出比监督式VOT方法更强的泛化能力，并在以复杂非线性运动为特点的反无人机数据集上取得了显著性能提升。

Insight: 论文的创新点在于将基础视觉模型SAM 2的隐式视频理解先验与面向跟踪的显式建模（包括非线性运动预测、语义线索检测和几何约束）相结合，从而弥合了通用视频理解与特定跟踪任务需求之间的差距，为构建更鲁棒、可泛化的跟踪器提供了新思路。

Abstract: Traditional visual object tracking (VOT) methods typically rely on task-specific supervised training, limiting their generalization to unseen objects and challenging scenarios with distractors, occlusion, and nonlinear motion. Recent vision foundation models, exemplified by SAM 2, learn strong video understanding priors from large-scale pretraining and offer a promising foundation for building more robust and generalizable trackers. However, directly applying SAM 2 to VOT remains suboptimal, as it does not explicitly model target motion dynamics or enforce geometric and semantic consistency across frames, both of which are essential for reliable tracking. To address this issue, we propose SAMOSA, a new tracking framework that adapts SAM 2 to complex VOT scenarios by explicitly leveraging motion, geometry, and semantic cues. Specifically, we introduce a lightweight nonlinear motion predictor to model target dynamics and guide mask selection as well as memory filtering. We further exploit semantic cues to detect target shifts and recover from tracking failures, while geometric cues are incorporated as structural constraints to improve tracking stability. In this way, SAMOSA bridges the gap between the implicit video understanding prior of SAM 2 and explicit tracking-oriented modeling. Extensive experiments show that SAMOSA consistently outperforms state-of-the-art SAM 2–based approaches on general benchmarks, demonstrates stronger generalization than supervised VOT methods, and achieves substantial gains on anti-UAV datasets, which typify complex nonlinear motion scenarios. Our code is available at https://github.com/DurYi/SAMOSA.

Yiming Xu, Yixuan Liu, Yuhang Zhang, Ling Zheng, Yihan Wang

TL;DR: 本文提出了一种基于多模态知识图谱和可靠性引导优化的病例感知医学图像分类框架，旨在模拟临床诊断中参考相似病例和症状的推理过程。该方法通过自适应检索相似病例构建多模态知识图谱，利用图注意力网络传播知识语义，并通过双向跨模态注意力机制将病例特征注入视觉表示，同时引入置信度校准的决策优化方案来减少噪声检索的影响。

Details

Motivation: 现有深度学习方法主要依赖孤立的视觉证据，无法有效利用相似病例或外部知识，而临床诊断通常需要参考历史相似病例及其相关症状，因此需要模拟这一诊断过程以实现可解释的医学图像诊断。

Result: 在多个医学影像数据集上的广泛实验表明，该方法持续优于强基线模型，消融研究验证了各模块的有效性。

Insight: 创新点包括通过自适应检索构建多模态知识图谱实现病例感知推理，设计知识传播与注入机制促进跨模态对齐，以及引入基于预测置信度和样本相似性的可靠性估计方案来优化决策并提供可解释的病例级证据。

Abstract: Deep learning has brought significant progress to medical image classification, yet most existing methods still rely on isolated visual evidence and cannot effectively leverage similar cases or external knowledge. In clinical practice, diagnosis is typically supported by historical similar cases and their associated symptoms. To simulate this diagnostic process, we propose a framework that performs case-aware reasoning using multimodal knowledge graphs for explainable medical image diagnosis. Given an input image, our method constructs a multimodal knowledge graph from adaptively retrieved similar cases, enabling more effective utilization of related samples. We further introduce a knowledge propagation and injection mechanism, where an image-centric Graph Attention Network propagates knowledge semantics to obtain case-based features, followed by a bidirectional cross-modal attention mechanism that injects these features into visual representations for cross-modal alignment. To mitigate noisy retrieval, we design a confidence-calibrated decision refinement scheme that estimates the reliability of each retrieved case by jointly considering prediction confidence and sample similarity, adaptively adjusting its contribution to the final prediction and providing interpretable case-level evidence. Extensive experiments on multiple medical imaging datasets show that our approach consistently outperforms strong baselines, and ablation studies validate the effectiveness of each component. The source code is publicly available at https://anonymous.4open.science/r/MKG-CARE-8B7B.

[89] SADGE: Structure and Appearance Domain Gap Estimation of Synthetic and Real Data cs.CVPDF

Patryk Bartkowiak, Bartosz Kotrys, Dominik Michels, Soren Pirk, Wojtek Palubicki

TL;DR: 本文提出了SADGE（结构与外观域差距估计）指标，用于量化评估合成图像数据集在计算机视觉任务（如目标检测、语义分割、姿态估计）中的实用性，无需下游模型训练。该指标通过非线性融合外观相似性（如DINOv3）与几何相似性（如MASt3R）来预测合成数据在真实数据上的性能表现。

Details

Motivation: 现有评估指标（如PSNR、FID、CLIP）主要关注语义对齐（外观相似性），而较少考虑结构相似性（几何相似性），但单独使用任一方面均无法可靠预测合成数据在下游任务中的性能，因此需要一种综合评估方法来解决这一瓶颈。

Result: 在五个公开的合成到真实基准系列（15个数据集变体，79k图像对）上，SADGE在线性（Pearson r=0.88）和基于排名的（Spearman rho=0.77）标准下均表现出与下游迁移性能最强的关联性，超越了仅使用外观或几何的基线方法。

Insight: 创新点在于首次系统性地评估了外观与几何相似性指标对合成数据效用的预测能力，并提出了通过约束双线性交互融合DINOv3外观相似性与MASt3R几何一致性的方法，揭示了二者非线性交互对合成数据实用性的决定性作用，为合成数据评估提供了新视角。

Abstract: We propose SADGE, a quantitative similarity metric that predicts the performance of synthetic image datasets for common computer vision tasks without downstream model training. Estimating whether a synthetic dataset will lead to a model that performs well on real-world data remains a bottleneck in model development. Existing evaluation metrics (e.g., PSNR, FID, CLIP) primarily measure semantic alignment between real and synthetic images (Appearance Similarity Score). Less commonly, structural similarity between images is considered to assess the domain gap (Geometric Similarity Score). However, to the best of our knowledge there exists no studies that evaluate which similarity metric is the best downstream predictor for a given synthetic dataset. In this paper, we show over a wide variety of different synthetic datasets and downstream tasks that neither appearance nor geometry alone can reliably predict downstream performance; rather, it is their non-linear interplay that dictates synthetic data utility. Specifically, we measure how commonly used Appearance and Geometric Similarity metrics computed between synthetic and real images correlate with downstream performance in object detection, semantic segmentation, and pose estimation. Across five public synthetic-to-real benchmark families and 15 dataset-level variants (79k image pairs), SADGE achieves the strongest association with downstream transfer performance under both linear and rank-based criteria, reaching Pearson r=0.88 and Spearman rho=0.77. We compute for each combination of geometry-based methods and appearance-based approaches SADGE scores across all benchmark families. The best configuration is obtained by fusing DINOv3 appearance similarity with MASt3R geometric consistency through a constrained bilinear interaction, outperforming both the strongest geometry-only baseline and the strongest appearance-only baseline .

[90] FashionLens: Toward Versatile Fashion Image Retrieval via Task-Adaptive Learning cs.CV | cs.MMPDF

Haokun Wen, Xuemeng Song, Xinghao Xie, Xiaolin Chen, Xiangyu Zhao

TL;DR: 本文提出了FashionLens，一个基于多模态大语言模型的统一框架，旨在实现多功能的时尚图像检索。为了支持多样化的查询格式和搜索意图，作者首先构建了U-FIRE基准测试，并设计了提案引导的球面查询校准器和梯度引导的自适应采样策略来处理不同的匹配目标和优化不平衡问题。

Details

Motivation: 现有时尚图像检索方法通常专注于狭窄的任务，无法充分捕捉现实应用中多样的查询格式和搜索意图，因此需要一个能够统一处理多样化现实检索场景的框架。

Result: 在提出的U-FIRE基准测试上的实验表明，FashionLens在多种检索场景中均达到了最先进的性能，并且对未见任务展现出强大的泛化能力。

Insight: 创新点在于通过提案引导的球面查询校准器动态地将查询表征对齐到任务特定的度量空间，以及通过梯度引导的自适应采样策略根据实时学习难度和数据规模先验自动重新加权任务，以解决多任务学习中的优化不平衡问题。

Abstract: Fashion image retrieval is a cornerstone of modern e-commerce systems. A unified framework that supports diverse query formats and search intentions is highly desired in practice. However, existing approaches focus on narrow retrieval tasks and do not fully capture such diversity. Therefore, in this work, we aim to develop a unified framework capable of handling diverse realistic fashion retrieval scenarios, achieving truly versatile fashion image retrieval. To establish a data foundation, we first introduce U-FIRE, a comprehensive benchmark that consolidates fragmented fashion datasets into a unified collection, supplemented by two manually curated datasets for testing generalization. Building upon this, we propose FashionLens, a unified framework based on Multimodal Large Language Models. To handle divergent matching objectives, we design a Proposal-Guided Spherical Query Calibrator that dynamically shifts query representations into task-aligned metric spaces via adaptive spherical linear interpolation. Additionally, to mitigate the optimization imbalance caused by varying task complexities and data scales, we develop a Gradient-Guided Adaptive Sampling strategy that automatically re-weights tasks based on realtime learning difficulty and the data scale prior. Experiments on U-FIRE show that FashionLens achieves state-of-the-art performance across diverse retrieval scenarios and generalizes robustly to unseen tasks. The data and code are publicly released at https://github.com/haokunwen/FashionLens.

[91] GeoWeaver: Grounding Visual Tokens with Geometric Evidence before Scene Reasoning cs.CVPDF

Deshui Miao, Xingsen Huang, Yameng Gu, Xin Li, Haijun Zhang

TL;DR: 本文提出了GeoWeaver框架，旨在解决视觉语言模型在时空推理中需要几何信息的问题。该框架在推理前构建多级几何信息库，并为每个视觉token自适应地分配最相关的几何证据，从而将几何信息作为表征基础融入视觉表示。实验表明，该方法能持续提升几何感知推理能力，同时保持通用的多模态能力。

Details

Motivation: 现有方法通常将几何线索作为所有视觉token的共享信号，忽略了不同视觉token因其空间角色不同而需要不同几何证据的细粒度挑战。本文旨在解决这一局限性，将几何信息视为时空推理的表征前提。

Result: 在空间推理基准测试上的广泛评估表明，GeoWeaver能持续增强几何感知推理能力，同时保持通用的多模态能力。

Insight: 创新点在于提出了一个推理前的几何基础框架，通过构建几何信息库和token自适应的几何证据分配，将几何信息作为表征基础而非后期融合的辅助信号。这为大型语言模型的推理提供了更坚实的几何表征基础。

Abstract: Spatio-temporal reasoning in vision-language models requires visual representations that preserve physical geometry rather than merely semantic appearance. Recent multimodal models incorporate geometric information through structural branches, 3D-aware supervision, reasoning-stage fusion, or long-horizon memory. While these approaches demonstrate the importance of geometry for spatial intelligence, they typically treat geometric cues as a shared signal across all visual tokens. We note that this overlooks a finer-grained challenge: different visual tokens require different geometric evidence depending on their spatial roles. To address this limitation, we introduce GeoWeaver, a pre-reasoning geometric grounding framework that treats geometry as a representational prerequisite for spatio-temporal reasoning. GeoWeaver constructs a multi-level geometry bank from a frozen geometry encoder and performs token-adaptive geometric evidence allocation, enabling each visual token to retrieve the most relevant geometric abstractions. The selected evidence is incorporated into visual tokens via a residual grounding operation prior to language modeling, yielding geometry-grounded representations for downstream reasoning. Extensive evaluations on spatial reasoning benchmarks demonstrate that GeoWeaver consistently enhances geometry-aware reasoning while retaining general multimodal capabilities. This indicates that geometric information yields the greatest benefit not as a late-fusion auxiliary signal but as a fundamental prerequisite that shapes the representational foundation on which large language models perform reasoning. All source code and models will be released at https://github.com/yahooo-m/GeoWeaver .

[92] SegGuidedNet: Sub-Region-Aware Attention Supervision for Interpretable Brain Tumor Segmentation cs.CVPDF

Hasaan Maqsood, Saif Ur Rehman Khan, Sebastian Vollmer, Andreas Dengel, Muhammad Nabeel Asim

TL;DR: 本文提出SegGuidedNet，一种用于脑肿瘤子区域分割的三维残差编码器-解码器网络。其核心创新是引入SegAttentionGate模块，通过轻量级辅助损失显式监督解码器为每个肿瘤子区域（坏死核心、瘤周水肿和增强肿瘤）生成空间可区分的注意力图，从而在保持高分割精度的同时提供内置的空间可解释性。

Details

Motivation: 解决多参数MRI中脑肿瘤子区域分割的挑战，包括形态学变异性、类别不平衡以及不同成像序列中肿瘤区域外观重叠的问题，旨在实现准确且可解释的分割。

Result: 在BraTS2021和BraTS2023 GLI两个基准数据集（各251个独立测试样本）上评估，平均Dice分数分别达到0.905和0.897。作为单一模型，其性能超越了基于集成的nnU-Net和HNF-Netv2，并以远低于Swin UNETR（10模型集成）的推理成本，在Dice分数上仅落后2-4个百分点。

Insight: 主要创新点在于通过SegAttentionGate模块和辅助损失进行子区域感知的注意力监督，这不仅能维持解码器对视觉模糊类别的区分能力，还能在推理时免费提供空间可解释性，无需任何后处理解释方法。该方法以极小的参数量开销（<0.2%）实现了竞争性的精度与内置可解释性，具有良好的泛化性和临床实用性。

Abstract: Accurate segmentation of brain tumour sub-regions from multi-parametric MRI is critical for treatment planning yet remains challenging due to morphological variability, class imbalance, and overlapping appearances of tumour regions across imaging sequences. We propose SegGuidedNet, a three-dimensional residual encoder–decoder network introducing a novel SegAttentionGate module that explicitly supervises the decoder to produce spatially discriminative attention maps for each tumour sub-region necrotic core, peritumoral oedema, and enhancing tumour via a lightweight auxiliary loss, adding less than 0.2% parameter overhead. This sub-region supervision maintains decoder discriminability between visually ambiguous classes while providing free-of-cost spatial interpretability at inference without any post-hoc explanation method. Evaluated independently on BraTS2021 and BraTS2023 GLI across 251 held-out subjects each, SegGuidedNet achieves mean Dice of 0.905 (ET= 0.873, TC=0.906, WT=0.935) and 0.897 (ET=0.859, TC=0.902, WT=0.931) respectively, surpassing ensemble-based nnU-Net and HNF-Netv2 as a single model and approaching Swin UNETR a 10-model ensemble within 2–4 Dice points at a fraction of the inference cost. The consistency of results across two benchmark editions further confirms the generalisability of the proposed approach, offering competitive accuracy with built-in interpretability in a lightweight, clinically practical framework.

[93] VGenST-Bench: A Benchmark for Spatio-Temporal Reasoning via Active Video Synthesis cs.CV | cs.AIPDF

Jinho Park, Youbin Kim, Hogun Park, Eunbyung Park

TL;DR: 本文提出了VGenST-Bench，一个用于评估多模态大语言模型时空推理能力的视频基准测试。该基准通过生成模型主动合成高度可控且多样化的评估场景，并采用包含人工质量控制的多智能体流程来确保视频和问答对的质量。它建立了一个全面的视频分类法，并设计了分层任务套件，以从被动策划转向主动合成，从而实现对模型时空理解能力的细粒度诊断。

Details

Motivation: 现有时空推理基准主要依赖静态图像集或被动策划的视频数据，限制了细粒度推理能力的评估。为了更精确地评估多模态大语言模型在现实世界操作中的核心时空推理能力，需要一种新的评估范式。

Result: 论文构建了VGenST-Bench基准，其通过主动合成视频场景，能够对多模态大语言模型的时空理解进行细粒度诊断。虽然没有提及具体的定量比较结果，但该基准旨在提供更全面和可控的评估环境。

Insight: 主要创新点在于将基准构建范式从被动数据策划转变为利用生成模型进行主动场景合成，从而实现对评估场景的高度控制和多样性覆盖。同时，提出的多智能体构建流程和分层任务设计，将低级视觉感知与高级时空推理解耦，有助于更精准地诊断模型能力。

Abstract: Spatio-temporal reasoning is a core capability for Multimodal Large Language Models (MLLMs) operating in the real world. As such, evaluating it precisely has become an essential challenge. However, existing spatio-temporal reasoning benchmark datasets primarily rely on static image sets or passively curated video data, which limits the evaluation of fine-grained reasoning capabilities. In this paper, we introduce VGenST-Bench, a video benchmark that employs generative models to actively synthesize highly controlled and diverse evaluation scenarios. To construct VGenST-Bench, we propose a multi-agent pipeline incorporating a human quality control stage, ensuring the quality of all generated videos and QA pairs. We establish a comprehensive 3x2x2 video taxonomy, encompassing Spatial Scale, Perspective, and Scene Dynamics to span diverse scenarios. Furthermore, we design a hierarchical task suite that decouples low-level visual perception from high-level spatio-temporal reasoning. By shifting the paradigm from passive curation to active synthesis, VGenST-Bench enables fine-grained diagnosis of spatio-temporal understanding in MLLMs.

[94] Cell Phantom Video Generation in Elliptical Fourier Descriptor Domain cs.CVPDF

Francesco Benedetto, Roberto Basla, Luca Magri, Giacomo Boracchi

TL;DR: 本文提出了一种在椭圆傅里叶描述符域生成细胞幻影视频的新框架，用于解决生物医学视频中细胞追踪任务标注数据稀缺的问题。该方法将细胞轮廓的时序演化建模为EFD系数的多元时间序列，以生成具有时间一致性和生物学合理性的合成视频，可用于构建带标注的合成数据集以训练深度神经网络。

Details

Motivation: 训练用于细胞追踪的深度神经网络需要大量标注数据，而人工标注耗时且需要专业知识，导致公开标注数据有限。生成带有真实标注的合成视频是解决该数据瓶颈的一个有前景的方案，其基础步骤是合成具有时间一致性的单细胞幻影。

Result: 实验验证表明，在EFD空间中建模时序演化能够生成生物学上合理的幻影视频。该方法可用于生成合成标注数据的流程中，从而极大减轻创建新数据集的标注工作量。

Insight: 创新点在于将细胞轮廓的时序演化建模为紧凑且几何可解释的椭圆傅里叶描述符系数的多元时间序列，这为细胞形态引入了强先验，并能高效生成时间上连贯演化的序列。从客观角度看，该方法提供了一种新的、基于形状参数化表示的合成数据生成范式，可能比直接在像素空间操作更高效和可控。

Abstract: Training Deep Neural Networks for tracking individual cells in biomedical videos requires a large amount of annotated data. The annotation of videos for cell tracking is very time consuming and often requires domain expertise; this explains the limited availability of public annotated data to address important medical problems like tissue repair or cancer treatment. Generating synthetic videos along with their Ground Truth annotations is a promising solution that relies, as a foundational first step, on the synthesis of single cell annotations (or phantoms). Phantoms need to be time consistent, as they have to replicate biological processes that are specific to the cell types. In this work, we propose a novel framework for generating videos of cell phantoms in the Elliptical Fourier Descriptors (EFDs) domain, a compact and geometrically interpretable representation for 2D closed contours. We represent the cell phantom evolution as a multivariate time series of EFD coefficients, introducing a strong prior for cell morphology and enabling the efficient generation of sequences that evolve coherently in time. Our experimental validation proves that modelling the temporal evolution in EFD space enables the generation of biologically plausible phantom videos. Our method can be used in generative pipelines for synthesizing annotated data for cell tracking, thus strongly mitigating the annotation effort for creating new datasets. Our code is available for download here: https://github.com/FrancescoBenedetto99/efd-cell-video-gen.

[95] MOTOR: A Multimodal Dataset for Two-Wheeler Rider Behavior Understanding cs.CVPDF

Varun A. Paturkar, Shankar Gangisetty, C. V. Jawahar

TL;DR: 本文介绍了MOTOR数据集，这是首个针对全球南方地区密集、非结构化交通中两轮车骑手行为的大规模、多视角、多模态数据集。该数据集包含1,629个序列（超过25小时视频），集成了同步的前、后、头盔视频，可穿戴眼动仪捕捉的骑手注视点，道路音频以及GPS、加速度计、陀螺仪等遥测数据，并提供了交通环境、骑手状态、12种骑行操作（包括常规和非常规行为）以及合法性标签的丰富标注。

Details

Motivation: 两轮车在全球南方道路死亡事故中占比过高，但针对其骑手行为的研究远落后于四轮车，后者已通过多模态数据集在高级驾驶辅助系统（ADAS）领域取得重大进展。为填补这一空白，本文旨在构建首个专门用于密集、非结构化交通中两轮车的多模态数据集，以推动骑手行为理解与安全研究。

Result: 研究使用最先进的视频动作识别骨干网络（基于CNN和Transformer）进行骑手行为识别和操作合法性分类的基准测试，并扩展了多模态融合方法。结果表明，结合RGB视频、注视点和遥测数据能始终获得最佳性能。

Insight: 论文的主要创新点是创建了首个针对两轮车在复杂交通场景下的多模态、多视角、大规模数据集MOTOR，并提供了全面的标注。从客观角度看，该数据集为研究两轮车骑手行为分析、合法性感知预测和智能交通系统模型提供了一个独特且宝贵的基准平台，有望推动该领域的安全关键性研究。数据集和代码已开源。

Abstract: Two-wheelers account for a disproportionately high share of road fatalities in the Global South. Research on two-wheeler rider behavior, however, lags far behind four-wheelers, where multimodal datasets have driven major advances in Advanced Driver Assistance Systems (ADAS). To address this gap, we present the MOtorized TwO-wheeler Rider (MOTOR) dataset, the first large-scale, multi-view, multimodal resource dedicated to two-wheelers in dense, unstructured traffic. MOTOR comprises 1,629 sequences (25+ hours of video data) collected from 16 riders and integrates synchronized front, rear, and helmet videos, rider eye-gaze from wearable trackers, on-road audio, and telemetry (GPS, accelerometer, gyroscope). Rich annotations capture traffic context, rider state, 12 riding maneuvers spanning conventional and unconventional behaviors, and legality labels (Legal, Illegal, Unspecified). We benchmark rider behavior recognition and maneuver legality classification using state-of-the-art video action recognition backbones (CNN and Transformer-based), extended with multimodal fusion, and find that combining RGB, gaze, and telemetry consistently yields the best performance. MOTOR thus provides a unique foundation for advancing safety-critical understanding of two-wheeler riding. It offers the research community a benchmark to develop and evaluate models for behavior analysis, legality-aware prediction, and intelligent transportation systems. Dataset and code is available at https: //varuniiith.github.io/MOTOR-Dataset/

[96] GLeVE: Graph-Guided Lesion Grounding with Proposal Verification in 3D CT cs.CVPDF

Shuo Jiang, Yuhao Hong, Chunbo Jiang, Weihong Chen, Huangwei Chen

TL;DR: 本文提出GLeVE框架，用于解决三维CT报告中自由文本描述与病灶三维空间定位之间的语义-空间鸿沟问题。该方法将每个病灶描述视为原子语义单元，通过关系感知图推理编码器官属性、特征及病灶间关系，生成具有区分性的病灶查询。结合解剖学先验的候选区域生成与验证，以及基于八叉树的自回归细化，实现了从文本到病灶的一对一精确对齐与边界勾勒。

Details

Motivation: 现有基于放射学报告的视觉-语言定位方法通常依赖短语级对齐或密集像素级监督，导致病灶级对应关系有限且定位精度欠佳。本文旨在弥合自由文本叙述与三维解剖体积之间的语义-空间差距，实现可验证的临床解释。

Result: 在AbdomenAtlas 3.0数据集上的实验表明，GLeVE在分割精度和病灶级定位方面均优于经典多模态基础模型和报告监督基线方法，取得了持续的性能提升。

Insight: 创新点包括将病灶描述作为原子单元进行图推理以生成判别性查询，以及通过解剖先验验证与八叉树层次细化实现精确的一对一文本-病灶对齐。从客观角度看，该方法通过结合结构化语义推理与渐进式空间细化，有效缩小了文本与三维视觉之间的模态鸿沟。

Abstract: Grounding radiology report descriptions to 3D CT volumes is essential for verifiable clinical interpretation, yet remains challenging due to the semantic-spatial gap between free-text narratives and volumetric anatomy. Existing report-assisted and vision-language grounding methods typically rely on phrase-level alignment or dense pixel supervision, resulting in limited lesion-wise correspondence and suboptimal localization accuracy. We propose GLeVE, a graph-guided lesion grounding framework with anatomical prior verification and octree-based autoregressive refinement. GLeVE treats each lesion description as an atomic semantic unit and encodes organ attribution, attributes, and inter-lesion relations through relation-aware graph reasoning to produce discriminative lesion-wise queries. Anatomy-aware proposal generation with region-level verification enforces one-to-one text-lesion alignment, while hierarchical octree refinement progressively improves boundary delineation. Experiments on AbdomenAtlas 3.0 demonstrate consistent gains over classical multimodal foundation models and report-supervised baselines in both segmentation accuracy and lesion-level localization.

Zhanbo Huang, Xiaoming Liu, Yu Kong

TL;DR: H-Flow是一种自监督的密集人体场景流估计方法，旨在从单目视频中同时捕捉人体骨骼运动（全局姿态）和衣物/软组织等非刚性表面变形。该方法通过一个统一的多头Transformer架构联合预测场景流、姿态和深度，并利用物理启发的几何、结构和生物力学先验作为跨模态训练目标来替代难以获取的真实标签。论文还引入了高保真合成基准DynAct4D用于评估。

Details

Motivation: 现有参数化人体模型能捕捉全局姿态但无法表示衣物和软组织的非刚性表面动态；而通用场景流方法在关节化人体上效果不佳，且密集像素级真实标签难以获取。因此，需要一种能统一估计骨骼运动与表面变形的密集人体场景流方法。

Result: 在标准基准测试中，H-Flow超越了现有的场景流和参数化基线方法，并能够零样本泛化到真实世界视频。论文提出的DynAct4D基准提供了跨不同主体、服装和动作的密集流标注。

Insight: 创新点在于将物理启发的人体运动先验（几何、结构、生物力学）编码为跨模态自监督训练目标，以解决缺乏真实标签的挑战；同时，联合多头Transformer架构统一估计场景流、姿态和深度，实现了骨骼运动与表面变形的协同建模。

Abstract: Parametric human models capture global pose but cannot represent the non-rigid surface dynamics of clothing and soft tissue. Generic scene flow estimates dense motion but breaks down on articulated bodies, where pixel-level supervision is also intractable to acquire. We introduce H-Flow, a dense human scene flow that captures both skeletal kinematics and surface deformation. A unified multi-head transformer estimates flow from monocular video, jointly predicting pose and depth as companion outputs. The challenge lies in the lack of supervision. In place of unattainable labels, we anchor the network in the physics of human motion, encoding geometric, structural, and biomechanical priors as cross-modal training objectives. We further introduce DynAct4D, a high-fidelity synthetic benchmark providing dense flow annotations across diverse subjects, garments, and motions. On standard benchmarks, H-Flow outperforms scene-flow and parametric baselines, and generalizes zero-shot to in-the-wild video. Code, models, and the DynAct4D benchmark will be released upon publication

[98] Enhancing Gaze Reasoning in Vision Foundation Models for Gaze Following cs.CVPDF

Shijing Wang, Yaping Huang, Chaoqun Cui, David Wong, Yihua Cheng

TL;DR: 本文针对基于视觉基础模型（VFMs）的视线跟随方法中存在的视线推理能力不足问题，提出了一种新颖的训练机制。该方法通过头部条件局部LoRA和锥外惩罚两项技术，增强模型对视线线索的学习，从而在目标物体语义不显著时也能准确定位。在GazeFollow和VAT数据集上的实验表明，该方法达到了最先进的性能。

Details

Motivation: 现有基于视觉基础模型的视线跟随方法虽然显著提升了场景理解能力，但对视线推理的贡献有限，导致模型过度依赖语义显著的物体而非真实的视线线索，当目标不显著时性能下降。

Result: 在GazeFollow和VAT数据集上的实验表明，该方法取得了最先进的（SOTA）性能，特别是在视线目标语义不显著的情况下，性能提升尤为显著。

Insight: 创新点在于提出了一个专门增强视线推理的训练机制，其核心是头部条件局部LoRA（实现局部适应以改进头部token学习）和锥外惩罚（将视线线索注入头部token并与场景token对齐）。这为未来视线跟随研究提供了有价值的见解，即可以在不破坏预训练VFMs强大场景理解能力的前提下，针对性提升其特定子任务（如视线推理）的能力。

Abstract: Gaze following requires both scene understanding and gaze reasoning to localize the gaze target of an in-scene person. Recently, vision foundation models (VFMs) have demonstrated strong performance on this task, enabling simpler architectures while outperforming prior methods. However, we observe a key limitation of VFM-based approaches: while VFMs substantially improve scene understanding, they contribute little to gaze reasoning. As a result, existing methods often rely on semantically salient objects rather than true gaze cues, leading to degraded performance when targets are not salient. To address this, we propose a novel training mechanism to enhance gaze reasoning in VFMs for gaze following. Our method includes: (1) a head-conditioned local LoRA, which enables localized adaptation to preserve scene token learning while improving head token learning for gaze reasoning; and (2) an out-of-cone penalty, which injects gaze cues into head tokens while aligning them with scene tokens. Experiments on the GazeFollow and VAT datasets demonstrate that our method achieves state-of-the-art performance, with particularly strong improvements when gaze targets are not semantically salient. Our findings offer valuable insights for advancing future gaze following research. We will release the code once the paper is accepted.

[99] Rethinking Noise-Robust Training for Frozen Vision Foundation Models: A Cross-Dataset Benchmark with a Case Study of Small-Loss Failure cs.CVPDF

Zitong Li, Haoyu Wang

TL;DR: 本文针对冻结视觉基础模型（VFMs）在医学图像中的噪声标签学习问题，提出了一个跨数据集的基准测试，涵盖8种方法、5个数据集、3种骨干网络、2种噪声类型和5种噪声率（共150种条件）。研究发现没有通用的最优方法，ELR在最多条件下获胜，而CUFIT获得最佳平均排名。研究还揭示了在冻结特征下小损失假设的失效，并建议将噪声标签学习视为一个依赖于具体场景的方法选择问题。

Details

Motivation: 动机在于当前针对冻结视觉基础模型的噪声标签学习方法研究不足，大多数方法仍依赖于端到端训练中的小损失假设，而该假设在冻结特征下可能不成立，因此需要系统评估现有方法在医学图像噪声标签场景下的表现。

Result: 在150种条件下，没有单一方法始终最优；ELR在49/150条件下获胜，CUFIT获得最佳平均排名（2.51）。方法选择对性能的影响随噪声严重性增加而急剧增大，在干净数据上差异为4.5个百分点，在40%非对称噪声下差异达18.8个百分点。在冻结DINOv2特征下，干净和噪声样本的损失分布重叠率达53-61%，且预测一致性比损失排序更稳定。

Insight: 创新点在于首次系统评估了冻结视觉基础模型下的噪声标签学习方法，揭示了小损失假设在该场景下的局限性，并提出了将问题重新定义为场景感知的方法选择问题，而非寻找单一主导算法。研究还提供了基于证据的实践指导和低后悔特征空间选择器。

Abstract: Frozen Vision Foundation Models (VFMs) with lightweight classification heads are increasingly used in medical imaging because they offer efficient and reproducible deployment. Yet noisy-label learning methods for this frozen-feature regime remain poorly understood, and most existing methods still rely on a small-loss assumption inherited from end-to-end training. We present a controlled benchmark of eight noisy-label methods across five medical datasets, three backbones, two noise types, and five noise rates (150 conditions, 6,000 training runs), evaluated with balanced accuracy. The benchmark shows that there is no universal winner: Friedman ranking over the 150 conditions yields $χ^2 = 333.2$ ($p = 4.77 \times 10^{-68}$), ELR wins the most conditions (49/150), while CUFIT attains the best mean rank (2.51). The practical cost of method choice grows sharply with noise severity, from 4.5pp on clean data to 18.8pp at asymmetric 40% noise. To explain these benchmark-level patterns, we revisit the small-loss assumption in a representative high-risk regime. Under frozen DINOv2 features, clean and noisy loss distributions overlap by 53–61%, and matched-rate clean-sample detection shows that prediction agreement is markedly more stable than loss ranking under asymmetric noise (3pp vs.\ 13pp precision drop). On ISIC2019 with asymmetric 40% noise, Co-Teaching reaches 68% overall accuracy while collapsing to 35.1% balanced accuracy with zero recall on three minority classes. Together, these results recast noisy-label learning for frozen VFMs as a regime-aware method-selection problem rather than a search for a single dominant algorithm. We conclude with evidence-based guidance and a low-regret feature-space selector for practical recommendation.

[100] From Abstraction to Instantiation: Learning Behavioral Representation for Vision-Language-Action Model cs.CVPDF

Bing Hu, Zaijing Li, Rui Shao, Junda Chen, April Hua Liu

TL;DR: 本文提出了BehaviorVLA框架，旨在解决视觉-语言-动作模型在分布偏移下性能下降的问题。该框架通过一个因果Mamba架构的视觉运动行为编码器来聚合长时程轨迹信息，以及一个相位条件行为解码器来动态对齐任务先验与实时执行进度，从而学习到时间上连贯的行为表示。

Details

Motivation: 现有VLA模型在分布偏移下泛化能力差，其行为表示方法常受限于短时程的时间碎片化和静态的执行对齐，导致在复杂场景中行为不一致。

Result: 在RoboTwin 2.0、LIBERO和CALVIN基准测试上分别达到了58%、98%的成功率和4.36的平均长度，取得了SOTA性能。在真实世界的仿真到现实迁移中，仅用50%的演示数据就达到了OpenVLA-OFT的性能。

Insight: 核心创新在于提出了一个对称的双组件框架，将长时程轨迹聚合与动态执行对齐相结合，以构建更鲁棒、时间一致的行为表示，从而显著提升了数据效率和泛化能力。

Abstract: Vision-Language-Action (VLA) models often suffer from performance degradation under distribution shifts, as they struggle to learn generalized behavior representations across varying environments. While existing approaches attempt to construct behavior representations through action-centric latent variables, they are often limited by short-horizon temporal fragmentation and static execution-alignment, leading to inconsistent behaviors in complex scenarios. To address these limitations, we propose \textbf{BehaviorVLA}, a framework that facilitates robust manipulation through the learning of a temporally coherent behavioral representations. Our approach features two symmetric components: (1) the \textbf{Visuomotor Behavior Encoder (VBE)}, which utilizes a causal Mamba-based architecture to aggregate long-horizon trajectory information into a unified behavior representation; and (2) the \textbf{Phase-conditioned Behavior Decoder (PBD)}, which decodes this representation into precise actions by dynamically aligning task-level priors with real-time execution progress. Experiments on RoboTwin 2.0, LIBERO, and CALVIN demonstrate state-of-the-art success rates of 58%, 98%, and 4.36 (Avg.Len), respectively. Notably, in real-world sim-to-real transfer, BehaviorVLA matches the performance of OpenVLA-OFT using only 50% of the demonstration data, showcasing its superior data efficiency and generalization.

[101] What Does the Caption Really Say? Counterfactual Phrase Intervention for Compositional Data Selection in Vision-Language Pretraining cs.CVPDF

Hyejin Go, Semi Lee, Hyesong Choi

TL;DR: 本文提出了一种名为Counterfactual Phrase Intervention (CPI)的短语级数据选择框架，用于改进视觉语言预训练中的数据筛选。该方法通过受控替换生成短语敏感性分数，以识别对图像-文本匹配有实质性贡献的短语，从而提升模型在组合泛化任务上的性能。

Details

Motivation: 现有CLIP式对比预训练通常使用基于图像-文本对整体对齐的样本级过滤信号，但这种信号在去除粗略不匹配后趋于饱和，无法有效评估caption中具体对象、属性和关系短语对图像-文本匹配的支持程度，而这正是组合泛化所必需的。

Result: 在CC3M规模上评估，使用CPI筛选出的50%数据子集在VL-CheckList-VG Relation任务上比全数据基线提升+1.91，比仅使用对齐过滤的方法提升+1.00；同时改善了SugarCrepe性能并保持通用迁移能力。应用于NegCLIP时，VL-CheckList-VG Relation进一步提升了+3.84。

Insight: 创新点在于将数据选择从样本级对齐转向短语级敏感性分析，通过受控替换量化短语对图像-文本匹配的贡献，从而更有效地支持组合泛化；该方法与损失函数正交，可灵活应用于不同预训练框架。

Abstract: CLIP-style contrastive pretraining typically curates web-scale image-text pairs using sample-level filtering signals, often based on pair-level alignment. We show that this signal saturates: once coarse mismatches are removed, stricter global filtering no longer tracks the compositional supervision provided by the retained captions. The reason is structural - a global score conflates whether a pair is broadly plausible with whether the individual object, attribute, and relation phrases inside the caption materially support the image-text match. The latter is what compositional generalization demands, yet pair-level filters are blind to it. We address this with Counterfactual Phrase Intervention (CPI), a phrase-level curation framework that converts controlled nonce-token substitutions into image-conditioned phrase-sensitivity scores. CPI uses global alignment only for coarse mismatch removal, then ranks the surviving pool by whether caption phrases measurably affect the image-text score under controlled substitution. We frame CPI as a first-order phrase-sensitivity signal rather than a grounding or identification result, and evaluate it at CC3M scale. Ranking by this signal yields a 50%-data subset that improves VL-CheckList-VG Relation by +1.91 over the full-data baseline and +1.00 over alignment-only filtering at matched budget, while improving SugarCrepe overall and preserving general transfer. CPI is loss-orthogonal: applied unchanged to NegCLIP, it further improves VL-CheckList-VG Relation by +3.84, with additional CE-CLIP gains in the main text.

[102] Conceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models cs.CV | cs.LGPDF

Piotr Kubaty, Patryk Marszałek, Łukasz Struski, Adam Wróbel, Jacek Tabor

TL;DR: 本文提出CEDAR（Conceptual Embedding Disentanglement via Adaptive Rotation），一种后处理方法，用于揭示预训练视觉-语言模型嵌入的组成结构，而无需增加维度。该方法通过可逆变换和top-k稀疏瓶颈，将语义信息集中到轴对齐的解耦坐标中，在CLIP和BLIP等模型上实现了可解释的概念对齐。

Details

Motivation: 视觉-语言模型学习到的多模态嵌入内部语义不透明，现有稀疏自编码器（SAEs）方法通过扩展表示维度来提取可解释特征，但会破坏原始几何结构并引入冗余，因此需要一种不增加维度的解耦方法。

Result: 实验表明，CEDAR在重建与稀疏性之间取得了有竞争力的权衡，产生的解释更可解释且更符合人类感知，在CLIP和BLIP等模型上验证了其有效性。

Insight: 创新点在于通过可逆的轴对齐变换实现嵌入的稀疏解耦，无需过完备扩展，揭示了视觉-语言表示中的表观纠缠可以通过合适的基变换解决，为模型可解释性提供了新思路。

Abstract: Vision-language models learn powerful multimodal embeddings, yet their internal semantics remain opaque. While sparse autoencoders (SAEs) can extract interpretable features, they rely on expanding the representation dimension, which compromises the original geometry and introduces redundancy. We introduce CEDAR (Conceptual Embedding Disentanglement via Adaptive Rotation), a post-hoc method that reveals the compositional structure of pretrained embeddings without increasing dimensionality. By learning an invertible transformation with a top-$k$ sparsity bottleneck, CEDAR concentrates semantic information into axis-aligned disentangled coordinates. In CLIP-like architecture, individual coordinates can be interpreted with textual concepts, while for generative models such as BLIP, they can be decoded into natural language descriptions. Experiments demonstrate that CEDAR achieves a competitive reconstruction-sparsity trade-off while producing explanations that are more interpretable and better aligned with human perception. Our results suggest that the apparent entanglement in vision-language representations can be resolved through a suitable change of basis, eliminating the need for overcomplete expansions.

[103] Swift Sampling: Selecting Temporal Surprises via Taylor Series cs.CV | cs.AIPDF

Dahye Kim, Bhuvan Sachdeva, Karan Uppal, Naman Gupta, Vineeth N. Balasubramanian

TL;DR: 本文提出Swift Sampling，一种无需训练、轻量级的帧选择算法，通过泰勒级数建模视频特征轨迹，自动识别视频中的时间惊喜时刻（即实际视觉特征偏离预测演化的关键帧）。该方法在三个长视频问答基准和10个下游任务中优于均匀采样和现有查询无关基线，尤其适用于帧预算有限的长视频，最高提升准确率12.5个百分点。

Details

Motivation: 解决长视频中大部分帧冗余、关键信息仅存在于时间惊喜时刻的问题，受人类大脑预测编码启发，旨在自动高效地选择高信息量帧。

Result: 在三个长视频问答基准和10个下游任务中超越均匀采样和先前的查询无关基线；计算开销极低，仅增加基线0.02倍成本，比领先基线便宜30倍；对帧预算有限的长视频效果显著，准确率最高提升+12.5点。

Insight: 创新点在于将视频建模为视觉潜在空间中的可微轨迹，利用泰勒展开预测特征路径并选择偏离预测的帧作为时间惊喜；无需训练、辅助网络或视频特定超参数调优，实现了轻量高效的帧选择。

Abstract: While most frames in long-form video are redundant, the critical information resides in temporal surprises: moments where the actual visual features deviate from their predicted evolution. Inspired by the human brain’s predictive coding, we introduce Swift Sampling, an elegant, training-free frame selection algorithm that automatically identifies high-information moments in a video. Specifically, we model a video as a differentiable trajectory in the visual latent space and compute the velocity and acceleration of its features. Then, we apply Taylor expansion to project the expected path of subsequent frames. Frames that diverge sharply from this predicted manifold are identified as temporally surprising frames and selected for sampling. Unlike prior training-free methods that rely on auxiliary networks or video-specific hyperparameter tuning, Swift Sampling is incredibly lightweight, adding only 0.02x additional computational cost over baseline making it 30x cheaper overhead than leading baselines. Across three long-video question answering benchmarks and 10 different downstream tasks, Swift Sampling outperforms uniform sampling and prior query-agnostic baselines. It is especially powerful for long videos with limited frame budgets improving accuracy by up to +12.5 points.

[104] Slimmable ConvNeXt: Width-Adaptive Inference for Efficient Multi-Device Deployment cs.CVPDF

Janek Haberer, Jon Eike Wilhelm, Olaf Landsiedel

TL;DR: 本文提出了Slimmable ConvNeXt，一种基于ConvNeXt架构的宽度自适应推理方法。该方法通过训练一组共享权重，使其包含多个容量递增的嵌套子网络，从而实现在不同计算资源约束下的高效部署。

Details

Motivation: 解决在不同资源约束设备（或同一设备因电池、热限制、延迟等因素导致计算能力波动）上部署视觉模型时，需要训练和维护多个独立模型的问题。

Result: 在ImageNet-1k上，Slimmable ConvNeXt-T在4.5 GMACs和1.2 GMACs下分别达到80.8%和77.4%的top-1准确率，超越了HydraViT、MatFormer-S和SortedNet-S等同类方法，达到SOTA水平。

Insight: 创新点在于发现ConvNeXt的现代设计（特别是LayerNorm和倒置瓶颈结构）使其特别适合通道宽度剪裁，消除了传统可瘦身网络的归一化开销，并提供了比先前CNN和ViT方法更简单的训练流程。

Abstract: Deploying vision models across devices with varying resource constraints, or even on a single device where available compute fluctuates due to battery state, thermal throttling, or latency deadlines, typically requires training and maintaining separate models. Width-adaptive inference addresses this by training a single set of shared weights containing multiple nested subnetworks of increasing capacity, but prior CNN-based approaches required switchable batch normalization, while recent scalable methods have focused on Vision Transformers. We present Slimmable ConvNeXt, which shows that ConvNeXt’s modern design, specifically LayerNorm and inverted bottlenecks, makes it particularly suited for channel-width slimming, eliminating the normalization overhead of classical slimmable networks and producing a simpler training pipeline than both prior CNN and ViT approaches. On ImageNet-1k, Slimmable ConvNeXt-T with 3 subnetworks achieves 80.8% top-1 accuracy at 4.5 GMACs and 77.4% at 1.2 GMACs, trained from scratch for 600 epochs. At comparable compute, this exceeds HydraViT’s 6-head subnetwork (78.4% at 4.6 GMACs) by 2.4 percentage points and its 3-head configuration (73.0% at 1.3 GMACs) by 4.4 percentage points, while also outperforming MatFormer-S (78.6%) and SortedNet-S (78.2%) at the same GMACs. Scaling to Slimmable ConvNeXt-B further improves maximum accuracy to 82.8% at 15.35 GMACs.

[105] Improving Viewpoint-Invariance and Temporal Consistency for Action Detection cs.CVPDF

Yannick Porto, Renato Martins, Thomas Chalumeau, Cedric Demonceaux

TL;DR: 本文提出了一种新颖的两阶段动作检测方法，旨在提升模型对视角变化的鲁棒性和动作的全局时间一致性。第一阶段从增强的虚拟视角提取运动特征（仅用于训练），第二阶段引入了一个基于选择性状态空间序列建模的、视角不变的多尺度时间编码器，以聚合跨视角和时间尺度的信息。

Details

Motivation: 现有基于外观的视频检测方法在训练时视角多样性有限，而基于运动的方法则难以建模连续运动窗口间的细粒度时间关系，因此需要同时提升视角不变性和时间一致性。

Result: 在PKU-MMD和BABEL基准测试上的实验表明，该方法在所有考虑的划分上都显著优于最先进的方法，达到了SOTA水平。

Insight: 创新点在于结合了虚拟视角增强的运动特征提取和基于选择性状态空间模型的多尺度时间编码，这为提升动作检测的视角不变性和长程时间建模提供了新思路。

Abstract: Viewpoint change invariance and action temporal consistency are critical aspects for the effective deployment of human action detection of untrimmed videos. Existing appearance-based video detection methods often struggle with limited viewpoint diversity during training, while motion-based detection approaches frequently fail to model fine-grained temporal relationships across consecutive motion windows. This paper introduces a novel two-stage action detection approach designed to improve both view-invariance and global temporal coherence properties. In the first stage, we extract motion features from augmented virtual viewpoints, solely used at training. Then, the second stage introduces a new view-invariant, multi-scale temporal encoder based on selective state-space sequence modelling to aggregate information across viewpoints and time scales. Experiments on PKU-MMD and BABEL benchmarks demonstrate that this approach significantly outperforms state-of-the-art methods in all considered splits. Code and trained models are available at: https://icb-vision-ai.github.io/HydraView-TAD

[106] SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentation cs.CV | cs.LG | cs.MM | eess.IVPDF

Zhenyu Lu, Liupeng Li, Jinpeng Wang, Haoqian Kang, Yan Feng

TL;DR: 本文提出SegCompass模型，旨在解决现有推理分割方法中推理过程与视觉感知之间缺乏透明连接的问题。该模型利用稀疏自编码器（SAE）构建了一个显式、可解释且可微的对齐路径，通过将思维链（CoT）和视觉标记映射到共享的高维稀疏概念空间，并利用查询码本选择关键概念，最终生成多槽热图来指导掩码解码器。

Details

Motivation: 现有推理分割方法（如潜在查询对齐）是端到端的’黑箱’，缺乏可解释性；而文本定位读出虽可读但不可解释，通常作为不受约束的后处理步骤。本文旨在弥合这一可解释性差距，构建一个透明、可追溯的连接。

Result: 在五个具有挑战性的基准测试上进行的大量实验表明，SegCompass达到或超越了最先进的性能。视觉和定量分析显示，学习到的稀疏概念质量与最终掩码精度之间存在强相关性。

Insight: 核心创新在于利用稀疏自编码器（SAE）作为’白盒’接口，在推理链与视觉感知之间建立显式、可解释的对齐。这种SAE驱动的对齐路径比潜在查询更可追溯，比文本读出更连贯，并通过联合训练（结合强化学习和标准分割监督）实现了性能与可解释性的统一。

Abstract: While large language models provide strong compositional reasoning, existing reasoning segmentation pipelines fail to transparently connect this reasoning to visual perception. Current methods, such as latent query alignment, are end-to-end yet opaque “black boxes”. Conversely, textual localization readout is merely readable, not truly interpretable, often functioning as an unconstrained post-hoc step. To bridge this interpretability gap, we propose SegCompass, an end-to-end model that leverages a Sparse Autoencoder (SAE) to forge an explicit, interpretable, and differentiable alignment pathway. Given an image-instruction pair, SegCompass first generates a chain-of-thought (CoT) trace. The core of our method is an SAE that maps both the CoT and visual tokens into a shared, high-dimensional sparse concept space. A query codebook selects salient concepts from this space, which are then spatially grounded by a slot mapper into a multi-slot heatmap that guides the final mask decoder. The entire model is trained jointly, unifying reinforcement learning for the reasoning path with standard segmentation supervision. This SAE-driven interface provides a “white-box” connection that is significantly more traceable than latent queries and more coherent than textual readouts. Extensive experiments on five challenging benchmarks demonstrate that SegCompass matches or surpasses state-of-the-art performance. Crucially, our visual and quantitative analyses show a strong correlation between the quality of the learned sparse concepts and final mask accuracy, confirming that SegCompass achieves superior results through its enhanced and inspectable alignment. Code is available at https://github.com/ZhenyuLU-Heliodore/SegCompass.

[107] Cross-Domain Human Action Recognition from Multiview Motion and Textual Descriptions cs.CVPDF

Yannick Porto, Renato Martins, Thomas Chalumeau, Cedric Demonceaux

TL;DR: 本文提出了一种新颖的面向跨域人类动作识别的方法，通过结合多视角运动信息和动作文本描述来提升零样本动作识别（ZSAR）的性能。该方法引入了一个方向感知的运动编码网络来学习不同的运动特征，并采用方向感知的文本提示来在推理时匹配相应特征。实验表明，该方法在多个基准数据集上超越了现有零样本方法，并展示了强大的表征迁移能力。

Details

Motivation: 解决零样本动作识别（ZSAR）在现实场景中因人体方向和相机视角变化导致的显著领域差距问题，旨在提升模型对未见动作和跨域变化的泛化能力，同时避免高成本的标注需求。

Result: 在NTU-RGB+D、BABEL、NW-UCLA以及两个监控数据集上进行了广泛实验，结果表明该方法一致性地提升了ZSAR性能，超越了近期的SOTA零样本方法，并且在已见动作的跨域和同域识别中也表现出有竞争力的性能。

Insight: 创新点在于将多视角运动线索与文本描述相结合，并专门设计了方向感知的运动编码网络和文本提示机制，以更好地建模和匹配不同方向下的动作特征，从而有效缓解视角变化带来的领域偏移问题。

Abstract: Robustness to domain changes is a key capability for effective deployment of human action recognition systems in real-world scenarios, where action categories at inference can present important domain shifts or even unseen actions from training. In this context, improving the recognition capabilities of Zero-Shot Action Recognition models (ZSAR), without requiring strong annotation efforts, remains a central challenge. Most ZSAR approaches assume that actions are observed under geometric conditions similar to those seen during training. In practice, variations in human body orientation and camera viewpoint add a significant domain gap in ZSAR, substantially limiting generalization to novel action-motion combinations. In this context, this paper presents a novel orientation-aware action recognition approach with improved cross-domain capabilities. Our approach combines motion cues of multiple camera viewpoints and text descriptions of human actions in the training phase. We present a new orientation-aware motion encoding network to learn different motion features, and adapt a specific orientation-aware text prompt to match the corresponding features at inference. Extensive experiments demonstrate that the proposed method consistently improves ZSAR performance across different recognition benchmarks, outperforming recent state-of-the-art zero-shot approaches on NTU-RGB+D, BABEL, NW-UCLA, and on two surveillance datasets. In addition, the learned representations exhibit strong transfer learning capabilities, yielding competitive performance on both cross-domain and same-domain recognition of seen actions. Code and trained models are available at: https://icb-vision-ai.github.io/OrientationAware-HAR

[108] WorldKV: Efficient World Memory with World Retrieval and Compression cs.CVPDF

Jung Yi, Minjae Kim, Paul Hyunbin Cho, Wooseok Jang, Sangdoo Yun

TL;DR: 本文提出了WorldKV框架，旨在解决自回归视频扩散模型中维持持久世界一致性的问题。该框架通过世界检索和世界压缩两个组件，在无需训练的情况下，高效管理KV缓存，以在保持实时生成的同时确保长期内容一致性。

Details

Motivation: 现有自回归视频扩散模型在生成实时动作条件世界时，难以维持持久世界的一致性：完整KV缓存注意力虽能保持一致性但破坏实时约束，而滑动窗口推理虽恢复吞吐量却丢弃长期一致性。

Result: 在Matrix-Game-2.0和LingBot-World-Fast基准测试中，WorldKV在吞吐量提升约2倍的情况下，匹配或超过了完整KV缓存的内存保真度，并且无需微调即可与经过记忆训练的基线方法竞争。

Insight: 创新点包括通过相机/动作对应关系选择性检索场景相关KV缓存块并无需重新编码插入注意力窗口的世界检索机制，以及基于关键帧相似性修剪冗余令牌以压缩存储的世界压缩方法，实现了训练无关的高效世界记忆管理。

Abstract: Autoregressive video diffusion models have enabled real-time, action-conditioned world generation. However, sustaining a persistent world, where revisiting a previously seen viewpoint yields consistent content, remains an open problem. Full KV-cache attention preserves this consistency but breaks real-time constraints: memory footprint and attention cost grow linearly with rollout length. Sliding window inference restores throughput but discards long-term consistency. We propose WorldKV, a training-free framework with two components: World Retrieval and World Compression. World Retrieval stores evicted KV-cache chunks in GPU/CPU memory and selectively retrieves scene-relevant chunks via camera/ action correspondence, inserting them back into the native attention window without re-encoding. World Compression prunes redundant tokens within each chunk via key-key similarity to an anchor frame, halving per-chunk storage to fit 2x more history under a fixed budget. On Matrix-Game-2.0 and LingBot- World-Fast, WorldKV matches or exceeds full-KV memory fidelity at roughly 2x the throughput, and is competitive with memory-trained baselines without any fine-tuning. Project Page: https://cvlab-kaist.github.io/WorldKV/

[109] Synthetic Data Alone is Enough? Rethinking Data Scarcity in Pediatric Rare Disease Recognition cs.CVPDF

Ganlin Feng, Yuxi Long, Erin Lou, Lianghong Chen, Zihao Jing

TL;DR: 本文研究了在儿科罕见病面部识别中，仅使用合成数据是否足以应对极端数据稀缺的挑战。通过在不同规模下仅使用表型感知的合成面部图像训练模型，发现当合成数据规模足够大时，其性能可与仅使用真实数据的基线模型相媲美。

Details

Motivation: 解决儿科罕见遗传病诊断中因数据极度稀缺、隐私限制和数据共享有限而导致的计算机视觉系统开发困难，以及临床遗传咨询中视觉资源不足的问题。

Result: 在多个骨干网络上的实验表明，仅使用合成数据训练在达到足够规模时，其性能与仅使用真实数据的基线相当，表明高保真合成数据可以近似具有临床意义的分布。

Insight: 论文的创新点在于系统性地探索了在超低资源儿科场景下‘仅用合成数据’的训练范式，并验证了其可行性；其核心洞察是，高质量、大规模的表型感知合成数据本身可以成为有效的学习资源，为隐私保护的遗传教育和临床工具开发提供了新途径。

Abstract: Children with rare genetic diseases often exhibit distinctive facial phenotypes, yet developing computer vision systems for early diagnosis remains challenging due to extreme data scarcity, privacy constraints, and limited data sharing in pediatric settings. These challenges not only hinder automated diagnosis but also restrict the availability of visual resources for clinical genetic counseling. While prior work has shown that synthetic data can augment real datasets and preserve phenotype-level semantics, it remains unclear whether synthetic data alone is sufficient for learning in ultra-low-resource pediatric settings. In this work, we study the synthetic-only regime for pediatric rare disease recognition. Under a controlled experimental setup, models are trained exclusively on phenotype-aware synthetic facial images at increasing scales. We find that synthetic-only training achieves performance comparable to real-data-only baselines at sufficient scale across multiple backbones, suggesting that high-fidelity synthetic data can approximate clinically meaningful distributions. These findings together further enable the use of synthetic pediatric facial images as privacy-preserving resources for genetic education and counseling, supporting clinician training and patient communication. Our results highlight the potential of computer vision to improve data efficiency and expand accessible visual tools in children’s healthcare.

[110] DecQ: Detail-Condensing Queries for Enhanced Reconstruction and Generation in Representation Autoencoders cs.CVPDF

Tianhang Wang, Yitong Chen, Wei Song, Zuxuan Wu, Min Li

TL;DR: 本文提出了DecQ框架，用于解决表示自编码器（RAEs）中重建与生成之间的权衡问题。该方法通过引入轻量级的细节凝聚查询，从冻结视觉基础模型的中间特征中提取细粒度信息，并将其整合到解码器中，从而同时提升图像重建质量和生成性能。

Details

Motivation: 冻结视觉基础模型作为分词编码器虽然能提供鲁棒的高层表示，但限制了空间重建能力，导致细粒度生成和图像编辑效果不佳；而通过微调引入重建信号又会破坏预训练的语义空间并降低生成保真度。DecQ旨在解决这一重建与生成之间的权衡问题。

Result: 实验表明，仅增加8个查询和3.9%的计算开销，DecQ将基于冻结DINOv2的RAE的重建PSNR从19.13 dB提升至22.76 dB；在生成建模中，DecQ实现了比RAE快3.3倍的收敛速度，无引导时FID达到1.41，有引导时达到1.05。

Insight: 创新点在于提出了轻量级的细节凝聚查询机制，通过凝聚模块从中间特征提取细粒度信息，并联合补丁令牌进行生成建模，有效聚合了浅层和深层信息。这为平衡自编码器的重建与生成性能提供了一种简单有效的设计思路。

Abstract: Representation Autoencoders (RAEs) leverage frozen vision foundation models (VFMs) as tokenizer encoders, providing robust high-level representations that facilitate fast convergence and high-quality generation in latent diffusion models. However, freezing the VFM inherently constrains its spatial reconstruction capacity, limiting fine-grained generation and image editing; in contrast, incorporating reconstruction-oriented signals via fine-tuning disrupts the pretrained semantic space and degrades generative fidelity. To address this trade-off, we propose DecQ, a simple yet effective framework for RAEs. Specifically, DecQ introduces lightweight detail-condensing queries that extract fine-grained information from intermediate VFM features through condenser modules. These queries are incorporated into the decoder to support reconstruction and are jointly generated with patch tokens during generative modeling. By aggregating information from both shallow and deep layers, DecQ effectively mitigates the reconstruction–generation trade-off, improving both reconstruction quality and generative performance. Our experiments demonstrate that: (1) with only 8 additional queries and 3.9% extra computation, DecQ improves reconstruction over the frozen DINOv2-based RAE, increasing PSNR from 19.13 dB to 22.76 dB; and (2) for generative modeling, DecQ achieves 3.3$\times$ faster convergence than RAE, attaining an FID of 1.41 without guidance and 1.05 with guidance.

[111] Spectral Tail Auxiliary Learning for AI-Generated Image Detection cs.CVPDF

Xingyi Li, Jiahui Zhang, Yiheng Li, Yun Cao, Wenhao Wang

TL;DR: 本文提出了一种名为谱尾辅助学习（STAL）的频率域辅助监督框架，用于检测AI生成的图像。该框架基于对真实图像和生成图像的一维径向对数功率谱的系统分析，发现生成图像在超高频率尾端存在异常的谱尾抬升现象。STAL在训练时通过一个尾部感知的频率教师将谱尾线索传递给空间检测器，而在推理时丢弃所有频率域模块，从而不引入额外开销。

Details

Motivation: 随着生成图像模型的快速发展，生成图像与真实图像之间的感知差距不断缩小，使得AI生成图像检测变得越来越困难。现有方法通常利用频域线索进行检测，但对其具体且反复出现的谱规律性理解不足，需要更深入的表征。

Result: 在9个公开数据集上的广泛实验表明，STAL在跨生成器、数据分布和真实场景中实现了强大的泛化能力和稳定性，达到了先进的检测性能。

Insight: 论文的核心创新点是首次系统性地识别并定义了生成图像中存在的“谱尾抬升”现象，并将其归因于训练生成模型中的非线性谐波累积，这为跨生成架构提供了一个结构性的检测线索。提出的STAL框架巧妙地将频域知识作为辅助监督迁移到空间检测器中，实现了零推理开销的泛化检测，为AI生成图像检测提供了新的视角和方法。

Abstract: As generative image models evolve rapidly, the perceptual gap between generated and real images continues to narrow, making AI-generated image detection increasingly challenging. Many existing methods exploit frequency-domain cues for detection, typically described as frequency-domain artifacts or high-frequency discrepancies. However, the specific and recurring spectral regularities remain insufficiently understood and characterized. In this paper, we systematically analyze the one-dimensional radial log-power spectra of real and generated images. We find that generated images do not necessarily exhibit higher or lower energy across the entire spectrum or high-band range. Instead, their spectra deviate from the power-law decay and show an anomalous uplift in the ultra-high-frequency tail. We term this phenomenon spectral tail uplift. We further attribute this phenomenon to nonlinear harmonic accumulation in trained generative models, suggesting that it can serve as a structural cue across generative architectures. Based on this observation, we propose Spectral Tail Auxiliary Learning (STAL), a frequency-domain auxiliary supervision framework for generalizable AI-generated image detection. STAL transfers spectral-tail cues from a tail-aware frequency teacher to a spatial detector during training, while all frequency-domain modules are discarded at inference time. Consequently, STAL introduces no inference overhead. Extensive experiments on 9 public datasets show that STAL achieves strong generalization and stability across generators, data distributions, and real-world scenarios.

[112] Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous Driving cs.CVPDF

Jiahao Wang, Bo Sun, Yijing Bai, Vincent Casser, Songyou Peng

TL;DR: 本文提出Sensor2Sensor，一种新颖的生成建模范式，用于将野外单目行车记录仪视频转换为高保真、多模态的自动驾驶传感器数据（包括多视角相机图像和LiDAR点云），以弥合自动驾驶系统训练和验证所需的大规模、多样化数据与现有数据源之间的差距。

Details

Motivation: 自动驾驶系统（ADS）的鲁棒训练和验证需要海量、多样的数据集。现有自动驾驶车队采集的专有数据虽然保真度高，但在规模、传感器配置多样性、地理和长尾行为覆盖上有限。而来自行车记录仪等来源的野外数据规模巨大、多样性丰富，能捕捉关键的长尾场景和新环境，但其非结构化视频数据与ADS期望的结构化、多模态传感器输入不兼容。

Result: 论文对生成传感器数据的保真度和真实感进行了全面的定量评估。通过将具有挑战性的互联网和行车记录仪片段转换为逼真的多模态数据格式，证明了Sensor2Sensor的实际效用，从而为自动驾驶开发解锁了庞大的外部数据源。

Insight: 核心创新在于提出了一种无配对训练数据的跨具身传感器转换生成范式。关键技术包括：利用4D高斯泼溅（4DGS）重建和新视角渲染将真实AV日志转换为行车记录仪风格视频以创建训练数据，以及采用扩散模型架构执行生成转换。这为解决数据稀缺和多样性不足问题提供了一种新思路。

Abstract: Robust training and validation of Autonomous Driving Systems (ADS) require massive, diverse datasets. Proprietary data collected by Autonomous Vehicle (AV) fleets, while high-fidelity, are limited in scale, diversity of sensor configurations, as well as geographic and long-tail-behavioral coverage. In contrast, in-the-wild data from sources like dashcams offers immense scale and diversity, capturing critical long-tail scenarios and novel environments. However, this unstructured, in-the-wild video data is incompatible with ADS expecting structured, multi-modal sensor inputs for validation and training. To bridge this data gap, we propose Sensor2Sensor, a novel generative modeling paradigm that translates in-the-wild monocular dashcam videos into a high-fidelity, multi-modal sensor suite (AV logs) comprising multi-view camera images and LiDAR point clouds. A core challenge is the lack of paired training data. We address this by converting real AV logs into dashcam-style videos via 4D Gaussian Splatting (4DGS) reconstruction and novel-view rendering. Sensor2Sensor then utilizes a diffusion architecture to perform the generative conversion. We perform comprehensive quantitative evaluations on the fidelity and realism of the generated sensor data. We demonstrate Sensor2Sensor’s practical utility by converting challenging in-the-wild internet and dashcam footage into realistic, multi-modal data formats, further unlocking vast external data sources for AV development.

[113] MotiMotion: Motion-Controlled Video Generation with Visual Reasoning cs.CVPDF

Lee Hsin-Ying, Hanwen Jiang, Yiqun Mei, Jing Shi, Ming-Hsuan Yang

TL;DR: MotiMotion是一个新颖的运动控制视频生成框架，它将运动控制重新定义为先推理后生成的问题。该框架利用免训练的视觉语言推理器来精炼主要轨迹的图像空间坐标并生成合理的次要运动，同时提出置信感知控制方案来调节引导强度，以提升运动自然度。

Details

Motivation: 现有运动控制的图像到视频生成模型严格遵循用户提供的轨迹，这些轨迹通常稀疏、不精确且因果不完整，导致生成结果不自然或不合逻辑，尤其缺乏次要的因果后果。

Result: 在作者构建的新图像到视频基准MotiBench上，基于VLM的评估和人工研究均表明，MotiMotion生成的视频具有更合理的物体行为和交互，优于现有方法。

Insight: 创新点在于将运动控制视为推理任务，使用免训练的视觉语言模型进行因果推理以生成次要运动，并引入置信感知控制机制来平衡外部引导与内部生成先验，从而提升生成视频的合理性和自然度。

Abstract: Current motion-controlled image-to-video generation models rigidly follow user-provided trajectories that are often sparse, imprecise, and causally incomplete. Such reliance often yields unnatural or implausible outcomes, especially by missing secondary causal consequences. To address this, we introduce MotiMotion, a novel framework that reformulates motion control as a reasoning-then-generation problem. To encourage causally grounded and commonsense-consistent interactions, we leverage a training-free vision-language reasoner to refine image-space coordinates of primary trajectories and to hallucinate plausible secondary motions. To further improve motion naturalness, we propose a confidence-aware control scheme that modulates guidance strength, enabling the model to closely follow high-confidence plans while correcting artifacts under low-confidence inputs with its internal generative priors. To support systematic evaluation, we curate a new image-to-video benchmark, MotiBench, consisting of interaction-centric scenes where new events are triggered by motion. Both VLM-based evaluation and a human study on MotiBench demonstrate that MotiMotion produces videos with more plausible object behaviors and interaction, and is preferred over existing approaches.

[114] Cambrian-P: Pose-Grounded Video Understanding cs.CVPDF

Jihan Yang, Zifan Zhao, Xichen Pan, Shusheng Yang, Junyi Zhang

TL;DR: 该论文提出了Cambrian-P，一种增强型视频多模态大语言模型，通过引入逐帧可学习的相机令牌和姿态回归头，将相机姿态作为轻量级监督信号。该方法在空间推理基准测试中取得显著提升，并在通用视频问答任务中表现出色，同时实现了ScanNet上的最先进流式姿态估计。

Details

Motivation: 现有视频多模态大语言模型将视频帧视为孤立的2D快照，忽略了相机姿态提供的共享空间坐标框架，导致缺乏对持续场景的感知能力。论文旨在利用相机姿态作为监督信号，增强模型对物理世界的推理能力。

Result: 在VSI-Bench等空间推理基准上获得4.5-6.5%的显著提升，在八个额外的空间和通用视频问答基准上具有良好泛化性，并在ScanNet上实现了最先进的流式姿态估计。使用野外视频的伪标注姿态训练进一步提升了通用视频问答性能。

Insight: 创新点在于将相机姿态作为轻量级监督信号集成到视频MLLMs中，通过可学习相机令牌和姿态回归头增强空间感知；客观分析表明，该方法不仅提升了空间推理，还意外改善了通用视频理解，证明了姿态信号在物理世界推理中的基础性作用。

Abstract: Camera pose matters. The position and orientation of each viewpoint define a shared spatial coordinate frame that relates observations across video frames. Yet this signal is largely absent from multimodal LLMs (MLLMs) for video understanding, which process frames as isolated 2D snapshots, instead of the persistent scene humans perceive. We revisit pose as a lightweight supervisory signal and introduce Cambrian-P, a video MLLM augmented with per-frame learnable camera tokens and a pose regression head. With a carefully designed sampling scheme, the model achieves substantial gains of 4.5-6.5% on spatial reasoning benchmarks such as VSI-Bench, generalizes across eight additional spatial and general video QA benchmarks, and, as a byproduct, achieves state of the art streaming pose estimation on ScanNet. Surprisingly, training on pseudo-annotated poses from in-the-wild video further improves general video QA benchmarks, showing pose helps beyond spatial reasoning. Together, these results position camera pose as a fundamental signal for video models that reason about the physical world.

[115] Which Way Did It Move? Diagnosing and Overcoming Directional Motion Blindness in Video-LLMs cs.CVPDF

Jongseo Lee, Hyuntak Lee, Sunghun Kim, Sooa Kim, Jihoon Chung

TL;DR: 该论文揭示了视频大语言模型（Video-LLMs）在感知基本方向运动（如物体向左、右、上、下移动）时存在严重缺陷，即‘方向运动盲区’。作者通过分析模型内部信息流，定位到问题源于方向信号与语言答案选项之间的‘绑定差距’。为应对此问题，论文提出了诊断驱动的DeltaDirect目标函数和MoDirect数据集，显著提升了模型在合成和真实视频上的方向运动识别准确率，同时保持了标准视频理解能力。

Details

Motivation: 尽管视频大语言模型在时序视频理解上进展迅速，但它们在一个基本的感知任务——判断图像平面内物体的带符号运动方向（如左、右、上、下）上表现糟糕，这被称为‘方向运动盲区’。论文旨在诊断这一失败的根本原因，并提出解决方案来克服这一基本能力的缺失。

Result: 在提出的合成基准MoDirect-SynBench上，使用DeltaDirect目标进行指令微调后，方向运动准确率从25.9%大幅提升至85.4%。在真实世界基准MoDirect-RealBench上，DeltaDirect在没有使用真实世界数据微调的情况下，比基线模型提升了21.9个百分点的准确率，并且保持了标准的视频理解性能。

Insight: 论文的创新点在于：1）系统性地诊断并揭示了Video-LLMs中方向信息与语言输出之间的‘绑定差距’这一核心问题；2）提出了DeltaDirect这一诊断驱动的、在投影器层面操作的训练目标，它通过预测相邻帧特征差值的归一化2D运动向量来直接强化方向信号；3）构建了MoDirect数据集家族，用于针对性的指令微调和评估。从客观角度看，其诊断方法（如概念向量分析）和针对‘绑定问题’的解决方案，为理解和改进多模态模型中的低级感知与高级推理的整合提供了有价值的思路。

Abstract: Video Large Language Models (Video-LLMs) have made rapid progress on temporal video understanding, yet many fail at a basic perceptual primitive: signed image-plane motion direction. On simple videos of a single object moving left, right, up, or down, most Video-LLMs perform near chance, with above-chance cases largely attributable to prediction biases rather than genuine direction understanding. We call this failure directional motion blindness. We localize the failure by tracing motion direction information through the Video-LLM pipeline. Motion direction remains linearly accessible from the vision encoder, projector, and LLM hidden states, but the readout fails to bind this signal to the correct verbal answer option, revealing a direction binding gap. Although synthetic motion direction instruction tuning reduces this gap on the source domain, motion direction concept vector analysis shows that visual complexity weakens the signal magnitude and limits out-of-domain generalization. We introduce MoDirect, a dataset family for motion direction instruction tuning and evaluation, and DeltaDirect, a diagnosis-driven, projector-level objective that predicts normalized 2-D motion vectors from adjacent-frame feature deltas. On MoDirect-SynBench, instruction tuning with DeltaDirect improves motion direction accuracy from 25.9% to 85.4%. On MoDirect-RealBench, DeltaDirect improves real-world motion direction accuracy by 21.9 points over the vanilla baseline without real-world tuning data, while preserving standard video-understanding performance. Code: https://github.com/KHU-VLL/DeltaDirect

cs.AI [Back]

[116] Efficient Agentic Reasoning Through Self-Regulated Simulative Planning cs.AI | cs.CL | cs.LG | cs.ROPDF

Mingkai Deng, Jinyu Hou, Lara Sá Neves, Varad Pimpalkhute, Taylor W. Killian

TL;DR: 本文提出了一种高效智能体推理框架SR²AM，通过将决策分解为模拟推理、自我调节和反应执行三个系统，实现可控的规划过程。该方法在数学、科学、表格分析和网络信息检索等任务上，以更少的推理token取得了与更大规模模型相当的性能。

Details

Motivation: 现有基于自适应计算（如思维链）的智能体通常缺乏对规划时机、结构和深度的控制，导致推理长度大幅增加且效率低下。本文旨在通过分解决策系统来解决这一问题，实现更高效的智能体推理。

Result: 在多个任务上，v0.1-8B和v1.0-30B模型分别达到了与120-355B和685B-1T参数系统相当的Pass@1性能，同时v1.0-30B比同类智能体LLM减少了25.8-95.3%的推理token。强化学习使平均规划范围增加了22.8%，而规划频率仅增长2.0%。

Insight: 创新点在于将智能体决策分解为三个可学习的系统，特别是通过自我调节机制动态控制规划时机和深度，实现了规划效率与性能的平衡。该方法为智能体在更广泛场景（如学习和适应）中的自我调控提供了可扩展的框架。

Abstract: How should an agent decide when and how to plan? A dominant approach builds agents as reactive policies with adaptive computation (e.g., chain-of-thought), trained end-to-end expecting planning to emerge implicitly. Without control over the presence, structure, or horizon of planning, these systems dramatically increase reasoning length, yielding inefficient token use without reliable accuracy gains. We argue efficient agentic reasoning benefits from decomposing decision-making into three systems: simulative reasoning (System II) grounding deliberation in future-state prediction via a world model; self-regulation (System III) deciding when and how deeply to plan via a learned configurator; and reactive execution (System I) handling fine-grained action. Simulative reasoning provides unified planning across diverse tasks without per-domain engineering, while self-regulation ensures the planner is invoked only when needed. To test this, we develop SR$^2$AM (Self-Regulated Simulative Reasoning Agentic LLM), realizing both as distinct stages within an LLM’s chain-of-thought, with the LLM as world model. We explore two instantiations: recording decisions from a prompted multi-module system (v0.1) and reconstructing structured plans from traces of pretrained reasoning LLMs (v1.0), trained via supervised then reinforcement learning (RL). Across math, science, tabular analysis, and web information seeking, v0.1-8B and v1.0-30B achieve Pass@1 competitive with 120-355B and 685B-1T parameter systems respectively, while v1.0-30B uses 25.8-95.3% fewer reasoning tokens than comparable agentic LLMs. RL increases average planning horizon by 22.8% while planning frequency grows only 2.0%, showing it learns to plan further ahead rather than more often. More broadly, learned self-regulation instantiates a principle we expect to extend beyond planning to how agents govern their own learning and adaptation.

[117] Search-E1: Self-Distillation Drives Self-Evolution in Search-Augmented Reasoning cs.AI | cs.CL | cs.IRPDF

Zihan Liang, Yufei Ma, Ben Chen, Zhipeng Qian, Xuxin Zhang

TL;DR: 本文提出Search-E1方法，一种仅通过标准GRPO（Group Relative Policy Optimization）与离线自蒸馏（OFSD）交替进行的自进化方法，使搜索增强推理智能体无需复杂外部机制即可自我提升。该方法在推理时通过前向KL目标将策略分布与一个暴露了更高效轨迹的特权上下文下的自身分布对齐，从而提供密集的每步监督。

Details

Motivation: 现有方法通过引入外部监督、辅助模块、复杂搜索或手工奖励设计来提升搜索增强推理性能，但这些机制增加了训练流程的复杂性和资源依赖。本文旨在探究这些机制是否必要，并提出一种更简洁的自进化方案。

Result: 在七个QA基准测试上，使用Qwen2.5-3B模型的Search-E1达到了0.440的平均EM分数，超越了所有同规模的开源基线方法，实现了SOTA性能。

Insight: 创新点在于仅通过GRPO与离线自蒸馏的简单交替实现自我进化，无需外部监督或复杂模块；其核心是利用特权上下文下的更优轨迹进行策略自对齐，为推理步骤提供密集监督，这是一种高效且资源友好的训练范式。

Abstract: Post-training has become the dominant recipe for turning a language model into a competent search-augmented reasoning agent. A line of recent work pushes its performance further by adding elaborate machinery on top of this standard pipeline. These augmentations import external supervision from stronger external systems, attach auxiliary modules such as process reward models or retrospective critics, restructure the rollout itself with tree search or multi-stage curricula, or shape the reward with hand-crafted bonuses and penalties. Each addition delivers a measurable gain, but each also inflates the training pipeline and ties the recipe to resources or designs that may not always be available. We take a step back and ask whether any of this machinery is actually necessary, and propose Search-E1, a self-evolution method that lets a search-augmented agent improve through only vanilla GRPO interleaved with offline self-distillation (OFSD). After each GRPO round, the policy rolls out on its own training questions. A token-level forward KL objective then aligns the policy’s inference-time distribution to its own distribution under a privileged context that exposes a more efficient sibling trajectory. Despite this simplicity, the procedure naturally provides dense per-step supervision. On seven QA benchmarks, Search-E1 reaches $0.440$ average EM with Qwen2.5-3B, surpassing all open-source baselines at both scales. Code and complete version will be made public soon.

[118] Beyond Acoustic Emotion Recognition: Multimodal Pathos Analysis in Political Speech Using LLM-Based and Acoustic Emotion Models cs.AI | cs.CL | cs.HC | cs.SD | eess.ASPDF

Juergen Dietrich

TL;DR: 本文研究在政治演讲的情感分析中，基于声学的情感识别模型能否作为LLM多模态分析中Pathos维度的替代。通过一个德国联邦议院演讲的案例研究，比较了三种分析模态：声学SER模型、多模态LLM（Gemini）以及TRUST-Pathos LLM监督集成。结果表明，Gemini的效价与TRUST-Pathos强相关，而声学模型则不然。研究还指出标准SER基准数据集存在表演性语音、文化偏见和类别不兼容等问题。

Details

Motivation: 探讨在政治演讲的Pathos（情感诉求）维度分析中，传统的声学情感识别模型是否足以替代基于LLM的多模态分析方法，以解决现有SER基准在真实、复杂场景下的局限性。

Result: 在Bundestag演讲案例中，Gemini 2.5 Flash的效价与TRUST-Pathos得分呈强正相关（Spearman rho = +0.664, p < 0.001），而emotion2vec模型的效价则无显著相关（rho = +0.097, p = 0.499）。对EMO-DB的系统评估进一步揭示了标准SER数据集的缺陷。

Insight: 创新点在于将LLM驱动的多模态分析（结合音频和文本）引入政治演讲的情感分析，证明了其在捕捉语义定义的政治情感上优于纯声学模型；同时，研究揭示了现有SER基准在真实世界应用中的局限性，为未来结合视觉模态的分析提供了方向。

Abstract: We investigate whether acoustic emotion recognition models can serve as proxies for the Pathos dimension in political speech analysis, as operationalised by the TRUST multi-agent large language model (LLM) pipeline. Using a Bundestag plenary speech by Felix Banaszak (51 segments, 245 s) as a case study, we compare three analysis modalities: (1) emotion2vec_plus_large, an acoustic speech emotion recognition (SER) model whose continuous Arousal and Valence values are derived via post-hoc Russell Circumplex projection; (2) Gemini 2.5 Flash, an LLM analysing the full speech audio together with its transcript in an open-ended, context-aware fashion; and (3) TRUST-Pathos scores from a three-advocate LLM supervisor ensemble. Spearman rank correlations reveal that Gemini Valence correlates strongly with TRUST-Pathos (rho = +0.664, p < 0.001), whereas emotion2vec Valence does not (rho = +0.097, p = 0.499). We further demonstrate, via a systematic quality evaluation of the Berlin Database of Emotional Speech (EMO-DB) using Gemini in an open-ended annotation paradigm, that standard SER benchmark corpora suffer from acted speech, cultural bias, and category incompatibility. Our results suggest that LLM-based multimodal analysis captures semantically defined political emotion substantially better than acoustic models alone, while acoustic features remain informative for low-level Arousal estimation. Future work will extend this approach to video-based analysis incorporating facial expression and gaze.

[119] Enhancing Visual Token Representations for Video Large Language Models via Training-Free Spatial-Temporal Pooling and Gridding cs.AI | cs.CVPDF

Bingjun Luo, Tony Wang, Hanqi Chen, Xinpeng Ding

TL;DR: 本文提出了一种名为ST-GridPool的无训练视觉令牌增强方法，用于提升视频大语言模型的性能。该方法通过金字塔时序网格化捕捉多粒度时空交互，并结合基于范数的空间池化保留高信息区域，从而高效压缩视觉令牌并改善其表示。

Details

Motivation: 现有MLLM方法（如LLaVA系列）在压缩视觉令牌时使用的池化或插值技术过于简单，忽略了视觉令牌复杂的时空动态交互，导致信息丢失。本文旨在解决这一效率与表示能力之间的差距。

Result: 在多个基准测试上的广泛实验表明，ST-GridPool能够持续提升视频LLMs的性能，且无需昂贵的重新训练，提供了一个即插即用的高效解决方案。

Insight: 创新点在于提出了一个完全无需训练的两阶段框架：PTG通过分层时序网格化结构化地建模时空关系，NSP则利用令牌范数与语义丰富度的相关性进行智能空间池化。这为改善视觉令牌表示提供了一种轻量且通用的新思路。

Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have significantly advanced video understanding tasks, yet challenges remain in efficiently compressing visual tokens while preserving spatiotemporal interactions. Existing methods, such as LLaVA family, utilize simplistic pooling or interpolation techniques that overlook the intricate dynamics of visual tokens. To bridge this gap, we propose ST-GridPool, a novel training-free visual token enhancement method designed specifically for Video LLMs. Our approach integrates Pyramid Temporal Gridding (PTG), which captures multi-grained spatiotemporal interactions through hierarchical temporal gridding, and Norm-based Spatial Pooling (NSP), which preserves high-information visual regions by leveraging the correlation between token norms and semantic richness. Extensive experiments on various benchmarks demonstrate that ST-GridPool consistently enhances performance of Video LLMs without requiring costly retraining. Our method offers an efficient and plug-and-play solution for improving visual token representations. Our code is available in https://github.com/bingjunluo/ST-GridPool.

[120] Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality? cs.AI | cs.CV | cs.CYPDF

Caixin Kang, Tianyu Yan, Sitong Gong, Mingfang Zhang, Liangyang Ouyang

TL;DR: 本文针对多模态大语言模型在人格感知任务中可能存在的偏见问题，提出了一个名为Grounded Personality Reasoning的新任务，并构建了MM-OCEAN数据集进行评测。研究发现，现有模型在给出正确人格评分时，往往缺乏基于可观察证据的推理，揭示了评分正确性与推理合理性之间的脱节。

Details

Motivation: 现有基准仅评估MLLMs在人格感知任务中的数值化大五人格得分预测能力，无法判断模型是通过行为理解真正感知人格，还是仅通过浅层模式匹配进行偏见性判断。

Result: 在包含27个MLLM的基准测试中，分析发现存在显著的偏见差距：整个领域内51%的正确评分并未基于检索到的线索，而整体接地率仅介于0-33.5%之间。

Insight: 创新点在于提出了结合评分、推理和证据接地的GPR任务框架，并设计了包含偏见率、虚构率等多维度细粒度评估指标，为MLLMs的接地社会认知研究提供了系统性的评估路线图。

Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in human-facing roles where personality perception is critical, yet existing benchmarks evaluate this capability solely on numerical Big Five score prediction, leaving open whether models truly perceive personality through behavioral understanding or merely prejudge through superficial pattern matching. We address this gap with three contributions. (i) A new task: we formalize Grounded Personality Reasoning (GPR), which requires MLLMs to anchor each Big Five rating in observable evidence through a chain of rating, reasoning, and grounding. (ii) A new dataset: we release MM-OCEAN (1,104 videos, 5,320 MCQs), produced by a multi-agent pipeline with human verification, with timestamped behavioral observations, evidence-grounded trait analyses, and seven categories of cue-grounding MCQs. (iii) Benchmark and analysis: we design a three-tier evaluation (rating, reasoning, grounding) plus four sample-level failure-mode metrics: Prejudice Rate (PR), Confabulation Rate (CR), Integration-failure Rate (IR), and Holistic-grounding Rate (HR), and benchmark 27 MLLMs (13 closed, 14 open). The analysis uncovers a striking Prejudice Gap: across the field, 51% of correct ratings are not grounded in retrieved cues, and the Holistic-Grounding Rate spans only 0-33.5%. These findings expose a disconnect between getting the right score and reasoning for the right reason, charting a roadmap for grounded social cognition in MLLMs.

[121] ST-SimDiff: Balancing Spatiotemporal Similarity and Difference for Efficient Video Understanding with MLLMs cs.AI | cs.CVPDF

Bingjun Luo, Tony Wang, Chaoqi Chen, Xinpeng Ding

TL;DR: 本文提出了一种名为ST-SimDiff的训练免费框架，旨在高效处理多模态大语言模型（MLLMs）中的长视频理解任务。该方法通过构建时空图来统一建模视觉标记的复杂关联，并采用并行的双选择策略：基于相似性的选择利用社区检测保留代表性标记以压缩静态信息，而基于时序差异的选择则精确定位内容变化点以保留捕捉关键动态转变的标记，从而以最少的标记数量同时保留静态和动态内容。

Details

Motivation: 现有方法主要通过基于重要性或相似性的剪枝或合并来减少冗余，以应对MLLMs处理长视频时因大量视觉标记带来的巨大计算开销，但这些方法忽视了视频内容的关键维度（如变化和转折点），且缺乏对时空关系的协同建模。

Result: 大量实验表明，该方法在显著降低计算成本的同时，性能大幅优于最先进的方法（SOTA）。

Insight: 创新点在于提出了“相似性用于识别冗余，差异性用于捕捉关键事件”的新视角，并设计了基于时空图的并行双选择策略，实现了对视频静态与动态内容的高效协同压缩，这是一种无需训练的高效框架设计思路。

Abstract: Multimodal Large Language Models (MLLMs) face significant computational overhead when processing long videos due to the massive number of visual tokens required. To improve efficiency, existing methods primarily reduce redundancy by pruning or merging tokens based on importance or similarity. However, these approaches largely overlook a critical dimension of video content, i.e., changes and turning points, and they lack a collaborative model for spatio-temporal relationships. To address this, we propose a new perspective: similarity is for identifying redundancy, while difference is for capturing key events. Based on this, we designed a training-free framework named ST-SimDiff. We first construct a spatio-temporal graph from the visual tokens to uniformly model their complex associations. Subsequently, we employ a parallel dual-selection strategy: 1) similarity-based selection uses community detection to retain representative tokens, compressing static information; 2) temporal difference-based selection precisely locates content-changing points to preserve tokens that capture key dynamic shifts. This allows it to preserve both static and dynamic content with a minimal number of tokens. Extensive experiments show our method significantly outperforms state-of-the-art approaches while substantially reducing computational costs. Our code is available in https://github.com/bingjunluo/ST-SimDiff.

[122] LACO: Adaptive Latent Communication for Collaborative Driving cs.AI | cs.CVPDF

Tianhao Chen, Yuheng Wu, Dongman Lee

TL;DR: 本文提出LACO，一种免训练的潜在通信范式，用于解决协作驾驶中语言通信带来的高延迟和信息损失问题。LACO通过迭代潜在协商、跨视野显著性归因和结构化语义知识蒸馏，使预训练驾驶模型能自适应协作场景。在CARLA仿真中的闭环实验表明，LACO显著降低了通信和推理延迟，同时保持了强大的协作驾驶性能。

Details

Motivation: 协作驾驶中，基于语言的通信虽直观但存在自回归解码导致的高延迟和内部表示压缩为离散令牌造成的信息损失两大挑战。多智能体设置下直接融合潜在状态会导致车辆间决策表示纠缠（智能体身份混淆），因此需要新的通信机制。

Result: 在CARLA仿真平台进行的闭环实验表明，LACO显著降低了通信和推理延迟，同时保持了强大的协作驾驶性能。

Insight: 创新点在于提出免训练的潜在通信范式，核心是迭代潜在协商实现潜在推理、跨视野显著性归因实现高效信息选择、结构化语义知识蒸馏稳定自我中心决策。从客观看，其将预训练模型自适应协作场景的思路和针对身份混淆的结构化设计具有借鉴意义。

Abstract: Collaborative driving aims to improve safety and efficiency by enabling connected vehicles to coordinate under partial observability. Recent approaches have evolved from sharing visual features for perception to exchanging language-based reasoning through foundation models for behavioral coordination. Though communicating in language provides intuitive information, it introduces two challenges: high latency caused by autoregressive decoding and information loss caused by compressing rich internal representations into discrete tokens. To address these challenges, we analyze latent communication in collaborative driving under inherent limitations of multi-agent settings. Our analysis reveals agent identity confusion, where direct fusion of latent states entangles decision representations across vehicles. Motivated by this, we propose LACO, a training-free \textbf{LA}tent \textbf{CO}mmunication paradigm that seamlessly adapts pretrained driving models to collaborative settings. LACO introduces Iterative Latent Deliberation (ILD) for latent reasoning, Cross-Horizon Saliency Attribution (CHSA) for communication-efficient information selection, and Structured Semantic Knowledge Distillation (SSKD) to stabilize ego-centric decision making. Closed-loop experiments in CARLA show that LACO notably reduces communication and inference latency while maintaining strong collaborative driving performance.

cs.LG [Back]

[123] Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation cs.LG | cs.AI | cs.CLPDF

Srujan P Mule, Aniketh Garikaparthi, Manasi Patwardhan

TL;DR: 本文研究了语言模型（LMs）在无需实际实验的情况下，预测研究想法在基准测试中相对成功的能力。通过构建基于PapersWithCode客观结果的数据集，研究发现监督微调（SFT）能显著提升模型性能，优于GPT-5等模型。此外，通过强化学习与可验证奖励（RLVR）框架将评估任务转化为推理任务，模型不仅能达到较高准确率，还能提供可解释的推理路径，展示了小模型作为高效、客观验证器的潜力。

Details

Motivation: 随着语言模型自动化生成科学假设，如何高效评估和筛选大量AI生成的想法成为新瓶颈。本文旨在探索语言模型是否能在实验前预测研究想法的实证成功，以加速自主科学发现。

Result: 在构建的11,488个想法对数据集上，现成的8B参数模型准确率仅30%，但监督微调（SFT）将准确率提升至77.1%，超过了GPT-5的61.1%。通过RLVR框架训练的模型达到71.35%准确率，并提供了可解释的推理。模型在分布外测试和跨领域时间分割测试集上表现出鲁棒性。

Insight: 创新点在于将研究想法评估定义为比较性实证预测任务，并利用强化学习与可验证奖励（RLVR）框架引导模型发现潜在推理路径，从而实现了高准确率与可解释性。这为小语言模型作为可扩展的客观验证器提供了新思路，有助于减少科学探索中的计算开销。

Abstract: As language models accelerate scientific research by automating hypothesis generation and implementation, a new bottleneck emerges: evaluating and filtering hundreds of AI-generated ideas without exhaustive experimentation. We ask whether LMs can learn to forecast the empirical success of research ideas before any experiments are run. We study comparative empirical forecasting: given a benchmark-specific research goal and two candidate ideas, predict which will achieve better benchmark performance. We construct a dataset of 11,488 idea pairs grounded in objective outcomes from PapersWithCode. While off-the-shelf 8B-parameter models struggle (30% acc.), SFT dramatically boosts performance to 77.1%, outperforming GPT-5 (61.1%). By framing evaluation as a reasoning task via Reinforcement Learning with Verifiable Rewards (RLVR), we train models to discover latent reasoning paths, achieving 71.35% acc. with interpretable justifications. Through additional ablations and out-of-distribution tests, we show robustness to surface-level heuristics and transfer to both a cross-domain time-split test set and an independently constructed test set. Our results demonstrate that compute-efficient small language models can serve as effective, objective verifiers, offering a scalable path for autonomous scientific discovery.

[124] HealthCraft: A Reinforcement Learning Safety Environment for Emergency Medicine cs.LG | cs.AI | cs.CLPDF

Brandon Dent

TL;DR: 本文提出了HealthCraft，这是首个用于评估语言模型在急诊医学场景下轨迹级安全性的公开强化学习环境。它基于FHIR R4标准构建，包含205个任务和2337个评估标准，并采用双层评分规则，一旦违反安全关键标准则奖励归零。在两个前沿模型上的测试结果显示，其在多步骤工作流上的性能近乎崩溃，凸显了现有静态基准的不足。

Details

Motivation: 当前前沿语言模型正被快速部署到临床工作流中，但评估其安全性的基础设施发展滞后。静态医学问答基准无法捕捉急诊医学中至关重要的失败模式，如轨迹级安全崩溃、工具误用以及在持续临床压力下的妥协。

Result: 在两个前沿模型上的V8版本测试结果显示，Claude Opus 4.6的Pass@1为24.8%，GPT-5.4为12.6%，安全失败率分别为27.5%和34.0%。在多步骤工作流（最接近真实急诊护理的代理任务）上，性能崩溃至接近零（Claude 1.0%， GPT-5.4 0.0%）。

Insight: 论文的核心创新在于构建了一个动态、交互式的强化学习环境来评估模型在复杂、高压急诊场景下的轨迹级安全性，这比静态QA基准更能反映真实风险。其双层评分规则（安全关键违规则奖励归零）和暴露大量工具（24个MCP工具）的设计，旨在模拟真实临床决策的约束和复杂性。研究还揭示了基础设施保真度本身是评估的一部分，并且奖励信号可能不适合直接用于训练，这为未来安全对齐研究提供了重要洞见。

Abstract: Frontier language models are being deployed into clinical workflows faster than the infrastructure to evaluate them safely. Static medical-QA benchmarks miss the failure modes that matter in emergency medicine: trajectory-level safety collapse, tool misuse, and capitulation under sustained clinical pressure. We present HealthCraft, the first public reinforcement-learning environment that rewards trajectory-level safety under realistic emergency-medicine conditions, adapted from Corecraft. It is built on a FHIR R4 world state with 14 entity types and 3,987 seed entities, exposes 24 MCP tools, and defines a dual-layer rubric that zeroes reward whenever any safety-critical criterion is violated. We release 195 tasks across six categories, graded against 2,255 binary criteria (515 safety-critical); a post-hoc 10-task negative-class slate extends this to 205 tasks and 2,337 criteria. V8 results on two frontier models show Claude Opus 4.6 at Pass@1 24.8% [21.5-28.4] and GPT-5.4 at 12.6% [10.2-15.6], with safety-failure rates of 27.5% and 34.0%. On multi-step workflows - the closest proxy to real emergency care - performance collapses to near zero (Claude 1.0%, GPT-5.4 0.0%) despite partial competence on individual steps. Six infrastructure bugs fixed between pilots v2 and v8 re-ordered which model “looks stronger,” evidence that infrastructure fidelity is part of the measurement. A deterministic LLM-judge overlay bounds evaluator noise, and a 60-run negative-class smoke pilot shows the reward signal is not drop-in training-safe: restraint criteria pass at 0.929 prevalence, a gameability an eval harness can tolerate but a training reward cannot. We scaffold coupling to a Megatron+SGLang+GRPO loop per Corecraft Section 5.2 and leave training-reward ablations as future work. Environment, tasks, rubrics, and harness are released under Apache 2.0.

[125] Value-Gradient Hypothesis of RL for LLMs cs.LG | cs.AI | cs.CLPDF

Arip Asadulaev, Daniil Ognev, Karim Salta, Martin Takac

TL;DR: 本文从价值梯度的角度分析了用于大语言模型后训练的免评论家强化学习方法（如PPO和GRPO）的工作机制。研究表明，在可微分展开和加性噪声参数化下，演员更新在期望上类似于价值梯度；对于离散Transformer策略，通过注意力机制的自动微分产生的经验协态近似于价值信号，误差由采样间隙和策略熵控制。

Details

Motivation: 旨在理解免评论家强化学习方法为何有效，以及何时能带来最大收益，从而为RL在大语言模型后训练中的应用提供理论依据。

Result: 理论分析表明，演员更新在期望上近似价值梯度，且通过注意力自动微分产生的经验协态能逼近价值信号，误差受控。

Insight: 提出了RL影响分解为价值梯度信号和可达奖励余量的框架，为判断RL在预训练轨迹中何时最有效提供了准则；揭示了免评论家RL方法通过近似价值梯度运作的机制，为算法设计和应用时机提供了新见解。

Abstract: Reinforcement learning substantially improves pretrained language models, but it remains understudied why critic-free methods such as PPO and GRPO work as well as they do, and when they should provide the largest gains. We develop a value-gradient perspective of critic-free RL for LLM post-training. First, under a differentiable rollout and additive-noise parameterization, we show that the actor update is value-gradient-like in expectation: the backward pass propagates costates whose conditional expectation equals the value gradient. Second, for discrete transformer policies, we show that autodifferentiation through attention produces empirical costates that approximate this value signal, with an error controlled by the sampling gap and policy entropy. These results motivate a decomposition of RL impact into value gradient signal and reachable reward headroom, yielding a criterion for when RL should be most effective along a pretraining trajectory.

[126] Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction cs.LG | cs.AI | cs.CLPDF

Alexander Smirnov

TL;DR: 该论文揭示了AI文本检测器并非学习区分AI与人类文本的边界，而是放大了预训练模型中的典型性轴。研究发现，在无监督的原始编码器上，通过投影到AI与人类文本质心差的方向，即可达到接近微调检测器的性能，甚至在某些情况下超过微调结果。论文还提出了一个封闭形式的雅可比预测器，能有效参数化操纵该轴的干预措施，并显著提升检测器性能。

Details

Motivation: 动机在于探究AI文本检测器的本质工作机制，挑战其通过微调学习AI与人类文本边界的传统观点，旨在证明检测器性能主要源于放大预训练模型中已有的典型性差异，而非学习新特征。

Result: 在多个架构（如RoBERTa-base）上，原始投影方法在NYT-vs-HC3数据集上达到AUROC 0.806至0.944，接近或超过微调检测器的性能上限（86-106%）。提出的雅可比预测器将ELECTRA-CE检测器在FPR=1%时的TPR从0.000提升至0.904，并在三个独立训练的第三方RoBERTa检测器上实现等效性能。

Insight: 核心创新点在于揭示了AI文本检测器的工作机制是放大预训练典型性轴，而非构建新边界，这通过多种操作上不同的探针（如几何消融、封闭形式预测器）的一致性得到验证。提出的雅可比预测器为操纵检测轴提供了可解释的数学工具，能有效提升和转移检测性能，对理解与改进文本检测方法具有重要启示。

Abstract: AI text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary. On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures (86-106% of the fine-tuned discrimination ceiling: on RoBERTa-base, raw projection exceeds fine-tuning); on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested. The same axis inverts on non-native ESL writing (AUROC 0.06-0.20) – a falsifiable prediction unique to the typicality reading. A 24-example frozen probe matches full fine-tuning (0.900 vs 0.895). A closed-form Jacobian predictor parameterises axis-manipulating interventions with R^2 = 1.000 universal, lifts ELECTRA-CE deployment TPR from 0.000 to 0.904 at FPR = 1%, and transfers to three independently-trained third-party RoBERTa detectors at 16/16 oracle-equivalence (57% NYT-FPR reduction on the OpenAI detector). Scope: encoder family; mechanism magnitude HC3-anchored; population-level shared axis with per-text mechanisms varying across architectures. Three operationally distinct probes – text-surface caps_rate residualisation, geometric signed-epsilon ablation, closed-form text-pair predictor – agree at cos 0.74/0.81/1.00 across three architectures, confirming observer-invariance. Under matched-TPR-0.90 evaluation, the published intervention zoo (CC, dealign-f2c) is calibration-equivalent across 27 cells (|Delta AUROC| <= 0.0081), and >= 97% of the LoRA->full-FT bias gap on ELECTRA is calibration shift, not learned representation – the central claim’s prediction confirmed.

[127] X-Token: Projection-Guided Cross-Tokenizer Knowledge Distillation cs.LG | cs.CLPDF

Sharath Turuvekere Sreenivas, Adithyakrishna Venkatesh Hanasoge, Mingyu Yang, Ali Taghibakhshi, Saurav Muralidharan

TL;DR: 本文提出X-Token方法，用于解决跨分词器知识蒸馏中基于完整输出分布的方法存在的两个关键缺陷：罕见词失败和过度保守匹配。该方法包含两种互补的损失函数P-KL和H-KL，通过稀疏投影矩阵W实现学生模型与教师模型分布的对齐，支持多教师蒸馏，在Llama-3.2-1B模型上取得了优于现有方法GOLD的效果。

Details

Motivation: 解决跨分词器知识蒸馏中，基于完整输出分布的logit方法存在的两个核心问题：一是罕见词失败，即关键标记因不匹配而被抑制；二是过度保守匹配，即严格的1对1匹配排除了表面形式相近的等价标记。

Result: 在Llama-3.2-1B模型上，使用Qwen3-4B教师时，X-Token比当前SOTA方法GOLD平均提升3.82分；使用Phi-4-Mini教师时提升0.5分。双教师设置（Phi-4-mini + Llama-3B）比单教师蒸馏进一步提升1.3分。

Insight: 创新点在于提出两种互补的损失函数（P-KL和H-KL）分别针对不同对齐场景，通过共享的稀疏投影矩阵W实现灵活匹配，既能处理罕见词错位问题，又能利用可靠的标记对齐关系，且天然支持多教师蒸馏。

Abstract: Cross-tokenizer knowledge distillation allows a student model to learn from teachers with incompatible vocabularies. Prior work operates on hidden states or logits; the latter is preferred as a drop-in replacement requiring no auxiliary components. Logit-based methods either use only the correct-token probability, missing the full ‘dark knowledge’ in the teacher’s distribution, or operate on the full output distribution, relying on strict token partitioning and/or unprincipled heuristic ranking. We identify two key shortcomings of full-distribution, logit-based methods: (i) an uncommon-token failure, where critical tokens fall into the unmatched subset (e.g., Llama’s 1100 multi-digit numerals under digit-splitting Qwen supervision) and are suppressed during training, reducing GSM8k from 12.89 to 2.56 compared to same-tokenizer KD from a weaker teacher; and (ii) over-conservative matching, where strict 1-to-1 matching excludes near-equivalent tokens across surface forms. These failures require distinct remedies: eliminating the partition when critical tokens are misaligned, and refining it when alignment is reliable. We propose X-Token, an approach with two complementary loss formulations targeting these issues. P-KL removes partitioning and aligns the student’s distribution with the teacher’s via a sparse projection matrix W (initialized from tokenizer-level string rules) to address the uncommon-token failure. H-KL retains the hybrid form while relaxing matching to align each student token with its top-ranked teacher mapping under W. Both objectives share W and extend naturally to multiple teachers. Empirically, on Llama-3.2-1B, X-Token outperforms the current state of the art GOLD by +3.82 average points with a Qwen3-4B teacher and by +0.5 with a Phi-4-Mini teacher. Further, a two-teacher setup (Phi-4-mini + Llama-3B) improves over single-teacher distillation by +1.3 points.

[128] Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimization cs.LG | cs.CLPDF

Zheyuan Zhang, Kaiwen Shi, Han Bao, Zehong Wang, Tianyi Ma

TL;DR: 本文针对大型语言模型后训练中基于不确定性的优化方法（如GRPO）存在的问题，提出了一个理论框架，将不确定性信号解释为表征和调节梯度方差与学习信号质量的机制。通过实证和理论分析，作者指出了当前基于熵的估计器存在的两个关键缺陷：各向异性差距和校准差距。为此，他们提出了几何感知校准策略优化（GCPO）框架，该框架结合了几何感知度量来捕捉语义分歧，并利用基于奖励的校准使不确定性与学习信号强度对齐。

Details

Motivation: 动机在于解决后训练中，无评论家模型虽然能从模型生成输出中进行可扩展学习，但缺乏区分信息性信号与噪声信号的原则性机制的问题。现有利用响应级度量作为不确定性信号来调节基于组的优化方法（如GRPO）的经验成功不稳定，且其如何影响优化动态尚不清楚。

Result: 在多个基准测试上的实验表明，GCPO方法能更准确地追踪梯度变异性，并持续提升后训练性能。

Insight: 创新点在于首次从原理上阐述了不确定性信号作为调节梯度方差和学习信号质量机制的作用，并具体提出了解决各向异性和校准差距的几何感知校准策略优化（GCPO）框架。其核心见解是设计的不确定性信号必须与优化动态对齐，这为鲁棒的后训练提供了一个原则性视角。

Abstract: Post-training has become central to improving reasoning and alignment in large language models, where critic-free models enable scalable learning from model-generated outputs but lack principled mechanisms to distinguish informative from noisy signals. Recent approaches leverage response-level measures as uncertainty signals to regulate group-based optimization methods such as GRPO. Yet their empirical success remains unstable and unclear in how they influence optimization dynamics. In this paper, we provide, to our knowledge, the first principled formulation that interprets uncertainty signals as mechanisms for characterizing and regulating gradient variance and learning signal quality. Based on both empirical and theoretical analysis, we identify two critical gaps of current entropy-based estimators: The anisotropic gap and The calibration gap. Motivated by this analysis, we propose Geometric-aware Calibrated Policy Optimization (GCPO), a novel framework integrating geometry-aware measures to capture semantic disagreement with reward-based calibration to align uncertainty with learning signal strength. Experiments on multiple benchmarks show that our approach more faithfully tracks gradient variability and consistently improves post-training performance. Our results highlight the importance of designing uncertainty signals that are aligned with optimization dynamics, offering a principled perspective for robust post-training.

[129] From Reasoning Chains to Verifiable Subproblems: Curriculum Reinforcement Learning Enables Credit Assignment for LLM Reasoning cs.LG | cs.AI | cs.CLPDF

Xitai Jiang, Zihan Tang, Wenze Lin, Yang Yue, Shenzhi Wang

TL;DR: 本文提出了一种名为SCRL（子问题课程强化学习）的课程强化学习框架，用于提升大语言模型在数学推理任务上的性能。该方法通过将参考推理链分解为可验证的子问题，并将原始问题作为最终子问题，从而将困难问题上的部分进展转化为可验证的学习信号。

Details

Motivation: 解决基于可验证奖励的强化学习在困难推理问题上效率低下的问题，因为正确答案的轨迹稀少，且样本级信用分配无法利用失败尝试中的部分进展。

Result: 在七个数学推理基准测试上，SCRL优于强课程学习基线，在Qwen3-4B-Base上平均准确率比GRPO提升4.1个百分点，在Qwen3-14B-Base上提升1.9个百分点。在AIME24、AIME25和IMO-Bench上，SCRL进一步提升了Qwen3-4B-Base的pass@1和pass@64指标，表明其在困难推理问题上具有更好的探索能力。

Insight: 核心创新在于通过构建子问题课程，将困难问题分解为可验证的子目标，从而将部分进展转化为有效的学习信号。算法上采用了子问题级归一化，独立归一化每个子问题位置的奖励，并将优势分配给对应的答案片段，实现了无需外部评分标准或奖励模型的细粒度信用分配。

Abstract: Reinforcement learning from verifiable rewards (RLVR) has shown strong promise for LLM reasoning, but outcome-based RLVR remains inefficient on hard problems because correct final-answer rollouts are rare and sample-level credit assignment cannot use partial progress in failed attempts. We introduce SCRL (Subproblem Curriculum Reinforcement Learning), a curriculum RL framework that derives verifiable subproblems from reference reasoning chains and fixes the final subproblem as the original problem. This turns partial progress on hard problems into verifiable learning signals. Algorithmically, SCRL uses subproblem-level normalization, which normalizes rewards independently at each subproblem position and assigns the resulting advantages to the corresponding answer spans, enabling finer-grained credit assignment without external rubrics or reward models. Our analysis shows that subproblem curricula lift hard problems out of gradient dead zones, with larger relative gains as the original problem becomes harder. Across seven mathematical reasoning benchmarks, SCRL outperforms strong curriculum-learning baselines, improving average accuracy over GRPO by +4.1 points on Qwen3-4B-Base and +1.9 points on Qwen3-14B-Base. On AIME24, AIME25, and IMO-Bench, SCRL further improves pass@1 by +3.7 points and pass@64 by +4.6 points on Qwen3-4B-Base, indicating better exploration on hard reasoning problems.

[130] Maestro: Reinforcement Learning to Orchestrate Hierarchical Model-Skill Ensembles cs.LG | cs.CLPDF

Jinyang Wu, Guocheng Zhai, Ruihan Jin, Yuhao Shen, Zhengxi Lu

TL;DR: 本文提出了Maestro，一个基于强化学习的编排框架，用于动态组合异构的专家模型和技能库，以解决多模态任务。该框架将任务视为对分层模型-技能注册表的顺序决策过程，训练一个轻量级策略来调用外部专家、选择模型-技能对并决定终止时机。

Details

Motivation: 现有框架通常依赖单一的大型语言模型和固定逻辑来调用技能，无法充分利用不同模型和技能在不同领域的互补优势，从而限制了在下游任务上的性能。

Result: 在涵盖数学推理、图表理解、高分辨率感知和领域特定分析等十个代表性多模态基准测试中，仅使用4B参数的编排器，Maestro实现了70.1%的平均准确率，超过了GPT-5（69.3%）和Gemini-2.5-Pro（68.7%）。其学习到的协调策略能够泛化到未见过的模型和技能，在四个具有挑战性的基准上，通过添加领域外专家实现了59.5%的平均准确率，优于所有闭源基线模型。

Insight: 核心创新在于将异构多模态任务重新定义为对分层模型-技能注册表的顺序决策问题，并通过基于结果的强化学习优化一个轻量级编排策略，无需逐步监督。这实现了对冻结专家模型和技能库的动态、自适应组合，有效利用了模型和技能的互补优势，并展示了良好的泛化能力和计算效率。

Abstract: The proliferation of large language models (LLMs) and modular skills has endowed autonomous agents with increasingly powerful capabilities. Existing frameworks typically rely on monolithic LLMs and fixed logic to interface with these skills. This gives rise to a critical bottleneck: different LLMs offer distinct advantages across diverse domains, yet current frameworks fail to exploit the complementary strengths of models and skills, thereby limiting their performance on downstream tasks. In this paper, we present Maestro (Multimodal Agent for Expert-Skill Targeted Reinforced Orchestration), a Reinforcement Learning (RL)-driven orchestration framework that reframes heterogeneous multimodal tasks as a sequential decision-making process over a hierarchical model-skill registry. Rather than consolidating all knowledge into a single model, Maestro trains a lightweight policy to dynamically compose ensembles of frozen expert models and a two-tier skill library, deciding at each step whether to invoke an external expert, which model-skill pair to select, and when to terminate. The policy is optimized via outcome-based RL, requiring no step-level supervision. We evaluate Maestro across ten representative multimodal benchmarks spanning mathematical reasoning, chart understanding, high-resolution perception, and domain-specific analysis. With only a 4B orchestrator, Maestro achieves an average accuracy of 70.1%, surpassing both GPT-5 (69.3%) and Gemini-2.5-Pro (68.7%). Crucially, the learned coordination policy generalizes to unseen models and skills without retraining: augmenting the registry with out-of-domain experts yields a 59.5% average on four challenging benchmarks, outperforming all closed-source baselines. Maestro further maintains high computational efficiency with low latency. The source code is available at https://github.com/jinyangwu/Maestro.

[131] Survive or Collapse: The Asymmetric Roles of Data Gating and Reward Grounding in Self-Play RL cs.LG | cs.CLPDF

Sophia Xiao Pu, Zhaotian Weng, Chengzhi Liu, Jayanth Srinivasa, Gaowen Liu

TL;DR: 本文研究了自博弈强化学习中数据门控和奖励信号对训练稳定性的不对称影响。通过控制实验发现，严格的数据门控足以保证稳定性，而奖励设计在缺乏门控时无法防止崩溃。研究揭示了‘接地提议者悖论’，即拥有真实答案的提议者反而会加速崩溃。

Details

Motivation: 自博弈强化学习在语言模型训练中表现出强推理能力，但普遍存在崩溃和不稳定问题。现有研究多将其归因于奖励设计问题，而本文主张稳定性由数据门控和奖励信号两个独立杠杆共同决定。

Result: 在Python输出预测任务和确定性DSL孪生任务上的实验表明，严格的数据门控在所有测试的奖励变体（包括无真实答案的自一致性奖励）下都能保持稳定；而一旦移除门控，任何奖励变体都无法防止崩溃。进一步发现，通过连续严格度参数ε揭示了训练指标与验证准确度的两阶段相变。

Insight: 创新点在于揭示了数据门控（而非奖励校准）是自博弈稳定性的关键约束，并提出了‘接地提议者悖论’这一反直觉耦合现象。从客观角度看，研究强调了在自博弈系统中数据筛选机制的基础性作用，为设计稳定训练框架提供了新视角。

Abstract: Self-play reinforcement learning trains language models on their own generated tasks, co-evolving a proposer and solver without human labels. Recent systems report strong reasoning gains, but collapse and instability are widely observed and poorly understood. The dominant response treats this as a reward-design problem. We argue instead that self-play stability is governed by two distinct levers: a data-level gate that decides which proposer-generated tasks enter the training pool, and the reward signal that updates the policy on tasks already admitted. Through controlled experiments on a Python output-prediction task and a deterministic-DSL twin task that strips pretraining priors, output ambiguity, and executor noise, we find the two levers are asymmetric. A strict gate is sufficient for stability under every reward variant we test, including a self-consistency reward with no access to ground truth; while no reward variant is sufficient once the gate is removed. This asymmetry exposes a counter-intuitive coupling we call the Grounded Proposer Paradox: a proposer with ground-truth access accelerates collapse faster than an ungrounded one when paired with a self-consistency solver, by concentrating training on clean tasks that form the fastest path to a spurious self-consistent attractor. Replacing the binary gate with a continuous strictness parameter $\varepsilon$ further reveals a two-stage phase transition: training-side metrics decouple at low $\varepsilon$, while validation accuracy holds until $\varepsilon$ is much higher. Data-level gating, not reward calibration, is the binding constraint on self-play stability.

[132] Two is better than one: A Collapse-free Multi-Reward RLIF Training Framework cs.LG | cs.CLPDF

Shourov Joarder, Diganta Sikdar, Ahsan Habib Akash, Binod Bhattarai, Prashnna Gyawali

TL;DR: 本文提出了一种多奖励内部反馈强化学习框架，通过分解训练信号为答案级和完成级两个互补组件，结合GDPO归一化和KL-Cov正则化，解决了单奖励RLIF方法中常见的奖励黑客、熵崩溃和推理结构退化问题。

Details

Motivation: 现有基于内部反馈的强化学习方法通常依赖单一内部奖励，容易导致奖励黑客、熵崩溃和推理结构退化，因此需要一种更稳定、无需外部监督的替代方案。

Result: 在数学推理和代码生成基准测试中，该方法相比先前的无监督RL方法提高了稳定性和鲁棒性，性能接近有监督的RLVR方法。

Insight: 创新点在于将内部奖励分解为基于集群投票的答案级奖励和基于逐令牌自确定性的完成级奖励，并引入GDPO归一化和KL-Cov正则化来防止熵崩溃，从而支持稳定的长程推理而无需外部真值监督。

Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved the reasoning ability of LLMs, but often depends on external supervision from human annotations or gold-standard solutions. Reinforcement learning from internal feedback (RLIF) has recently emerged as a scalable unsupervised alternative, using signals extracted from the model itself. However, existing RLIF methods typically rely on a single internal reward, which can lead to reward hacking, entropy collapse, and degraded reasoning structure. We propose a multi-reward RLIF framework that decomposes the training signal into two complementary components: an answer-level reward based on cluster voting and a completion-level reward based on token-wise self-certainty. To combine these signals robustly, we apply GDPO-based normalization to reduce reward-scale imbalance. We further introduce KL-Cov regularization, which targets low-entropy token distributions responsible for disproportionate entropy reduction, preserving exploration and preventing late-stage collapse. Across mathematical reasoning and code-generation benchmarks, our method improves stability and robustness over prior unsupervised RL approaches, while achieving performance close to supervised RLVR methods. These results show that complementary internal rewards, combined with targeted regularization, can support stable long-horizon reasoning without relying on external ground-truth supervision. Code will be released soon.

[133] The Double Dilemma in Multi-Task Radiology Report Generation: A Gradient Dynamics Analysis and Solution cs.LG | cs.CL | cs.CVPDF

Erjian Zhang, Yatong Hao, Liejun Wang, Zhiqing Guo

TL;DR: 本文针对多任务学习的放射学报告自动生成（RRG）中线性标量化策略的局限性，从梯度动力学角度分析了其失败机制，并提出了一种与主干网络无关的优化器CAME-Grad。该优化器通过冲突规避的方向校正和幅度增强的能量注入，解决了梯度漂移项偏差和扩散项衰减的“双重困境”，并在多个RRG方法上带来了显著且一致的性能提升。

Details

Motivation: 当前多任务RRG方法大多采用粗糙的线性标量化策略，无法有效平衡判别性临床监督的硬约束与报告生成的平滑性要求，导致性能受限。

Result: 实验表明，CAME-Grad作为通用即插即用优化器，在八种不同的RRG方法上均带来实质性改进，在MIMIC-CXR和IU X-Ray数据集上平均分别提升了2.3%和1.9%的整体临床效能。

Insight: 创新点在于从随机微分方程（SDE）框架分析梯度动态，揭示了线性标量化的“双重困境”机制，并设计了冲突规避和幅度增强的优化策略，以及自适应梯度融合机制来平衡理论最优方向与任务特定归纳偏差，为多任务优化提供了新视角和通用解决方案。

Abstract: While multi-task learning based automatic radiology report generation (RRG) is widely adopted to ensure clinical consistency, most focus on architectural designs yet remain limited to coarse linear scalarization strategies. These strategies cannot effectively balance the hard constraints of discriminative clinical supervision with the smoothness requirements of report generation. To address these problems, we analyze the failure mechanism of linear scalarization from the perspective of gradient dynamics, utilizing the stochastic differential equation (SDE) framework to characterize it as a “Double Dilemma” of drift term deviation and diffusion term decay. Based on this, we propose a backbone-agnostic optimizer named Conflict-Averse Magnitude-Enhanced Gradient Descent (CAME-Grad). Through conflict-averse direction rectification and magnitude-enhanced energy injection, the algorithm not only ensures geometric validity, but also avoids local optimal solutions. Then, the adaptive gradient fusion mechanism is used to establish a dynamic balance between the theoretical optimal direction and the task-specific inductive bias. Experiments show that as a universal plug-and-play optimizer, CAME-Grad brings substantial and consistent improvements across eight diverse RRG methods, elevating overall clinical efficacy performance by an average of 2.3% on MIMIC-CXR and 1.9% on IU X-Ray. Our code is available at https://github.com/vpsg-research/CAME-Grad.

cs.RO [Back]

[134] Imagine2Real: Towards Zero-shot Humanoid-Object Interaction via Video Generative Priors cs.RO | cs.CVPDF

Jiahe Chen, ZiRui Wang, Feiyu Jia, Xiao Chen, Xiaojie Niu

TL;DR: Imagine2Real是一个零样本全身人形机器人-物体交互框架，旨在解决高质量3D数据稀缺的问题。它通过将机器人和物体运动统一建模为4D点轨迹，并利用视频生成先验，仅跟踪稀疏关键点，从而绕过了复杂的重定向过程。该框架利用行为基础模型的潜在空间作为搜索域，通过渐进式训练实现零样本物理部署。

Details

Motivation: 动机是解决全身人形机器人-物体交互中高保真3D数据稀缺的瓶颈，以及现有方法因依赖几何先验导致的表示错位和因密集形变与形态不匹配带来的重定向复杂性。

Result: 论文提出的方法在运动捕捉系统中实现了零样本物理部署，通过简单的跟踪奖励学习到了鲁棒的行为，但摘要中未提及具体的定量基准测试结果或与SOTA的比较。

Insight: 创新点包括将机器人和物体运动统一为4D点轨迹以解决表示错位，仅跟踪稀疏关键点（底座、手和物体）以避免重定向误差，以及利用行为基础模型的潜在空间来维持自然步态。从客观角度看，其几何无关的交互框架和渐进式训练策略具有借鉴意义。

Abstract: Whole-body Humanoid-Object Interaction (HOI) is bottlenecked by the scarcity of high-fidelity 3D data. While video generative priors offer a promising alternative, existing methods suffer from \textit{Representation Misalignment} due to their reliance on geometric priors (e.g., explicit CAD models), and \textit{Retargeting Complexity} arising from intensive morphing and morphological mismatch. We propose Imagine2Real, a zero-shot HOI framework for flexible, geometry-free interaction. To resolve misalignment, we formulate robot and object motions as unified 4D point trajectories. To overcome retargeting complexity, our Keypoints Tracker tracks only sparse critical points (base, hands, and object), entirely bypassing the error-amplifying retargeting process. To maintain natural gaits despite these sparse signals, we utilize the latent space of a Behavior Foundation Model (BFM) as the tracker’s search domain. Using a progressive training strategy, Imagine2Real learns robust behaviors with simple tracking rewards, enabling zero-shot physical deployment within a motion capture(mocap) system.

[135] Decoupling Ego-Motion from Target Dynamics via Dual-Interval Motion Cues for UAV Detection cs.RO | cs.CVPDF

Liuyang Wang, Feitian Zhang

TL;DR: 本文提出了一种仅基于视觉的运动引导检测框架，用于无人机视频中的目标检测。该框架通过全局运动补偿对齐相邻帧，并采用双区间运动提取策略捕获短期和长期运动线索，再通过轻量级运动引导注意力模块增强特征表示，以解决无人机检测中由严重自运动、相机抖动和大尺度变化带来的挑战。

Details

Motivation: 解决无人机视频目标检测中因严重自运动、相机抖动和大尺度变化导致的性能下降问题，特别是针对动态场景中的小目标检测。

Result: 在VisDrone-VID数据集上的实验表明，该方法在严重自运动条件下相比强大的YOLOv8基线模型取得了持续的性能提升。消融研究进一步证实了双区间设计和运动引导注意力机制的有效性。

Insight: 创新点在于通过全局运动补偿和双区间运动提取策略解耦目标运动与相机扰动，并利用轻量级运动引导注意力模块整合多尺度运动线索，为动态视频中的鲁棒目标检测提供了新思路。

Abstract: Object detection from Unmanned Aerial Vehicles (UAVs) is challenged by severe ego-motion, camera jitter, and large scale variations. While modern detectors perform well on static images, their direct application to UAV video often fails, particularly for small objects in dynamic scenes. Existing motion-based methods either rely on computationally expensive optical flow or use single-interval differencing, which is sensitive to jitter and limited in capturing diverse motion patterns. We propose a vision-only motion-guided detection framework that decouples target motion from camera-induced disturbances. A homography-based Global Motion Compensation (GMC) first aligns adjacent frames. We then introduce a Dual-Interval Motion Extraction strategy that captures both short-term and long-term motion cues. To integrate these cues, a lightweight Motion-Guided Attention (MGA) module enhances feature representations within a Feature Pyramid Network. Experiments on the VisDrone-VID dataset demonstrate consistent improvements over a strong YOLOv8 baseline under severe ego-motion. Ablation studies further confirm the effectiveness of the dual-interval design and the proposed motion-guided attention mechanism.

[136] GesVLA: Gesture-Aware Vision-Language-Action Model Embedded Representations cs.RO | cs.CVPDF

Wenxuan Guo, Ziyuan Li, Meng Zhang, Yichen Liu, Yimeng Dong

TL;DR: 本文提出GesVLA模型，通过引入手势作为并行指令模态，来解决现有视觉-语言-动作模型在复杂场景中因空间模糊性（如存在多个相似物体）而导致的指令理解困难问题。该模型将手势特征编码到潜在空间，使其参与高层推理和低层动作生成，并采用双视觉语言模型架构实现手势表征与动作策略的紧密耦合。

Details

Motivation: 现有VLA模型主要依赖文本指令，在具有多个相似物体的复杂场景中难以解决空间模糊性问题，限制了机器人操作的精确性。

Result: 在多个真实世界机器人任务（包括受控的积木操作任务以及更实用的产品和农产品选择场景）上的实验结果表明，引入手势指令能持续提高目标定位准确性和人机交互效率，尤其是在复杂和杂乱的环境中。

Insight: 创新点在于将手势作为一种新的指令模态与VLA模型深度融合，提出了将手势特征直接编码到潜在空间的双VLM架构，并设计了一个通过将手部模型渲染到真实场景图像上来生成可扩展手势数据的流水线，以减小模拟到真实的视觉差距。

Abstract: Vision-Language-Action (VLA) models have shown strong potential for general-purpose robot manipulation by unifying perception and action. However, existing VLA systems primarily rely on textual instructions and struggle to resolve spatial ambiguity in complex scenes with multiple similar objects. To address this limitation, we introduce gesture as a parallel instruction modality and propose a Gesture-aware Vision-Language-Action model (GesVLA). Our approach encodes gesture features directly into the latent space, enabling them to participate in both high-level reasoning and low-level action generation, and adopts a dual-VLM architecture to achieve tight coupling between gesture representations and action policies. At the data level, we construct a scalable gesture data generation pipeline by rendering hand models onto real-world scene images. This reduces the sim-to-real visual gap while producing rich data with diverse motion patterns and corresponding pointing annotations. In addition, we employ a two-stage training strategy to equip the model with both gesture perception and action prediction capabilities. We evaluate our approach on multiple real-world robotic tasks, including a controlled block manipulation task for validation and more practical scenarios such as product and produce selection. Experimental results show that incorporating gesture consistently improves target grounding accuracy and human-robot interaction efficiency, especially in complex and cluttered environments. Project page: https://gwxuan.github.io/GesVLA/.

Wenxuan Guo, Xiuwei Xu, Yichen Liu, Xiangyu Li, Hang Yin

TL;DR: 本文提出AwareVLN，一种用于视觉语言导航（VLN）的新框架，旨在通过引入自我感知推理机制，让智能体以端到端和数据驱动的方式理解自身状态与任务进度，从而弥合了现有基于视觉语言模型（VLM）的端到端方法与显式启发式规划方法之间的差距。

Details

Motivation: 当前最先进的VLN方法虽然利用视觉语言模型（VLM）进行端到端动作预测，但缺乏对智能体、指令和场景之间关系的显式且可解释的理解；而显式构建场景图进行启发式规划则依赖额外3D传感器且不利于大规模视觉语言预训练。

Result: 在Habitat模拟器的多个数据集上进行的大量实验表明，AwareVLN显著优于之前最先进的视觉语言导航方法。

Insight: 主要创新点包括：1）一个促进空间和任务导向自我感知的结构化推理模块；2）一个带有进度划分的自动数据引擎，用于有效训练。这为VLN任务提供了一种兼具可解释性和端到端学习能力的新思路。

Abstract: Vision-and-Language Navigation (VLN) requires an agent to ground language instructions to its own movement within a visual environment. While state-of-the-art methods leverage the reasoning capabilities of Vision-Language Models (VLMs) for end-to-end action prediction, they often lack an explicit and explainable understanding of the relationships between the agent, the instruction, and the scene. Conversely, explicitly building a scene map for heuristic planning is intuitively appealing but relies on additional 3D sensors and hinders large-scale vision-language pre-training. To bridge this gap, we propose AwareVLN, a novel framework that equips the navigation model with a self-aware reasoning mechanism, enabling it to understand the agent’s state and task progress in a fully end-to-end and data-driven manner. Our approach features two key innovations: (1) a structural reasoning module that fosters spatial and task-oriented self-awareness, and (2) an automatic data engine with progress division for effective training. Extensive experiments on various datasets in Habitat simulator show our AwareVLN significantly outperforms previous state-of-the-art vision-language navigation methods. Project page: https://gwxuan.github.io/AwareVLN/.

eess.IV [Back]

[138] A Task-Agnostic Algebraic Integrity Metric for Event-Camera Streams Toward SOTIF-Compliant Perception using Pearson Correlation Coefficient eess.IV | cs.CVPDF

Arthur de Miranda Neto

TL;DR: 本文提出了一种任务无关的代数完整性度量框架，用于直接评估事件相机异步事件流的质量。该框架将皮尔逊相关系数（PCC）推广到时间表面、事件帧和体素网格三种标准事件表示上，生成了三种度量：用于流完整性监控的r-TS、用于自适应ROI选择的r2-EF以及用于时间冗余门控的r-VG。

Details

Motivation: 事件相机在自动驾驶系统中具有高带宽、低延迟等优势，但目前缺乏直接作用于异步事件流的任务无关质量度量方法。现有方法依赖下游任务性能评估，不符合ISO 21448（SOTIF）等安全标准的认证要求，BiasBench基准也明确指出了这一空白。

Result: 在程序合成的模拟事件流上演示了每种度量的行为，例如在隧道下降完整性异常场景中，r_C值从0.93（相干流）降至0以下（触发警报），验证了其异常检测能力。

Insight: 创新点在于建立了事件相机对比度阈值机制与基于PCC的变化准则之间的结构同构，并形式化了三种可直接在事件流上操作的代数度量，为符合SOTIF的安全感知提供了无需下游任务评估的完整性监控工具。

Abstract: Event cameras have emerged as a high-bandwidth, low-latency sensing modality for safety-critical perception in automated driving systems (ADS), offering microsecond temporal resolution, 120-140 dB dynamic range, and intrinsic absence of motion blur. However, no task-agnostic quality metric currently operates directly on the asynchronous event stream: state-of-the-art proxies require a downstream task (e.g., detection accuracy, tracking error) to assess stream integrity, which is incompatible with the certification requirements of ISO 21448 (SOTIF) and ISO/PAS 8800:2024. The recent BiasBench benchmark (CVPR 2025) explicitly identifies this gap. This work proposes a unified algebraic framework that lifts the Pearson Correlation Coefficient (PCC), historically used in two prior works for redundancy filtering and ROI selection on frame-based images, to the three standard event representations: Time Surface, Event Frame, and Voxel Grid. The framework yields three metrics: (i) r-TS for stream integrity monitoring against an ego-motion-predicted Time Surface, (ii) r2-EF for adaptive ROI selection requiring only integer comparisons, and (iii) r-VG for temporal redundancy gating. A structural isomorphism is established between the contrast-threshold mechanism of the event camera (|Delta L| >= C) and the PCC-based change criterion, the three lifted metrics are formalized, and pipeline latency and information loss are analyzed symmetrically against the raw stream. Illustrative behavior of each metric is demonstrated on a procedural-synthetic event stream, generated by direct simulation of the emission model rather than drawn from any real or video-derived dataset, including a tunnel-dip integrity-anomaly scenario in which r_C drops from 0.93 (coherent flow) to below 0 (alarm). An explicit epistemic convention ([ESTABLISHED], [SOLID], [HYPOTH.], [OPEN]) delineates the status of every contribution.

Farzaneh Barzegar, Tobias Bolch, Norbert Kuehtreiber, Silvia L. Ullo

TL;DR: 本文提出了一个名为CryoNet的深度学习框架，用于多模态的冰碛覆盖冰川制图。该框架融合了Sentinel-2光学影像、DEM地形变量、光谱指数、PCA、InSAR相干性与相位、缨帽特征和GLCM纹理等多种数据，通过一个基于ResNet101编码器并带有嵌套跳跃连接和空间-通道注意力（scSE）的编码器-解码器CNN，来区分洁净冰、冰碛覆盖冰川和冰湖。研究以喜马拉雅中部的波曲流域为案例，并评估了模型向阿尔卑斯山勃朗峰地区的可迁移性。

Details

Motivation: 冰川是重要的淡水储备和气候变化指标，但其自动描绘，特别是冰碛覆盖冰川，由于与周围地形光谱相似而极具挑战。本研究旨在解决这一难题，实现稳健的冰川测绘。

Result: 在喜马拉雅中部波曲流域的案例研究中，CryoNet取得了总体IoU 90.52%、平均召回率98.08%和平均精确率92.26%的优异结果。针对冰碛覆盖冰川，其IoU达到90.46%，召回率95.79%，精确率94.21%。在各类指标上，CryoNet均超越了作为SOTA参考的DeepLabV3+、SegFormer和U-Net模型。

Insight: 论文宣称的创新点在于提出了一个综合利用丰富多模态数据的深度学习框架CryoNet，其网络结构结合了嵌套跳跃连接和空间-通道注意力机制以增强特征提取。从客观角度看，其系统性的多模态数据融合策略、针对特定地物（冰碛覆盖冰川）的精细分类能力，以及对模型可迁移性的评估，都是值得借鉴的研究思路。

Abstract: Glaciers play a critical role as freshwater reserves and indicators of climate change, yet their automatic delineation, especially for debris-covered glaciers, remains challenging due to spectral similarity with surrounding terrain. This study introduces CryoNet, a deep learning framework that leverages a rich multi-modal dataset combining Sentinel-2 optical imagery, DEM-derived topographic variables, spectral indices, Principal Component Analysis (PCA), InSAR coherence and phase, tasseled-cap features, and GLCM texture to discriminate clean-ice glaciers, debris-covered glaciers, and glacial lakes. CryoNet is an encoder-decoder CNN with nested skip connections and spatial-channel Squeeze-and-Excitation (scSE) attention, built upon a ResNet101 encoder to capture hierarchical contextual and spatial features. The study is conducted in the Poiqu Basin in the central Himalaya, and transferability is evaluated by applying the trained model to the Mont Blanc Massif in the Alps. We additionally analyse the importance of each data layer in improving glacier mapping performance. The proposed model achieves an overall IoU of 90.52%, mean Recall of 98.08%, and mean Precision of 92.26%. For debris-covered glaciers specifically, CryoNet obtains an IoU of 90.46%, a recall of 95.79%, and a precision of 94.21%. Across both per-class and overall metrics, CryoNet surpasses DeepLabV3+, SegFormer, and U-Net, taken as state-of-the-art (SOTA) references, demonstrating its effectiveness for robust glacier mapping in complex high-mountain environments.

[140] Time-varying rPPG signal separation via block-sparse signal model eess.IV | cs.CVPDF

Kosuke Kurihara, Yoshihiro Maeda, Daisuke Sugimura, Takayuki Hamamoto

TL;DR: 本文提出了一种基于块稀疏信号模型的时变远程光电容积描记（rPPG）信号分离方法，用于从面部视频中提取心率信号。该方法利用rPPG信号的准周期性特征，将其建模为时频域中的块稀疏结构，并构建了一个时变信号分离框架以适应光照波动。

Details

Motivation: 远程光电容积描记（rPPG）技术通过分析面部视频中的细微颜色变化实现非接触式心率测量，但由于信号极其微弱且易受光照噪声干扰，提取rPPG信号仍具挑战性。

Result: 在公共数据集上的实验证明了该方法的有效性，但摘要未具体说明定量结果或是否达到SOTA水平。

Insight: 创新点在于将rPPG信号的准周期性（源于稳定的心动周期）建模为时频域的块稀疏结构，并构建了时变信号分离框架以自适应处理光照波动，这为弱信号分离提供了新思路。

Abstract: Remote photoplethysmography (rPPG) enables non-contact measurement of cardiac pulse signals by analyzing subtle color changes in facial videos. Nevertheless, extracting rPPG signals remains challenging because of their extremely weak signal strength and susceptibility to illumination noise. In this paper, we propose an rPPG signal extraction method that exploits the quasi-periodic characteristics of rPPG signals. Our approach models quasi-periodicity of the rPPG signal, which arises from the stable cardiac cycle, as a block-sparse structure in the time-frequency domain. To incorporate a block-sparse model and enable adaptive signal separation under illumination fluctuations, we construct a time-varying signal separation framework. Experiments using a public dataset demonstrate the effectiveness of our method.

Table of Contents

cs.CL [Back]

[1] Reflective Prompt Tuning through Language Model Function-Calling cs.CLPDF

[2] When Cases Get Rare: A Retrieval Benchmark for Off-Guideline Clinical Question Answering cs.CLPDF

[3] ACC: Compiling Agent Trajectories for Long-Context Training cs.CL | cs.AIPDF

[4] LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning cs.CL | cs.CVPDF

[5] Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention cs.CL | cs.CVPDF

[6] ArabDiscrim: A Decade-Long Arabic Facebook Corpus on Racism and Discrimination cs.CLPDF

[7] Psy-Chronicle:A Structured Pipeline for Synthesizing Long-Horizon Campus Psychological Counseling Dialogues cs.CLPDF

[8] Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models? cs.CLPDF

[9] Unified Data Selection for LLM Reasoning cs.CLPDF

[10] DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Memory QA cs.CL | cs.AI | cs.LGPDF

[11] GHI: Graphormer over Conditioned Hypergraph Incidence for Aspect-Based Sentiment Analysis cs.CLPDF

[12] Polite on the Surface, Wrong in Practice: A Curated Dataset for Fixing Honorific Failures in Multilingual Bangla Generation cs.CLPDF

[13] LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance cs.CLPDF

[14] Seeing the Poem: Image-Semantic Detection of AI-Generated Modern Chinese Poetry with MLLMs cs.CL | cs.CVPDF

[15] Self-Policy Distillation via Capability-Selective Subspace Projection cs.CLPDF

[16] ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoning cs.CLPDF

[17] Evaluating Commercial AI Chatbots as News Intermediaries cs.CLPDF

cs.CV [Back]

[18] GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillation cs.CVPDF

[19] Lens: Rethinking Training Efficiency for Foundational Text-to-Image Models cs.CVPDF

[20] Flat-Pack Bench: Evaluating Spatio-Temporal Understanding in Large Vision-Language Models through Furniture Assembly cs.CV | cs.AI | cs.CLPDF

[21] PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects cs.CV | cs.ROPDF

[22] UniVL: Unified Vision-Language Embedding for Spatially Grounded Contextual Image Generation cs.CV | cs.LGPDF

[23] AVI-HT: Adaptive Vision-IMU Fusion for 3D Hand Tracking cs.CV | cs.ROPDF

[24] BEiTScore: Reference-free Image Captioning Evaluation with an Efficient Cross-Encoder Model cs.CV | cs.CL | cs.LGPDF

[25] Improving 3D Labeling in Self-Driving by Inferring Vehicle Information using Vision Language Models cs.CV | cs.ROPDF

[26] Look-Closer-Then-Diagnose: Confidence-Aware Ultrasound VQA via Active Zooming cs.CV | cs.AIPDF

[27] BodyReLux: Temporally Consistent Full-Body Video Relighting cs.CV | cs.GRPDF

[28] Ablate-to-Validate: Are Vision-Language Models Really Using Continuous Thought Tokens? cs.CVPDF

[29] MM-Conv: A Multimodal Dataset and Benchmark for Context-Aware Grounding in 3D Dialogue cs.CV | cs.CLPDF

[30] CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models cs.CV | cs.AIPDF

[31] Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding cs.CVPDF

[32] Two-Stage Multimodal Framework for Emotion Mimicry Intensity Prediction cs.CV | cs.AI | cs.HCPDF

[33] Learning Emergent Modular Representations in Multi-modality Medical Vision Foundation Models cs.CV | cs.AIPDF

[34] MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks cs.CV | cs.AIPDF

[35] SDGBiasBench: Benchmarking and Mitigating Vision–Language Models’ Biases in Sustainable Development Goals cs.CV | cs.AIPDF

[36] SceneGraphGrounder: Zero-Shot 3D Visual Grounding via Structured Scene Graph Matching cs.CV | cs.ROPDF

[37] Thermo-VL: Extending Vision-Language Models to Thermal Infrared Perception cs.CVPDF

[38] Universal CT Representations from Anatomy to Disease Phenotype through Agglomerative Pretraining cs.CVPDF

[39] EvoVid: Temporal-Centric Self-Evolution for Video Large Language Models cs.CVPDF

[40] MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cues cs.CV | cs.AIPDF

[41] Bounding-Box Trajectories Matter for Video Anomaly Detection cs.CVPDF

[42] Foresee-to-Ground: From Predictive Temporal Perception to Evidence-Driven Reasoning for Video Temporal Grounding cs.CVPDF

[43] Video as Natural Augmentation: Towards Unified AI-Generated Image and Video Detection cs.CV | cs.AIPDF

[44] Interpreting and Enhancing Emotional Circuits in Large Vision-Language Models via Cross-Modal Information Flow cs.CV | cs.AIPDF

[45] Learning Spatiotemporal Sensitivity in Video LLMs via Counterfactual Reinforcement Learning cs.CV | cs.AIPDF

[46] Visual-Advantage On-Policy Distillation for Vision-Language Models cs.CVPDF

[47] PointLLM-R: Enhancing 3D Point Cloud Reasoning via Chain-of-Thought cs.CV | cs.GR | cs.LGPDF

[48] ORBIS: Output-Guided Token Reduction with Distribution-Aware Matching for Video Diffusion Acceleration cs.CV | cs.ARPDF

[49] Diverse Yet Consistent: Context-Guided Diffusion with Energy-Based Joint Refinement for Multi-Agent Motion Prediction cs.CVPDF

[50] FRED: A Multi-Modal Autonomous Driving Dataset for Flooded Road Environments cs.CV | cs.AI | cs.ROPDF

[51] AgroVG: A Large-Scale Multi-Source Benchmark for Agricultural Visual Grounding cs.CV | cs.AIPDF

[52] Distributed Image Compression with Multimodal Side Information at Extremely Low Bitrates cs.CVPDF

[53] GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigation cs.CV | cs.AIPDF

[54] Echo4DIR: 4D Implicit Heart Reconstruction from 2D Echocardiography Videos cs.CV | cs.AIPDF

[55] COCOTree: A Dataset and Benchmark for Open Tree-Structured Visual Decomposition cs.CVPDF

[56] TWINGS: Thin Plate Splines Warp-aligned Initialization for Sparse-View Gaussian Splatting cs.CV | cs.LGPDF

[57] JMed48k: A Multi-Profession Japanese Medical Licensing Benchmark for Vision-Language Model Evaluation cs.CV | cs.AIPDF

[58] LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model cs.CV | cs.AIPDF

[59] TextTeacher: What Can Language Teach About Images? cs.CV | cs.AI | cs.LGPDF

[60] VISTA: Validation-Guided Integration of Spatial and Temporal Foundation Models with Anatomical Decoding for Rare-Pathology VCE Event Detection – after competition results cs.CVPDF

[61] AesFormer: Transform Everyday Photos into Beautiful Memories cs.CVPDF

[62] OPERA: An Agent for Image Restoration with End-to-End Joint Planning-Execution Optimization cs.CVPDF

[63] Accelerating Vision Foundation Models with Drop-in Depthwise Convolution cs.CVPDF

[64] EventGait: Towards Robust Gait Recognition with Event Streams cs.CVPDF

[65] One Sentence, One Drama: Personalized Short-Form Drama Generation via Multi-Agent Systems cs.CVPDF

[66] Balancing Uncertainty and Diversity of Samples: Leveraging Diversity of Least, High Confidence Samples for Effective Active Learning cs.CVPDF

[67] Enhancing Multimodal Large Language Models for Safety-Critical Driving Video Analysis cs.CV | cs.LGPDF

[68] No Pose, No Problem in 4D: Feed-Forward Dynamic Gaussians from Unposed Multi-View Videos cs.CVPDF

[69] OSS: Open Suturing Skills Vision-Based Assessment Challenge 2024-2025 cs.CV | cs.AI | cs.LGPDF

[70] Zero-Shot Temporal Action Localization Through Textual Guidance cs.CVPDF

[71] MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering cs.CV | cs.AI | cs.MMPDF

[72] EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning cs.CVPDF

[73] GALAR-TemporalNet v2: Anatomy-Guided Dual-Branch Temporal Classification with Bidirectional Mamba and Dual-Graph GCN for Video Capsule Endoscopy – after competition results cs.CVPDF

[74] Exposing Vulnerabilities in Visible-Infrared VLMs: A Unified Geometric Adversarial Framework with Cross-Task Transferability cs.CVPDF

[75] AgroTools: A Benchmark for Tool-Augmented Multimodal Agents in Agriculture cs.CVPDF

[76] Bernini: Latent Semantic Planning for Video Diffusion cs.CV | cs.AI | cs.MMPDF

[77] Translating Signals to Languages for sEMG-Based Activity Recognition cs.CVPDF