cs.CL [Total: 32]
cs.CV [Total: 204]
cs.AI [Total: 6]
cs.RO [Total: 7]
eess.IV [Total: 5]
q-bio.QM [Total: 1]
cs.LG [Total: 13]
cs.CR [Total: 2]
cs.HC [Total: 1]
cs.GR [Total: 1]
cs.SE [Total: 3]
cs.IR [Total: 2]

cs.CL [Back]

[1] Gemma 4 Technical Report cs.CL | cs.AIPDF

Gemma Team, Sherif El Abd, Vaibhav Aggarwal, Robin Algayres, Alek Andreev

TL;DR: Gemma 4是Gemma模型家族的新一代开放权重、原生多模态语言模型。它采用密集和混合专家架构，参数范围从2.3B到31B，并引入了改进的视觉和音频编码器、无编码器的统一架构（用于12B模型处理原始音频和图像块）以及思维模式。该模型在推理速度、内存和计算效率以及长上下文能力方面均有提升，在STEM、多模态和长上下文基准测试中实现了性能飞跃，并在人类评估任务中可与更大的前沿开放模型相媲美。

Details

Motivation: 旨在推进计算效率和推理能力，开发新一代高效、通用的多模态基础模型，以应对STEM、多模态理解和长上下文处理等复杂任务。

Result: 在STEM、多模态和长上下文基准测试中实现了性能飞跃，在人类评估任务中与更大的前沿开放模型表现相当。

Insight: 主要创新点包括：1) 提出无编码器的统一架构，使模型能直接处理原始音频和图像块，简化多模态处理流程；2) 集成“思维模式”，允许模型在响应前生成推理轨迹，增强可解释性和分步推理能力；3) 在架构设计（如MoE）和编码器改进上优化了效率与长上下文能力。

Abstract: We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemma 4 model suite features dense and Mixture-of-Experts architectures, ranging from 2.3B to 31B parameters. Alongside improved vision and audio encoders for all model sizes, we propose a unified, encoder-free architecture for our 12B model, which ingests raw audio and image patches. Furthermore, we integrate a thinking mode, enabling Gemma models to generate reasoning traces prior to responding. We improve inference speed, memory, and compute efficiency, as well as long-context abilities through critical design choices. Gemma 4 establishes a leap in performance across STEM, multimodal, and long-context benchmarks, and rivals larger, frontier open models in human-rated tasks.

[2] Echoes of Unrest: A Multimodal NLP Framework for Early Warning of Fake News and Violence-Driven Mob Activity cs.CL | cs.AIPDF

Md. Maruf Bangabashi, Tahmid Hasan, Golam Mahmud, Md. Mostafijur Rahman, Md. Toufiqur Rahman

TL;DR: 本文提出了一种多语言多模态NLP框架，用于早期检测虚假新闻和暴力倾向动态。该框架融合了XLM-RoBERTa进行多语言文本表示、CLIP进行视觉嵌入，并采用多头注意力机制进行多模态融合，同时加入了讽刺和地理空间元数据等辅助特征。在包含138,256个孟加拉语和英语样本的融合数据集上，实验在30%的分层子集上取得了98%的测试准确率。

Details

Motivation: 社交媒体上虚假信息、操纵内容和煽动性叙事的快速传播，常与社会动荡、政治不稳定和暴民暴力相关联，现有事实核查难以跟上其传播速度，因此需要开发早期预警系统。

Result: 在分层抽取的30%测试子集上，模型达到了98%的测试准确率，并具有较高的精确率和召回率，证明了多模态方法在早期虚假信息检测中的有效性。

Insight: 创新点在于构建了一个融合多语言文本、视觉信息和地理空间元数据的多模态框架，并证明了地理空间信号对于预测现实世界事件升级具有附加价值；从客观角度看，将讽刺检测等辅助特征整合到早期预警系统中是一个值得借鉴的思路。

Abstract: Rapid growth in social media has transformed global communication by enabling fast information exchange, but it has also accelerated the spread of misinformation. Fake news, manipulated content, and provocative narratives are increasingly linked to social unrest, political instability, and mob violence. Incidents in South Asia and elsewhere demonstrate how false information disseminated via platforms such as Facebook and WhatsApp can trigger real-world harm, often spreading faster than fact-checking efforts can respond. To address this challenge, this chapter presents a multilingual, multimodal Natural Language Processing (NLP) framework for early detection of misinformation and violence-prone dynamics. A fused dataset of 138,256 Bangla and English samples was created by combining multiple benchmark datasets. The framework integrates XLM-RoBERTa for multilingual text representation, CLIP for visual embedding, and a multi-head attention mechanism for multimodal fusion, enhanced with auxiliary features such as sarcasm and geospatial metadata. Experiments on a stratified 30% subset achieved 98% test accuracy with strong precision and recall. The outcomes show the efficacy of multimodal approaches in early misinformation detection and highlight the added value of geospatial signals for anticipating real-world escalation.

[3] Reinforcement Learning for Data-Efficient Code-Switched ASR cs.CL | cs.SDPDF

Ziwei Ye, Peter Vickers

TL;DR: 本文提出了一种名为RLVR的强化学习方法，用于数据高效地适配音频-语言模型以进行语码转换（code-switched）语音识别。该方法结合了错误率奖励和脚本保真度奖励，并使用两阶段草稿-精炼流程，在仅使用10%的合成语音数据训练时，其性能即可与使用全数据集进行LoRA监督微调的模型相媲美，尤其在类型学距离较远的语言对上提升显著。

Details

Motivation: 现有的音频-语言模型虽然能处理语码转换语音，但其解码过程并未针对语码转换进行优化，常在语言边界处失败。因此，需要一种数据高效的方法来专门优化模型在语码转换场景下的性能。

Result: 在Qwen2-Audio模型和10个语言对上进行的实验表明，所提出的RLVR方法仅使用10%的合成语音数据训练，其性能即可匹配使用全部数据进行LoRA监督微调的模型。该方法在类型学距离远的语言对上收益最大，并且其性能可以零样本迁移到真实录制的语码转换语料库上。

Insight: 创新点在于提出了一种结合可验证奖励（错误率奖励和脚本保真度奖励）的强化学习配方，以及两阶段的草稿-精炼流程，实现了数据高效的模型适配。脚本保真度奖励能独立地减少脚本污染而不降低性能，这是一个有效的设计。

Abstract: Audio-language models can be prompted for code-switched speech, but their decoding is not optimized for code-switching and often fails at language boundaries. We propose a practical reinforcement learning with verifiable rewards recipe for data-efficient adaptation of audio-language models to code-switched ASR using group relative policy optimization, combining an error rate reward with a script fidelity reward that penalizes wrong writing systems and a two-pass draft-and-refinement procedure. Using Qwen2-Audio as a reproducible testbed across 10 language pairs, training on only TTS code-switched speech, we show that RLVR with 10% of the data matches LoRA supervised fine-tuning trained on the full dataset, with the largest gains on typologically distant pairs. The error rate reward eliminates translation errors while the script fidelity reward separately reduces script contamination without degradation. These gains transfer zero-shot to a human-recorded code-switching corpus.

[4] Jointly Improving Dialect Identification and ASR in Indian Languages using Multimodal Feature Fusion cs.CL | eess.ASPDF

Saurabh Kumar, Amartyaveer, Prasanta Kumar Ghosh

TL;DR: 本文提出了一种多模态框架，联合改进印度语言的自动语音识别（ASR）和方言识别（DID）。该方法通过瓶颈编码器从基于Conformer的语音表示中提取方言特征，并通过RoBERTa编码器处理ASR生成的CTC嵌入，使用门控机制融合这些特征，再通过注意力编码器精炼表示，最后将学习到的嵌入与Conformer输出拼接以增强ASR特征。

Details

Motivation: 针对印度语言中许多资源匮乏且方言差异显著的问题，现有方法通常单独优化ASR或DID，导致性能权衡，因此需要一种联合建模方法以同时提升两者性能。

Result: 在包含八种印度语言和三十三种方言的数据集上评估，该方法实现了平均DID准确率81.63%，平均字符错误率（CER）和词错误率（WER）分别为4.65%和17.73%，展示了联合ASR-DID建模的有效性。

Insight: 创新点在于通过多模态特征融合（结合语音和文本嵌入）和门控注意力机制，实现ASR与DID的协同优化，这为低资源语言处理提供了可借鉴的联合学习框架。

Abstract: Automatic Speech Recognition (ASR) and Dialect Identification (DID) are crucial for Indian languages, many of which are low-resource and exhibit significant dialectal differences. Existing methods often optimize ASR or DID individually, resulting in performance trade-offs. In this work, we propose a multimodal framework that jointly improves ASR and DID. Our method employs a Bottleneck Encoder to extract dialectal features from Conformer-based speech representations and a RoBERTa encoder to process ASR-generated CTC embeddings. A gating mechanism merges these features, followed by an attention encoder to refine the representations. The learned embeddings are concatenated with Conformer outputs to enhance ASR features. Evaluated on eight Indian languages with thirty-three dialects, our method achieves an average DID accuracy of 81.63% and average CER and WER of 4.65% and 17.73%, respectively. These results highlight the effectiveness of our method for joint ASR-DID modeling.

[5] Where do LLMs Fall Short in CBT-Guided Affective Reasoning? cs.CL | cs.AI | cs.HC | cs.IRPDF

Vaishnavi Sinha, Pooja Guttal, Pranay Deep Reddy Katike, Vishal Sinha, Gerald Ndawula

TL;DR: 本文探讨了大型语言模型在认知行为疗法引导的情感推理中的局限性。研究发现，尽管LLMs在CBT理论知识上表现出色，但在实际应用中倾向于固守‘验证与反思’策略，无法有效应用CBT框架。作者提出了一个知识引导的框架，通过将用户叙述分解为贝克认知概念化结构，并采用多链思维策略选择，但干预效果有限。

Details

Motivation: 解决LLMs在认知行为疗法对话中，虽然能流畅共情地回应，但倾向于固守‘验证与反思’策略，无法根据用户实际需求有效应用结构化CBT知识的问题。

Result: 在三个开源权重LLM和14个源自RealCBT的案例研究中，通过专家评估、效价-唤醒轨迹和语言趋同进行评估。引入的协议杠杆力指标显示，仅通过单链思维提示引入协议定义无法改变LLM行为，而多链思维策略选择略有改善，但效果仅在约1.2-1.3%范围内，所有模型仍偏向验证与反思。

Insight: 创新点在于提出了一个将CBT对话视为受控情感推理的知识引导框架，结合了临床SNOMED CT概念和自然语言推理进行落地，并引入了协议杠杆力这一行为层面的新度量来量化干预效果。客观来看，研究揭示了LLMs在理论知识与实际应用间存在显著差距，为情感计算社区提供了测量LLMs不足之处的工具。

Abstract: Cognitive Behavioral Therapy (CBT) provides a structured framework for understanding a user’s mental state by examining the interaction between cognitive and behavioral factors. However, out-of-the-box LLMs respond fluently and empathetically, yet collapse into validation & reflection, regardless of what the user actually needs. They know theoretical CBT (scoring up to 96% accuracy on licensing exam questions) but fail to apply it effectively. We explore this gap with a knowledge-guided framework that treats CBT dialogue as controlled affective reasoning: user narratives are decomposed into Beck’s Cognitive Conceptualization structure, grounded in clinical SNOMED CT concepts validated via Natural Language Inference, and a Multiple Chain-of-Thought (MCoT) strategy selection between Validation & Reflection, Socratic Questioning, or Alternative Perspectives. To measure whether such guidance actually changes behavior, we introduce the Protocol Leverage Force (F), a behavior-level metric that captures how far an intervention shifts a model away from its default response. Across three open-weight LLMs and 14 RealCBT-derived case studies, evaluated with human experts, valence-arousal trajectories, and linguistic entrainment, F shows that simply introducing protocol definitions via single chain-of-thought prompting fails to change LLM behavior, while MCoT on these definitions guides strategy selection better. Still, the effect stays within 1% (approx. 1.2-1.3%), and all models remain biased toward Validation & Reflection. These results show CBT knowledge alone does not ensure effective application, giving the affective-computing community instrumentation to measure where LLMs fall short.

[6] Seduced by the Narrative: Assessing Rule Adherence in Semi-Open Textual Sandboxes cs.CL | cs.AIPDF

Weiying Chen, Junlong Shen, Zhanyuan Guo, Xiaoou Zhou

TL;DR: 本文提出了一种名为CoC-Seduce的多智能体对抗性基准测试，用于评估大型语言模型在半开放文本游戏环境中作为自主裁决者时的规则遵守能力。研究发现，当前前沿模型在面对利用伪逻辑推理和权威胁迫等叙事框架技术进行的‘修辞注入’攻击时，其裁决鲁棒性存在显著缺陷，且模型规模或显式推理机制均不能可靠地提升其表现。

Details

Motivation: 随着LLM越来越多地被部署为半开放文本游戏环境中的自主裁决者，当用户意图与系统规则冲突时，确保模型对规则的鲁棒遵守变得至关重要。然而，这些模型被训练得乐于助人且顺从，使其容易受到一类名为‘修辞注入’的攻击，即对抗性用户利用叙事框架技术来绕过裁决逻辑。

Result: 在基于TRPG机制构建的CoC-Seduce基准测试中，使用GPT-5.4、Claude Sonnet 4.6和Gemini 3.5 Flash作为对抗生成器，产生了5,376个样本。对20个目标裁决模型的评估表明，模型规模和显式推理机制均不能可靠地赋予裁决鲁棒性，其中伪逻辑成为最主要的攻击向量，跨文化设定则暴露了所有被评估模型家族的系统性知识差距。

Insight: 论文的创新点在于提出了‘修辞注入’这一攻击形式，并构建了一个基于TRPG的、规则明确但交互完全自然的半开放环境基准（CoC-Seduce），用于系统性地评估LLM的规则遵守鲁棒性。客观来看，其核心洞察是揭示了当前LLM在遵循规则与满足用户请求之间存在根本性脆弱性，且这种脆弱性不能简单地通过扩大模型规模或引入推理机制来解决，这为未来开发更鲁棒的AI系统指明了关键挑战。

Abstract: As LLMs are increasingly deployed as autonomous adjudicators in semi-open textual game environments, robust rule adherence becomes critical when user intent conflicts with system rules. However, these models are trained to be helpful and compliant, leaving them vulnerable to a class of attacks we term \textit{Rhetorical Injection}, where adversarial users exploit narrative framing techniques such as pseudo-logical reasoning and authoritative coercion to bypass adjudication logic. We present CoC-Seduce, a multi-agent adversarial benchmark built on Tabletop Role-Playing Game (TRPG) mechanics, an ideal instantiation of semi-open environments where rules are explicit for adjudication, yet interaction remains entirely in natural language. Three frontier models, i.e., GPT-5.4, Claude Sonnet 4.6, Gemini 3.5 Flash, serve as adversarial generators producing 5,376 samples across 4 world settings and 16 skill categories. We then benchmark 20 target adjudicators against this corpus. Evaluation across 20 models reveals that neither model scale nor explicit reasoning mechanisms reliably confer adjudication robustness, with \textsc{Pseudo-Logic} emerging as the dominant attack vector and cross-cultural settings exposing systematic knowledge gaps across all evaluated families. Project page: https://github.com/answerrtx/CoC-Seduce

[7] Distill Where the Student Goes: Teacher-Regularized RL for English-Evidence Cross-Lingual RAG cs.CLPDF

Haotian Zhou, Weiran Huang, Siqi Liu, Xiting Wang, Xin Zhang

TL;DR: 本文提出了一种名为TR-RAG的教师正则化强化学习方法，用于解决英语证据跨语言检索增强生成中的语言漂移和证据使用不可靠问题。该方法通过结合奖励优化和基于学生访问前缀的策略蒸馏，利用一个更强的冻结教师模型作为锚点来稳定训练。

Details

Motivation: 动机是解决在英语证据跨语言RAG设置中，尽管基础模型强大，但生成仍可能失败的问题，具体表现为语言漂移（输出英语或语码转换）以及模型在生成非英语答案时不可靠地使用证据。作者将这些失败归因于两个后训练挑战：前缀依赖的错误和序列级奖励带来的噪声信用分配。

Result: 在BioASQ-ENKB5、Hotpot-ENKB5和MKQA三个基准测试以及两种骨干模型上，TR-RAG在语言遵循性和证据基础正确性的综合指标上优于强基线。教师锚点起到了安全网作用，防止了奖励优化RL可能出现的语言一致性大幅下降，并在字符3-gram召回率上，紧凑的学生模型有时能超越其70B参数的教师模型。

Insight: 创新点在于提出了一个耦合奖励优化与策略蒸馏的教师正则化RL框架，并引入了针对英语证据多语言生成的奖励分解（包括语言一致性、字符3-gram召回率和LLM评判的证据基础正确性得分）。其核心思想是利用冻结教师模型在学生访问的前缀上提供反向KL锚点，以稳定训练并防止性能崩溃。

Abstract: Cross-lingual retrieval-augmented generation (RAG) is often deployed in an English-evidence regime, where users query in diverse languages but retrieved passages remain English. In this setting, generation can fail despite strong base models: English evidence induces language drift (English or code-switching outputs) and models use evidence unreliably when producing non-English answers. We attribute these failures to two post-training challenges: (i) errors are prefix-dependent, so fixed-trajectory supervision suffers from prefix mismatch; and (ii) sequence-level (partly discrete / judge-based) rewards yield noisy credit assignment and high-variance updates. We propose TR-RAG, a teacher-regularized RL recipe that couples reward optimization with on-policy distillation on student-visited prefixes. A compact student samples on-policy answers, while a stronger frozen teacher is queried only on those prefixes and provides a prefix-wise student-to-teacher reverse-KL anchor. We further introduce a reward decomposition for English-evidence multilingual generation, combining language consistency, character 3-gram recall, and an LLM-judge score for evidence-grounded correctness. Across three benchmarks – BioASQ-ENKB5, Hotpot-ENKB5, and naturally multilingual MKQA – and two backbones, TR-RAG improves the composite of language adherence and evidence-grounded correctness over strong baselines. Crucially, the teacher anchor acts as a safety net: on in-domain languages it prevents the large language-consistency collapses (up to ~27 percentage points) that reward-only RL can suffer by drifting below even the base model, while on distant out-of-distribution languages – where reward-only RL stalls at the base model’s ceiling – it still improves evidence grounding; and on character 3-gram recall the compact student sometimes surpasses its 70B teacher.

[8] Don’t Wait to Reply: Towards Responsive yet Thoughtful Dialogue through Proactive Thinking cs.CL | cs.AIPDF

Ante Wang, Jiaqi Fu, Xuanyi Chen, Ruotian Ma, Zhaopeng Tu

TL;DR: 本文提出了一种名为’主动思考’的框架，旨在解决大语言模型在对话中因被动触发思考而引入延迟的问题。该框架允许模型在对话间隙预计算潜在的回应元素，而非空闲等待下一个输入，从而在不牺牲性能的前提下提升交互效率。

Details

Motivation: 当前LLMs的思考过程是反应式的，仅在收到用户回复后才被动触发推理，这不可避免地引入了延迟，破坏了对话的流畅性。这与人类在自然停顿中主动预测和规划未来内容以确保无缝交互的方式形成鲜明对比。

Result: 研究将三个不同复杂度的基准测试调整为模拟实时对话流的时间感知环境进行评估。结果表明，主动思考能有效提升交互效率，且不损害模型性能。

Insight: 核心创新点在于将模型的思考模式从被动反应转变为主动预测，提出了’主动思考’框架。一个可借鉴的技术点是其无需训练的基线方法，该方法通过预测未来状态进行前瞻性思考，并在推测性持续思考中平衡效率与质量。

Abstract: Thinking has emerged as a critical capability for Large Language Models (LLMs) tackling complex tasks. However, its reactive nature, where reasoning is passively triggered only upon receiving a user response, inevitably introduces latency that compromises conversational fluidity. This stands in sharp contrast to human dialogue, where speakers proactively anticipate and plan future content during natural pauses to ensure seamless interaction. To bridge this gap, we propose Proactive Thinking, a framework that empowers models to pre-compute potential response elements during conversational downtime instead of waiting idly for the next input. We then introduce a training-free baseline that can think ahead by anticipating future states, balancing efficiency and quality through speculative continual thinking. To evaluate this approach in practice, we adapt three benchmarks of varying complexity into time-aware environments that simulate real-time conversational flow. We demonstrate that proactive thinking effectively improves interaction efficiency without compromising performance. Ultimately, this work advocates for a fundamental shift toward more intelligent, anticipatory, and real-time conversational AI.

[9] KARMA: Knowledge graph-based Automated Reasoning Materialization and Alignment cs.CL | cs.AI | cs.LGPDF

Jinkyeong Choi, Chaebin Jeong, Donghyeon Park

TL;DR: 本文提出了KARMA方法，以解决基于模板的对比合成中存在的‘分辨率不匹配问题’。该方法通过枚举领域知识图谱中受模式约束的路径，并将其转化为槽位对齐的对比候选，然后利用槽位并行对齐（SPA）技术，将偏好监督路由到具有区分性的实体槽位。

Details

Motivation: 动机在于解决基于模板的对比合成方法中，候选样本往往仅在少数实体槽位上存在差异，而序列级优化却将监督信号分散在大部分共享模板上的问题，即‘分辨率不匹配问题’。

Result: 在生物医学、计算机科学和化学领域的基准测试中，KARMA的表现优于基础大语言模型（LLM）和相同数据的监督微调（SFT）基线，并且与序列级和令牌级的偏好学习方法相比具有优势。

Insight: 核心创新点在于形式化了分辨率不匹配问题，并提出了通过知识图谱路径枚举和槽位对齐的候选生成方法，以及采用解耦的槽位级目标（SPA）来更精确地施加偏好监督。其槽感知掩码注意力机制为高效实现提供了可选方案。

Abstract: Template-based contrastive synthesis is scalable, but its candidates often differ only in a few entity-slots while sequence-level optimization spreads supervision over mostly shared templates. We formalize this as the Resolution Mismatch Problem and propose KARMA, which enumerates schema-constrained paths over domain knowledge graphs and verbalizes them into slot-aligned contrastive candidates. Slot-Parallel Alignment (SPA) then applies a decoupled slot-level objective to route preference supervision to discriminative entity-slots, with slot-aware masked attention serving as an optional packed-evaluation implementation. Across biomedical, computer-science, and chemistry benchmarks, KARMA outperforms base LLM and same-data SFT baselines, and compares favorably with sequence and token-level preference methods.

[10] From Judgments to Issues: Structured Extraction of Legal Reasoning with Citation-Hallucination Control cs.CL | cs.AI | cs.IRPDF

Giovanni Piccioli, Alessia Fidelangeli, Piera Santin, Pierpaolo Vivo

TL;DR: 本文提出了一种自动化流程，用于将意大利税务法院判决书分解为独立的法律争议点，并为每个争议点提取基于IRAC框架和法律三段论的结构化XML表示。该流程针对约33万份意大利税务法院一审和二审判决书，采用高效且成本可控的通用模型DeepSeek V3，并引入自动幻觉检测过滤器来确保法律引用的准确性。

Details

Motivation: 解决从大量法律判决书中自动化提取结构化法律推理信息的需求，同时应对大型语言模型在法律引用上容易产生幻觉（即生成错误引用）的可靠性问题。

Result: 在50份由税法博士标注的判决书上验证了流程，评估了标注者间一致性、模型与专家间一致性，以及幻觉过滤器的独立性能。这是首个针对意大利税务法院判决的、专家验证的、具备幻觉控制功能的争议点级结构化提取流程。

Insight: 创新点在于将通用LLM（DeepSeek V3）与专用的法律引用解析器（Linkoln）及标准化标识符（URN-NIR, ECLI, CELEX）相结合，构建了一个包含幻觉控制的可扩展提取管道，为法律信息检索、引文网络分析和大规模法律推理数据集构建提供了基础。

Abstract: We present an automated pipeline that decomposes Italian tax-court judgments into individual legal issues and extracts, for each issue, a structured XML representation grounded in the IRAC framework and the legal syllogism. The pipeline targets a corpus of approximately $330{,}000$ first- and second-instance decisions of the Italian tax courts and is built around a capable yet cost-efficient general-purpose model (DeepSeek V3), a choice driven by the need to process several hundred thousand documents at a sustainable cost. To address the well-documented unreliability of large language models on legal citations, we couple the extraction step with an automatic hallucination-detection filter that compares the references produced by the model with those identified in the judgment text by a dedicated parser (Linkoln), normalised to standard identifiers (URN-NIR, ECLI, CELEX). We validate the pipeline on $50$ judgments annotated by two PhDs in tax law, computing inter-annotator agreement and LLM-vs-expert agreement on both issue extraction and legal citations, together with a stand-alone evaluation of the hallucination filter. To the best of our knowledge, this is the first issue-level, expert-validated structured extraction pipeline with hallucination control for Italian tax-court decisions, and it provides a concrete starting point for downstream applications such as issue-level retrieval, citation-network analysis, and the construction of large-scale datasets of legal reasoning.

[11] Efficient Decentralized Multi-task Dataset Valuation via Model Merging cs.CL | cs.AI | cs.CV | cs.LGPDF

Mohammadsajad Alipour, Mohammad Mohammadi Amiri

TL;DR: 本文提出了一种名为DMVM的去中心化多任务数据集估值框架，通过模型融合技术来评估不同数据集在多任务模型训练中的贡献，避免了传统方法中需要重新训练或共享原始数据的高计算成本和隐私问题。

Details

Motivation: 现有数据集估值方法主要针对单任务场景，且基于Shapley值或重训练的方法计算成本高，难以适应去中心化环境中的多任务优化需求和严格隐私约束。

Result: 在计算机视觉和自然语言处理任务上的综合实验验证了DMVM的有效性，并提供了理论误差界来表征其近似质量。

Insight: 创新点在于利用任务算术从模型组合中直接推断数据集贡献，实现了可扩展、计算高效且与多任务泛化行为明确对齐的估值过程，并引入了安全聚合协议以支持去中心化部署。

Abstract: Accurate and efficient dataset valuation is essential for enabling fair and transparent data marketplaces, especially when multiple contributors provide data for training multi-task models. Most existing valuation methods, however, are limited to single-task settings, overlooking scenarios where a buyer aims to optimize performance across multiple downstream tasks. Moreover, traditional valuation approaches, such as Shapley-based or retraining-based methods, are computationally expensive and poorly suited for decentralized environments without a trusted central coordinator and with strict privacy constraints. We propose DMVM (Decentralized Multi-task Valuation via Model Merging), a novel framework that bypasses retraining and data sharing by leveraging task arithmetic to infer dataset contributions directly from model combinations. Instead of retraining or sharing raw data, DMVM quantifies how models trained on different datasets combine in parameter space to infer each dataset’s marginal utility across multiple tasks. This formulation yields a valuation process that is scalable, computationally efficient, and explicitly aligned with multi-task generalization behavior. To support decentralized deployment, we introduce a secure aggregation protocol that enables collaborative valuation without revealing individual model parameters or private data. We also provide theoretical error bounds characterizing the approximation quality of DMVM and validate our framework through comprehensive experiments on computer vision and natural language processing tasks.

[12] Reading Between the Dots: Decoding Hidden Computation across Filler Tokens cs.CL | cs.AI | cs.LGPDF

Kaley Brauer, Claudio Mayrink Verdun, Samuel Marks

TL;DR: 前沿大语言模型（LLMs）能够在内容无关的填充标记（如点或计数序列）上进行多步推理，生成正确答案而无需可见的思维链（CoT）。本文通过分析两个开源前沿模型（DeepSeek V3、Kimi K2）在四个任务族（事实检索、并行数值组合、字符串操作和上下文计算）上的行为，发现模型在填充标记区域的计算是结构化且可读的。作者提出了一种无监督解码流程，仅利用隐藏状态就能以80-95%的准确率恢复中间值，表明即使表面标记无法监控，模型的完整计算轨迹仍具有可监测性。

Details

Motivation: 研究前沿LLMs在内容无关的填充标记上进行隐藏计算的现象，这构成了行为监督的极限情况，因为表面标记不携带底层推理信息。本文旨在探究这种隐藏计算是否真的无法被观测和理解。

Result: 在两个开源前沿模型（DeepSeek V3、Kimi K2）和四个任务族上，通过注意力分析、logit-lens读取和KV-cache移植等实验，证明了计算在填充标记区域是结构化且可读的。提出的无监督解码流程仅使用隐藏状态作为输入，无需真实标签或训练，在两个模型和所有四个任务上，以最佳LLM评判器评估，恢复了80-95%准确率的中间值。

Insight: 论文的创新点在于揭示了LLMs在填充标记上的隐藏计算并非完全不可观测，而是可以通过分析模型的内部状态（如残差流）进行解码。这挑战了仅依赖表面标记进行行为监督的局限性，表明模型的可监测性应基于其完整的计算轨迹，为理解和监控LLMs的内部推理过程提供了新视角和方法。

Abstract: Frontier LLMs can perform multi-step reasoning over content-free filler tokens like dots or counting sequences, producing correct answers with no visible chain-of-thought (CoT). This is a limit case for behavioral oversight, where surface tokens carry no information about the underlying reasoning. But hidden from the output is not the same as hidden from us. On four task families (fact retrieval, parallel numeric composition, string manipulation, and in-context computation), two open-weights frontier models (DeepSeek V3, Kimi K2) compute over filler tokens in a structured, legible way: attention routes the question through the filler region to the answer, logit-lens readouts show retrieved facts emerging early and their composition crystallizing in late layers, and KV-cache transplants at filler positions causally swap outputs between examples. We introduce an unsupervised decoding pipeline that takes only hidden states as input and recovers intermediate values with 80-95% accuracy (best LLM judge) across both models and all four tasks, without ground-truth labels or training. Hidden computation that defeats behavioral CoT monitoring is, on these tasks, directly readable from the residual stream, suggesting monitorability is a property of the model’s full computational trace, not just its surface tokens.

Haoming Li, Jessica Ouyang

TL;DR: 本文提出GRASP框架，通过结合LLM规划与图算法来生成高质量的文献综述相关章节。该框架采用双层图结构（思维图和论证-反驳规划网络）表示文献间的多层次关系，并利用Steiner树进行拓扑感知剪枝以提取核心关联。基于引文分析的评估表明，GRASP生成的文献综述在论述角色、意图和引文分组方面与人工撰写的结果高度吻合。

Details

Motivation: 解决传统文献综述撰写中难以深度理解文献间关系（如继承、挑战或提供替代视角）的问题，旨在自动化生成高保真度的相关章节。

Result: 在基于引文分析的评估中，GRASP生成的文献综述章节在论述角色、意图和引文分组方面与人工撰写目标高度匹配，表明其能有效模拟人类写作逻辑。

Insight: 创新点在于将LLM规划与图算法结合，通过双层图结构和Steiner树剪枝实现文献关系的多粒度建模与核心关联提取，为自动化学术写作提供了可解释的结构化方法。

Abstract: Writing a literature review requires a deep understanding of the relationships among cited papers: how they build on, challenge, or offer alternative perspectives to one another. We present Graph-Reasoning Aided Survey Planning (GRASP), a framework combining LLM planning for related work generation with graph algorithms to extract key relationships among cited papers. Our two-layer graph structure consists of a Graph of Thoughts and an Argument-Counterargument Planning Network, representing the cited papers at different levels of granularity, and we apply topology-aware pruning via a Steiner tree to identify the core inter-paper relationships captured in our graph. Our citation analysis-based evaluation shows that GRASP generates related work sections (RWS) that closely match human-written targets in terms of the discourse roles, intents, and grouping of citations.

[14] The Remarkable Effectiveness of Providing AI Agents with Natural Language Tools: A Replication Study Validating NLT Performance Across 14 Models cs.CL | cs.AIPDF

Alexander Somma, Isabelle Plante, Fred Premji

TL;DR: 本研究独立复现并扩展了Johnson等人（2025）提出的自然语言工具（NLT）框架，该框架质疑在大语言模型（LLM）智能体系统中使用结构化工具调用的做法。通过对14个模型进行8560次实验，研究发现NLT总体上将工具调用准确率提升了14.9个百分点（62.3% vs 47.4%），并将关键错误减少了93%，同时降低了25.2%的令牌使用量。NLT的优势因模型能力而异，对非原生工具调用模型、推理模型和小模型提升显著，而对经过强化的前沿模型（如GPT-5）优势较小甚至为负。

Details

Motivation: 论文的动机是验证并扩展NLT框架的有效性，该框架旨在解决LLM智能体系统中结构化工具调用存在的脆弱性和可靠性问题，为生产系统提供一个更可靠的替代方案。

Result: 在涵盖14个模型的广泛评估中，NLT将整体工具调用准确率从47.4%提升至62.3%，关键错误减少了93%（从755个降至51个），令牌使用量降低了25.2%。其优势在非原生工具调用模型、推理模型和小模型上尤为显著（提升24.0至43.1个百分点），但在经过强化学习优化的前沿模型上优势较小。

Insight: 论文的核心创新点在于首次通过开源工具独立验证了NLT框架，并量化了其可靠性优势（错误减少93%），这为重视可靠性的生产系统提供了关键依据。研究还揭示了模型能力是调节NLT优势的关键因素，这一发现对智能体系统架构设计具有重要指导意义。

Abstract: This study independently replicates and extends the Natural Language Tools (NLT) framework of Johnson et al.~(2025), which questions the use of structured tool calling in large language model (LLM) agentic systems. We evaluated NLT across 14 models and 8,560 trials, adding newer frontier, reasoning, and open-weight models to the original set. The results confirm the core findings and add detail. NLT improves tool-calling accuracy by 14.9 percentage points overall (62.3% versus 47.4% structured) and reduces critical errors by 93% (51 versus 755 errors). The gains depend on model capability: models without native tool calling, reasoning models, and smaller models gain substantially (+24.0pp to +43.1pp), while heavily optimized frontier models (GPT-5, Gemini 2.5 Pro) show smaller or reversed advantages. This matches recent analyses of reinforcement-learning-optimized tool use (Martinez, 2025). NLT also cuts token usage by 25.2%. The reliability and efficiency advantages compound in recursive agentic workflows, where agents chain many tool calls across sub-agents: a structured failure triggers retries, fallback routing, and coordination overhead, while NLT avoids most of that cost at the source. This work makes three contributions: (1) the first independent validation of NLT using open-source tooling, (2) evidence that model capability moderates NLT’s advantages (Chen et al., 2025; Zhang et al., 2025), and (3) a measurement of NLT’s reliability benefit (93% fewer errors), its most deployment-relevant property given the known fragility of structured tool calling. NLT is a practical alternative to structured tool calling, especially for production systems that value reliability over parseability.

[15] NormWorlds-CF: Solver-Verified Counterfactual Normative Reasoning with Metamorphic-Relation GRPO cs.CL | cs.AIPDF

Xinqi Zhang

TL;DR: 本文介绍了NormWorlds-CF，一个用于可执行规则世界中反事实规范性推理的求解器验证环境，其确定性求解器可生成答案、证明和证伪证书等，无需LLM评判即可进行监督和评估。研究通过SFT诊断和配对世界任务实验，发现仅监督最终答案存在风险，而结合证明与证伪的训练能提升整体准确性。此外，提出了基于变形关系的GRPO（MR-GRPO）方法，在结构化变更任务中相比稀疏和仅答案的奖励，能改善关系准确性并减少错误。

Details

Motivation: 解决语言模型在规范性推理中可能因错误原因得出正确结论的问题，通过构建可验证的反事实推理环境，确保模型推理过程的可靠性和可解释性。

Result: 在1.7B和Qwen3-4B模型上的实验表明，MR-GRPO在保持答案变更字段、支持变更、状态变更及软根级变形关系指标上实现了最均衡的性能，优于稀疏和仅答案的GRPO方法。

Insight: 创新点在于引入求解器验证的环境和MR-GRPO奖励机制，强调结构化变更监督的重要性，而非仅依赖最终答案；但完全变更记录生成、不变子类型识别和OOD迁移仍是待解决的开放问题。

Abstract: Language models can reach the right normative verdict for the wrong reason. We introduce NormWorlds-CF, a solver-verified environment for counterfactual normative reasoning in executable rule worlds. Its deterministic solver produces final answers, proof and falsification certificates, argument statuses, support sets, and paired-world change labels, enabling supervision and evaluation without LLM judges. The benchmark contains staged SFT diagnostics and a compact paired-world task with 270 root families and 1080 canonical-to-variant pairs. The SFT diagnostics show that final-answer supervision is an unsafe proxy: answer-only SFT reaches perfect accuracy on answer tasks but scores zero on falsification, while proof-plus-falsification training with targeted replay reaches strong all-task accuracy. For the structured-change task, we introduce metamorphic-relation GRPO (MR-GRPO), a class-conditioned reward for GRPO that gives partial credit for relation families and solver-visible change fields. In matched 1.7B continuation experiments, MR-GRPO improves held-out relation accuracy and relation-family correctness, and reduces wrong-family error, compared to sparse and answer-only GRPO. In Qwen3-4B three-seed validation, answer-only reward improves answer-change fields but weakens relation-family structure, sparse reward preserves coarse relation labels best, and MR-GRPO delivers the strongest balanced performance across answer-change, support-change, status-change, and soft root-level metamorphic-relation metrics. These results show that verified counterfactual structure can shape post-training beyond final answers, while exact full change-record generation, invariant subtype recognition, and out-of-distribution (OOD) transfer remain open problems.

[16] BanglaMemeEvidence: A Multimodal Benchmark Dataset for Explanatory Evidence Detection in Bengali Memes cs.CL | cs.AI | cs.CVPDF

Fatema Tuj Johora Faria, Mukaffi Bin Moin, Md. Mahfuzur Rahman, Pronay Debnath, Asif Iftekher Fahim

TL;DR: 本文提出了BanglaMemeEvidence数据集和BengaliMemeEvidenceNet模型，用于孟加拉语表情包的解释性证据检测任务。该任务旨在分析表情包及其上下文信息，以识别解释其含义和幽默的特定句子。

Details

Motivation: 现有研究主要关注表情包的情感维度，但在有害内容检测、网络欺凌识别和准确情感分析等方面仍面临挑战，这主要源于对更深层次上下文理解的需求。本文旨在通过证据检测任务来提升对表情包的理解。

Result: 提出的BengaliMemeEvidenceNet混合多模态框架在BanglaMemeEvidence数据集上取得了0.74的F1分数，这是首个专注于孟加拉语表情包证据检测的研究。

Insight: 创新点在于引入了“解释性证据检测”这一混合任务，并构建了首个包含孟加拉语表情包、OCR文本、上下文和证据句子的标注数据集。模型通过整合文本和视觉特征进行综合表示，为低资源语言的表情包分析提供了新思路。

Abstract: Memes have become influential communication tools on social media, combining viral visuals with concise messaging to convey impactful ideas. While substantial research has examined the affective dimensions of memes, key challenges such as detecting harmful content, identifying cyberbullying, and performing accurate sentiment analysis remain critical, largely due to the need for deeper contextual understanding. In this paper, we introduce MemeEvidenceDetect, a hybrid task aimed at analyzing a meme and its contextual information to identify specific sentences that explain or elucidate its meaning and humor. To support this task, we present BanglaMemeEvidence, a curated dataset of 2,917 Bengali memes, emphasizing its significance as a resource for the Bangla language. Each meme is annotated with natural language explanations, including Meme OCR, Meme Context, and Evidence Sentences, alongside relevance scores that reflect the relationship between a meme and its corresponding annotations. To address the gap in dynamically inferring a meme’s context, we propose BengaliMemeEvidenceNet, a hybrid multimodal framework that integrates textual and visual features for comprehensive meme representation. Our experiments demonstrate the effectiveness of BengaliMemeEvidenceNet, achieving an F1 score of 0.74. To the best of our knowledge, this is the first study to focus on evidence detection in Bengali memes, marking a notable step forward in the analysis of memes in low-resource languages.

[17] Beyond Multilingual Averages: MTEB-PT, a Benchmark for Portuguese Sentence Encoders cs.CL | cs.AI | cs.LGPDF

Lucas Hideki Takeuchi Okamura, Alexandre Alcoforado, Anna Helena Reali Costa

TL;DR: 本文针对葡萄牙语在文本嵌入评估中代表性不足的问题，提出了MTEB-PT基准，该基准从MMTEB中选取了14个数据集，覆盖语义文本相似性、分类、检索和重排序等任务。作者使用该基准评估了17个开源和闭源嵌入模型，发现葡萄牙语性能高度依赖任务类型，多语言排名无法可靠预测葡萄牙语特定性能，且语言特定的微调仍能提升模型表现。

Details

Motivation: 葡萄牙语作为全球使用最广泛的语言之一，在文本嵌入评估中代表性不足，导致模型选择常依赖英语或多语言指标，其在葡萄牙语上的有效性不明确。

Result: 在MTEB-PT基准上评估了17个模型，结果显示葡萄牙语性能强烈依赖任务，多语言排名无法可靠预测葡萄牙语特定性能，且长上下文能力强的模型在检索和重排序等长输入任务中表现更优。通过葡萄牙语对比监督和Matryoshka表示学习微调的模型在语义文本相似性任务上提升最显著。

Insight: 创新点在于构建了首个针对葡萄牙语的综合文本嵌入基准MTEB-PT，揭示了多语言平均性能无法代表特定语言表现，并验证了语言特定微调的有效性，尤其是在任务类型与微调数据匹配时。该方法强调了语言特定评估的重要性，并为其他低资源语言提供了基准构建范例。

Abstract: Portuguese remains underrepresented in text embedding evaluation, despite being one of the most widely spoken languages in the world. As a result, embedding models are often selected based on English or multilingual metrics, while their effectiveness in Portuguese remains unclear. We present MTEB-PT, a Portuguese benchmark constructed from a subset of MMTEB, comprising 14 existing datasets across Semantic Textual Similarity (STS), classification, retrieval, and reranking. We use this benchmark to evaluate 17 open- and closed-source embedding models under a unified protocol. Our results show that Portuguese performance is strongly task-dependent: multilingual rankings do not reliably predict Portuguese-specific performance across task families, no single model dominates all settings, and models with stronger long-context capacity are particularly advantageous on longer-input tasks such as retrieval and reranking. The benchmark also shows that language-specific fine-tuning still improves model performance in Portuguese, especially on task types that match the adaptation data most closely. To examine this effect, we fine-tune three representative backbone models with Portuguese contrastive supervision and Matryoshka Representation Learning (MRL). These benchmark-informed baselines yield their strongest gains on STS, consistent with the predominantly symmetric supervision used during training, while also improving retrieval and remaining competitive under dimensional truncation. We release the MTEB-PT benchmark, the fine-tuned models, and the training and evaluation code.

[18] CausalGame: Benchmarking Causal Thinking of LLM Agents in Games cs.CL | cs.AI | cs.LG | stat.MLPDF

Zhenhao Chen, Yongqiang Chen, Chenxi Liu, Junchi Yu, Xiangchen Song

TL;DR: 本文提出了CausalGame基准测试，用于评估LLM智能体在游戏化场景中的因果思维能力。该基准通过14个模拟真实科学发现挑战（如选择偏差、测量误差和隐藏混杂因素）的交互式游戏，要求智能体主动设计实验、收集数据并生成解释报告。实验表明，当前30个LLM智能体均未展现出可靠的因果推理能力，最佳模型生存率仅68.0%，远低于理论最优值78-85%，且仅5-7%的会话在因果推理评分中获得认可。

Details

Motivation: 现有AI科学家基准缺乏对选择偏差、测量误差和隐藏混杂因素等现实科学发现挑战的显式评估，而因果思维（区分因果关系与相关性、识别隐藏偏差）是LLM智能体进行科学发现的核心能力，因此需要构建专门评测因果思维的基准。

Result: 在CausalGame基准的14个场景中，测试的30个LLM智能体表现均不理想：最佳模型生存率为68.0%（理论最优为78-85%），仅5-7%的会话在因果推理评分标准中获得分数，表明当前LLM智能体尚未具备可靠的因果思维能力。

Insight: 创新点在于将因果思维评估嵌入交互式游戏框架，通过模拟真实科学发现中的复杂偏差场景（选择偏差、测量误差、隐藏混杂因素）构建可扩展的评测环境；客观而言，该工作首次系统量化了LLM智能体在主动实验设计与因果推理中的能力缺陷，为AI科学家智能体的因果推理研究提供了标准化测试平台。

Abstract: Building AI Scientist agents with Large Language Models (LLMs) has recently attracted growing attention. Since scientific discovery fundamentally relies on uncovering causal relationships from observations, the capability of causal thinking, i.e., distinguishing causation from correlation and recognizing hidden biases, is essential to LLM agents. Although a number of benchmarks exist for AI Scientists, none explicitly incorporate challenges from selection bias, measurement error, and hidden confounders that widely exist in real-world scientific discovery. To this end, we present CausalGame, a benchmark that evaluates the causal thinking capabilities of LLM agents through interactive games. CausalGame asks LLM agents to actively design experimental protocols, collect observation data, and derive a final solution with an explanation report. To emulate realistic scientific discovery challenges, we design 14 scenarios that incorporate selection bias, measurement error, and hidden confounders. Across 30 LLM agents, none demonstrates reliable causal thinking: the best model reaches only 68.0% survival against analytical optima of 78-85%, and merely 5-7% of sessions receive credits on the causal-reasoning rubrics. CausalGame provides a scalable and controlled testbed for evaluating the causal thinking of AI Scientist agents.

[19] Spinning Straw into Gold: Relabeling LLM Agent Trajectories in Hindsight for Successful Demonstrations cs.CLPDF

Zichao Li, Gang Wu, Zichao Wang, Ruiyi Zhang, Wanrong Zhu

TL;DR: 本文提出了一种称为后见监督学习（HSL）的方法，用于解决大语言模型（LLM）智能体在部分可观测、长周期任务中监督数据获取困难的问题。该方法利用智能体轨迹中无意间达成的成功目标作为监督信号，通过一个辅助LLM对已完成轨迹进行重新标注，生成（轨迹，达成目标）对用于微调，并结合了无关动作掩码和样本重加权技术来优化学习过程。

Details

Motivation: 动机在于解决LLM智能体在长周期、部分可观测环境中，获取高质量监督信号（如成功演示）成本高昂且稀缺的核心瓶颈。

Result: 实验表明，HSL方法灵活且与现有后训练流程兼容，能有效提升监督微调（SFT）和直接偏好优化（DPO）的性能，在目标空间更复杂的长周期任务上提升更显著。在ALFWorld基准测试中，HSL仅使用四分之一的地面真实演示数据，其性能就超过了使用全部数据训练的基线模型。

Insight: 核心创新点在于利用智能体轨迹中‘无意达成的成功目标’这一被忽视的监督信号源，通过后见之明进行重新标注以生成训练数据。从客观角度看，其提出的无关动作掩码和样本重加权技术，为处理此类重新标注数据中的次优性问题提供了有效解决方案，提升了方法的样本效率。

Abstract: Large language model agents operate in partially observable, long-horizon settings where obtaining supervision remains a major bottleneck. We address this by utilizing a source of supervision overlooked in existing post-training methods: unintended yet successful goals embedded within agent rollouts. Specifically, we introduce Hindsight Supervised Learning (HSL), where an auxiliary LLM reviews each completed trajectory and relabels it with all of the natural-language goals the agent actually achieved. HSL then pairs the trajectory with its relabeled goals and uses these pairs for additional fine-tuning. To mitigate suboptimality in the relabeled data, we propose two learning techniques for HSL, irrelevant-action masking and sample reweighting. Our experiments show that HSL is flexible and compatible with existing post-training pipelines. It improves both SFT and DPO, with larger gains on long-horizon tasks with more diverse goal spaces. Moreover, HSL is sample-efficient: on ALFWorld, it surpasses baselines trained on the full dataset while using only one quarter of the ground-truth demonstrations.

[20] AI Wizards at EXIST 2026: Hierarchical Soft-Label Learning for Multimodal Sexism Identification in Memes cs.CLPDF

Matteo Fasulo, Antonio Gravina, Luca Tedeschini, Luca Babboni

TL;DR: 本文介绍了AI Wizards团队为EXIST 2026任务提交的多模态迷因性别歧视识别系统。该方法将任务建模为基于经验标注者分布的分层条件软标签预测，使用轻量级门控MLP处理固定的Gemini Embedding 2视觉语言表征，并采用KL散度和同方差不确定性加权进行训练。

Details

Motivation: 解决多模态迷因中性别歧视识别这一复杂任务，该任务包含三个难度递增的子任务，旨在更准确地建模标注者之间的分歧和不确定性。

Result: 在官方Soft-Soft排行榜上，该方法在Task 2.3上排名第一，在Task 2.1和Task 2.2上排名第四。

Insight: 创新点在于将多标签分类任务分层建模为条件软标签预测，直接学习标注者分布，并结合同方差不确定性加权来平衡多任务损失，这是一种处理标注不确定性和多任务学习的新颖方法。

Abstract: We present the AI Wizards submission to EXIST 2026 for multimodal sexism identification in memes. The task is composed of three, increasingly harder subtasks. We model them hierarchically as conditional soft-label prediction over empirical annotator distributions. Our system maps fixed Gemini Embedding 2 vision-language representations through a lightweight Gated MLP trained with KL divergence and homoscedastic uncertainty weighting. Our submissions ranked first on Task 2.3 and fourth on Tasks 2.1 and 2.2 on the official Soft-Soft leaderboards. The code is available at https://github.com/NLP-AI-Wizards/EXIST-2026

[21] UI-MOPD: Multi-Platform On-Policy Distillation for Continual GUI Agent Learning cs.CL | cs.AI | cs.CV | cs.LG | cs.MMPDF

Niu Lian, Alan Chen, Zhehao Yu, Chengzhen Duan, Fazhan Liu

TL;DR: 本文提出了UI-MOPD方法，一种用于持续GUI智能体学习的多平台同策略蒸馏框架。该方法通过构建高质量跨平台交互数据集Uni-GUI，并利用平台条件化的多教师蒸馏，将特定平台的行为先验知识迁移到共享策略中，以解决跨平台GUI智能体面临的平台覆盖不足、行为模式混合和灾难性遗忘等挑战。

Details

Motivation: 构建多平台GUI智能体面临两大挑战：一是高质量、可执行的跨平台交互轨迹数据稀缺且平台覆盖有限；二是不同平台交互惯例差异导致联合或持续训练时易出现行为模式混合、平台特定能力退化及灾难性遗忘问题。

Result: 在OSWorld和MobileWorld基准测试上，UI-MOPD分别实现了38.2%和12.0%的任务成功率，证明了其在平衡跨平台能力保持与新平台适应方面的有效性。

Insight: 创新点在于首次将多教师同策略蒸馏引入GUI智能体的持续学习，通过动态选择平台特定教师进行平台条件化蒸馏，使共享策略能适应新平台同时保留现有平台能力；客观来看，其构建的Uni-GUI数据集和平台条件化蒸馏机制为解决跨平台行为迁移与遗忘问题提供了可借鉴的框架。

Abstract: Recent advances in multimodal foundation models and agent systems have driven GUI agents from single-platform task execution toward cross-platform interaction. However, building multi-platform GUI agents remains challenging. On one hand, high-quality and executable cross-platform interaction trajectories are still scarce, and existing data often suffer from limited platform coverage. On the other hand, different platforms exhibit distinct interaction conventions, making joint or continual training prone to behavioral pattern mixing, platform-specific capability degradation, and catastrophic forgetting. To address these challenges, we construct Uni-GUI, a high-quality cross-platform GUI interaction dataset, and propose UI-MOPD, the first method that incorporates multi-teacher on-policy distillation into continual learning for GUI agents. UI-MOPD dynamically selects a platform-specific teacher according to the current environment and transfers platform-specific behavioral priors to a shared policy through platform-conditioned distillation, enabling adaptation to new platforms while preserving capabilities on existing ones. Experiments on OSWorld and MobileWorld show that UI-MOPD achieves task success rates of 38.2% and 12.0%, respectively, demonstrating its effectiveness in balancing cross-platform capability retention and new-platform adaptation. Project page: https://elispectre.github.io/UI-MOPD/.

[22] dOPSD: On-Policy Self-Distillation for Diffusion Language Models cs.CL | cs.AIPDF

Phuong Tuan Dat, Qi Li, Xinchao Wang

TL;DR: 本文提出了一种名为dOPSD的新方法，用于改进扩散大语言模型（dLLMs）的推理能力。该方法通过利用模型自身去噪轨迹中的信息，为教师模型提供特权信息，从而实现了有效的策略内自蒸馏，避免了对外部真实标签的依赖。

Details

Motivation: 扩散大语言模型在推理任务上表现不佳，传统的监督微调存在策略外和暴露偏差问题，而强化学习则面临稀疏奖励和序列似然难处理的问题。现有的策略内自蒸馏方法因依赖推理时不可用的特权信息而效果有限。

Result: 在Dream和LLaDA基准测试上，dOPSD在领域内数学推理和领域外代码生成任务上均取得了提升，性能超过了监督学习和策略内基线方法。

Insight: 核心创新在于从学生模型自身的去噪轨迹中直接推导教师特权信息，利用同一轨迹中后续更解码的步骤来评估掩码位置，使教师优势源于模型自身的解码过程，从而实现了无需外部标签的有效自蒸馏。

Abstract: Diffusion large language models (dLLMs) generate text by iteratively denoising a masked sequence, offering a parallel alternative to autoregressive models, but eliciting strong reasoning through post-training remains difficult: supervised fine-tuning is off-policy and suffers from exposure bias, while reinforcement learning gives only sparse, sequence-level rewards and is hard to apply without tractable sequence likelihoods. On-policy self-distillation (OPSD) offers a promising alternative, using one model as both student and teacher to provide dense, token-level, on-policy supervision, but its effectiveness hinges on giving the teacher privileged information (PI) - typically an instance-specific ground-truth reference unavailable at inference - so the student ends up distilling a weak PI-free consensus policy that yields little improvement on dLLM reasoning. We introduce dOPSD, which instead derives the teacher’s privilege directly from the student’s own denoising trajectory, evaluating masked positions using later, more-decoded steps of that same trajectory rather than an external label, so the teacher’s advantage emerges from the model’s own decoding process; on Dream and LLaDA, dOPSD improves both in-domain math reasoning and out-of-domain code generation, outperforming supervised and on-policy baselines.

[23] Don’t Commit Alone: Joint Token Commitment in Diffusion Large Language Models cs.CLPDF

Lin Yao

TL;DR: 本文提出CoCommit方法，用于解决扩散大语言模型（dLLMs）在去噪步骤中独立解码多个位置时因位置依赖而产生的因子化误差问题。该方法通过一个标记门控的协调阶段，在贪婪解码前重新应用骨干网络的最后n层，使标记位置能够协调，近似联合模式解码，从而提升模型性能。

Details

Motivation: 扩散大语言模型在每步去噪中独立解码多个选定位置，当这些位置存在依赖关系时，基于置信度的选择无法从边缘分布中观察到条件总相关性所捕获的因子化误差，这影响了解码的准确性。

Result: 在LLaDA2.1-mini模型上使用LoRA适配器进行匹配贪婪推理，联合承诺方法在评估的六个基准测试中均提高了准确性，尤其在推理和精确答案任务上取得了最大增益。

Insight: 创新点在于引入标记门控协调阶段，通过延迟承诺和重新应用网络层来实现位置间的协调，近似联合解码，无需额外模型权重，仅增加一次部分前向传递，有效减少了因子化误差。

Abstract: Diffusion large language models (dLLMs) commit multiple tokens per denoising step by decoding each selected position independently from the shared context; when those positions are dependent, the resulting factorization error is captured by conditional total correlation, which confidence-based selection cannot observe from marginals alone. We propose CoCommit, a marker-gated coordination pass that briefly defers commitment: after the usual bundle selection, a learned marker announces the commit set and the backbone’s last-$n$ layers are re-applied so marked positions coordinate – approximating joint-mode decoding – before greedy argmax writes tokens. The method reuses existing weights with one extra partial forward pass and no auxiliary model. On LLaDA2.1-mini with LoRA adapters and matched greedy inference, joint commitment improves accuracy on all six benchmarks we evaluate, with the largest gains on reasoning and exact-answer tasks.

[24] FormalRx: Rectify and eXamine Semantic Failures in Autoformalization cs.CLPDF

Haocheng Wang, Baiyu Huang, Yingjia Wan, Xiao Zhu, Xiaoyang Liu

TL;DR: 本文提出了FormalRx，一个用于诊断和评估自动形式化（autoformalization）系统语义对齐错误的综合框架。该框架包含一个层次化的SCI错误分类法，将错误分解为28个类别，并提供对齐判定、错误分类、错误定位和修正四项诊断能力。作者基于此框架训练了诊断模型FormalRx-8B，并发布了首个细粒度诊断基准FormalRx-Test。

Details

Motivation: 现有自动形式化评估方法仅提供不透明的二元判定或标量分数，无法解释翻译失败的具体位置和原因，这严重限制了人类理解和自动化系统的改进。本文旨在填补这一空白，将评估从黑盒判断转变为可操作的反馈。

Result: 在FormalRx-Test基准上，诊断模型FormalRx-8B在判定（F1=0.88）、分类（F1=0.71）、定位（准确率0.75）和修正（准确率0.73）任务上均显著优于通用大语言模型和专用基线模型，实现了SOTA性能。

Insight: 论文的核心创新在于提出了一个系统化的、可解释的错误分类法（SCI Error Taxonomy）和相应的诊断框架，将评估与具体改进路径连接起来。这为自动形式化领域提供了首个细粒度诊断基准和工具，推动了该领域从单纯性能评估向可解释、可诊断的系统性改进转变。

Abstract: The veracious semantic alignment in autoformalization is significant for formal mathematical reasoning. However, existing evaluations provide only opaque binary verdicts or scalar scores, offering no interpretable insight into where or why translations fail. This opacity severely limits both human understanding and automated system improvement. To bridge this gap, we introduce FormalRx, a comprehensive diagnostic evaluation framework that transforms autoformalization assessment from black-box judgments into actionable feedback. At its core is SCI Error Taxonomy, a hierarchical classification scheme decomposing autoformalization errors into 28 distinct categories with strict priority ordering. Building on this taxonomy, FormalRx provides four critical diagnostic capabilities: alignment verdicts, error categorization, error localization, and correction. We instantiate the framework with a diagnostic model FormalRx-8B, trained on 56,287 NL-FL pairs with fine-grained diagnostic annotations, and release FormalRx-Test as the first fine-grained diagnostic benchmark. FormalRx-8B achieves F1-scores of 0.88 (verdict) and 0.71 (categorization), along with accuracies of 0.75 (localization) and 0.73 (correction), substantially outperforming both general-purpose LLMs and specialized baselines. By connecting evaluation with actionable insights, FormalRx enables systematic diagnosis and improvement of autoformalization systems.

[25] What You See Is What You Get: Observation-Aligned Supervision for Chart-to-Code Generation cs.CL | cs.LGPDF

Tianhao Niu, Qingfu Zhu, Wanxiang Che

TL;DR: 本文针对图表到代码生成任务中监督学习目标与视觉观察不一致的问题，提出了一种名为Observation-Aligned supervision的重写框架。该框架将训练数据中不可从图像中唯一恢复的原始数据目标（如箱线图的原始样本），替换为受视觉观察约束的量（如箱线图的统计量），从而减少模型幻觉和过度指定代码的生成。实验表明，该方法在多个视觉语言模型和基准测试上均能一致地提升可观测值的恢复能力。

Details

Motivation: 当前图表到代码生成任务通常使用参考绘图脚本进行监督微调，隐含地将黄金代码视为完全可观测的目标。然而，许多图表程序包含无法从渲染图像中唯一恢复的潜在原始变量（如箱线图的原始样本），监督模型去复现这些不可识别的量会鼓励模型产生幻觉和生成过度指定的代码。

Result: 在ChartMimic和ChartX基准测试上，对多个视觉语言模型应用该方法进行实验，结果表明在可观测值恢复方面取得了持续改进，包括在可执行性评估下。

Insight: 核心创新点在于提出了一个对齐视觉观察的监督重写框架，将训练目标从不可识别的原始数据替换为可从图像中推断的量。这揭示了提升图表到代码模型性能的关键不仅在于更多数据或更先进的算法，还在于监督目标必须尊重从图表图像中可识别的内容。

Abstract: Chart-to-code generation is commonly trained with supervised fine-tuning on reference plotting scripts, implicitly treating the gold code as a fully observable target. We argue that this assumption is often invalid: many chart programs contain latent raw variables that cannot be uniquely recovered from the rendered image. For example, a boxplot exposes summary statistics rather than original samples, a pie chart reveals proportions rather than arbitrary raw values, and a histogram shows bin-level mass rather than individual observations. Supervising models to reproduce such non-identifiable quantities encourages hallucination and over-specified code generation. We introduce Observation-Aligned supervision, a rewriting framework that replaces latent raw-data targets with quantities constrained by the visual observation: box statistics for boxplots, wedge percentages for pie charts, and bin weights for histograms. Applying this framework to chart-to-code training data from two sources, we obtain the Observation-Aligned supervision target data. Experiments across multiple VLMs on ChartMimic and ChartX demonstrate consistent improvements in observable value recovery, including under both-executable evaluation. Our results suggest that improving chart-to-code models requires not only more data or advanced learning objectives or algorithms, but also supervision targets that respect what is identifiable from the chart image.

[26] Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment cs.CL | cs.AI | cs.LGPDF

Yu Li, Xiuyu Li, Mingyang Yi, Jiaxing Wang, zhangliangxu

TL;DR: 本文提出了一种名为选择性重要性采样（SIS）的插件方法，用于改进大型语言模型（LLM）的强化学习对齐训练。该方法通过将离策略令牌转化为在策略令牌，避免了传统重要性采样中因长序列导致的方差爆炸问题，从而稳定了训练过程。

Details

Motivation: 动机在于解决LLM强化学习后训练中“采样后更新”范式产生的离策略数据问题。传统重要性采样方法在令牌级别上会因长序列导致方差严重膨胀，影响训练稳定性和效率。

Result: 在数学和智能体基准测试中，对密集和混合专家（MoE）LLM的实验表明，SIS能持续改进所有目标，并在离策略数据下提供显著更强的鲁棒性。

Insight: 创新点在于受拒绝采样启发，提出了令牌级别的拒绝测试机制，将部分离策略令牌视为在策略令牌，从而无需重要性校正。该方法理论上减小了令牌级与序列级离策略梯度估计器之间的差距，且作为插件仅修改策略损失中的重要性比率，计算开销极低，可与多种RL后训练算法结合。

Abstract: Reinforcement learning (RL) post-training for large language models (LLMs) follows a efficient paradigm of “rollout then update”, which inevitably results in off-policy training data. To resolve this, Importance sampling (IS) is proposed, while the token-level ratios compound over long sequences, causing severe variance exploded. A natural idea is “transferring” these off-policy token into on-policy token, so that the importance scores for correction are unnecessary. Following this idea, we propose Selective Importance Sampling (SIS), which is inspired by rejection sampling. Concretely, SIS implements by viewing off-policy model as proposal distribution, and implement a token-level rejection test: accepted tokens are viewed as on-policy, so that receive unit importance score, while rejected tokens retain the standard IS correction. Our proposed SIS is theoretically proved reducing the gap between token-level and sequence-level off-policy gradient estimators. The SIS acts as a plug-in that only modifies the importance ratio in the policy loss, adding negligible wall-clock overhead, and can be combine with a vast vary of RL post-training algorithms. Experiments on dense and MoE LLMs across math and agent benchmarks show that SIS consistently improves all objectives, while providing substantially stronger robustness under off-policy data.

[27] LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure cs.CL | cs.LGPDF

Yueyang Wang, Baolong Bi, Shuo Lu, Jingyuan Zhang

TL;DR: 本文提出了一种名为LP-SFT的局部保持监督微调方法，旨在解决标准监督微调在提升目标领域性能时损害模型原有能力的问题。该方法通过分析预训练模型的多模态熵结构，设计了一个自适应损失函数来保护模型内部的局部偏好结构，从而在微调过程中更好地平衡准确性与多样性。

Details

Motivation: 标准监督微调（SFT）在适应下游任务时，通常会以牺牲模型预训练阶段已习得的丰富能力为代价，因为它只优化监督标签token，而可能破坏预训练模型学到的局部概率分布结构。

Result: 在混合领域和单一领域的微调实验中，LP-SFT在整体性能上优于标准SFT和近期SFT增强基线，在pass@1准确率和pass@k性能之间取得了最佳平衡。

Insight: 创新点在于首次通过香农熵和Renyi熵分析揭示了预训练模型固有的多模态熵结构，并据此设计了一个局部归一化的保持损失，以显式保护模型内部的相对概率结构，从而在微调中缓解能力退化并保持采样多样性。

Abstract: Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities. Standard cross-entropy fine-tuning promotes only the observed label token and leaves unconstrained how probability mass is redistributed over other plausible alternatives, potentially distorting the rich local preference structure learned during pretraining. We first analyze next-token predictions using Shannon and Renyi entropies, revealing that pretrained models exhibit a regular multimodal entropy structure. These entropy peaks correspond to varying numbers of plausible alternatives, indicating that the base model intrinsically encodes rich distributional knowledge beyond the single supervised token. Motivated by this observation, we propose LP-SFT, a Local-Preserving Supervised Fine-Tuning objective designed to explicitly protect this inherent entropy structure. At each step, LP-SFT constructs an adaptive support of alternative tokens and applies a locally normalized preservation loss to maintain the base model’s relative structure among them, while standard cross-entropy independently optimizes the supervised token. Across mixed-domain and single-domain fine-tuning experiments, LP-SFT improves overall performance over vanilla SFT and recent SFT-enhancement baselines, achieving the best balance between pass@1 accuracy and pass@k performance. These results suggest that local preservation helps mitigate capability degradation without collapsing sampling-accessible diversity.

[28] You Frame It: How Conceptual Representations Shape LLM Detection and Reasoning about Antisemitism cs.CLPDF

Katharina Soemer, Helena Mihaljević

TL;DR: 该研究探讨了不同概念表征形式（定义性、细粒度分类、示例增强和大上下文）如何影响大型语言模型（LLM）对反犹太主义的检测和推理能力。通过使用两个专家标注的数据集，在四个SOTA LLM上进行实验，发现细粒度分类表征能显著提高召回率但会降低精确度，而提供大量概念资源并未带来额外定量收益。分析还揭示了模型在解释时存在系统性局限。

Details

Motivation: 研究动机是利用LLM在推理时整合外部概念资源的能力，以检测意识形态和历史背景复杂的现象（如反犹太主义），并探索不同概念表征形式对检测和解释行为的影响。

Result: 在专家标注数据集上的实验结果表明，细粒度分类表征能大幅提升召回率，但会牺牲精确度；提供大量概念资源未带来额外定量提升；大屠杀后的反犹太主义对所有模型和配置都是最持久的挑战。

Insight: 创新点在于系统比较了不同概念表征对LLM检测复杂社会现象的影响。客观分析认为，研究揭示了概念增强LLM在检测反犹太主义方面的潜力与局限，特别是模型在解释中存在的系统性偏差（如过度引用概念、依赖词汇线索、过度自信及处理微妙或辩护形式反犹太主义的困难），这对构建更可靠的社会内容分析模型具有借鉴意义。

Abstract: LLMs enable the integration of external conceptual resources at inference time, creating new opportunities for detecting ideologically and historically complex phenomena such as antisemitism. We investigate how different forms of conceptual grounding affect antisemitism detection and explanation behavior across four state-of-the-art LLMs. Using two expert-annotated datasets, we compare definitional, fine-grained taxonomic, example-augmented, and large-context representations of antisemitism. We find that fine-grained taxonomic representations substantially improve recall, while simultaneously reducing precision. Surprisingly, supplying substantially larger conceptual resources yields no additional quantitative benefit. Post-Holocaust antisemitism poses the most persistent challenge across models and configurations. Analysis of explanations further reveals systematic limitations including overproduction of conceptual references, reliance on lexical cues, overconfidence, and difficulties with subtle or justificatory forms of antisemitism. Our findings highlight both the potential and the remaining limitations of conceptually grounded LLMs for antisemitism detection and reasoning.

[29] Knowledge Knows, Verbalization Tells: Disentangling Latent Directions for Mathematical Solvability in LLMs cs.CL | cs.LGPDF

Nikolaos Xiros, Maria-Eleni Zoumpoulidi, Georgios Paraskevopoulos

TL;DR: 该论文提出了一种在大型语言模型（LLM）中分离数学问题可解性知识与模型言语化表达的方法。研究表明，模型内部隐藏状态中，关于问题是否可解的知识和模型如何表达这一判断（即言语化）是两种不同且可线性解码的表征。通过分析提示和激活引导，论文展示了如何通过操纵言语化表征来减少模型在不可解问题上的捏造行为，从而提高模型的弃答能力。

Details

Motivation: 尽管LLMs在数学推理上取得了进展，但判断一个数学问题是否可解仍然是一项基础且具有挑战性的能力。现有研究主要从行为层面分析模型的言语化输出，而缺乏对其内部表征的探究，这限制了对该能力的分析和操控。

Result: 在多个LLMs上的实验表明，可解性知识和言语化表达在隐藏状态中被编码为不同的、可线性解码的方向。通过使用不可解性提示进行引导，可以主要通过改变言语化表征来减少捏造；激活引导实验进一步证明，操纵这些表征可以机制性地改善模型的弃答行为。

Insight: 创新点在于首次在模型内部表征层面分离了“知识”和“言语化”，揭示了模型捏造行为主要源于言语化方向的改变而非底层知识的改变。这为通过直接干预内部表征来提升模型的可信度和弃答能力提供了新的机制性途径。

Abstract: Although LLMs have made significant progress in mathematical reasoning, determining whether a mathematical problem is solvable remains a fundamental yet challenging capability. While recent studies have probed internal representations of model solvability beliefs, verbalization has primarily been studied behaviorally rather than as an internal representation, limiting its analysis and manipulation. We address this gap by separately probing representations of solvability knowledge and verbalization, allowing us to disentangle the two within model hidden states. Across multiple LLMs, we show that knowledge and verbalization are encoded as distinct, linearly decodable representations and that fabrication is primarily associated with changes in verbalization rather than the underlying knowledge. Prompting with unsolvability cues reduces fabrication primarily by shifting verbalization, while activation steering demonstrates that these representations can be echanistically manipulated to improve model abstention.

[30] Multi-Large Language Model Orchestrated Severity Assessment of Clinical Records (MOSAIC) cs.CLPDF

Manuela Del Castillo Suero, Arnault-Quentin Vermillet, Nicole Sonne Heckmann, Darmendra Ramcharran, Maurizio Sessa

TL;DR: MOSAIC是一个用于临床记录严重性评估的两阶段智能体大语言模型（LLM）框架，以2型糖尿病（T2D）为概念验证。该框架通过合成临床证据和推理电子健康记录（EHR）来生成严重性表型，其开放权重版本与专有管道性能相当，并与现有算法基准达到中等至公平的一致性，同时能有效区分全因死亡率和并发症风险。

Details

Motivation: 基于规则的电子健康记录（EHR）方法难以捕捉疾病严重性这一多维结构，而智能体LLM系统在合成临床证据和推理EHR方面的潜力尚未在此任务中得到充分评估。

Result: 在合成队列（SyntheticMass）上，开放权重的MOSAIC与专有管道性能匹配（加权Kappa = 0.773），并与Cooper（Kappa = 0.597）、DCSI（Kappa = 0.534）和DiSSCo（Kappa = 0.320）算法基准达到中等至公平的一致性。基于智能体的分级显著区分了全因死亡率（对数秩检验 p < 0.001）和并发症发生率（对数秩检验 p < 0.001）。

Insight: 创新点在于提出了一个可解释的、两阶段的智能体LLM框架，能够生成并应用涵盖血糖分期、β细胞功能和社会健康决定因素等新领域的临床严重性表型，其推理能力超越了固定规则（与冻结版本相比Kappa = 0.428），展示了智能体系统从结构化EHR数据中进行临床推理的潜力。

Abstract: Background: Disease severity is a multidimensional construct difficult to capture with rule-based approaches in Electronic Healthcare Records (EHR). Agentic large language model (LLM) systems could synthesise clinical evidence and reason over EHRs, but remain unevaluated for this task. Methods: MOSAIC is a two-phase agentic LLM framework for severity phenotyping, using type 2 diabetes (T2D) as a proof-of-concept. MOSAIC was evaluated on a synthetic cohort (SyntheticMass; open-weight N = 4,886; closed-weight N = 200) against three algorithmic ground truths (DCSI, DiSSCo, Cooper) and against all-cause mortality and incident complications. Open-weight (locally deployable) and proprietary pipelines were also compared. Results: The generated framework spanned domains absent from the comparators, including biomarker-based glycaemic staging, beta-cell function, and social determinants of health. Open-weight MOSAIC matched the proprietary pipeline (closed- vs open-weight weighted kappa = 0.773) and reached moderate agreement with Cooper (kappa = 0.597) and DCSI (kappa = 0.534) and fair agreement with DiSSCo (kappa = 0.320). Agent-based (Type 1) tiers showed significant separation of all-cause mortality (log-rank p < 0.001; crude hazard ratios 1.6-2.4 for non-Baseline tiers), with non-monotonic separation at the upper tiers, and an inverse gradient for incident complications (log-rank p < 0.001) consistent with depletion of susceptibles. Agentic classification also diverged from deterministic execution of the same rubric (MOSAIC Frozen; kappa = 0.428), indicating reasoning beyond fixed rules. Conclusion: MOSAIC shows agentic LLM systems can generate and apply clinically meaningful severity phenotypes from structured EHR data in T2D. Extending it to other diseases with similarly multidimensional severity warrants further research.

[31] Unified Audio Intelligence Without Regressing on Text Intelligence cs.CL | cs.AI | cs.LG | cs.SD | eess.ASPDF

Zhifeng Kong, Sang-gil Lee, Jaehyeon Kim, Boxin Wang, Zihan Liu

TL;DR: 本文提出了Nemotron-Labs-Audex-30B-A3B (Audex)，一个基于纯文本MoE大语言模型构建的统一音频-文本大语言模型。它采用简单的统一解码器架构，将音频编码并投影到文本嵌入空间，实现了强大的音频-文本融合与多模态生成能力。通过精心策划的大规模数据集和多阶段训练，该模型在音频理解、语音识别与翻译、文本转语音、音频生成等任务上达到SOTA，同时基本保持了其纯文本骨干模型的推理、对齐和知识能力。

Details

Motivation: 旨在构建一个统一的音频智能模型，能够同时理解和生成音频与语音，并解决现有模型在融合音频与文本能力时可能出现的性能退化问题。

Result: 在音频理解、语音识别与翻译、文本转语音、音频生成和语音到语音生成等多个任务上实现了最先进的性能，同时其纯文本骨干模型的推理、对齐、知识和长上下文等能力仅有轻微或没有退化。

Insight: 其核心创新在于采用简单的统一解码器架构，将音频输入映射到文本嵌入空间，实现了音频与文本的无缝融合与统一生成，这种设计使其能兼容标准LLM基础设施。此外，大规模多模态数据集的精心策划和多阶段训练策略（包括纯文本强化学习和多领域策略蒸馏）也是其成功的关键。

Abstract: Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM. Audex adopts a simple unified design with a single Transformer decoder: audio inputs are encoded and projected into the text embedding space, while text tokens and quantized audio output tokens are treated uniformly during generation. This architecture enables strong audio-text fusion, seamless multimodal generation, and compatibility with standard LLM training and inference infrastructure. For training, we meticulously curate audio-text datasets comprising 157.4B audio tokens and 320.5B text tokens. We apply multi-stage supervised training on these datasets, followed by text-only Cascade RL and multi-domain on-policy distillation. Audex delivers state-of-the-art audio understanding, speech recognition and translation, text-to-speech, audio generation, and speech-to-speech generation, while preserving very compelling reasoning, alignment, knowledge, long-context, and agentic capabilities of its text-only LLM backbone with marginal or no regression. We release the model checkpoints to facilitate open research.

[32] REDDIT: Correcting Model-Generated Timestamp Drift in ASR without Forgetting via Replay-Based Distribution Editing cs.CL | cs.AI | cs.SDPDF

Cheng-Kang Chou, Ming-To Chuang, Ke-Han Lu, Chan-Jan Hsu, Hung-yi Lee

TL;DR: 本文提出了一种名为REDDIT的轻量级后训练框架，用于纠正自回归ASR系统在长非语音段中生成的时间戳漂移问题。该方法通过两阶段重放式分布编辑，在修正时间戳的同时避免了灾难性遗忘，无需人工标注即可构建校正监督数据。

Details

Motivation: 现代自回归ASR系统虽然能输出时间戳，但在长非语音段上会出现时间轴漂移，导致转录文本与音频时间线错位。直接进行时间戳校正微调会严重损害模型原有的ASR性能，引发灾难性遗忘问题。

Result: 在Whisper-tiny模型上，仅使用34.9小时校正音频并更新1.6%参数，长间隙mIoU从38.7%提升至95.0%，混合间隙域外AAS从2752ms降至223ms，同时保持CV-en MER在41.3%（普通SFT解码器调优则恶化至524.2%）。

Insight: 创新点在于提出无需人工标注的校正监督构建方法（结合VAD修剪语音段与插入非语音间隙），以及通过重放上下文编辑目标分布的两阶段框架，实现了参数高效的时间戳校正且避免灾难性遗忘。

Abstract: Modern autoregressive ASR systems can emit timestamps as decoded tokens, enabling timestamped transcription without frame-level aligners or inference-time post-processing. We show that these generated timestamps can drift across long non-speech spans: the transcript may remain plausible, but the decoded time axis drifts away from the audio. We study this non-speech-induced timestamp drift with self-built gap and long-gap benchmarks across 15 evaluated timestamp-producing ASR and audio-language systems. Naive timestamp-corrected fine-tuning improves alignment but can severely degrade non-target ASR behavior, exposing a forgetting problem. We propose REDDIT(REplay-based Distribution eDITing), a lightweight two-stage post-training framework that corrects timestamps while avoiding this catastrophic forgetting: it first edits timestamp targets under the model’s own replayed decoder context while matching the frozen base distribution on non-timestamp tokens, then applies a short edited-prefix refinement stage. In this framework, we construct correction supervision without human transcripts or human timestamp annotations by combining VAD-trimmed speech spans with inserted non-speech gaps and known concatenation offsets. On Whisper-tiny, 34.9 hours of targeted correction audio used and only 1.6% of model parameters updated, raising long-gap mIoU from 38.7% to 95.0% and reducing mixed-gap out-of-domain AAS from 2752 ms to 223 ms while preserving CV-en MER at 41.3% (versus 524.2% for ordinary SFT decoder tuning).

cs.CV [Back]

[33] Learning 3D Affordances for Blade Insertion in Cluttered Stowing cs.CV | cs.ROPDF

Tianyu Li, Harpreet Sawhney, Minju Jung, Aditya Mehrotra, Kunal Mehrotra

TL;DR: 本文提出VulcanVoxel方法，用于学习三维自由空间可供性，以解决在杂乱仓储箱中执行刀片插入任务的问题。该方法通过基于三维占据场的掩码自编码器，从单模态数据中重建刀片占据情况，从而在空间域直接推理几何可行性。

Details

Motivation: 解决机器人仓储任务中，需要推理刚性工具在杂乱环境中的自由导航空间（即可供性）的问题。传统方法将可供性建模为SE(3)位姿分布，但这是一个在错误域中提出的几何问题，无法从单模态数据中恢复多模态预测。

Result: 在10,000个真实仓库仓储场景（无需人工标注）上训练，VulcanVoxel的top-5覆盖率达到0.89，优于最佳基于位姿的基线（0.71）。其蒸馏后的学生模型能在30毫秒内实现RGB到体素的推理，而体素到体素推理需1.4秒。

Insight: 创新点在于将刀片可供性定义为三维空间的子集（空间对象），直接在空间域进行推理，而非学习位姿分布。该方法利用掩码自编码器从单模态数据中恢复多模态几何可行性预测，解决了传统生成目标（如流匹配）无法恢复几何替代方案的问题。

Abstract: Many manipulation tasks require reasoning about free-space affordances: discovering volumes where an extended rigid tool can safely navigate, complementary to surface contact affordances for grasping. Robotic stowing is a canonical instance, where a blade must sweep items aside inside cluttered fabric bins to create insertion space. Production stow systems generate millions of such episodes, but standard approaches with unimodal data infer affordances as SE(3) pose distributions, a geometric question asked in the wrong domain. VulcanVoxel keeps inference spatial: a masked autoencoder over 3D occupancy fields reconstructs blade occupancy conditioned on scene geometry, computing feasibility locally at each voxel and recovering multi-modal predictions from unimodal data. Blade affordances are spatial objects, subsets of 3D space defined by geometric feasibility. Pose parameters carry no structure for reasoning whether unobserved placements are feasible, and standard generative objectives including flow matching faithfully learn the unimodal distribution produced by execution policies and cannot recover geometric alternatives. Trained on 10,000 real warehouse stow episodes without human annotation, VulcanVoxel achieves top-5 coverage of 0.89 versus 0.71 for the best pose-based baseline, with a distilled student providing RGB-to-voxel inference in 30 ms. vs. 1.4 s. for voxel-to-voxel. We have released a dataset of real blade insertion cycles with RGB-D observations and pose trajectories at https://www.armbench.com/blade_insertion. html.

[34] DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differences cs.CV | cs.AIPDF

Yankai Yang, Yancheng Long, Bin Wen, Fan Yang, Tingting Gao

TL;DR: 本文提出DELTAVID框架，通过跨视频差异任务增强视频多模态大语言模型的细粒度时空感知能力。该框架将跨视频找差异转化为可训练的感知信号，并构建了包含10K样本的训练集和评估基准，显著提升了模型在细粒度视频理解任务上的性能。

Details

Motivation: 现有视频多模态大语言模型在开放视频理解方面取得进展，但缺乏精确的局部时空感知能力，当两个视频全局语义相似仅局部存在细微差异时，模型难以可靠识别变化并提供证据。

Result: 实验表明，DELTAVID在跨视频差异理解任务上性能大幅提升，并将学习到的局部证据能力迁移到多个通用视频理解基准（如MMVU、MLVU、Video-MME等），实现了SOTA或相当水平的性能。

Insight: 创新点在于将跨视频差异作为可扩展的代理监督信号，将视频MLLM从粗粒度语义理解推向细粒度时空证据推理；客观来看，该方法通过可控的局部差异构建训练数据，为解决细粒度感知提供了可泛化的框架。

Abstract: Video multimodal large language models have made strong progress on open-ended video understanding, but they still lack precise local spatiotemporal perception. When two videos share almost the same global semantics and differ only in a short time span or a small region, current models often fail to find the change and provide reliable evidence. We propose DELTAVID, a verifiable proxy-task framework that enhances fine-grained spatiotemporal perception with cross-video differences. The key idea is to turn cross-video spot-the-difference into a trainable perception signal, where a model identifies local changes, judges temporal boundaries, and organizes spatial evidence by comparing similar videos. To make this signal scalable to train and reliable to evaluate, we further introduce DELTAVID-10K and DELTAVID-Bench, which convert controllable local differences in real videos into evidence-labeled training and test samples. Experiments show that DELTAVID substantially improves performance on cross-video difference understanding and transfers the learned local evidence ability to general video understanding benchmarks, including MMVU, MLVU, Video-MME, VideoHolmes, VideoMMMU, LVBench, TempCompass, and LongVideoBench. These results show that cross-video differences are not only an effective way to diagnose fine-grained perception failures, but also a scalable proxy supervision that moves Video MLLMs from coarse semantic understanding toward fine-grained spatiotemporal evidence reasoning.

[35] Reliability-Aware Monocular Depth Supervision for Sparse-View Neural Reconstruction cs.CV | cs.GRPDF

Wei-Teng Chu, Yashasvini Gopalan, Changju Yuan

TL;DR: 本文研究了在稀疏视角神经重建中如何利用单目深度估计器提供几何先验，并提出了基于可靠性的选择性深度监督方法。通过使用Depth Anything V2作为密集单目深度先验，并利用尺度-偏移拟合将其对齐到度量深度，再通过RGB-only基线模型生成的光度掩码选择性地应用深度监督。该方法在Mip-NeRF-360和Splatfacto两种场景表示上进行了评估，结果表明选择性深度监督对Splatfacto的提升更明显，而对Mip-NeRF-360的渲染增益有限。

Details

Motivation: 在室外驾驶场景中，稀疏视角神经重建面临挑战，因为相机通常沿狭窄的前向轨迹移动，提供的多视角重叠有限。虽然单目深度估计器能提供密集几何先验，但其预测存在噪声，且不同图像区域的可靠性不均匀，因此需要研究如何有效利用这些先验进行监督。

Result: 在KITTISeq02的every2稀疏视角设置下，掩码单目深度监督对Mip-NeRF-360仅带来边际渲染增益，且未改善度量几何；而对Splatfacto则明显提升PSNR从14.903到15.932，降低RMSE从0.542到0.100。在KITTISeq05和Bicycle场景的额外实验中进一步验证了选择性监督的有效性，并发现当多视角覆盖已较强时，深度监督可能改善几何但损害RGB渲染质量。

Insight: 创新点在于提出了基于可靠性的选择性深度监督策略，通过光度掩码筛选低误差区域进行监督，而非简单减少监督像素数量。客观分析表明，单目深度先验对约束不足的稀疏视角重建有用，但应选择性应用并适度加权，以避免在已有强多视角覆盖时对渲染质量产生负面影响。

Abstract: Sparse-view neural reconstruction is challenging in outdoor driving scenes, where cameras usually move along a narrow forward-facing trajectory and provide limited multi-view overlap. Although monocular depth estimators can provide dense geometric priors, their predictions are noisy, and not uniformly reliable across image regions. In this work, we study monocular depth supervision for sparse-view neural reconstruction. We use Depth Anything V2 as a dense monocular depth prior, align its predictions to metric depth using scale-shift fitting, and apply depth supervision selectively through photometric masks generated from an RGB-only baseline model. We evaluate this strategy on two representative scene representations: Mip-NeRF-360 and Splatfacto. On KITTISeq02 under an every2 sparse-view setting, masked monocular depth supervision gives only marginal rendering gains for Mip-NeRF-360 and does not improve metric geometry. In contrast, Splatfacto benefits more clearly, improving PSNR from 14.903 to 15.932 and reducing RMSE from 0.542 to 0.100. Additional KITTISeq05 experiments and matched-ratio mask ablations further show that the gains for Splatfacto come from selecting reliable low-error regions rather than simply reducing the number of depth-supervised pixels. Additional experiments on the Bicycle scene show that depth supervision can improve geometry while hurting RGB rendering quality when multi-view coverage is already strong. Overall, our results suggest that monocular depth priors are useful for under-constrained sparse-view reconstruction, but should be applied selectively and with moderate weighting.

[36] Interpretable machine learning predicts Parkinson’s disease severity using motion-corrected QSM MRI and multiband multiecho fMRI features cs.CV | cs.LG | q-bio.NCPDF

Aixa X. Andrade

TL;DR: 该研究使用可解释机器学习方法，结合定量磁化率成像（QSM）和多波段多回波静息态功能磁共振成像（fMRI）衍生的局部一致性（ReHo）特征，预测帕金森病患者的运动严重程度（MDS-UPDRS Part III评分）。研究发现，成像模型具有有意义的预测信号，而结合QSM与临床变量的模型能提供最接近临床实际的预测。

Details

Motivation: 研究动机在于寻找客观的神经影像学生物标志物，以捕捉临床检查无法直接观察到的脑部变异，从而改进帕金森病运动功能的评估。

Result: 在多个模型中，完整的fMRI、完整的QSM与临床变量组合提供了最强的全局拟合，解释了运动严重程度45.4%的方差。而选定的QSM加临床变量模型产生了最接近临床的预测，在表现最佳的模型中，其预测误差最低（MAE最低），且75.0%参与者的预测值在真实值±5分以内。

Insight: 论文的创新点在于将可解释机器学习应用于多模态神经影像（QSM和fMRI）特征，以预测帕金森病严重程度，并强调了结构和功能成像根据不同的临床预测目标贡献不同。从客观角度看，其方法通过SHAP等技术提供了特征重要性解释，突出了小脑、丘脑、纹状体、岛叶和运动皮层等区域的特征，增强了模型的可解释性和临床相关性。

Abstract: Introduction: Objective neuroimaging biomarkers may improve Parkinson’s disease motor assessment by capturing brain variation not directly observable from clinical examination. We used interpretable machine learning to predict current motor severity, measured by MDS-UPDRS Part III, from QSM and multiband multi-echo resting-state fMRI-derived ReHo features. Methods: Regional QSM and ReHo features were extracted from 28 participants, including 24 individuals with Parkinson’s disease and 4 controls. Thirteen feature-set experiments evaluated imaging-only, clinical-only, imaging-plus-clinical, full, reduced, and multimodal inputs. Support vector regression, Elastic Net, Random Forest, and XGBoost models were trained using nested cross-validation. Performance was assessed using pooled held-out R^2, RMSE, MAE, Pearson correlation, permutation testing, and the proportion of participants predicted within +/-5 MDS-UPDRS Part III points. Results: Imaging-only models carried meaningful predictive signal, whereas the clinical-only model performed weakly. Full fMRI, full QSM, and clinical variables provided the strongest global fit, explaining 45.4% of variance in motor severity. Selected QSM plus clinical variables produced the most clinically close predictions, with 75.0% of participants predicted within +/-5 points and the lowest MAE among top-performing models. SHAP highlighted cerebellar, thalamic, striatal, insular, and motor cortical features. Conclusion: QSM and multiband multi-echo fMRI-derived ReHo capture distinct, interpretable dimensions of Parkinson’s disease motor severity. These findings show that structural and functional imaging contribute differently depending on the clinical prediction goal.

[37] How many labels do you need? A decision framework for cross-habitat marine species recognition cs.CVPDF

Alzayat Saleh, Mostafa Rahimi Azghadi

TL;DR: 本文提出了一个决策框架，用于量化跨海洋栖息地迁移视觉系统时标注工作量与识别准确率之间的权衡关系。研究通过涵盖五个数据集、三个海洋区域和三个生物类别的基准测试，发现冻结的自监督基础模型特征（DINOv2 + 线性分类器）在未见过的珊瑚礁站点上泛化性能至少与完全微调的卷积基线相当，且每个物种仅需10-20张标注图像即可在新站点部署可靠的识别系统。

Details

Motivation: 生态监测中自动化图像识别需要扩展到新站点，但生态学家缺乏关于可靠部署所需标注工作量的实证指导。本文旨在解决跨栖息地迁移视觉系统时标注成本与识别性能之间的平衡问题。

Result: 在跨20个珊瑚礁站点的栖息地内迁移、跨数据集地理迁移以及少样本适应曲线三个协议下评估，冻结的DINOv2特征（仅1538个可训练参数）在未见站点上的泛化性能与完全微调的卷积基线相当或更好。每个物种仅需10-20张标注图像即可实现可靠识别，将标注工作量减少约一个数量级。

Insight: 创新点在于提出了一个量化标注工作量与识别准确率权衡的决策框架，并发现自监督基础模型能学习物种诊断性、栖息地不变的特征表示，而传统基线模型容易编码栖息地特定的捷径特征导致在新站点失效。实践上，结合冻结基础模型与简单线性分类器可大幅降低部署成本。

Abstract: Automated image recognition is increasingly used to scale ecological monitoring beyond manual annotation, yet ecologists lack evidence-based guidance on how much labelling effort reliable deployment at new sites requires. We present a decision framework quantifying the trade-off between labelling effort and recognition accuracy when transferring vision systems across marine habitats. The benchmark spans five datasets, three oceans, and three taxonomic groups (fish, corals, invertebrates), from tropical reefs in the Great Barrier Reef and French Polynesia to a temperate Danish fjord. We evaluated four recognition models (DINOv2, CLIP, ResNet-50, EfficientNet-B4) under four adaptation strategies (linear probing, LoRA, Visual Prompt Tuning, full fine-tuning) across three protocols: within-habitat transfer across 20 reef sites (240 runs), cross-dataset geographic transfer along a difficulty gradient (40 runs), and few-shot adaptation curves with 0-100 labelled samples per class (648 runs). Frozen self-supervised foundation features (DINOv2 + linear classifier, 1,538 trainable parameters) generalised to unseen reef sites at least as well as fully fine-tuned convolutional baselines four orders of magnitude larger; they learned species-diagnostic, habitat-invariant representations, whereas baselines encoded habitat-specific shortcuts that fail at new sites. As few as 10-20 labelled images per species sufficed to deploy reliable recognition at a new site, cutting annotation effort by roughly an order of magnitude. Solution. Programmes expanding to new sites can deploy reliable recognition by pairing a frozen, open foundation model (DINOv2) with a simple linear classifier and annotating only 10-20 images per species - roughly 1-4 hours per site. The framework lets programmes budget labelling effort against expected accuracy across sites, ecosystems, and platforms.

[38] Entropy-Coded MS-VQ-VAE with Learned Priors for Ultra-Low Bitrate Video Compression cs.CVPDF

Manikanta Kotthapalli, Banafsheh Rekabdar

TL;DR: 该论文提出了一种基于熵编码的多尺度向量量化变分自编码器（MS-VQ-VAE）与学习先验的超低比特率视频压缩方法。通过离散潜在表示和自回归先验模型，有效解决了连续潜在表示在极低比特率（<0.1 bpp）下无法可靠工作的问题，实现了比特率低于传统编码器（H.264/H.265）3.3-7.6倍的压缩性能。

Details

Motivation: 基于连续潜在表示的学习视频编解码器在低于0.1 bpp的极低比特率下难以可靠工作，因为缺乏可微分的率信号，拉格朗日优化无法在极端压缩比下有效权衡重建质量与比特率。论文旨在通过离散潜在表示绕过这一限制。

Result: 在UCF101测试集（64×64分辨率，32帧）上，模型在0.043-0.064 bpp的比特率下运行，比H.264和H.265的实用下限低3.3-7.6倍；所有配置在感知质量（LPIPS）上均优于H.265 CRF 36，其中K=1024时在LPIPS上绝对优势达0.072，同时比特节省5.1倍。

Insight: 创新点包括：利用向量量化的离散潜在表示硬性限制信息上限，结合学习自回归先验挖掘码本使用的幂律分布以进一步降低比特率；通过EMA稳定码本更新和死码重启解决训练不稳定性；验证了码本幂律分布和70-85%的熵效率，建立了原则性的学习熵编码流程。

Abstract: Learned video codecs based on continuous latent representations struggle to operate reliably below 0.1 bits per pixel(bpp): without a differentiable rate signal, Lagrangian optimisation cannot effectively trade reconstruction quality for bitrate at extreme compression ratios. We demonstrate that discrete latent representations sidestep this limitation entirely. In a vector-quantized(VQ) codec, the codebook size$K$ imposes a hard information ceiling of $\log_2 K$ bits per symbol; a learned autoregressive prior then exploits the non-uniform distribution of code usage – which we show follows a power law – to push actual bitrates well below this ceiling, without any rate-penalty tuning. Building on the MS-VQ-VAE architecture introduced in\cite{kotthapalli2026msvqvae}, we sweep $K \in {128, 256, 512, 1024}$ under a uniform training protocol to trace four operating points on the rate-distortion(RD) curve. We identify and resolve a critical training instability: gradient-based VQ collapses catastrophically at $K \leq 512$, whereas EMA-stabilised codebook updates with dead-code restart maintain full utilisation across all configurations. On 500 UCF101 test clips ($64!\times!64$, 32frames), our models operate at 0.043-0.064~bpp – 3.3-5$\times$ below H.264’s practical floor and $5$-$7.6\times$ below H.265’s floor at this resolution. Every MS-VQ-VAE configuration outperforms H.265 CRF,36 on perceptual quality (LPIPS) despite using $5$-$7.6\times$ fewer bits. At $K{=}1024$, the model surpasses H.265 CRF,36 on LPIPS by a margin of 0.072 absolute while using $5.1\times$ fewer bits. Codebook analysis confirms power-law index distributions and 70-85% entropy efficiency, establishing the pipeline as a principled learned entropy coder.

[39] Coordinate Singularities Break Conformal Coverage for Gaze and Head Pose cs.CV | cs.HCPDF

Mohammadreza Jamalifard, Yaxiong Lei, Parastoo Azizinezhad, Javier Andreu-Perez

TL;DR: 本文揭示了在视觉任务中，当输出空间为弯曲流形（如球面上的视线方向或三维头部旋转）时，使用平坦坐标图（如偏航-俯仰角或欧拉角）定义保形预测的评分函数，会在坐标奇点附近引入系统性几何失真，导致条件覆盖率的严重下降。作者通过理论分析和在四个数据集（ETH-XGaze, Gaze360, BIWI, AFLW2000-3D）上的实验验证了这一问题，并提出使用与坐标无关的测地线评分可以消除失真，且无需重新训练模型。

Details

Motivation: 保形预测为视觉系统提供了分布无关的可靠性保证，但这些保证依赖于输出空间中预测误差的度量方式。许多视觉任务的输出位于弯曲空间，而常用的评分函数（如残差、不确定性估计）却定义在平坦的坐标图上，这可能在坐标奇点附近引入失真，破坏条件覆盖保证。

Result: 在四个数据集上的实验表明，在坐标奇点区域（如球面上俯仰角大于70度或头部姿态俯仰角大于60度接近万向节锁时），名义目标为90%的切片条件覆盖率会下降30-50个百分点，例如在ETH-XGaze上降至38.9%，在Gaze360上降至42.0%。尽管边际覆盖率仍接近90%，但条件覆盖率严重崩溃。

Insight: 核心创新点在于揭示了保形预测在弯曲输出空间中，因使用平坦坐标评分而导致的系统性条件覆盖率失真这一隐藏失效模式。一个关键的诊断工具是使用黎曼体积密度这一几何量来识别覆盖率崩溃区域。提出的解决方案是采用与坐标无关的测地线评分，该方法无需重新训练模型，计算成本可忽略，并能从根本上消除几何失真，恢复正确的条件覆盖保证。

Abstract: Conformal prediction provides distribution-free reliability guarantees for vision systems, but these guarantees depend on how prediction errors are measured in the output space. Many vision tasks produce outputs on curved spaces (e.g. gaze directions on the sphere or 3D head rotations), yet intermediate prediction heads, residuals, uncertainty estimates, or conformal scores are often defined in flat coordinate charts such as yaw-pitch or Euler angles. We show that this scoring choice introduces systematic geometric distortion near coordinate singularities (large pitch angles on the sphere and poses approaching gimbal lock in 3D rotations). Across four datasets (ETH-XGaze, Gaze360, BIWI, AFLW2000-3D), slice-conditional coverage at a nominal 90% target drops by 30-50 percentage points in these regions, falling to 38.9% on ETH-XGaze and 42.0% on Gaze360 at gaze pitch above 70 degrees, and to 57.5% on BIWI and 55.2% on AFLW2000-3D at head pose pitch above 60 degrees near gimbal lock, despite marginal coverage remaining near 90%. We prove that this is structural. Scalar thresholding changes the size of chart-coordinate prediction sets but leaves their distorted axis ratios unchanged. To diagnose this hidden failure mode, we show that a simple geometric quantity, the Riemannian volume density, strongly correlates with where coverage collapse occurs. Finally, we show that coordinate-free geodesic scoring removes this distortion. It requires no retraining and adds negligible computational cost.

[40] MAGE: View-guided Point Cloud Completion with Efficient Modality Alignment and Adaptive Geometry Enhancement cs.CVPDF

Weize Quan, Zhengwei Wu, Kai Wang, Dong-Ming Yan

TL;DR: 本文提出了一种名为MAGE的视图引导点云补全框架，通过高效的模态对齐和自适应几何增强来解决跨模态几何不一致问题，在合成和真实数据集上均取得了优于现有方法的性能。

Details

Motivation: 现有基于视图的点云补全方法由于模态对齐能力弱和自几何增强有限，导致性能受限，本文旨在解决这些挑战。

Result: 在合成和真实世界数据集上的大量实验表明，该方法实现了优于现有方法的性能。

Insight: 创新点包括：提出几何感知的模态对齐机制，结合共享自注意力Transformer和跨模态重建监督；设计自适应几何感知自注意力模块，同时考虑局部几何感知注意力计算和空间变体特征融合；引入几何感知锚点细化模块来重组锚点以提升补全性能。

Abstract: View-based point cloud completion aims to recover a complete 3D shape from a partial point cloud, guided by a single-view image. However, existing approaches often suffer from limited performance due to weak modality alignment and limited self-geometry enhancement. To overcome these challenges, we propose a unified geometry-aware framework that integrates efficient modality alignment and adaptive geometry enhancement, mainly to address cross-modal geometric inconsistency of view-guided point cloud completion. Specifically, we propose a geometry-aware modality alignment by integrating a shared self-attention Transformer and cross-modality reconstruction supervision, which aims to bring features of the image and point cloud close to each other in a shared latent space describing the 3D object. To enhance the perception of global shape and local geometric details, we propose an adaptive geometry-aware self-attention module, which simultaneously considers local geometry-aware attention computation and the spatially-variant feature fusion. In addition, we apply a geometry-perceptive anchor refinement module to reorganize the anchor points (representing a local region of the shape) under appropriate supervision, further boosting the completion performance of our method. Extensive experiments on both synthetic and real-world datasets demonstrate that our method achieves superior performance over existing approaches. Our code will be available at https://github.com/weizequan/MAGE.

[41] Dual-Adaptive SAM3: Hierarchical Routing over Low-Rank Expert Layers for Parameter-Efficient Medical Image Segmentation cs.CVPDF

Ying Chen, Jinyue Li, Kun Wang, Qiankun Li, Yang Liu

TL;DR: 本文提出了一种名为Dual-Adaptive SAM3（DA-SAM3）的新型框架，旨在高效地将强大的视觉-语言模型SAM3适配到多样化的医学图像分割任务中。该框架通过任务感知的动态专家路由器和参数感知的分解参数化专家设计，在保持高分割精度的同时，实现了极致的参数效率。

Details

Motivation: 动机在于解决将SAM3这类强大的开放词汇分割模型有效适配到复杂多样的医学影像领域所面临的挑战。传统的全参数微调参数效率低下，而标准的混合专家方法又引入了过高的计算开销，限制了其临床应用。

Result: 在多个公开的医学分割基准测试上的广泛实验表明，DA-SAM3不仅匹配或超过了全参数微调的SAM3和标准MoE基线的精度，而且比当前最先进的方法取得了显著的5%性能提升。

Insight: 创新点在于其双自适应专业化机制：一是任务感知的动态专家路由器，通过联合推理视觉输入和文本概念提示来稀疏激活最相关的专家；二是参数感知的分解参数化专家设计，将每个专家表示为一个共享的冻结基座和一个轻量级可训练的低秩增量，将MoE参数开销降低了80%以上，实现了参数效率与性能的平衡。

Abstract: The Segment Anything Model with Concepts (SAM3) heralds a new paradigm for open-vocabulary segmentation through natural language interaction, offering significant potential for medical image analysis. However, effectively adapting such a powerful vision-language model to the diverse and nuanced domain of medical imaging remains a key challenge. Naive fine-tuning is parameter-inefficient, while standard Mixture-of-Experts (MoE) methods introduce prohibitive computational overhead, limiting their clinical applicability. To address this, we propose Dual-Adaptive SAM3 (DA-SAM3), a novel framework that achieves both high segmentation accuracy and extreme parameter efficiency via a dual-adaptive specialization mechanism. Our first adaptation is task-aware: a Dynamic Expert Router (DER) that sparsely activates the most relevant experts by jointly reasoning about the visual input and the textual concept prompt, mimicking a clinical consultation process. Our second adaptation is parameter-aware: a Decomposed Parameterized Experts (DPE) design that represents each expert as a shared frozen base (inherited from the pretrained SAM3) and a lightweight trainable low-rank delta, reducing MoE parameter overhead by over 80%. Extensive experiments on multiple public medical segmentation benchmarks demonstrate that Dual-Adaptive SAM3 not only matches or exceeds the accuracy of fully fine-tuned SAM3 and standard MoE baselines, but also achieves a notable 5% gain over current state-of-the-art methods, with interpretable results validating its effectiveness. The code is available at: https://github.com/Reconsider80/DA-SAM3.

[42] Uncertainty-Aware Last-Layer Adaptation of RETFound for Referable Diabetic Retinopathy Screening Under Dataset Shift cs.CV | cs.LGPDF

Karim Mardhani

TL;DR: 本文对RETFound（一种自监督视觉Transformer视网膜基础模型）在可转诊糖尿病视网膜病变筛查任务中，针对数据集偏移下的不确定性感知最后一层适应方法进行了以安全为中心的实证评估。研究比较了缓存特征softmax头、后处理温度缩放、变分贝叶斯最后一层头、对角拉普拉斯最后一层近似和SNGP风格缓存特征头等方法。在APTOS数据集上，不确定性感知操作点提高了敏感性和选择性转诊行为；但在DDR数据集上，贝叶斯方法表现较弱且迁移效果不佳。

Details

Motivation: 解决在数据集偏移下，糖尿病视网膜病变筛查模型的安全性和可靠性问题，特别是通过不确定性感知的最后一层适应方法来提升模型在安全导向操作点上的性能。

Result: 在APTOS 2019数据集上，最优选择性转诊结果可延迟约20%的病例，将接受病例的假阴性降至零，同时保持高特异性；但在DDR数据集上，贝叶斯方法仅定性复现了APTOS的趋势，但权衡效果较弱，且SNGP检查点迁移失败。

Insight: 创新点在于对不确定性感知最后一层适应方法进行以安全为中心的评估，强调超越聚合准确率的安全覆盖评估和跨数据集验证的重要性；客观分析表明，贝叶斯建模本身并非假阴性减少的唯一因素，阈值调优同样关键，且模型在数据集偏移下的泛化能力仍需加强。

Abstract: This paper presents a safety-centered empirical evaluation of uncertainty-aware last-layer adaptation for referable diabetic retinopathy screening using RETFound, a self-supervised vision-transformer retinal foundation model used here as a frozen feature encoder, and the public APTOS 2019 and DDR diabetic retinopathy fundus image datasets. We compare a cached-feature softmax head, post-hoc temperature scaling, variational Bayesian last-layer heads, a diagonal Laplace last-layer approximation, and an SNGP-style cached-feature head. On APTOS, uncertainty-aware operating points improved sensitivity and selective-referral behavior. The strongest APTOS selective-referral result deferred approximately 20 percent of cases and reduced accepted-case false negatives to zero while preserving high accepted-case specificity. However, threshold tuning also reduced false negatives at high false-positive cost, so false-negative reduction alone was not unique to Bayesian modeling. On DDR, native Bayesian heads qualitatively reproduced the APTOS direction but with weaker tradeoffs, while the APTOS-trained SNGP checkpoint transferred poorly and failed to provide useful external selective-referral behavior. These results highlight the value of safety-centered evaluation beyond aggregate accuracy: uncertainty-aware last-layer heads can improve internal safety-oriented operating points, but trustworthy retinal screening claims require explicit safety-coverage evaluation and second-dataset validation under shift.

[43] Criterion-Conditional In-Context Learning: Evaluating Criterion-Shift Adaptation in Vision-Language Models cs.CV | cs.AIPDF

Kaiyun Yang, Ruilin Yang, Zhimin Yao, J. Wang, Wei Ge

TL;DR: 本文提出了一个名为准则条件上下文学习的新评估设置，用于衡量视觉语言模型在任务语义固定但决策准则变化时的适应能力。作者构建了CC-Bench多领域基准，并设计了准则不变性和准则敏感性两个互补的评估指标。实验发现现有模型普遍存在僵化的边界偏差，而简单的多准则训练策略能显著提升模型对准则变化的敏感性。

Details

Motivation: 现实应用中许多任务的高层意图稳定，但具体决策准则会随需求变化，而标准上下文学习设定中决策准则固定，无法评估模型对此类准则偏移的适应能力。

Result: 在CC-Bench基准上的实验表明，大多数模型表现出僵化的边界偏差，难以根据潜在准则调整预测。通过简单的多准则训练，7B规模模型的准则敏感性得到显著提升，甚至能超越某些专有模型，且不损害通用多模态性能。

Insight: 创新点在于提出了准则条件上下文学习这一新评估范式，并设计了能合法生成基于活动准则的真实标签变化的数据层次结构。研究揭示了模型对准则偏移的适应不足，并证明通过多准则暴露训练可以有效缓解此问题，为提升模型在实际动态场景中的实用性提供了方向。

Abstract: Vision-language models can perform new tasks without parameter updates through in-context learning (ICL), whose core mechanism is utilizing the support set for task induction. In the standard ICL setting, once the task is induced, its decision criterion remains fixed. However, in real-world applications, many tasks exhibit a stable high-level intent, while their decision criteria shift according to specific requirements. Thus, we introduce a new setting, denoted as Criterion-Conditional In-Context Learning (CC-ICL), where models must infer the latent criterion from context and adjust predictions accordingly under fixed task semantics. To evaluate this capability, we propose two complementary metrics, Criterion Invariance and Criterion Sensitivity, capturing the model’s robustness and adaptability under criterion shifts. We further construct CC-Bench, a multi-domain benchmark that supports evaluation under the CC-ICL setting. By employing a dual-level data hierarchy, CC-Bench enables legitimate ground-truth variation conditioned on the active criterion even when the task remains fixed. Experiments on CC-Bench reveal that most models exhibit a rigid boundary bias, struggling to align their decisions with the latent criterion. We also find that even a simple multi-criterion training strategy can significantly reduce this bias, improving Criterion Sensitivity and enabling 7B-scale models to surpass proprietary models without degrading general multimodal performance.

[44] Reliability-Aware CT-MRI Registration: A Quality Engineering Framework with Stability Analysis and Risk Classification cs.CVPDF

Nisreen Albzour

TL;DR: 本文提出了一种可靠性感知的CT-MRI多模态图像配准框架，该框架通过数据学习到的阈值将配准质量转化为绿/黄/红风险等级。在包含脑部、腹部和颈部的90对患者图像切片上，评估了刚性和仿射变换的配准性能，发现仿射配准在NMI和SSIM指标上优于刚性配准，并获得更高的绿色分类比例。该框架为多模态配准提供了一个可解释的质量控制层。

Details

Motivation: 当前多模态CT-MRI配准流程通常只报告聚合质量指标，缺乏针对每个具体病例的可靠性信号，这在实际临床应用中存在风险。本文旨在解决这一问题，为图像引导放疗、手术导航和诊断工作流程提供更可靠的配准质量评估。

Result: 在包含18名患者、90对切片的脑部、腹部和颈部数据集上，仿射配准在NMI和SSIM指标上优于刚性配准，获得了44%的绿色分类（刚性为33%）。经过可靠性过滤的配准方法比未过滤方法提高了平均对齐效果。腹部配准的可靠性明显强于脑部配准。

Insight: 创新点在于将配准可靠性量化为单一风险评分R，并基于数据学习到的阈值进行风险分类，为临床工作流提供了一个可解释的质量控制层。从客观角度看，其结合了Delta NMI、Delta SSIM、Dice重叠度、配准稳定性和逆一致性误差等多个指标，并通过权重敏感性分析识别出Dice重叠度是可靠性的主导成分，这种多指标融合与风险分类框架具有借鉴意义。

Abstract: Multimodal CT-MRI registration is central to image-guided radiotherapy, surgical navigation, and diagnostic workflows, but most pipelines report only aggregate quality metrics without per-case reliability signals. We propose a reliability-aware framework that converts registration quality into Green/Yellow/Red risk categories using data-learned thresholds. CT images were registered to T1-weighted MRI using rigid and affine transformations on 90 paired slices from 18 patients across brain, abdominal, and neck anatomies. Reliability was assessed using Delta NMI, Delta SSIM, Dice overlap, registration stability, and inverse consistency error, combined into a single score R. Thresholds learned from training patients were applied unchanged to held-out test patients. Affine registration outperformed rigid registration on NMI and SSIM, yielding 44% Green classifications versus 33% for rigid. Reliability-filtered registrations improved the average alignment profile compared with unfiltered methods. Per-anatomy analysis showed substantial variation, with stronger reliability for abdominal registrations than brain registrations. Weight sensitivity analysis identified Dice overlap as the dominant reliability component. The proposed framework provides an interpretable quality-control layer for multimodal registration, while risk thresholds reflect statistical rather than clinical validation.

[45] RotateAttention: RoPE-Aware Rotation and Range Rectification for INT4 Quantized Attention in Video Generation cs.CVPDF

Yaofu Liu, Wanli Lan, Jinxi Li, Binhang Yuan, Harry Yang

TL;DR: 本文提出了RotateAttention，一个专为配备3D RoPE的DiT视频生成模型设计的高效混合精度INT4 FlashAttention框架。它通过RoPE感知的旋转和范围优化的P量化技术，解决了现有低比特量化方法在应用在线旋转矩阵和量化非负注意力矩阵时面临的挑战，在保持视频生成质量的同时实现了显著的加速。

Details

Motivation: 在配备3D RoPE的DiT视频生成模型中，注意力机制因其序列长度的二次复杂度成为主要计算瓶颈。现有低比特量化方法难以协调在线旋转矩阵与RoPE，且对称量化会浪费非负注意力矩阵P的一半4比特动态范围。

Result: 实验表明，RotateAttention在保持视频生成质量与全精度基线几乎相同的同时，实现了高达1.68倍的端到端加速和2.2倍的内核级加速。

Insight: 创新点在于观察到Q和K的异常值分布受3D RoPE维度划分的强烈影响，并据此提出了可融合到RoPE中的可合并旋转矩阵（RoPE-aware Rotation）以及充分利用INT4数值范围的范围优化P量化（Range-optimized P Quantization）。

Abstract: In \textbf{DiT-based video generation models equipped with 3D Rotary Position Embeddings (3D RoPE)}, the attention mechanism remains a primary computational bottleneck due to its quadratic complexity with respect to sequence length. While quantized \textbf{FlashAttention} offers a promising path toward hardware acceleration, existing low-bit quantization methods overlook two critical challenges in this setting: \textbf{1)} applying online rotation matrices – a widely used technique for mitigating outliers in Queries ($Q$) and Keys ($K$) – is difficult to reconcile with \textbf{RoPE}; and \textbf{2)} the non-negative attention matrix $P = \exp(QK - \max(QK))$ makes symmetric quantization waste half of the 4-bit dynamic range. In this work, we observe that the outlier distributions of $Q$ and $K$ are strongly affected by the dimensional partitioning of \textbf{3D RoPE}. Based on this finding, we propose \textbf{RotateAttention}, an efficient \textbf{mixed-precision INT4 FlashAttention} framework tailored for \textbf{DiT-based video generation models with 3D RoPE}, using selective \textbf{FP16 fallback} for accuracy-sensitive attention blocks and denoising steps. RotateAttention introduces two core techniques: \textbf{1) RoPE-aware Rotation}, which employs either mergeable rotation matrices that can be fused into RoPE or negligible-overhead matrices to mitigate RoPE-induced outliers in $Q$ and $K$; and \textbf{2) Range-optimized $P$ Quantization}, which uses fixed scales and zero-points to fully exploit the \textbf{INT4 numerical range} with minimal computational overhead. Experiments show that \textbf{RotateAttention} preserves video generation quality nearly identical to full-precision baselines while achieving up to 1.68$\times$ end-to-end speedup and 2.2$\times$ kernel-level acceleration.

[46] Classroom Behavior Monitoring with YOLO An Empirical Study in Higher Education Settings cs.CV | cs.CYPDF

Sinh Vu Trong, Dung Nguyen Manh, Hieu Hoang Minh, Hieu Pham Trung, Thu Pham Ha

TL;DR: 本研究提出了一种基于计算机视觉的课堂行为监控方法，利用在越南银行学院收集的真实课堂视频数据集（BAV-Classroom数据集），该数据集标注了九种不同的行为类别。研究评估并比较了最先进的计算机视觉模型，其中YOLOv11取得了最佳性能。实验结果表明，学生在课程后半段的注意力显著下降，这为学术质量管理提供了有价值的见解。

Details

Motivation: 传统课堂行为观察方法主观性强且难以扩展，本研究旨在利用计算机视觉技术实现自动化、可扩展的课堂行为监控，以评估学生参与度并提升教学效果。

Result: 在提出的BAV-Classroom数据集上，YOLOv11模型取得了最佳性能，达到了最先进的水平。定量实验揭示了学生在课程后期注意力显著下降的模式。

Insight: 论文的创新点在于构建并发布了一个真实场景下的课堂行为视频数据集，并系统评估了YOLO等目标检测模型在该任务上的可行性。从客观角度看，将通用目标检测模型（YOLOv11）应用于特定教育监控场景，并得出可量化的行为模式结论，具有实践借鉴意义。

Abstract: Classroom behavior monitoring plays a vital role in evaluating student engagement and improving teaching effectiveness. Traditional observation methods remain subjective and lack scalability. This study introduces a real-world dataset of classroom videos collected at the Banking Academy of Vietnam (BAV-Classroom dataset), annotated with nine distinctive behavioral categories. State-of-the-art Computer Vision models were evaluated and compared, with YOLOv11 achieving the best performance. Experimental results indicate that students’ concentration often decreases notably during the final part of lectures, highlighting challenges in sustaining engagement. Our findings demonstrate the feasibility of applying computer vision for automated classroom monitoring, providing valuable insights for academic quality management.

[47] Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoning cs.CV | cs.AI | cs.CLPDF

Yixin Ji, Fanghua Ye, Juntao Li, Bo Zhao, Zexuan Qiu

TL;DR: 本文提出了Homer框架，旨在解决多模态大语言模型在处理长视频时面临的在线处理和有限内存挑战。该框架通过构建层次化记忆结构来存储视频的多尺度信息，并采用智能体推理机制进行多轮记忆检索和验证，以提升对长视频内容的理解和问答能力。

Details

Motivation: 现有在线方法在处理长视频时，要么保留缺乏语义结构的紧凑视觉表示，要么构建基于时间邻近性而非显式因果链接的高级记忆，导致多跳叙事推理能力不足。

Result: Homer在M3-Bench-robot、M3-Bench-web和Video-MME-Long基准上分别比先前最佳智能体方法提升了5.5、10.8和4.4个百分点，并持续提升了三种不同LLM骨干的性能，显示出其在长视频检索任务上的模型无关结构能力。

Insight: 创新点在于构建了模拟长视频多尺度结构的层次化记忆，以及采用类似人类推理方式的智能体探索机制，通过多轮检索和验证步骤实现更精确的因果推理和答案生成。

Abstract: Multimodal large language models excel on short clips but struggle on hour-long videos in an online setting, where frames are processed incrementally under limited memory. Existing online methods either retain compact visual representations that lack semantic structure, or build higher-level memory stores organized around temporal proximity rather than explicit causal links, leaving multi-hop narrative reasoning to be reconstructed by the LLM at every query. We bridge this gap with \textsc{Homer}, a Hierarchical Online Memory Exploration and Reasoning framework. \textsc{Homer}’s memory mirrors the multi-scale structure of long videos, ranging from raw perception, to recurring entities, to events connected by explicit temporal and causal relations. Its agentic reasoner then explores this memory the way humans do, locating the relevant scene, looking up details, and composing the answer through multi-round memory retrieval, with a harness that verifies and corrects each step. \textsc{Homer} outperforms the previous best agent method by $+5.5$, $+10.8$, and $+4.4$ points on M3-Bench-robot, M3-Bench-web, and Video-MME-Long, and consistently lifts three various LLM backbones, indicating a model-agnostic structural capability for grounded retrieval over long videos.

[48] H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation cs.CV | cs.LGPDF

Qixiang Yin, Huanjin Yao, Yuchen Cai, Jianghao Chen, Ziyi Wang

TL;DR: 本文提出了一种名为H-OPD的置信度感知异构多教师策略蒸馏框架，用于解决多模态推理任务中现有方法依赖静态教师路由（如按模态或任务类型分配单一教师）的不足。该方法通过验证异构教师在相同推理过程中的互补性，在共享的学生轨迹上实现了基于token级别的教师仲裁，并利用视觉到语言的描述迁移使纯文本教师能够获取关键视觉语义。

Details

Motivation: 现有基于策略蒸馏的多模态推理方法通常采用静态教师路由，为每个样本分配单一教师，这忽略了视觉定位和抽象推理可能主导不同解码步骤，导致单一教师不足以覆盖完整轨迹。

Result: 在11个广泛使用的推理基准测试上进行的大量评估表明，该方法取得了优越的性能。

Insight: 核心创新点在于用token级别的动态教师仲裁取代了任务或样本级别的静态教师路由，并引入了置信度感知的仲裁机制，在每一步解码时动态结合视觉-语言教师和纯文本教师。这充分利用了异构教师的互补性，并通过视觉到语言的描述迁移扩展了纯文本教师的能力范围。

Abstract: On-policy distillation (OPD) has recently emerged as an effective post-training paradigm by providing supervision on student-generated trajectories. However, existing OPD methods for multimodal reasoning usually rely on a static teacher routing, assigning each sample to a single teacher based on modality or task type. This ignores that visual grounding and abstract reasoning may dominate different decoding steps, making a single teacher insufficient for the full trajectory. To this end, H-OPD is proposed as a confidence-aware heterogeneous multi-teacher OPD framework for multimodal reasoning. By verifying the complementarity of heterogeneous teachers in the same reasoning process, H-OPD replaces task or sample level teacher routing with token-level teacher arbitration along the shared student trajectory. H-OPD employs vision-to-language description transfer to enable text-only teachers to access key visual semantics, and uses a confidence-aware arbitration mechanism to dynamically combine vision-language teacher and text-only teachers at each token. Extensive evaluations over 11 widely-used reasoning benchmarks showcase the superior performance of our method.

[49] Evaluating Agentic Harness Systems for Autonomous Computational Pathology cs.CVPDF

Jie Lin, Zongyi Chen, Qiaoling Zheng, Liuyi Wang, Hengyi Jiang

TL;DR: 本文提出了ACP-Bench框架，用于评估自主计算病理学（ACP）中智能体系统将高级病理分析目标转化为可执行、可追溯且临床边界明确的工作流程的能力。该基准测试了41个病理工作流程任务，涵盖多个身体系统组和终点家族，并对9个模型和3个智能体系统组进行了全面评估。

Details

Motivation: 动机是实现自主计算病理学（ACP），这需要将通用的智能体系统适配到病理学特定的任务、工具、证据标准和临床声明边界中，以评估这些系统能否在声称可靠的临床自主性之前，实际可操作化病理工作流程。

Result: 在ACP-Bench上评估了9个模型和3个智能体系统组（Claude Code、Codex和Open Code），产生了369条完整轨迹。评估发现，工作流启动、任务解释和诊断报告比工具绑定执行、结果绑定和反射性工作流修订更成熟，而正式的端到端完成仍然罕见。

Insight: 创新点在于提出了一个可重复使用的标准框架ACP-Bench，它结合了专家裁决的过程审计、诊断评估和病理学家验证的安全审查，以全面评估智能体系统在病理工作流程中的执行、诊断性能和临床边界对齐能力，为临床自主性提供了审计基准。

Abstract: Autonomous computational pathology (ACP) converts high-level pathology analysis goals into executable, traceable and clinically bounded workflows. Realizing this capability requires adapting general agentic harness systems to pathology-specific tasks, tools, evidence standards and clinical claim boundaries. We contribute ACP-Bench, a framework that adapts existing harness systems from computational pathology support toward ACP workflow capability. ACP-Bench evaluates 41 pathology workflow tasks, including 24 biomarker, 7 morphology and 10 prognosis tasks spanning 6 body-system groups and 9 endpoint families. The benchmark evaluates 9 models and 3 harness groups (Claude Code, Codex and Open Code), yielding 369 complete trajectories. ACP-Bench evaluates each trajectory across workflow execution, diagnostic performance and clinical-boundary alignment, combining expert-adjudicated process audits, diagnostic assessment and pathologist-validated safety review. Across evaluated systems, workflow initiation, task interpretation and diagnostic reporting were more mature than tool-bound execution, result binding and reflective workflow revision, and formal end-to-end completion remained rare. ACP-Bench provides a reusable standard for auditing whether agentic systems can operationalize pathology workflows before claims of reliable clinical autonomy.

[50] Token-level Response-visual Attention Guidance for Multimodal LLMs Knowledge Distillation cs.CV | cs.CLPDF

Jaehyun Jang, Eunseop Yoon, Hee Suk Yoon, SooHwan Eom, Mark A. Hasegawa-Johnson

TL;DR: 本文提出了一种名为Token-level Response-visual Attention Guidance（TRAG）的知识蒸馏方法，用于压缩多模态大语言模型（MLLMs）。该方法通过将蒸馏重点从传统的提示-视觉注意力转向响应-视觉注意力，并采用基于注意力熵的自适应加权KL散度进行token级别的目标优化，从而更有效地指导学生模型模仿教师模型的视觉关注模式。

Details

Motivation: 现有知识蒸馏方法在压缩多模态大语言模型时，仅依赖输出token分布已被证明不足，而直接蒸馏注意力信号的方法中，应蒸馏哪些注意力信号及其效用尚未得到充分探索。本文旨在解决这一问题，通过分析发现下游性能与响应-视觉注意力的相似性高度相关，而与提示条件注意力的相关性可忽略，从而提出更精确的蒸馏目标。

Result: 在多个基准测试上的广泛实验结果表明，TRAG显著优于先前的知识蒸馏基线方法，验证了其有效性。

Insight: 创新点在于揭示了响应-视觉注意力（而非提示-视觉注意力）对下游任务性能的关键影响，并提出了token级别的自适应注意力熵加权机制，以更精细地指导学生对教师视觉关注模式的模仿，这为多模态模型压缩提供了新的视角和实用技术。

Abstract: While knowledge distillation (KD) is widely adopted for training lightweight models by leveraging supervision from larger teacher models, relying solely on output token distributions has proven insufficient for compressing Multimodal Large Language Models (MLLMs). Since output tokens are a byproduct of the model attending to visual inputs, prior works have explored explicitly distilling attention to provide a direct supervisory signal. While promising, the precise utility of which attention signals to distill remains under-explored. In this work, we challenge the conventional reliance on prompt-to-vision attention by revealing that downstream performance correlates strongly with response-to-vision attention similarity to the teacher, but negligibly with that of prompt-conditioned attention. Furthermore, we observe that attention distributions exhibit significant variance across individual tokens, indicating that a uniform distillation objective is suboptimal. To this end, we introduce Token-level Response-visual Attention Guidance (TRAG), a distillation objective that 1) shifts the focus to response-to-vision signals and 2) employs token-specific objectives by adaptively weighting the Kullback-Leibler divergence based on attention entropy, effectively guiding the student to mirror the teacher’s precise visual focus. Extensive experimental results on multiple benchmarks demonstrate that TRAG significantly outperforms prior distillation baselines.

[51] Privacy-Preserving Industrial Ergonomics: mmWave-Based Automated REBA Scoring and Pose Estimation cs.CV | cs.LGPDF

Xuhan Zhang, Zhuangzhuang Dai, Luis J. Mans, Victor Chang

TL;DR: 本文提出了一种基于毫米波雷达的隐私保护工业人机工程学评估框架，通过端到端多任务学习从稀疏点云重建3D人体骨架并自动生成REBA风险评分，在MMFi数据集上实现了77.78%的分类准确率和5.70毫秒的实时推理性能。

Details

Motivation: 解决传统人工REBA评估效率低下、基于视觉的自动化方法存在隐私泄露风险的问题，利用毫米波雷达实现非接触式、保护隐私的工效学评估。

Result: 在MMFi数据集上达到77.78%分类准确率，高风险REBA场景的平均绝对误差为0.93，显著优于直接回归和两阶段方法，推理延迟仅5.70毫秒。

Insight: 创新点包括：1) 结合生物力学约束与时间平滑性的多目标损失函数处理雷达点云稀疏性；2) 针对高风险姿势不平衡的过采样策略；3) 端到端多任务架构同步完成姿态估计与风险评估。

Abstract: Work-related Musculoskeletal Disorders (WMSDs) require continuous ergonomic assessments. While Rapid Entire Body Assessment (REBA) is a gold-standard observation tool, manual monitoring is labor-intensive, and vision-based automation leads to privacy concerns. This paper proposes a novel end-to-end multi-task learning framework for privacy-preserving ergonomic assessment using millimetre-wave (mmWave) radar. A spatio-temporal backbone reconstructs 3D human skeletons, which serves as the biomechanical foundation for a subsequent regression head to generate REBA risk scores. To overcome the sparsity of radar point clouds, we utilise a multi-objective loss function incorporating biomechanical limits and temporal smoothness constraints. Furthermore, we implement an oversampling strategy to address the imbalance of high-risk postures in existing datasets. Experimental results on MMFi dataset demonstrate that our framework achieves a Categorical Accuracy of 77.78% and real-time performance with an inference latency of 5.70 ms. Our method reaches a High-risk REBA MAE of 0.93, which significantly outperforms both direct regression and two-stage pipelines in high-risk scenarios, providing a robust solution for non-invasive industrial ergonomic assessment.

[52] Latent Visual Cache for Video Reasoning cs.CV | cs.CLPDF

Yongheng Zhang, Zhipeng Xu, Hao Wu, Yinghui Li, Di Yin

TL;DR: 本文提出Latent Video Cache（Latent-VC），一种用于视频推理的循环潜在视觉缓存机制，旨在解决大型多模态模型在生成过程中视觉基础信息衰减（Visual Anchoring Decay）的问题。该方法通过将紧凑的视觉记忆持续注入解码器，增强了模型在长视频和密集基础任务中的推理能力。

Details

Motivation: 现有视频推理系统多采用’读取一次，生成多次’的范式，导致生成过程中视觉基础信息逐渐弱化，即视觉锚定衰减问题。本文旨在填补这一空白，使模型在长序列推理中保持对视觉证据的持续关注。

Result: 基于Qwen3.5-9B构建的Latent-VC在六个视频基准测试中持续优于强CoT和SFT+GRPO基线，尤其在需要密集基础和长视频任务上提升显著。同时，该方法能以更短的响应获得更高的准确率。

Insight: 创新点在于引入循环潜在视觉缓存来持久保存紧凑的视觉记忆，并通过监督对比缓存对齐和带有潜在基础奖励的视觉基础GRPO进行训练。客观来看，该方法通过潜在状态而非冗长文本链来维持视觉证据，为视频推理提供了更高效的记忆机制。

Abstract: Video reasoning requires Large Multimodal Models (LMMs) to remain grounded in dense evidence, yet existing systems largely adopt “read-once, generate-many” paradigm, in which visual grounding weakens during generation. This phenomenon has been widely observed and is known as Visual Anchoring Decay. To fill this gap, we introduce Latent Video Cache (Latent-VC), a recurrent latent visual cache inserted into the decoder to preserve compact visual memories throughout reasoning. The cache is trained with supervised contrastive cache alignment and vision-grounded GRPO with a latent grounding reward, while maintaining strict train-inference alignment through native decoder hidden states. Built on Qwen3.5-9B, Latent-VC consistently outperforms strong CoT and SFT+GRPO baselines across six video benchmarks, with especially clear gains on grounding-intensive and long-video tasks. In addition, it also achieves higher accuracy with substantially shorter responses, suggesting that latent visual caching improves video reasoning by preserving visual evidence rather than relying on longer textual chains.

[53] DynaWM: A Base-VLA-Guided World Foundation Model for Moving-Object Manipulation cs.CV | cs.ROPDF

Chongkei Chang, Zhidong Deng

TL;DR: 本文提出DynaWM，一种基于基础VLA引导的世界基础模型，用于动态移动物体的操作。该模型通过Mamba-3动作编码器、V-JEPA 2.1视觉编码器和本体感觉状态编码器提取特征，并利用流匹配DiT生成运动感知的动作轨迹，以适应多种微调或粗调的基础VLA检查点。

Details

Motivation: 现有VLA模型在操作动态移动物体时面临挑战，将端到端前向或逆向动力学模型（即世界模型）集成到高性能基础VLA架构中可能因不恰当的微调而降低预训练模型的性能。

Result: 在构建的DynaGrasp-32基准测试（涵盖速度变化、轨迹变化和多物体操作等六类任务）上，对于微调的基础VLA检查点，DynaWM相比SmolVLA、X-VLA、π0和π0.5分别提升了7.19%、45.31%、1.88%和10.94%；对于粗调检查点，性能分别提升35.13%、44.06%、35.69%和26.13%。消融实验表明视觉编码提升成功率27.50%，而去除动作条件则降低45.44%。

Insight: 创新点在于提出一种基础VLA引导的世界模型架构，通过分离动作编码、视觉编码和本体感觉编码来条件化流匹配DiT，避免直接微调基础VLA导致的性能退化，并构建了系统的动态抓取基准和数据集用于评估。

Abstract: Although vision-language-action (VLA) models have received widespread attention, many challenges remain in manipulating dynamic moving objects. In most existing approaches, end-to-end forward or inverse dynamics models, i.e., world models, are incorporated into high-performance base VLA architectures, which may degrade the performance of well-pretrained base VLA models due to inappropriate fine-tuning. In this paper, we propose DynaWM, a base-VLA-guided world foundation model that adapts to a wide variety of fine-tuned and coarse-tuned base-VLA checkpoints for moving-object manipulation. DynaWM uses a Mamba-3-based action encoder to encode the base action chunk produced by the base VLA into an action-conditioning representation, a V-JEPA 2.1 vision encoder to extract features from multi-view observation history, and a proprioceptive state encoder to encode robotic-arm proprioceptive states. These feature representations jointly condition a flow-matching DiT to regenerate motion-aware action trajectories for moving-object manipulation. For systematic evaluation, we construct the DynaGrasp-32 benchmark, covering six categories of moving-object manipulation tasks, including velocity variation, trajectory variation, and multi-object manipulation, as well as the DynaGrasp-1600 dataset, which consists of 32 scenarios, 1,600 demonstration trajectories, and approximately 1.53M images. For fine-tuned base-VLA checkpoints, DynaWM achieves percentage improvements of 7.19, 45.31, 1.88, and 10.94 over SmolVLA, X-VLA, π0, and π0.5, respectively. For coarse-tuned base-VLA checkpoints, performance increases by 35.13, 44.06, 35.69, and 26.13 percentage, respectively. Ablation experiments show that visual encoding enhances success by 27.50%, while reducing success by 45.44% if action conditioning is removed.

[54] Fusion: A Framework for Unified Sequential Token AdaptatIon in VisiOn TraNsformers cs.CV | cs.AI | cs.AR | cs.DC | cs.LGPDF

Aravind Pradeep, Samira Nazari, Mahdi Taheri, Christian Herglotz

TL;DR: 本文提出了Fusion框架，旨在统一协调视觉Transformer中的自适应推理机制，包括token合并、提前退出和token剪枝，通过分阶段设计使它们协同工作而非竞争，从而在保持精度的同时显著降低计算成本和能耗。

Details

Motivation: 视觉Transformer对所有图像区域进行几乎相同的计算，存在冗余，而现有的自适应推理方法（如token压缩和提前退出）结合使用时会导致中间表示不稳定和精度下降，因此需要一种统一框架来协调这些机制。

Result: 在ImageNet-1k上使用DeiT-S时，Fusion在可比计算预算下达到或超越了最先进的自适应ViT方法，同时将校准误差降低高达4倍，推理能耗减少48%，并在多个数据集和ViT骨干上展示了无需调优的稳定可迁移性。

Insight: 创新点包括分阶段设计（先合并token，再评估置信度，最后剪枝）确保机制协同，以及轻量级路由模块支持输入自适应压缩和推理时精度-延迟权衡调整，无需重新训练。

Abstract: Vision Transformers achieve strong image classification accuracy but process all image regions with nearly the same computation, even when many regions are redundant or uninformative. Recent adaptive inference methods reduce this cost by selectively compressing tokens or terminating inference early, but combining these mechanisms often causes unstable intermediate representations and accuracy degradation. We introduce Fusion, a unified adaptive inference framework that coordinates token merging, early exiting, and token pruning through a simple staged design: tokens are merged first, confidence is evaluated next, and pruning is applied only to samples that continue inference. This ordering allows the three mechanisms to operate cooperatively rather than competitively. Fusion further includes lightweight routing modules that adapt compression strength to each input and support inference-time adjustment of the accuracy–latency trade-off without retraining. On ImageNet-1k with DeiT-S, Fusion matches or surpasses state-of-the-art adaptive ViT methods at comparable compute budgets while reducing calibration error by up to $4\times$ and inference energy by $48%$. Experiments across ImageNet-100, CIFAR-100, and ImageNette with multiple ViT backbones demonstrate consistent transferability without dataset-specific tuning.

[55] EmoteGPT: 3D Human Facial Expressions from Natural Language Descriptions cs.CVPDF

Haoran Wang, Mohit Mendiratta, Christian Theobalt, Adam Kortylewski

TL;DR: 本文提出了EmoteGPT，一个从自然语言描述生成3D人脸表情的框架。为了解决现有方法难以进行细粒度表情控制的问题，作者将文本驱动表情合成定义为在解耦的3D形变模型参数空间中的回归问题。为此，他们首先创建了一个名为Txt2Emote的基准数据集，该数据集包含由GPT-4o生成细粒度文本标注的多样化3D表情。基于此数据集，EmoteGPT利用一个带有专用标记的多模态大语言模型来语义化地锚定表情表示，并将其解码为3DMM参数。

Details

Motivation: 现有文本生成3D人脸的方法通常联合生成身份、表情和纹理，导致难以对表情进行精确、细粒度的控制，这限制了其在虚拟化身、动画和人机交互中的应用。

Result: 通过利用大规模图像到3DMM数据进行增强训练，EmoteGPT在情感识别指标和感知表现力上超越了最先进的文本到3D人脸合成方法。该方法可集成到化身管线中，实现从文本输入生成逼真或风格化的3D化身，以及具有3D一致性的富有表现力的2D人脸。

Insight: 核心创新点在于将问题重新定义为在解耦的3DMM参数空间中进行回归，并为此构建了首个细粒度文本-3D表情配对数据集。方法上，利用带有专用标记的MLLM来桥接语言与表情参数，并通过数据增强策略提升了性能，实现了对表情的精确、可控生成。

Abstract: Precise control of 3D facial expressions from text is crucial for virtual avatars, animation, and human-computer interaction, yet existing text-to-3D methods jointly generate identity, expression, and texture, making fine-grained expression control difficult. We instead formulate text-driven expression synthesis as a regression problem in the disentangled parameter space of a 3D Morphable Model (3DMM). This setting, however, requires paired data linking detailed language to precise expression parameters, which are missing from existing resources. To fill this gap, we introduce Txt2Emote, a benchmark of diverse 3D facial expressions with fine-grained textual annotations obtained from GPT-4o and a high-fidelity face tracker, providing both explicit descriptions detailing facial features and implicit descriptions referencing the situational context behind the expression. Leveraging this dataset, we present EmoteGPT, a text-to-3D expression framework based on a Multimodal Large Language Model (MLLM) with a dedicated token to semantically ground expression representations, which are then decoded into 3DMM parameters. We further improve EmoteGPT by augmenting training with large-scale image-to-3DMM data, enabling it to surpass state-of-the-art text-to-3D face synthesis methods on emotion recognition metrics and in perceived expressiveness. Integrated into avatar pipelines, our method enables photorealistic and stylized 3D avatars, as well as expressive 3D-consistent 2D face synthesis from textual input.

[56] K9-Bench: Evaluating Multimodal LLMs on Canine-Centric Videos cs.CV | cs.AIPDF

Khush Attarde, Yusuf Ali, Megha Thukral, Divye Bhutani, Thomas Ploetz

TL;DR: 本文提出了K9-Bench，一个专注于现实世界家犬视频的新型基准测试，旨在评估多模态大语言模型在犬类为中心的长序列多模态推理任务上的能力。该基准包含907个视频和约5000个问答对，涵盖五个任务类别，并采用了一个由VLM/LLM驱动的、可扩展的数据生成流程来构建数据集。实验发现，前沿的MLLMs在犬类任务上的零样本性能有限，即使是顶级闭源模型也难处理长时程的细微姿态和交互线索的组合推理。

Details

Motivation: 动机在于，尽管MLLMs在多种输入上展现出强大的零样本能力，但在理解和建模以动物为中心的场景这一关键应用上仍未得到充分探索。鉴于动物是数百万家庭的重要组成部分，在宠物相关任务上对下一代AI模型进行基准测试，对于构建能与人类协同工作的AI系统至关重要。

Result: 实验结果表明，前沿MLLMs在犬类中心任务上表现出有限的零样本性能：最先进的闭源模型优于开源模型，但在处理跨越长时程的细微姿态和交互线索的组合推理时仍然存在困难。通用的思维链提示对此类长时程推理仅能带来有限的性能提升。

Insight: 创新点包括：1) 提出了首个专注于犬类视频理解的多模态基准测试K9-Bench；2) 设计了一个可扩展的、由VLM/LLM驱动的数据生成流程，可自动从网络挖掘视频并生成需要细粒度、多跳推理的问答对；3) 实施了偏见缓解策略以消除VLM在数据集构建过程中引入的偏差；4) 该基准和构建流程可推广至其他数据稀缺领域进行定量分析。

Abstract: MLLMs have shown strong zero-shot capabilities across diverse inputs such as across images, video, audio, and text. A crucial, yet underexplored, application of these models lies in understanding and modeling animal-centric scenarios. As animals are integral to millions of households, benchmarking next-generation AI models on pet-focused tasks, ranging from recognizing distress signals to enabling responsive robotic companions, is essential for building AI systems that can work alongside us. We introduce K9-Bench, a novel benchmark focused on real-world domestic dog videos, specifically targeting canine action and interaction understanding via approximately 5000 question-answer pairs across 907 videos spanning 5 distinct task categories that test long-form, canine-centric multimodal reasoning in MLLMs. To create this dataset, we propose a scalable, VLM/LLM-powered data generation pipeline that automatically mines canine-centric videos from the web and curates QA pairs requiring fine-grained, multi-hop reasoning over canine actions and temporally extended interaction sequences. We implement bias mitigation strategies designed to eliminate biases introduced by VLMs during dataset curation. Through extensive experimentation, we find that frontier MLLMs exhibit limited zero-shot performance on canine-centric tasks: although state-of-the-art closed-source models outperform open-source counterparts, they still struggle with compositional reasoning over subtle posture and interaction cues spread over long horizons. We observe that generic chain-of-thought prompting provides only modest performance for such long-horizon reasoning. Beyond a novel dataset for canine activity analysis, K9-Bench provides a general-purpose dataset construction pipeline that can be adapted to other low-data domains for quantitative analysis. Our project website is available at: https://ogmenrobotics.github.io/K9Bench.

[57] S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval cs.CV | cs.AIPDF

Xiaodong Wang, Xuanyi Zhao, Pedro Rodriguez, Devendra Singh Sachan, Barlas Oguz

TL;DR: S-EMBER是一个用于流式自我中心记忆检索的大规模基准测试，包含超过388小时、3141个由智能眼镜拍摄的第一人称视频。该基准模拟了可穿戴设备连续视频流的现实场景，要求AI助手在流式数据中基于视觉事件触发，进行因果、主动的记忆检索，而非传统的全局离线搜索。它提供了9448个需要手动视觉验证的问答对，用于评估模型在精确时间定位和灵活响应方面的能力。

Details

Motivation: 当前基准测试通常基于离线评估，能够访问整个视频文件，这无法模拟可穿戴智能设备面临的连续流式数据现实。因此，需要一个新的基准来推动AI助手在真实、连续的流式环境中进行长时程经验回忆（情景记忆）的能力。

Result: 对前沿模型进行广泛基准测试后发现了一个定位悖论：虽然语义推理能力随着模型参数规模增加而提升，但时间定位精度却成为一个停滞的架构瓶颈，无法通过简单地增加模型大小、分辨率或帧密度来改善。

Insight: 该论文的创新点在于提出了‘基于视觉事件触发的流式情景检索’这一范式转变，从全局离线搜索转向因果、主动的回忆。S-EMBER基准为开发下一代可穿戴AI代理提供了硬件真实、可靠的情景记忆基础。从客观角度看，其揭示的‘定位悖论’对当前大模型在视频时序理解方面的能力提出了重要质疑，强调了专门架构设计而非单纯规模扩展的必要性。

Abstract: As wearable devices enable continuous first-person recording, AI assistants must reason across long time horizons to recall past experiences-a capability known as episodic memory. Current benchmarks often rely on offline evaluation with access to entire video files, failing to simulate the streaming reality of wearable intelligence. We introduce S-EMBER (Streaming Egocentric Memory Benchmark for Episodic Retrieval), a large-scale benchmark comprising 3,141 videos totaling 388 hours of organic activity captured via Ray-Ban Meta smart glasses. S-EMBER formalizes grounded streaming episodic retrieval, a paradigm shift from global offline search to causal, active recall triggered by visual events in a continuous stream. We provide 9,448 QA pairs requiring manual visual proof through precise temporal localization and supporting flexible response lengths to simulate natural human-AI interaction. Our extensive benchmarking of frontier models uncovers a localization paradox: while semantic reasoning improves with parameter scale, temporal grounding precision remains a stagnant architectural bottleneck that does not benefit from brute-force increases in model size, resolution, or frame density. S-EMBER establishes a hardware-authentic foundation for developing grounded, reliable episodic memory in the next generation of wearable AI agents.

[58] VLRC: Vision-Language Reprojection Consistency as a scalable signal for better feed-forward 3D pretraining cs.CVPDF

Marwane Hariat, David Filliat, Antoine Manzanera

TL;DR: 本文提出了一种名为视觉语言重投影一致性（VLRC）的可扩展辅助目标，利用冻结的视觉语言表示作为语义多视图监督，以改进前馈3D模型的预训练。该方法无需额外3D标注，通过重投影密集视觉语言特征并强制对应图像位置的特征一致性，将几何与语言基础特征对齐，从而提升深度和相机估计，并实现更连贯的多视图语义融合，以支持开放词汇的3D场景理解。

Details

Motivation: 前馈3D模型通常依赖昂贵的几何监督或自监督光度目标，这些信号不完整，因此需要一种可扩展的辅助目标来利用视觉语言表示提供语义多视图监督，以改进3D预训练。

Result: 在室内和室外基准测试中，实验表明VLRC在3D重建精度和零样本开放词汇3D语义分割方面均取得一致提升。

Insight: 创新点在于引入VLRC作为辅助目标，利用冻结的视觉语言特征提供语义监督，无需额外3D标注，可无缝集成到自监督单目重建和监督预训练的前馈3D模型中，促进几何与语义的对齐，提升开放词汇3D理解能力。

Abstract: Feed-forward 3D models are commonly trained using either expensive geometric supervision or self-supervised photometric objectives, both of which provide incomplete learning signals. We introduce Vision-Language Reprojection Consistency (VLRC), a scalable auxiliary objective that exploits frozen vision-language representations as semantic multi-view supervision. Given a predicted 3D reconstruction, VLRC reprojects dense vision-language features across views and enforces feature consistency between corresponding image locations, requiring no additional 3D annotations. The objective integrates seamlessly with both self-supervised monocular reconstruction and supervised-pretrained feed-forward 3D models during unlabeled adaptation. By aligning geometry with language-grounded features, VLRC not only improves depth and camera estimation but also enables more coherent multi-view semantic fusion for open-vocabulary 3D scene understanding. Experiments on indoor and outdoor benchmarks demonstrate consistent gains in 3D reconstruction accuracy and zero-shot open-vocabulary 3D semantic segmentation.

[59] An Automated Multimodal Glaucoma Detection Framework Using ViT and a Stacking-Based Ensemble cs.CVPDF

Ishrat Jahan, Muhammad E. H Chowdhury, Murugappan Murugappan, Kanchon Kanti Podder, Tawsifur Rahman

TL;DR: 本文提出了一种基于Vision Transformer和堆叠集成学习的自动化多模态青光眼检测框架，该框架整合了眼底图像和临床数据，在样本级和患者级两种评估设置下进行青光眼检测。在公开的PAPILA数据集上，该框架在样本级多模态分类中取得了97.47%的准确率和97.50%的F1分数，在患者级检测中取得了98.97%的准确率和F1分数，并最终部署为一个端到端的网络平台。

Details

Motivation: 青光眼是一种可导致不可逆视力丧失的进行性疾病，传统诊断方法耗时且依赖专家解读，难以进行大规模筛查。本研究旨在开发一个自动化、可扩展的检测框架来解决这一问题。

Result: 在PAPILA数据集上的实验结果表明，所提框架在样本级和患者级检测中均取得了优异的性能（准确率和F1分数均超过97%），展现了强大的诊断能力。

Insight: 创新点在于提出了一个结合ViT提取深度视觉特征、整合多模态数据（眼底图像与临床信息）、并采用堆叠集成策略优化分类性能的自动化框架。其将检测任务明确区分为样本级和患者级两种评估设置，并最终实现为端到端的网络应用，具有临床实用化潜力。

Abstract: Glaucoma is a progressive eye disease that can lead to irreversible vision loss if not detected at an early stage. Conventional diagnostic procedures are often time-consuming and rely heavily on expert interpretation, limiting their scalability for large-scale screening. In this study, glaucoma detection is investigated under two evaluation settings: sample-wise, where individual samples are analyzed independently, and patient-wise, where data from each patient are aggregated for final prediction. An automated multimodal framework is proposed that integrates fundus images with clinical data. Under the sample-wise setting, detection is performed using fundus images and clinical features individually, as well as through their multimodal combination. Under the patient-wise setting, predictions are obtained by aggregating multiple fundus image representations with corresponding clinical information for each patient. Deep visual features are extracted using a Vision Transformer (ViT) architecture and classified using classical machine-learning models, with a stacking-based ensemble of the three best-performing classifiers employed to optimize performance. Experiments conducted on the publicly available PAPILA dataset demonstrate strong diagnostic performance, achieving 97.47% accuracy and a 97.50% F1-score for sample-wise multimodal classification, and 98.97% accuracy and F1-score for subject-wise detection. The proposed framework is further deployed as an end-to-end web-based platform to support automated glaucoma screening and clinical decision support.

[60] When Does Resolution Help a Frozen Backbone? Global Attention at Resolution Predicts Scalable Adaptation for Camouflaged and Marine Animal Segmentation cs.CVPDF

Tyler Rust, Chandra Kambhamettu

TL;DR: 该论文研究了在冻结视觉基础模型上进行细粒度分割时，骨干网络选择的影响，特别是分辨率与全局注意力的关系。研究发现，各向同性的ViT模型由于在整个高分辨率网格上应用全局注意力，其性能随分辨率提升而持续改善；而分层骨干网络因早期注意力局限于局部窗口并在全局阶段前进行池化，性能在较低分辨率即达到瓶颈。论文通过一个名为SALT的统一流程验证了这一点，并在伪装和海洋动物分割任务上取得了新的SOTA结果。

Details

Motivation: 动机在于探究如何有效地将冻结的视觉基础模型适配到细粒度分割任务中，特别是理解骨干网络结构（尤其是全局注意力机制与分辨率处理方式）如何影响低秩适配器的性能，以解决现有方法在分辨率提升时性能饱和的问题。

Result: 在六个骨干网络的对照研究中确立了上述模式，并通过编辑骨干网络验证了原因。在统一的SALT流程下，使用一个强大的各向同性骨干网络（仅RGB输入）在四个数据匹配的伪装分割数据集上取得了最佳S-measure，并在所有海洋和显著物体分割数据集上领先，特别是在海洋动物基准MAS3K上达到了新的SOTA（mIoU 0.878）。

Insight: 创新点在于揭示了全局注意力在高分辨率令牌集上的应用是低秩适配器将分辨率转化为准确率的关键预测因素，而池化操作有助于保持这一效益。这为选择或设计适用于细粒度分割的冻结骨干网络提供了明确的指导原则，即优先考虑各向同性、具有全局注意力的架构。

Abstract: Adapting frozen vision foundation models to fine-grained segmentation now largely depends on backbone selection. Whether the backbone applies global attention to a high-resolution token set predicts whether a low-rank adapter turns resolution into accuracy. Isotropic ViTs attend globally over the full grid and keep improving with resolution; hierarchical backbones confine early attention to local windows and pool the grid before their global stages, plateauing at lower resolutions. A controlled six-backbone study establishes the pattern, and editing the backbone points to the cause: pooling keeps the benefit, removing global attention does not. The effect is specific to low-rank adaptation. Under one fixed pipeline, SALT (Side-stem, Attention-gated U-Net, Low-rank Tuning), one RGB-only pass on a strong isotropic backbone wins the best S-measure on the four data-matched camouflaged sets, and leads every marine and salient set. It reaches a new state of the art on both marine-animal benchmarks (MAS3K mIoU 0.878).

[61] Diagnosing Aerial-View Object Detectors with Foundational Image Generative Models cs.CV | cs.AI | cs.LGPDF

Stanislav Panev, Minhyek Jeon, Vaishnavi Khindkar, Ahish Deshpande, Celso M de Melo

TL;DR: 本文提出了一种基于大规模图像生成模型的合成诊断框架，用于评估和诊断航空视角下的车辆检测器。该框架结合文本引导生成、属性控制编辑和自动属性验证，构建可控的合成测试环境，以精细评估预训练检测器在不同场景和环境条件下的性能。实验表明，合成场景的性能趋势能准确反映真实世界的弱点，并指导高效的数据收集，实现针对性增强。

Details

Motivation: 动机在于利用大规模图像生成模型作为诊断工具，解决航空和遥感领域中真实数据集难以隔离特定场景和环境条件的问题，从而对训练好的视觉系统进行细粒度评估。

Result: 在三种检测架构和三个真实航空数据集上，合成场景的性能趋势与真实世界弱点高度匹配；通过针对性补充弱类别的小规模真实数据，AP50指标提升高达13%，且所需额外样本远少于非针对性增强方法。

Insight: 创新点在于将生成模型作为可控合成测试平台，用于预测真实域性能差距并指导数据收集；框架模块化设计允许集成替代生成或视觉-语言模型，具有可扩展性。

Abstract: Recent advances in large-scale image generative models enable photorealistic scene synthesis with controllable attributes. Beyond data augmentation, their potential as diagnostic tools for trained vision systems remains unexplored in the aerial and remote sensing domains. We introduce a synthetic diagnostic framework for aerial-view vehicle detection that combines text-guided generation, attribute-controlled editing, and automated attribute verification to construct a controllable synthetic testbed. This enables fine-grained evaluation of pretrained detectors under diverse scene types and environmental conditions that are difficult to isolate in real datasets. Across three detection architectures and three real aerial datasets, synthetic scene-wise performance trends closely match real-world weaknesses. Guided by these diagnostics, targeted supplementation with small real datasets from the identified weak categories yields improvements of up to 13% AP50 while requiring substantially fewer additional samples than non-targeted augmentation. Our results show that controlled synthetic probing can predict real-domain performance gaps and guide efficient data collection. The proposed diagnostic framework is modular and can incorporate alternative generative or vision-language models as capabilities evolve. Our code and datasets are available here: https://humansensinglab.github.io/AVODDiag/

[62] Global Pose Control for Generative View Synthesis in Normalized Object Coordinate Space cs.CVPDF

Zhibing Li, Amogh Gupta, Behnoosh Parsa, Dan Casas

TL;DR: 本文提出了一种在归一化物体坐标空间（NOCS）中实现精确相机姿态控制的新方法，用于从单张或多张无姿态图像生成新视角。该方法将新视角合成重新定义为图像编辑问题，通过上下文多模态条件策略注入相机信息，并结合文本描述来消除NOCS的歧义性，从而实现对目标视角的全局、直观控制。

Details

Motivation: 现有生成式新视角合成方法难以提供全局且直观的相机姿态控制，因为它们通常使用输入相关的相对相机姿态或只能生成稀疏的全局视图，这限制了NVS在下游任务中的应用潜力。

Result: 大量实验表明，该方法能够从多样类别的任意无姿态图像中，鲁棒地生成具有准确、一致朝向的新视角图像，在图像质量和保真度方面达到了最先进（SOTA）水平。

Insight: 创新点在于将NVS任务重新定义为图像编辑问题，并利用最先进的编辑模型；提出了在NOCS中使用绝对相机姿态进行控制，结合文本描述来定义物体的规范坐标系以消除歧义；采用了上下文多模态条件策略，将相机信息作为专用相机令牌注入。从客观角度看，其构建高质量、方向对齐的数据集以及将姿态控制与文本语义相结合的策略，为解决姿态控制的模糊性问题提供了新思路。

Abstract: Novel View Synthesis (NVS) enables the generation of unseen views of a scene from a single or multiple images, allowing users to freely explore an object from any viewpoint. Despite the recent impressive qualitative improvements of generative models for this task, existing methods struggle to provide global and intuitive control of target viewpoints because they either use input-relative camera poses or are limited to generating sparse global views. This lack of global pose control severely limits the number of downstream tasks potentially enabled by NVS. To address this limitation, we propose a novel approach for precise camera control in a customizable Normalized Object Coordinate Space (NOCS), requiring single or few unposed images. Our method operates solely on the absolute camera pose of the target view in NOCS, eliminating the need for a relative world frame or camera poses of the input images. Unlike previous methods that treat NVS as a standalone generation task, we formulate it as an image editing problem and build upon state-of-the-art editing models to leverage their superior generalization capability. Camera information is injected as dedicated camera tokens via an in-context multi-modal conditioning strategy. To alleviate the inherent ambiguity of NOCS, we incorporate text descriptions that explicitly define the object’s canonical coordinate frame, which also enhances generalization to unseen object categories. Furthermore, we curate a high-quality dataset with consistently aligned orientations and corresponding NOCS text definitions. Extensive experiments demonstrate that our method robustly generates novel views with accurate and consistent orientations from arbitrary unposed images across diverse categories, achieving state-of-the-art image quality and fidelity.

[63] Signal from Space: Detecting Schools and Towers to Bridge the Digital Divide cs.CV | cs.AI | cs.CYPDF

Zakarya Elmimouni, Sandor Farkas, Fares Fourati, Vladimir Daigele, Walid Mathlouthi

TL;DR: 本文提出了一种可扩展的纯视觉框架，利用高分辨率卫星图像和迁移学习，同时完成学校与信号塔的检测任务，以评估学校周边的网络连接基础设施，从而支持大规模基础设施测绘和数据驱动的连接投资优先级排序。

Details

Motivation: 解决发展中国家因学校未联网而导致数百万学龄儿童无法上网的问题，Giga倡议旨在连接每所学校，但大规模实施需要高效方法在不依赖稀疏或嘈杂的第三方数据的情况下测绘学校和评估周边连接基础设施。

Result: 在莱索托的真实卫星图像上进行了演示，显示出在该地区的强大性能，但摘要未提及具体定量结果或基准测试比较。

Insight: 创新点在于将预训练的目标检测模型通过少量标注数据适应新地理区域，直接从太空检测学校和信号塔，并分析其空间关系作为连接可用性的代理，从而减少对辅助数据的依赖，实现纯图像驱动的大规模测绘。

Abstract: Reliable internet access is essential for modern education, yet millions of school-aged children especially in developing regions remain offline due to unconnected schools. The Giga Initiative aims to connect every school to the internet, but doing so at scale requires efficient methods to map schools and assess surrounding connectivity infrastructure without relying on sparse or noisy third-party datasets. In this work, we propose a scalable, vision-only framework that uses high-resolution satellite imagery and transfer learning to address both tasks simultaneously. By adapting pre-trained object detection models to new geographical regions with minimal labeled data, we detect schools and cell towers directly from space. We then analyze the spatial relationship between detected schools and nearby towers as a proxy for connectivity availability. This purely imagery-driven pipeline enables large-scale infrastructure mapping, reduces dependency on auxiliary data, and supports data-driven prioritization of connectivity investments in underserved areas. Our approach is demonstrated on real satellite imagery from Lesotho, showing strong performance across this region.

[64] Biomechanics-aware Multi-view Markerless Motion Capture of Dexterous Hand Movements cs.CVPDF

Pouyan Firouzabadi, J. D. Peiffer, Kunal Shah, Anton Sobinov, Lee E. Miller

TL;DR: 本研究提出了一种基于梯度优化的生物力学重建方法，用于无标记运动捕捉（MMC）下灵巧手部运动的追踪，并通过8相机系统采集了121段视频数据。该方法与传统的两阶段重建方法（先三角化2D关键点再施加生物力学约束）进行了对比，在生物力学合理性和遮挡鲁棒性方面表现更优。

Details

Motivation: 现有无标记运动捕捉技术在手部复杂运动分析中的应用滞后于其他肌肉骨骼系统，本研究旨在开发一种能够有效追踪灵巧、无约束手部运动的生物力学重建方法。

Result: 在包含6种手部姿势和5种物体操作任务的121段视频中，所提方法对所有视频均生成了解，而两阶段方法有15%未收敛。在其余视频上，所提方法产生了更具生物力学合理性的手部运动学数据，且在涉及物体的任务中对遮挡效应更具鲁棒性。

Insight: 创新点在于将梯度优化方法与生物力学模型在环路中结合，实现端到端的重建，从而更有效地利用2D关键点信息，提高了对灵巧手部运动追踪的自动化和生物力学意义。

Abstract: Markerless motion capture (MMC) techniques have been widely beneficial in biomechanical analysis of human movement; however, application to complex motions of the hand lags other musculoskeletal systems. The primary goal of this study was to evaluate the performance of a biomechanical reconstruction method that implements a gradient-based optimization approach with a biomechanical model in the loop for tracking dexterous, unconstrained hand movements using MMC. Using a custom, 8-camera setup, we acquired 121 video recordings from 6 participants performing 11 different tasks that spanned 6 hand postures, 5 object manipulation tasks, and involved motion of the proximal upper limb joints. Performance of the proposed MMC pipeline was directly compared to a more commonly adopted two-stage reconstruction method that first triangulates 2D keypoints from computer vision pose estimation algorithms to 3D and then enforces biomechanical constraints by solving a constrained inverse kinematics problem. Relative performance was assessed qualitatively by visual inspection and quantitatively using a computer vision metric. Our method generated solutions for all 121 video recordings; the two-stage method did not converge for 15% of the recordings. Across the remaining videos, our method produced more biomechanically plausible hand kinematics than the two-stage method and was more robust to occlusion effects during tasks that involved objects. The relative robustness of the end-to-end method suggests that it is more effective in utilizing the available 2D digital keypoint information. Automatic and biomechanically meaningful tracking of hand kinematics during dexterous movements has the potential to support clinical evaluation, rehabilitation monitoring, and studies of human motor control.

[65] GRCD: Grounded Region Change Detection for Multi-Finding Chest X-Ray Pairs cs.CVPDF

OFM Riaz Rahman Aranya, Peyman Najafirad, Kevin Desai

TL;DR: 本文提出了GRCD框架，用于从多发现场景下的胸部X光图像对中生成具有空间定位的报告。该框架通过构建一个经过严格清洗的时序胸部X光对数据集，并引入区域引导变化标记模块来编码解剖结构的区域级时序变化，通过结合前置空间标记与门控交叉注意力的双路径策略将变化信号注入语言模型。

Details

Motivation: 放射科医生常规比较当前和先前的胸部X光片以追踪疾病进展，生成描述多个发现（每个发现定位到解剖区域并标注时序变化状态）的随访报告。现有自动化方法要么从单张图像生成报告而未建模时序上下文，要么结合了时序信息但未对其输出进行空间定位。少数将时序推理与空间定位结合的方法仅限于单发现描述，未能处理具有混合变化方向的多发现报告。

Result: 在多发现测试集上，GRCD在文本生成和临床准确性指标上优于现有基线方法，并在变化检测方面取得提升。消融研究证实，双路径设计在文本和临床指标上优于任一单独集成策略，并且区域级变化编码对于多发现生成是必要的。

Insight: 主要创新点包括：1) 通过识别和纠正源标注中的两个系统性标注错误，构建了严格清洗的时序胸部X光对数据集；2) 提出了区域引导变化标记模块，用于编码跨解剖结构的区域级时序变化；3) 采用结合前置空间标记与门控交叉注意力的双路径策略，将区域变化信号有效注入语言模型，以支持多发现报告的生成。

Abstract: Radiologists routinely compare current and prior chest X-rays to track disease progression, producing follow-up reports that describe multiple findings, each localised to an anatomical region and annotated with a temporal change status. Existing automated methods either generate reports from a single image without modelling temporal context, or incorporate temporal information but do not ground their outputs spatially. The few approaches that combine temporal reasoning with spatial grounding are restricted to single-finding descriptions, leaving multi-finding reports with mixed change directions unaddressed. We present GRCD, a framework for grounded report generation from chest X-ray pairs in the multi-finding setting. We first construct a rigorously cleaned dataset of temporal chest X-ray pairs by identifying and correcting two systematic labelling errors in the source annotations. We then introduce a Region-Guided Change Token module that encodes per-region temporal change across anatomical structures and injects this signal into a language model through a dual-pathway strategy combining prepended spatial tokens with gated cross-attention. On a multi-finding test set, GRCD outperforms existing baselines on text generation and clinical accuracy metrics, with gains in change detection. Ablation studies confirm that the dual-pathway design outperforms either integration strategy in isolation on text and clinical metrics, and that region-level change encoding is necessary for multi-finding generation. Code is available at https://github.com/UTSA-VIRLab/GRCD

[66] Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generation cs.CV | cs.GRPDF

Long Vu, Tan Ngo, Animesh Karnewar, Amir Habibian, Binh-Son Hua

TL;DR: 本文提出UniCaMo框架，通过构建扩散模型的输入噪声，实现对视频生成中物体轨迹和相机视角的统一控制。该方法利用3D点轨迹对参考帧噪声进行局部扭曲，并结合虚拟球形噪声表示处理相机运动下的新场景区域，从而在保持几何与时间一致性的同时，无需修改底层视频扩散模型架构。

Details

Motivation: 现有图像-文本到视频扩散模型缺乏在单一生成过程中对物体运动和相机运动的精确统一控制，本文旨在解决这一问题。

Result: 在Wan 2.1等大型预训练视频扩散模型上进行轻量级LoRA微调后，UniCaMo在标准可控视频生成基准测试中取得了视频质量和运动可控性的最先进（SOTA）结果。

Insight: 创新点在于构建了3D基础的运动一致噪声空间，通过局部轨迹引导的噪声扭曲与全局球形噪声采样的结合，实现了对物体和相机运动的统一控制，且仅需修改输入噪声而无需额外适配器或模型架构改动。

Abstract: Modern image-and-text-to-video diffusion models can synthesize highly realistic videos by iteratively denoising an initial Gaussian noise tensor conditioned on reference image and text inputs. However, existing approaches still lack precise and unified controllability over both object motion and camera motion within a single generation process. We present UniCaMo, a unified framework that enables simultaneous control of object trajectories and camera viewpoints by directly constructing the input noise of the diffusion model. Specifically, UniCaMo builds a shared 3D-grounded motion-consistent noise space across latent video frames. Sparse 3D point tracks are used to warp the Gaussian noise of the reference frame along desired object trajectories, while a virtual spherical noise representation provides globally consistent noise values for newly revealed scene regions under camera motion. By combining local track-guided noise warping with global sphere-based noise sampling, UniCaMo maintains geometric and temporal consistency under both object movement and viewpoint changes. Because UniCaMo modifies only the input noise, it requires no auxiliary adapters, control branches, or architectural changes to the underlying video diffusion model. With lightweight LoRA fine-tuning on large pretrained video diffusion models, including Wan 2.1 (14B), UniCaMo achieves state-of-the-art results in both video quality and motion controllability on standard controllable video generation benchmarks.

[67] Prior Bias in Vision Language Models on UML Diagram Interpretation cs.CV | cs.SEPDF

Zaiyu Cheng, Khai-Nguyen Nguyen, Antonio Mastropaolo

TL;DR: 这篇论文研究了视觉语言模型在解释UML类图时存在的先验偏见问题。作者创建了一个受控的UML基准测试，其中每个符合先验的图表都配有一个冲突先验的对应图表，仅反转关系箭头方向但保持类名和布局不变。评估发现，开源模型在箭头反转后准确率大幅下降，即使是前沿的闭源模型也存在显著差距，表明VLMs可能依赖预训练先验而非真正理解图表。

Details

Motivation: 探究视觉语言模型在处理UML类图等软件工程工件时，是否真正读取了视觉符号的含义，还是仅仅基于预训练先验（例如类之间的典型关系）来回答问题。

Result: 在基准测试中，八个开源模型在箭头反转后关系方向准确率平均下降33.48%，GPT-5.4 Mini仍有10%的差距；在更复杂的三类条件下，开源模型准确率骤降45.28%，GPT-5.4家族平均下降18.62%。缩放模型规模仅带来有限且依赖模型家族的改进。

Insight: 论文的创新点在于设计了一个诊断性的基准测试来揭示VLMs在基于图表的软件理解中的先验驱动失败；从客观角度看，该方法提供了一种系统评估模型是否真正理解视觉符号（如UML箭头）而非依赖统计先验的工具，对VLMs在专业领域的可靠性评估具有借鉴意义。

Abstract: Vision Language Models (VLMs) are increasingly applied to software engineering artifacts, especially UML class diagrams whose meaning depends on visual notation. Yet, it is unclear whether VLMs actually read such diagrams or instead answer from pretrained priors about how classes typically relate. We introduce a controlled UML benchmark in which each prior-conforming diagram is paired with its prior-conflicting counterpart that (1) preserves the same class names and layout while (2) reverses only the relation arrow. We evaluate eight open-source VLMs from two model families, InternVL3.5 and Qwen3, alongside two closed-source frontier models GPT-5.4 and GPT-5.4 Mini. Across the eight open-source models, reversing the arrow reduces relation-direction accuracy by 33.48% on average, while GPT-5.4 Mini retains a 10% gap. In the harder three-class condition, accuracy drops sharply by 45.28% for open-source models, and even 18.62% for the GPT-5.4 family on average. Scaling provides only limited improvements and is family-dependent. Our benchmark presents a diagnostic prior-driven failure in diagram-grounded software understanding. Our artifact is available at https://anonymous.4open.science/r/UMLKnowledgeConflict-8461.

[68] SPLIT: Training-Free AI-Generated and Partially Edited Video Detection via Spatial Patch-Level Incoherence and Temporal Roughness cs.CV | cs.AI | cs.LGPDF

Jongyeop Hyun, Hyounghun Kim

TL;DR: 本文提出了SPLIT，一种无需训练的AI生成和部分编辑视频检测方法，通过空间块级不一致性和时间粗糙度来识别伪造视频。该方法利用冻结视觉编码器的块令牌，计算两步时间粗糙度（TTR）和局部空间运动不一致性（LSMI）两个互补信号，并通过伽马校正融合以在严格阈值下增强真假分离。

Details

Motivation: 在现实服务中部署AI生成视频检测器需要极低的误报率，以避免错误拒绝真实内容，而标准指标（如AUROC）无法反映实际运行行为。

Result: 在FakeParts、GenVideo和ViF-Bench三个基准测试中，SPLIT在FPR=0.1%时实现了最高的假视频召回率，显著优于有监督和无训练的基线方法，并对后处理保持鲁棒性。

Insight: 创新点包括提出基于固定FPR下假视频召回率的服务对齐评估协议，以及通过TTR和LSMI的乘法融合与伽马校正来优化低误报率下的检测性能，无需训练即可实现高效检测。

Abstract: Deploying AI-generated video detectors in real-world services demands an ultra-low false positive rate (FPR) on real videos to avoid falsely rejecting authentic content, a regime where standard metrics such as AUROC fail to reflect actual operating behavior. We introduce Spatial Patch-Level Incoherence and Temporal Roughness (SPLIT), a training-free detector that operates on patch tokens from a frozen vision encoder to detect both fully generated and partially edited videos. SPLIT computes two complementary signals: Two-step Temporal Roughness (TTR), capturing non-smooth patch trajectories via one-step and two-step feature variation contrast, and Local Spatial Motion Incoherence (LSMI), measuring spatially inconsistent temporal changes through gradients of a feature-space motion field. The two are fused multiplicatively with gamma correction to sharpen real-fake separation at strict thresholds. We further propose a service-aligned evaluation protocol based on Fake Recall at fixed FPR with real-only threshold calibration and cross-real threshold transfer. Across three benchmarks (FakeParts, GenVideo, and ViF-Bench), SPLIT achieves the highest Fake Recall at FPR = $0.1%$, substantially outperforming supervised and training-free baselines while remaining robust to post-processing with negligible overhead. The code is publicly available at https://github.com/mldljyh/SPLIT .

[69] ProLaViT: Learning Progressive Latent Visual Thoughts in Structured Latent Space cs.CV | cs.CLPDF

Peiming Li, Yifan Wang, Xiaotian Zhang, Zhiyuan Hu, Shiyu Li

TL;DR: 本文提出了ProLaViT框架，旨在增强多模态大语言模型在复杂视觉推理任务中的能力。该框架通过在连续潜在空间中进行结构化视觉推导，利用内生自蒸馏机制和可扩展的程序化合成流程，设计了两种推理范式（从粗到细的因果链和辩证推理链），并引入距离加权的多样性损失来防止特征退化。实验表明，ProLaViT在视觉中心基准测试中优于基线方法，实现了更高的准确性、可解释性和效率。

Details

Motivation: 当前多模态大语言模型在需要多步感知和逻辑推导的复杂视觉推理任务上仍存在困难，而显式视觉生成计算成本过高，现有潜在方法则常依赖外部专家或缺乏严格的认知逻辑。

Result: 在多个视觉中心基准测试上的广泛实验表明，ProLaViT超越了基线模型，取得了更高的准确性和可解释性，同时保持了高效率。

Insight: 创新点包括：1. 提出在连续潜在空间中进行结构化视觉推导的内生自蒸馏框架，无需依赖异构外部模型；2. 设计了两种结构化的推理范式（空间任务的从粗到细因果链和逻辑任务的辩证推理链）以增强认知逻辑；3. 引入距离加权的多样性损失来施加拓扑感知约束，防止特征退化并确保语义区分性。

Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress but still struggle with complex visual reasoning tasks requiring multi-step perception and logical deduction. While explicit visual generation incurs prohibitive computational costs, existing latent approaches often rely on external experts or lack rigorous cognitive logic. In this paper, we introduce ProLaViT (Progressive Latent Visual Thought), a framework empowering MLLMs to perform structured visual derivation in the continuous latent space. Unlike works dependent on heterogeneous external models, ProLaViT leverages an endogenous self-distillation mechanism, utilizing the model’s own visual encoder to supervise latent thoughts. To facilitate this, we construct a scalable programmatic synthesis pipeline enabling the model to internalize algorithmic precision without inference time tools. We design two reasoning paradigms: (1) Coarse-to-Fine Causal Chain for spatial tasks, guiding attention from global context to local targets. (2) Dialectical Reasoning Chain for logical tasks, incorporating counter-factual thinking for verification. Furthermore, we propose a Distance-Weighted Diversity Loss to impose topology-aware constraints, preventing feature degeneration by enforcing semantic distinctiveness. Extensive experiments demonstrate that ProLaViT outperforms baselines on vision-centric benchmarks, achieving superior accuracy and interpretability with high efficiency.

[70] Learning Taxonomic Trees with Hierarchical Representation Regularization for Large Multimodal Models cs.CV | cs.AIPDF

Hulingxiao He, Zhi Tan, Yuxin Peng

TL;DR: 本文提出了一种名为Hierarchical Representation Regularization（HiR^2）的即插即用正则化方法，旨在提升大型多模态模型（LMMs）的层次视觉识别一致性。该方法通过构建语义感知的视觉树，并结合双曲蕴含锥损失与判别性分散损失，在多种LMMs和微调方法中有效捕获了分类学结构。

Details

Motivation: 大型多模态模型通常缺乏分类学知识，导致其层次视觉识别一致性较低，这主要是因为它们在微调时仅依赖语言建模目标，而缺乏显式的、感知分类学结构的正则化。

Result: 大量实验表明，HiR^2方法能有效提升多种LMMs和微调方法在捕获分类学结构方面的性能，具体结果在论文中进行了广泛验证。

Insight: 创新点在于提出了一个结合双曲空间（洛伦兹模型）中蕴含锥约束与单位球面上角度分离约束的正则化框架，从而在不破坏径向层次结构的前提下，显式地建模了概念间的语义层次关系。

Abstract: Taxonomies provide key information about the semantic relationships between concepts and the inherent organization of vision and language. Despite their impressive capabilities, large multimodal models (LMMs) often lack taxonomic knowledge, leading to low hierarchical visual recognition (HVR) consistency. These models typically only rely on language modeling objectives during fine-tuning and lack explicit taxonomy-aware regularization. To address this, we propose Hierarchical Representation Regularization ($HiR^2$), a simple plug-and-play regularizer that improves hierarchical consistency in LMMs. Specifically, we introduce a semantic-aware visual tree construction framework that extracts coarse-to-fine visual features from intermediate LLM layers guided by textual cues. The regularizer combines two complementary objectives: a taxonomic entailment loss that enforces hierarchy via hyperbolic entailment cones in the Lorentz model, and a discriminative dispersive loss that promotes angular separation of semantically similar embeddings on the unit sphere without disturbing the radial hierarchical structure. Extensive experiments demonstrate that $HiR^2$ effectively captures taxonomic structures across diverse LMMs and fine-tuning methods. Code is available at https://github.com/PKU-ICST-MIPL/HiR2_ICML2026.

[71] R3D: Quantitative 3D Spatial Reasoning for Egocentric Wearables cs.CV | cs.AIPDF

Maxwell Horton, Wei Lu, Quan Tran, Yury Astashonok, Kirmani Ahmed

TL;DR: 本文提出了R3D-Bench基准数据集和R3D框架，用于解决第一人称视角RGB-D视频的定量3D空间推理问题。R3D-Bench包含3,033个问题，涵盖15种类型，基于Aria Digital Twin的57个视频序列构建。R3D是一个模型无关的空间工具调用框架，通过分割和深度提升的对象表示构建3D场景，并为LLM提供八种可组合的空间工具。

Details

Motivation: 现有基准无法有效处理自然第一人称视频、带姿态的RGB-D输入以及具有挑战性的定量3D空间推理问答，因此需要新的基准和方法来推动下一代可穿戴助手的发展。

Result: 在R3D-Bench上，结合Qwen3-VL 235B的R3D模型取得了73.5%的平均相对准确率，显著优于最佳支持深度的基线（CuTR+Tools，61.9%）和最佳仅RGB基线（Gemini 3 Flash，46.5%），达到了新的SOTA水平。

Insight: 创新点在于提出了一个专门针对第一人称3D空间推理的基准，以及一个模型无关的工具调用框架。该方法不直接将3D信息嵌入模型输入，而是通过构建显式的3D场景表示并提供可组合的工具，让LLM进行推理，这种解耦设计提高了灵活性和性能。

Abstract: Quantitative 3D spatial reasoning from egocentric RGB-D video is a critical capability for next-generation wearable assistants. Yet existing benchmarks do not reflect the challenges of handling (1) natural egocentric video, (2) posed RGB-D video inputs, and (3) challenging quantitative 3D spatial reasoning Q&A. To fill this gap, we introduce R3D-Bench (Reasoning in 3D), a benchmark of 3,033 quantitative spatial reasoning questions across 15 types – spanning multiple-choice, distance-based, and volumetric reasoning questions – built on top of 57 egocentric video sequences from Aria Digital Twin. To set a strong baseline on this dataset, we introduce R3D, a model-agnostic spatial tool-calling framework. In contrast to existing approaches that directly embed 3D information into the model’s input representation, R3D constructs a 3D scene from video using segmentation and depth-lifted object representations. It provides this information to an LLM through eight composable spatial tools. On R3D-Bench, R3D with Qwen3-VL 235B achieves 73.5% mean relative accuracy, substantially outperforming the best depth-enabled baseline (CuTR+Tools, 61.9%) and the best RGB-only baseline (Gemini 3 Flash, 46.5%).

[72] STAC: Selective Spatiotemporal Aggregation and Compression for Video Reasoning Segmentation cs.CVPDF

Syed Ariff Syed Hesham, Yun Liu, Guolei Sun, Jing Yang, Henghui Ding

TL;DR: STAC提出了一种选择性时空聚合与压缩方法，用于视频推理分割任务。该方法通过解耦的双向空间扫描和因果时间扫描来增强特征表示，并利用状态空间模型的线性递归特性在低成本下评估内容冗余度，从而实现层次化压缩。在零样本流式兼容设置下，STAC在推理分割基准上超越了无压缩基线，同时实现了85%的令牌减少和1.8倍的加速。

Details

Motivation: 视频推理分割需要在复杂自然语言查询下跨数百帧进行像素级精确的对象跟踪，产生密集的时空令牌，其二次自注意力成本使得长视频处理难以实现。现有方法通过令牌压缩来解决，但通常在缺乏时间上下文编码器特征上操作，限制了在可靠评估内容冗余之前的压缩选择。

Result: STAC在视频推理分割基准测试中，在零样本流式兼容设置下超越了无压缩基线方法，同时实现了85%的令牌减少和1.8倍的推理速度提升。

Insight: 创新点在于利用状态空间模型的线性递归（O(T)成本）来选择性条件化每个令牌的时间上下文，从而在低成本下获得可评估内容冗余的表示。在此基础上，通过解耦的双向空间和因果时间扫描来丰富特征，并利用递归衍生的冗余进行具有自适应阈值（通过分割目标优化）的层次化压缩。

Abstract: Video reasoning segmentation demands pixel-accurate object tracking across hundreds of frames under complex natural language queries, producing dense spatiotemporal tokens whose quadratic self-attention cost makes long-video processing prohibitive. Existing methods address this through token compression, yet typically operate on encoder features lacking temporal context, constraining selection before content redundancy can be reliably assessed. Informed compression requires contextual awareness, but acquiring that awareness at full resolution incurs the same quadratic cost compression aims to reduce. State-space models resolve this constraint, as their linear recurrence selectively conditions each token on temporal context at $\mathcal{O}(T)$ cost, producing representations where content redundancy becomes assessable. Building on this, Selective SpatioTemporal Aggregation and Compression (STAC) enriches features via decoupled bidirectional spatial and causal temporal scanning, leveraging recurrence-derived redundancy for hierarchical compression with adaptive thresholds optimised with segmentation objective. STAC achieves 85% token reduction and 1.8$\times$ speedup while surpassing compression-free baselines on reasoning segmentation benchmarks in a zero-shot streaming-compatible setting. Code is available \href{https://github.com/MCG-NKU/nku-video}{here}.

[73] CL-Anomaly: Layer-Adaptive Mixture-of-Experts with Multimodal Large Language Model for Continual Learning in Anomaly Detection cs.CVPDF

Wen Dong, Zhao Wang, Shuangqing Zhang, Kai Sun, Ben Li

TL;DR: 本文提出了CL-Anomaly，一个用于异常检测持续学习的参数高效微调框架。该框架基于隔离-共享协作机制，通过任务私有专家PrivLoRA隔离任务特定参数空间以防止语义纠缠，并通过层自适应共享专家在统一特征空间中维护跨任务表示以实现知识共享。此外，还引入了层自适应知识转移策略，通过动量机制动态选择更新每层的共享专家，促进相关异常检测任务间的有效知识转移。

Details

Motivation: 解决多模态大语言模型在异常检测任务中进行持续学习时面临的计算成本高、参数空间语义纠缠以及异常检测任务在模态、领域和缺陷规模上的三重异构性所带来的知识迁移困难。

Result: 在类别增量、跨领域和跨模态三种异常检测持续学习场景下的广泛实验表明，CL-Anomaly的性能优于最先进的方法。

Insight: 创新点在于提出了一个结合物理隔离（PrivLoRA）与动态共享（层自适应共享专家）的混合专家框架，并设计了基于动量的层自适应知识转移策略，以应对异常检测中复杂的异构性，实现高效且抗遗忘的持续学习。

Abstract: Multimodal Large Language Models (MLLMs) excel in diverse vision tasks, but full-parameter retraining is computationally expensive as real-world knowledge evolves. Existing continual learning methods often suffer from semantic entanglement in parameter spaces across tasks, impeding the continuous deployment of models. This challenge is especially pronounced in Anomaly Detection (AD), which exhibits triple heterogeneity across modalities, domains, and defect scale variability, significantly complicating multi-task knowledge transfer. In this paper, we propose CL-Anomaly, a parameter-efficient fine-tuning framework based on an isolation-sharing collaboration to enable continual learning for anomaly detection with MLLMs. We introduce the task-private expert PrivLoRA, which physically isolates task-specific subspaces in the parameter space to prevent semantic entanglement of anomaly knowledge in diverse scenarios. The Layer-Adaptive Shared Experts maintain cross-task representations within a unified feature space, enabling knowledge sharing between previous and new tasks. Furthermore, we propose a Layer-Adaptive Knowledge Transfer strategy that automatically selects and dynamically updates the layer-wise key shared experts of each task via a momentum-based mechanism, promoting effective knowledge transfer across related anomaly detection tasks. Extensive experiments across three continual learning scenarios for anomaly detection, including class-incremental, cross-domain, and cross-modal, demonstrate that CL-Anomaly outperforms state-of-the-art methods. Code is available at https://github.com/WenDongyp/CL-Anomaly.

[74] MatPhaseBench: A Semantics-Guided Benchmark for Materials Phase Diagrams Understanding cs.CV | cond-mat.mtrl-sci | cs.AIPDF

Hanwen Wang, Sihan Liang, Zhiwei Liu, Yangang Wang, Wei Yan

TL;DR: 本文提出了MatPhaseBench，一个专注于材料相图理解的高质量、高可靠性基准测试，用于评估视觉语言模型在复杂科学图像理解方面的能力。该基准包含从3681篇材料科学论文中精选的200个高质量图文对，覆盖189个材料体系和70种元素，强调深度理解和语义对齐。实验表明，当前视觉语言模型在专家级理解方面仍有显著差距，主要局限于表面视觉感知，缺乏基于热力学机制的深度推理。

Details

Motivation: 材料相图是材料科学中的核心知识表示，其完全理解需要热力学机制分析和科学推理。尽管视觉语言模型在科学图像理解方面显示出潜力，但针对此类逻辑复杂、需要深度机制解释的图像的系统性评估仍然有限，相图为这一目的提供了一个具有挑战性的测试平台。

Result: 实验结果表明，当前的视觉语言模型在专家级理解方面仍然大幅落后：它们主要局限于表面视觉感知，缺乏基于热力学机制的深度推理，领域意识和专家分析经验有限，并且在区分复合或多图设置中的细粒度差异方面表现不佳。

Insight: 论文的创新点在于构建了一个语义引导的、专注于复杂科学图像（材料相图）理解的基准测试，其关键特征包括：超越简单客观测试的开放式任务、全面的图文语义对齐以及高质量的人工监督文本获取。这为复杂科学图像理解、相图分析和可信赖的科学多模态AI提供了一个基础平台。

Abstract: Materials phase diagrams are a core knowledge representation in materials science, encoding temperature,composition, phase stability, and phase transformation pathways, with their full understanding requiring thermodynamic mechanism analysis and scientific reasoning. Although VLMs have shown promise in scientific image understanding, their systematic evaluation on such logically complex images demanding deep mechanistic interpretation remains limited, and phase diagrams provide a challenging testbed for this purpose. We introduce MatPhaseBench, a high-quality, high-reliability benchmark for complex scientific image understanding, focused on materials phase diagrams. MatPhaseBench is constructed from 3681 papers in classical materials science journals, from which 200 high-quality diagram-text pairs were selected, covering 189 material systems and 70 elements. The benchmark has three key features: (1)targeting complex scientific image understanding-it moves beyond simple objective tests to open-ended tasks requiring deep comprehension; (2)comprehensive image-text alignment-semantic information associated with images is fully preserved during literature mining and matching; (3) high-quality human-supervised text acquisition-all descriptions undergo strict manual validation. Experimental results show that current VLMs remain substantially behind expert-level understanding: they are largely limited to surface visual perception, lack deep reasoning grounded in thermodynamic mechanisms, have limited domain awareness and expert analytical experience, and perform poorly in distinguishing fine-grained differences in composite or multi-diagram settings. Overall, MatPhaseBench constitutes a challenging research-grade benchmark, providing a foundational platform for complex scientific image understanding, phase diagram analysis, and trustworthy multi-modal AI in science.

[75] VideoSearcher: Empowering Video Deep Research with Multi-Tool Agentic Reasoning via Reinforcement Learning cs.CV | cs.AIPDF

Zhenkun Gao, Yicheng Bao, Jinlong Peng, Xueheng Li, Theo Huang

TL;DR: 本文提出了VideoSearcher，一个用于视频深度研究的闭环智能体框架。它通过强化学习驱动的多工具推理，将时序定位、空间聚焦和多模态搜索统一起来，以渐进式地定位视觉线索、检索证据并合成答案。同时，论文还构建了首个用于评估开放世界视频信息定位和多模态搜索推理的基准VideoSearch-QA。

Details

Motivation: 当前视频理解正从封闭场景感知转向开放世界证据探索，但现有多模态搜索智能体主要针对静态图像，且现有视频深度研究基准依赖丢弃关键视觉信息的文本中心检索，存在局限性。

Result: 大量实验表明，VideoSearcher在各种面向搜索和多模态理解的基准上显著优于先前的开源智能体基线。

Insight: 主要创新点包括：1）将时序定位、空间聚焦和多模态搜索统一在单一推理轨迹中的闭环智能体框架；2）用于优化知识密集型推理轨迹的双分支序列策略优化算法，将工具调用优化与答案准确性优化解耦；3）构建了首个针对开放世界视频信息定位和多模态搜索推理的基准VideoSearch-QA。

Abstract: Video understanding is moving beyond closed-context perception toward open-world evidence exploration, a paradigm formalized as Video Deep Research (VDR). However, existing multimodal search agents primarily target static images, and the current VDR benchmark relies on text-centric retrieval that discards crucial visual information. To address these limitations, we propose VideoSearcher, a closed-loop agentic framework that empowers Vision-Language Models with multi-tool reasoning for VDR. VideoSearcher unifies temporal localization, spatial focusing, and multimodal search within a single reasoning trajectory, enabling agents to progressively ground visual clues, retrieve relevant evidence, and synthesize answers. To optimize knowledge-intensive reasoning trajectories, we propose Bi-branch Sequence Policy Optimization (BiSPO), a reinforcement learning algorithm that decouples tool-invocation optimization from answer-accuracy optimization. This design provides stable learning signals for both evidence-grounded reasoning and purposeful tool use. Furthermore, we construct VideoSearch-QA, the first benchmark designed to evaluate open-world video information grounding and multimodal search-based reasoning. Extensive experiments demonstrate that VideoSearcher significantly outperforms prior open-source agentic baselines across various search-oriented and multimodal understanding benchmarks.

[76] MORE: A Multilingual Document Parsing Benchmark and Evaluation cs.CV | cs.CLPDF

Long Xu, Binghong Wu, Tinghao Yu, Hao Feng, Zhenyu Huang

TL;DR: 本文介绍了MORE，一个用于评估多语言文档解析的大规模基准测试。它覆盖149种语言，包含代码块、表格和目录等复杂结构，数据源自真实文档并通过模型辅助、人工精炼的流程标注。作者使用MORE评估了最先进的模型，为长尾语言建立了性能基线，并验证了基准在诊断模型真实多样场景能力方面的有效性。

Details

Motivation: 现有基准主要关注英语和中文等高资源语言，导致模型在其他语言上的性能存在评估盲点，且缺乏真实标注数据来实证验证声称支持数百种语言的视觉语言模型（VLMs）的能力。

Result: 使用MORE评估了最先进的模型，为长尾语言建立了新的性能基线，并验证了基准在诊断模型于真实、多样场景下能力的有效性。

Insight: 创新点在于构建了一个前所未有的多语言（149种语言）、结构复杂（包含代码、表格等）且数据真实（源自现实文档）的文档解析基准，填补了现有评估体系的空白，为模型能力提供了更全面的实证检验标准。

Abstract: Multilingual documents encapsulate rich regional cultures, scientific discoveries, and historical records. Parsing this content into structured, machine-readable formats is critical for unlocking global knowledge. However, existing benchmarks predominantly focus on high-resource languages like English and Chinese, creating an evaluation blind spot concerning model performance on other languages. While recent Vision-Language Models (VLMs) claim support for hundreds of languages, the lack of ground truth makes it impossible to empirically verify these capabilities. To bridge this gap, we introduce MORE, a large-scale benchmark designed for multilingual document parsing evaluation. MORE distinguishes itself through three key dimensions: (1) Unprecedented Scale: It covers 149 languages, making it the most linguistically diverse benchmark to date; (2) Structural Complexity: Unlike previous works, it extends evaluation beyond plain text to include structural elements such as code blocks, tables, and catalogs; and (3) Data Authenticity: All samples are curated from real-world documents via a model-assisted, human-refined annotation pipeline. We evaluate state-of-the-art models using MORE, establishing new performance baselines for long-tail languages and validating the benchmark’s effectiveness in diagnosing model capabilities in realistic, diverse scenarios. The MORE dataset will be available at https://github.com/zimoqingfeng/MORE.

[77] Incentivizing Vision Language Models to Search for Long Video Question Answering cs.CV | cs.LGPDF

Harsh Goel, S P Sharan, Sahil Shah, Minkyu Choi, Joungbin An

TL;DR: 本文提出了VSeek，一个将长视频问答任务从被动的单次感知转变为多轮检索过程的智能体框架。它利用自然语言驱动的搜索在长视频中定位相关上下文，并通过强化学习进行后训练，以联合制定有针对性的搜索查询并对检索到的片段进行推理。为了解决长视频理解中缺乏已验证奖励信号的问题，作者提出了一种新颖的神经符号方法，将开放式自然语言查询编译为时序逻辑规范，从而生成一个包含所需原子视觉基元及其时序顺序的确定性检查清单，为强化学习提供密集且可验证的奖励信号。

Details

Motivation: 动机在于将长视频问答从被动、单次感知任务转变为主动、多轮检索过程，并解决强化学习后训练在长视频理解领域因缺乏已验证奖励信号而受阻的问题。

Result: 在长视频理解基准测试中，VSeek相比基础模型，Pass@1分数提升了高达8%，Pass@4分数提升了15%。

Insight: 创新点在于提出了一个结合主动检索与强化学习的智能体框架，并引入了一种新颖的神经符号方法，通过将自然语言问题编译为形式化的时序逻辑规范来生成可验证的视觉基元检查清单，从而为强化学习训练提供密集、可靠的奖励信号，优化了模型寻求可验证证据的行为。

Abstract: We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process. VSeek utilizes a natural language-driven search to identify relevant context within long videos and is post-trained with reinforcement learning (RL) to jointly formulate targeted search queries and reason over retrieved clips for LVQA. While RL post-training has revolutionized reasoning in symbolic domains such as mathematics and code, its application to long-video understanding remains hindered by a lack of verified rewards. To ensure that the retrieved context is relevant, we propose a novel neuro-symbolic approach that bridges open-ended natural language with discrete visual verification. Specifically, complex user queries are compiled into formal temporal logic specifications for systematically decomposing natural language questions into a definitive checklist of required atomic visual primitives, such as key objects and activities, along with their temporal ordering. These systematically derived grounding events provide the critical feedback signal for RL post-training, enabling dense, verifiable rewards based on the successful retrieval of these specific visual elements rather than relying entirely on outcome-only answer accuracy. By explicitly optimizing for this verifiable evidence-seeking behavior, VSeek improves Pass@1 scores by up to 8% and Pass@4 scores by 15% on long-video understanding benchmarks compared to base models. We open-source our code at https://utaustin-swarmlab.github.io/VSeek.

[78] Overloading Large Vision-Language Models for Jailbreaking cs.CV | cs.CRPDF

Haoyu Zhang, Yangyang Guo, Mohan Kankanhalli

TL;DR: 本文提出了一种针对大型视觉-语言模型（LVLM）的新型越狱攻击方法——信息过载攻击。该方法通过结合大量文本和多维图像攻击，并采用基于递归的图像-排版布局，指数级增加多模态信息的复杂性，从而破坏LVLM的安全对齐机制。实验表明，该方法在开源和商业LVLM上均实现了最高的攻击成功率，并揭示了复杂多模态输入如何通过加剧跨模态处理来削弱模型的安全响应能力。

Details

Motivation: 现有LVLM越狱攻击方法（如使用简短文本或分布外图像）设计简单，且随着大语言模型主干和多模态机制的进步，其可迁移性（尤其是在不同模型架构间）受到削弱。因此，需要一种更强大的攻击方法来克服这一限制。

Result: 在开源模型上，该方法平均攻击成功率（ASR）达到88.6%；在商业LVLM上，平均ASR达到84.0%，比最佳基线方法高出48.7%。该方法在开源代理模型上优化的提示词能有效跨模型家族迁移，确立了新的SOTA LVLM越狱攻击。

Insight: 创新点在于提出信息过载攻击范式，通过递归式图像-排版布局组合大量文本和多维图像攻击，指数级增加多模态复杂性以破坏安全对齐。客观分析认为，其核心洞察是复杂多模态组合会加剧跨模态处理，降低模型生成拒绝响应的确定性，这揭示了LVLM部署中一个实际且新兴的安全风险。

Abstract: Large Vision-Language Models (LVLMs) exhibit remarkable vision-language capabilities and are increasingly deployed in real-world applications such as personal assistants, document analysis systems, and embodied agents. However, their dual-modal attack surfaces make them vulnerable to jailbreak attacks. Existing LVLM jailbreaks rely on simple designs, e.g., short text and out-of-distribution images. Nevertheless, recent advancements in both large language model backbones and multimodal mechanisms undermine these attacks, particularly their transferability among model architectures. To overcome this limitation, we propose a novel information overloading method that is equipped with both extensive text and multi-dimensional image attacks. These components are arranged in recursion-based image-typography layouts to exponentially increase multimodal information complexity. This overloading approach amplifies the cross-modal processing required, which undermines the safety alignment in LVLMs. Extensive experiments on both open-sourced and commercial LVLMs establish our method as a new state-of-the-art LVLM jailbreak attack. On open-source models, our method achieves an average ASR of 88.6%; on commercial LVLMs, it reaches an average ASR of 84.0%, exceeding the best baseline by 48.7%. Moreover, our prompts optimized on open-source surrogate models transfer effectively across model families. Beyond empirical results, we probe the safety-critical information flows within victim LVLMs. Our observations reveal that complex image-typography compositions induce intensified cross-modal processing and reduce the model’s certainty in generating refusal responses. Together, these findings highlight information overloading as a practical and emerging safety risk for real-world LVLM deployments, underscoring the need for stronger defenses against complex multimodal jailbreak inputs.

[79] GuideMe: Multi-Domain Task Guidance and Intervention in Streaming Video cs.CVPDF

Fang Liu, Jinpeng Chen, Ke Xu, Yuhao Liu, Huankang Guan

TL;DR: 本文提出了GuideMe，首个用于流媒体视频多领域任务指导的基准测试集，支持多模态大语言模型（MLLMs）在闭环交互式任务指导中的训练与评估。该基准包含2,458个视频，涵盖烹饪、物体操作、日常生活指导和健身等多个领域，并设计了包含序列对齐、干预时机判断和内容质量评估的三组件评估框架。实验表明，现有MLLMs在提供指令方面表现出色，但在错误检测和纠正反馈方面存在显著不足。

Details

Motivation: 研究动机是探索多模态大语言模型（MLLMs）在实时程序性教练角色中的能力差距，即模型能否在流媒体视频中持续监控执行过程、检测错误并提供闭环交互式纠正指导。

Result: 在GuideMe基准上的广泛实验揭示了关键的性能不对称性：尽管现有MLLMs在提供指令方面表现优异，但在识别执行错误和生成纠正反馈方面持续失败。评估使用了序列级对齐的时间-语义二分匹配、干预时机的行为分类以及基于LLM-as-a-Judge的内容质量评估。

Insight: 论文的创新点在于构建了首个支持闭环交互式任务指导的多领域流媒体视频基准测试集（GuideMe），并设计了综合评估框架来系统衡量MLLMs的实时指导能力，揭示了模型在错误检测与纠正反馈方面的能力短板，为未来研究指明了方向。

Abstract: While multimodal Large Language Models (MLLMs) excel at offline video understanding, an interesting question of how far they are from serving as a real-time procedural coach remains unknown. Such a role typically requires an MLLM to continuously monitor the execution, detect mistakes, and provide corrective guidance in a closed-loop interaction. In this paper, we construct GuideMe, the first multi-domain benchmark for streaming video that supports training and evaluation of MLLMs for closed-loop interactive task guidance. It comprises 2,458 videos spanning 223.7 hours across diverse domains (\eg, cooking, object manipulation, daily-life guidance, and fitness), with 47,775 interaction samples covering next-step instructions, completion feedback, error detection, and corrective guidance. To evaluate existing models on GuideMe, we design a three-component assessment framework to measure the capabilities of representative MLLMs, which consists of temporal-semantic bipartite matching for sequence-level alignment, behavioral classification for intervention timing, and LLM-as-a-Judge for content quality. Extensive experiments highlight a critical performance asymmetry: despite excelling at providing instructions, existing MLLMs consistently fail to identify execution errors and respond with corrective feedback. Code and data are released at https://fawnliu.github.io/project/guideme.

Wenzheng Zeng, Siyi Jiao, Chen Gao, Hwee Tou Ng, Mike Zheng Shou

TL;DR: 本文提出了一种并行化自回归解码框架，用于提升全模态密集视频描述任务的生成效率和性能。该方法通过利用时间上不同事件间的弱局部依赖性来重构因果依赖图，实现无损并行生成，其中跨事件依赖弱的令牌可并行解码，而每个事件内紧密耦合的令牌则保持顺序解码以维持局部语义连贯性。

Details

Motivation: 密集视频描述旨在生成视频事件的时间定位描述，但现有基于令牌逐个生成的自回归视频大语言模型在推理效率上严重受限，尤其当视频长度和事件密度增加时，可扩展性受阻。

Result: 在多个基准测试上的实验表明，该方法在全模态事件定位和描述任务中，在效率和性能上均展现出明显优势。

Insight: 创新点在于提出了一种无损并行解码框架，其核心是通过潜在全局规划机制自动学习事件级结构并生成编码全局事件间因果关系的紧凑令牌，以及事件分解的并行解码机制，有效平衡局部聚焦与全局事件间感知，从而在保持语义质量的同时大幅提升生成速度。

Abstract: Dense video captioning aims to generate temporally grounded descriptions of video events, benefiting both event-level video understanding and generation. In this domain, autoregressive video large language models have emerged as a prevalent paradigm due to their strong generative and cross-modal modeling capacity. However, generating dense captions under the token-by-token paradigm severely limits inference efficiency and hinders scalability as video length and event density increase. In this work, we propose a parallelized autoregressive framework that not only improves generation efficiency but also enhances temporally grounded captioning performance. Our key insight is to exploit the weak local dependencies across temporally distinct events to restructure the causal dependency graph, thereby enabling lossless parallel generation. Specifically, tokens with weak cross-event dependencies can be decoded in parallel, while tightly coupled tokens within each event retain sequential decoding to preserve local semantic coherence. To realize this insight, we introduce two key components for lossless parallel decoding: (1) a latent global planning mechanism that automatically learns the event-level structure and produces compact tokens encoding global inter-event causality while adaptively aggregating event-level audio-visual semantics, guiding subsequent dependency restructuring and parallel decoding; and (2) an event-factorized parallel decoding mechanism that effectively balances local focus with global inter-event awareness. Experiments on various benchmarks demonstrate the clear advantage of our approach in both efficiency and performance in omni-modal event grounding and captioning. Project website: https://github.com/showlab/PadCaptioner.

[81] RePos: Relative-to-Absolute Output Factorization for Cross-Environment WiFi-Based 3D Human Pose Estimation cs.CVPDF

Zhangcheng Hou, Tomoaki Ohtsuki

TL;DR: 本文提出RePos框架，通过将基于WiFi CSI的3D人体姿态估计分解为根节点相对姿态估计和根节点定位两个子任务，以解决现有方法在跨环境泛化中的性能下降问题。该框架利用身体部位感知的潜在令牌和骨架引导建模来学习环境不变的姿态表示，同时通过基于幅度的网络和可微分空间分解模块估计根节点位置。

Details

Motivation: 现有基于商用WiFi CSI的无设备3D人体姿态估计方法在跨环境部署时泛化能力较差，因为CSI测量缺乏与身体部位的空间局部对应性且受多径传播影响严重，导致绝对姿态回归模型将身体结构与环境特定的位置线索纠缠在一起。

Result: 在Person-in-WiFi-3D数据集上，RePos-D（端到端绝对姿态变体）达到86.9 mm MPJPE，比之前最好的WiFi方法DT-Pose提升3.4%。在严格的MM-Fi跨环境协议下，RePos的MPJPE为254.4-296.1 mm，比现有基于WiFi的方法降低10-21%，且在多种活动协议、留一环境划分和无泄漏少样本迁移中保持一致的改进。

Insight: 创新点在于将绝对姿态估计分解为相对姿态和根定位两个解耦任务，通过防止绝对姿态监督影响结构分支来学习环境不变的姿态表示。客观分析表明，相对姿态表示基本与位置无关，而根定位保留环境依赖性，这揭示了结构和定位任务具有不同的泛化行为。

Abstract: Device-free 3D human pose estimation using commodity WiFi Channel State Information (CSI) enables privacy-preserving and illumination-robust human sensing, but its deployment is limited by poor cross-environment generalization. Unlike images, CSI measurements do not have a spatially localized correspondence to body parts and are heavily affected by multipath propagation, causing models that regress absolute poses to entangle body structure with environment-specific location cues. Within a single environment this coupling is benign: an end-to-end absolute-pose variant, RePos-D, already achieves state-of-the-art accuracy on Person-in-WiFi-3D (86.9 mm MPJPE, a 3.4% gain over the previous best WiFi method, DT-Pose). Across environments, however, the same model overfits position and suffers significant performance degradation. We therefore propose RePos, a factorized framework that separates root-relative pose estimation from root localization. By preventing absolute-pose supervision from affecting the structure branch, RePos learns environment-invariant pose representations. Specifically, it groups CSI features into body-part-aware latent tokens that skeleton-guided modeling refines into the pose, while a separate amplitude-based network estimates the root position through a differentiable spatial-decomposition module. Under the strict MM-Fi cross-environment protocol, RePos achieves MPJPEs of 254.4-296.1 mm, a 10-21% reduction over existing WiFi-based methods. The improvement remains consistent across activity protocols, leave-one-environment-out splits, and leakage-free few-shot transfer. Analysis of the learned features shows that relative-pose representations remain largely position-agnostic, while root localization retains environment dependence, indicating distinct generalization behavior for structure and localization.

[82] VISTA: Auditing Semantic Divergence in Vision-Language Models cs.CV | cs.AIPDF

Junchi Liao, Jiawen Deng, Fuji Ren

TL;DR: 本文提出了VISTA方法，用于审计视觉语言模型中的语义分歧。该方法通过结合语义熵和基于分布的散度，以黑盒跨模型审计的方式，检测模型在特定视觉概念（如人口特征、企业标志或意识形态符号）下产生的异常一致响应。

Details

Motivation: 动机在于视觉语言模型可能表现出视觉概念条件化的分歧行为，即面对包含特定视觉概念的图像时，某些模型会产生与同类模型不同的异常一致响应，而这类行为无法通过纯文本审计方法检测。

Result: 在受控研究中，通过在三个VLM上使用小型有偏数据集进行微调以植入概念条件化立场，VISTA成功检测到了这些植入行为。对六个VLM在19个主题上的审计中，VISTA发现了142个（1.2%）高可疑案例，并识别出选择性拒绝这一先前未报告的差异模式。

Insight: 创新点在于提出了一种专门针对视觉语言模型的黑盒跨模型审计框架VISTA，它能够检测由视觉概念引发的语义分歧，并揭示了模型在人口统计查询上存在0%到65%不等的选择性拒绝率这一新发现的分歧模式。

Abstract: Vision-language models can exhibit visual concept-conditioned divergence: given images containing demographic features, corporate logos, or ideological symbols, some models produce unusually uniform responses that differ from what peer models say about the same input. These behaviors evade text-only audits because visual concepts cannot be isolated or substituted the way text tokens can. We present VISTA (Visual Inconsistency Screening Through Analysis), a black-box cross-model audit that couples semantic entropy with distribution-based divergence to flag model-specific anomalies. In a controlled study, we implant concept-conditioned stances in three VLMs via fine-tuning on small biased datasets and confirm that VISTA detects them. Auditing six VLMs across 19 topics, VISTA surfaces 142 high-suspicion cases (1.2%) and identifies selective refusal as a previously unreported divergence pattern, where models refuse demographic queries at rates varying from 0 to 65% across groups.

[83] CONFLUX: A Latent Diusion Model for 3D Chest-CT Synthesis with RL Post-Training cs.CV | cs.AI | cs.LGPDF

Max Van Puyvelde, Halil Ibrahim Gulluk, Wim Van Criekinge, Olivier Gevaert

TL;DR: CONFLUX是一个用于生成3D胸部CT图像的潜在扩散模型，它结合了3D变分自编码器进行体积压缩和整流流变换器在潜在空间生成。模型通过自适应层归一化实现基于放射学元数据的条件控制，并引入在线强化学习后训练阶段以增强生成图像对指定临床属性的忠实度。

Details

Motivation: 旨在开发可控的3D医学图像生成模型，以合成具有指定临床属性的高保真、原生3D且忠实于条件信息的医学体积数据，解决现有方法在同时满足这些要求方面的不足。

Result: 在tri-planar Fréchet距离指标上显著优于基线模型（FID 32.3 vs. MAISI的74.6），后训练阶段通过独立分类器评估，将生成图像与真实扫描在可靠性上的差距缩小了47%。

Insight: 创新点包括将整流流变换器与3D VAE结合用于3D医学图像生成，以及采用基于强化学习的后训练策略（group-relative policy optimization）来优化条件控制的可靠性；其发布的带条件元数据的大规模合成数据集也具有重要价值。

Abstract: Controllable generative models of 3D medical images can synthesize volumes with specified clinical attributes, but this demands samples that are simultaneously high-fidelity, natively 3D, and faithful to the requested conditioning. We present CONFLUX, a latent diffusion model for chest computed tomography (CT): a 3D variational autoencoder compresses each volume, and a rectified-flow transformer generates in the latent space. Generation is conditioned on structured radiological metadata (18 abnormality findings, sex, age, and reconstruction kernel) through adaptive layer normalization. The model leads strong volumetric baselines on tri-planar Frechet distance (FID 32.3 vs. 74.6 for MAISI) while exposing direct control over clinical attributes. To strengthen that control we add an online reinforcement-learning post-training stage (group-relative policy optimization) that rewards how reliably a classifier recovers the requested findings from each generated volume. Judged by a separate, independent classifier, post-training removes 47% of the shortfall relative to real-scan reliability. We release the model and a ~200k synthetic chest-CT dataset with conditioning metadata spanning a wide variety of clinical findings.

[84] PosterHarness: Turning Scientific Poster Generation into an Auditable Instruction-Following Benchmark cs.CV | cs.AI | cs.SEPDF

Tianyi Yang, Dawei Fu, Youpeng Wu, Zixun Kou, Linrui Chen

TL;DR: 本文提出了POSTERHARNESS，一个将科学海报生成转化为可审计的指令遵循任务的框架。它通过‘占位符优先’的契约将视觉摘要设计与数据图形绘制分离，确保模型不伪造科学图形，并建立了一个包含基准和失败分类法的可测量评估体系。

Details

Motivation: 当前文本丰富的图像模型能够设计海报级布局，但缺乏衡量其是否遵守科学传播契约（如清晰标签、规定宽高比、最重要的是避免伪造科学图形）的方法。

Result: 在12篇论文（6篇高能物理，6篇AI/ML相关）上实例化该框架，发现占位符契约将VLM检测到的合成图形数量从34个降至0个；与Paper2Poster基线比较显示，PosterHarness能产生更高分辨率的输出、更低的空白画布比例和更强的VLM视觉偏好，但基线在保留信息和运行速度上略有优势。

Insight: 核心创新在于‘占位符优先’的契约设计，将布局设计与数据图形插入解耦，使得模型输出（如占位符数量、ID准确性、图形空白性、宽高比合规性、避免合成图形等）变得可测量和可审计，为生成模型的可靠性和安全性评估提供了新范式。

Abstract: Text-rich image models can now design poster-scale layouts, but we lack ways to measure whether they honor scientific communication contracts: legible labels, prescribed aspect ratios, and – above all – abstaining from fabricated scientific figures. We present POSTERHARNESS, an auditable harness reframing poster generation as measurable instruction-following tasks, with a pilot benchmark and failure taxonomy. POSTERHARNESS uses a placeholder-first contract to separate two jobs models otherwise conflate. The model performs visual-summary design: typography, reading path, color, and background – but never draws data-bearing figures. Every figure region must be an empty labeled placeholder; a deterministic compositor inserts real source-paper figures at detected coordinates. This makes properties measurable: placeholder count and ID accuracy, blankness, aspect-ratio compliance, abstention from synthesized graphics, public-text hygiene, and source-figure provenance – with failures logged as explicit rejections, not hidden in plausible-looking output. We instantiate the harness on 12 papers (6 HEP, 6 AI/ML-adjacent) and report three findings. (i) A counterfactual probe shows the placeholder contract drives VLM-counted synthesized figures from 34 to 0 across three papers. (ii) A failure taxonomy identifies blocking contracts: placeholder geometry, placeholder QA, template critic, and public text. (iii) Comparison with Paper2Poster shows a trade-off: PosterHarness yields higher-resolution artifacts, lower white-canvas fraction, and stronger VLM visual preference; the deterministic baseline retains slightly more PosterQuiz-style information and runs faster. We report this as regime characterization, not a superiority claim. All artifacts, prompts, manifests, and audit scripts are released as a reusable evaluation component.

[85] HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion cs.CV | cs.AI | cs.LGPDF

Dongyeun Lee, Amir Zandieh, Vahab Mirrokni, Junmo Kim, Insu Han

TL;DR: 本文提出HyperVAttention（HVA），一种无需训练的稀疏注意力框架，旨在解决视频扩散模型中自注意力机制的二次计算复杂度问题。通过引入3D局部窗口聚类和硬件感知的簇合并策略，HVA显著降低了聚类开销并提高了GPU计算单元（CTA）的利用率，从而在保持生成质量的同时大幅提升了长视频生成的推理效率。

Details

Motivation: 视频扩散变换器（VDiTs）在生成高质量视频方面表现出色，但其生成长视频的能力受限于自注意力机制的二次计算复杂度。现有的基于聚类的稀疏注意力方法虽然改善了质量与速度的权衡，但仍面临聚类开销大和由不规则簇结构导致的GPU CTA利用率低两大瓶颈。

Result: 在文本到视频生成任务上的实验表明，HVA为无需训练的稀疏注意力方法建立了新的帕累托前沿，在保持甚至提升生成保真度的同时，将端到端延迟降低了最高2.13倍，优于现有的无需训练稀疏注意力基线方法。

Insight: 主要创新点包括：1）利用视频令牌的时空局部性，提出3D局部窗口聚类以限制质心搜索范围，并配合自定义Triton内核高效执行；2）利用去噪步骤间聚类分配的时序稳定性，提出混合聚类策略（仅在锚点步骤进行完整聚类，在中间步骤仅更新子集令牌）；3）提出硬件感知的簇合并策略，通过并行凝聚合并最小化CTA对齐的执行成本，提高块密度和近似保真度。这些设计共同优化了稀疏注意力在现代GPU固定瓦片结构上的执行效率。

Abstract: Video Diffusion Transformers (VDiTs) have demonstrated significant capabilities in high-fidelity video generation. However, their ability to produce long-duration videos is fundamentally constrained by the quadratic complexity of the self-attention mechanism. Recent clustering-based sparse attention methods improve the quality-speed trade-off by grouping semantically similar tokens, but their practical efficiency remains limited by two bottlenecks: substantial clustering overhead and low CTA utilization caused by irregular cluster-induced blocks. We propose HyperVAttention (HVA), a training-free sparse attention framework that addresses both bottlenecks jointly. To reduce clustering overhead, we introduce 3D local-window clustering, which exploits the spatio-temporal locality of video tokens to restrict centroid search to fixed local neighborhoods, and implement it with a custom Triton kernel for efficient execution. We further propose a hybrid clustering strategy that performs full clustering only at anchor steps and updates only subset tokens at intermediate steps, leveraging the temporal stability of cluster assignments across denoising steps. To improve CTA utilization, we present hardware-aware cluster merging that minimizes CTA-aligned execution cost through parallel agglomerative merging, improving block density and approximation fidelity by utilizing idle tile capacity. Together, these components reduce clustering overhead, avoid redundant updates, and better align sparse attention with the fixed tile structure of modern GPU kernels. Experiments on Text-to-Video generation show that HVA establishes a new Pareto frontier for training-free sparse attention in video diffusion, reducing end-to-end latency by up to $2.13\times$ while improving fidelity over existing training-free sparse attention baselines.

[86] MambaLIE: Scene Light Intensity-Boosted Low-Light Image Enhancement with State Space Model cs.CV | cs.AIPDF

Wanshu Fan, Xiangyu Li, Cong Wang, Kin-man Lam, Xin Yang

TL;DR: 本文提出MambaLIE方法，一种基于状态空间模型（SSM）的场景光强增强低光图像增强方法。该方法通过引入场景光强来改善光照结构分布，并设计局部增强状态空间模型（LESSM）来高效建模光照，在保持线性时间复杂度的同时增强局部特征表示。

Details

Motivation: 解决现有低光图像增强方法中CNN无法建模长距离依赖、Transformer计算成本高的问题，旨在开发一种既高效又能捕获全局光照关系的增强方法。

Result: 在四个合成基准和五个真实世界基准上，MambaLIE在精度、速度和模型大小方面均优于当前最先进的基于CNN和Transformer的低光增强方法，适合在资源受限设备上部署。

Insight: 创新点包括引入场景光强作为引导信号，以及设计双分支LESSM模块（SSM分支建模长距离依赖，局部增强分支强化局部特征），在计算效率和性能间取得平衡。

Abstract: Images captured by consumer electronic devices, such as mobile phones and digital cameras, often suffer from low-light degradation due to sensor limitations and imaging pipelines, which degrades visual quality and affects downstream vision tasks. Existing methods based on Convolutional Neural Networks (CNNs) and Transformers have dominated current low-light image enhancement (LIE) due to their excellent ability to model hierarchical features. However, CNNs operate in local receptive fields that cannot model long-range dependencies, while Transformers overcome this problem but incur substantial computational costs. To address these challenges, we propose MambaLIE, a Scene Light Intensity-Boosted Low-Light Image Enhancement method based on a State Space Model (SSM). We first introduce scene light intensity to improve the structural distribution of illumination, which is then gated with the low-light input to guide enhancement. To better model the illumination while maintaining computational efficiency, we propose the Locally Enhanced State Space Model (LESSM) for efficient light enhancement. Our LESSM contains two branches: an SSM branch and a Local Enhanced branch, where the former is used to model the long-range dependencies with linear time complexity, while the latter is used to enhance local feature representations. Extensive experiments demonstrate that MambaLIE outperforms state-of-the-art CNN-based and Transformer-based LIE methods on four widely used synthetic benchmarks and five publicly available real-world benchmarks in terms of accuracy, speed, and model size, making it suitable for practical deployment on resource-constrained devices.

[87] $C^3$ASD: Multi-Level Consistency-Driven Representation Learning cs.CV | cs.SDPDF

Jin Hong, Jisoo Park, Junseok Kwon

TL;DR: 本文提出C^3ASD框架，一种用于主动说话人检测的多级一致性驱动表示学习方法。该方法通过嵌入级、序列级和预测级三个互补的一致性约束，增强模型在音频、视觉及联合模态损坏下的鲁棒性，同时保持干净数据上的竞争力。

Details

Motivation: 现有视听融合方法在干净数据上表现良好，但在现实世界损坏（如背景噪声、遮挡或模态同时退化）下性能下降，这归因于缺乏显式的一致性约束来促进跨模态的鲁棒、语义对齐表示。

Result: 大量实验表明，该方法在多种音频、视觉和联合损坏下取得显著改进，同时在干净数据上保持有竞争力的性能。

Insight: 创新点在于引入多级一致性约束：嵌入级跨模态一致性对齐语音期间的视听表示；序列级模态内一致性通过轨迹感知对比学习分离说话与非说话簇；预测级一致性通过知识蒸馏稳定融合过程，从而提升模型对损坏的鲁棒性。

Abstract: Active Speaker Detection determines whether a visible person in a video is speaking at each moment. While recent audio-visual fusion methods perform well on clean data, they degrade under real-world corruptions such as background noise, occlusion, or simultaneous modality degradation. We attribute this limitation to the absence of explicit consistency constraints that promote robust, semantically aligned representations across modalities. Without such guidance, models tend to learn fragile modality-specific shortcuts that fail under corrupted conditions. We propose $C^3$ASD, a multi-level consistency-driven framework with three complementary constraints: embedding-level inter-modality consistency aligns audio-visual representations during speech; sequence-level intra-modality consistency separates speaking and non-speaking clusters via track-aware contrastive learning; and prediction-level consistency stabilizes fusion through knowledge distillation. Extensive experiments demonstrate significant improvements under diverse audio, visual and joint corruptions, while maintaining competitive performance on clean data.

[88] Natural Language Camera Movement Understanding cs.CVPDF

Yuwen Tan, Joey Huang, Jin Huang, Haoxiang Li, Boqing Gong

TL;DR: 本文提出自然语言相机运动理解作为独立研究任务，针对现有视觉语言模型在区分相机平移/旋转、左右方向及物体/相机运动方面的严重缺陷，构建了包含真实与合成视频的双层电影学分类基准与大规模增强训练集。微调后的VLM-8B模型在基准测试中超越Gemini 3.1 Pro达10-11%，但仍与人类表现存在显著差距。

Details

Motivation: 解决现有视觉语言模型无法准确理解自然语言描述的相机运动问题，该能力对视频生成模型的训练评估等应用至关重要。

Result: 在构建的真实与合成视频基准测试中，微调VLM-8B模型分别以10%和11%的优势超越Gemini 3.1 Pro，但性能仍远低于人类水平。

Insight: 通过建立电影学双层分类体系与原子化基准，结合针对性数据增强构建训练集，为相机运动理解任务提供了系统化评估框架与数据解决方案。

Abstract: Understanding camera movement in natural language is critical for training and evaluating video generation models, among other applications. However, we demonstrate that existing vision-language models (VLMs) fail this task in surprising ways, frequently confusing translation with rotation, left with right, and object movement with camera movement. To address these limitations, we establish natural language camera movement understanding as a standalone research task. We introduce a two-level cinematographic taxonomy and an extensive, atomic benchmark featuring both real and synthetic videos. Furthermore, we curate a large-scale, multi-source training set enhanced by targeted camera movement augmentation. Our fine-tuned VLM-8B outperforms Gemini 3.1 Pro by 10% and 11% on our benchmark’s real and synthetic videos, respectively. Despite these gains, a significant gap remains relative to human performance, underscoring the need to promote and facilitate future research on natural language camera movement understanding.

[89] PixCon: Clean-Positive Contrastive Learning for Foundation-Model Semi-Supervised Segmentation cs.CV | cs.AI | cs.LG | eess.IVPDF

Ebenezer Tarubinga

TL;DR: 本文提出PixCon，一种基于干净正样本的像素对比学习框架，用于基础模型半监督语义分割。该方法通过构建无污染的正样本集（ρ_F=0）来优化嵌入空间结构，在Pascal VOC、Cityscapes和ADE20K数据集上匹配或超越了基于DINOv2的UniMatch V2基线模型。

Details

Motivation: 在基础模型（如DINOv2）作为教师的半监督分割中，严格的置信度阈值已能获得高纯度（约98%）的伪标签，因此核心问题从过滤置信度转向如何通过类别结构优化嵌入空间。现有对比学习方法（如ReCo、U²PL）基于置信度过滤的伪标签构建记忆库，仍存在污染风险。

Result: 在Pascal VOC、Cityscapes和ADE20K数据集上，PixCon在计算匹配的单次切换协议中匹配或改进了强基线UniMatch V2。例如，在Pascal-1/8设置下所有种子均提升（每种子约+0.2 mIoU），三种子平均达到87.90 mIoU，与已发表的UniMatch V2-B结果相当。

Insight: 创新点在于构建了严格无污染（ρ_F=0）的每类记忆库，仅包含学生已正确分类的标注像素，避免了传统对比方法中伪标签污染问题。理论分析揭示了污染损失项与ρ_F/(1-ρ_F)成正比，而PixCon通过干净正样本监督提升鲁棒性，为基于基础模型的半监督分割提供了低成本、高稳健的默认方案。

Abstract: Semi-supervised semantic segmentation (SSSS) has long turned on one question, which pseudo-labels to trust, and answered it with ever more careful confidence filtering. Foundation backbones change the regime: with a DINOv2 teacher a strict threshold already retains a measured 98%-clean pseudo-label set, so the accuracy that remains lives not in the filter but in how the embedding space is structured by class. We propose PixCon, a clean-positive pixel-contrastive framework. PixCon maintains a per-class memory bank that admits only labeled pixels the student already classifies correctly, guaranteeing a contamination-free positive set ($ρ_F=0$) by construction, unlike prior contrastive SSSS banks (ReCo, U$^2$PL) built from confidence-filtered pseudo-labels. It is a single branch over a consistency backbone, adds no inference-time parameters, and needs no bank-specific threshold. A first-order analysis of the supervised-InfoNCE gradient explains why contamination hurts: its false-positive term scales as $ρ_F/(1-ρ_F)$, which we measure (0.018 on Pascal, 0.106 on ADE20K) rather than assume. Across Pascal VOC, Cityscapes, and ADE20K, PixCon matches or improves a strong DINOv2-based UniMatch V2 baseline in a compute-matched one-switch protocol: it improves every Pascal-1/8 seed (a per-seed gain of about +0.2 mIoU) and its three-seed mean reaches 87.90, the published UniMatch V2-B figure. Because contamination is already rare under foundation-model teachers, our analysis indicates the $ρ_F=0$ guarantee acts chiefly as robustness as teachers weaken, while the accuracy gain comes from cleaner positive supervision, making clean-positive contrast a robust, low-cost default for foundation-model SSSS.

Wenlin Wu, Sheng Zhou, Peipei Song, Wenhao Wang, Junbin Xiao

TL;DR: 本文提出SafeGuard，一个多智能体感知-推理框架，用于检测具有社会风险的AI生成视频。该框架通过分层感知求解器提取细粒度取证证据，并结合自反思验证器确保语义推理与物理合理性的一致性，以弥补现有方法在感知与推理之间的鸿沟。

Details

Motivation: 随着视频生成技术从局部操纵发展到全场景合成，AI生成视频的检测变得更具挑战性，因为伪造视频展现出连贯的全局结构和高度感知真实性。现有基准偏向于感知保真度，主要基于感知伪影评估检测器，对需要推理物理定律、结构连贯性或社会逻辑违规的场景覆盖有限，导致感知与推理之间存在差距。

Result: 在提出的SafeVid基准（包含20K视频，涵盖10个社会风险类别）上，SafeGuard将准确率提升了+18.7%，并在四个公共基准上持续优于先前方法，展示了其泛化能力。

Insight: 创新点在于通过多智能体协作专门化，结合取证感知与语义推理，形成可解释的证据链，以解决AI生成视频检测中的感知-推理差距。同时，引入了专注于物理合理性、结构一致性和社会行为合理性的新基准SafeVid，以更全面地评估检测方法。

Abstract: As video generation paradigms evolve from localized manipulation to full-scene synthesis, AI-generated video detection becomes increasingly challenging, as forgeries exhibit coherent global structure and high perceptual realism. However, existing benchmarks are biased toward perceptual fidelity and primarily evaluate detectors based on perceptual artifacts, providing limited coverage of scenarios that require reasoning about violations of physical laws, structural coherence, or social logic. This dataset bias shapes current approaches and results in a Perception-Reasoning Gap: artifact-centric models capture low-level statistical irregularities yet lack semantic inference, whereas vision-language models perform semantic reasoning but remain insensitive to fine-grained forensic cues. To bridge this gap, we propose SafeGuard, a multi-agent framework that enables collaborative specialization between forensic perception and semantic reasoning. A hierarchical perceptual solver extracts fine-grained forensic evidence, while a self-reflective verifier enforces consistency between semantic inference and physical plausibility, forming an interpretable evidence chain. To support evaluation, we introduce SafeVid, a novel AI-generated video detection benchmark comprising 20K videos spanning 10 social risk categories, designed to evaluate physical plausibility, structural consistency, and the rationality of social behaviors. Extensive experiments demonstrate the generalization of SafeGuard, improving accuracy on SafeVid by +18.7% and consistently outperforming prior methods across four public benchmarks.

[91] Vidu S1: A Real-Time Interactive Video Generation Model cs.CV | cs.LGPDF

Jintao Zhang, Kai Jiang, Jintao Chen, Xu Wang, Yang Luo

TL;DR: Vidu S1是一个支持语音控制的实时交互式视频生成模型，用户可通过语音指令随时控制视频生成内容。该模型基于TurboDiffusion和TurboServe构建，能在普通消费级GPU上以高达42 FPS的速度生成540p的实时视频，支持无限长度生成且无模糊、漂移或视觉失真。

Details

Motivation: 解决现有视频生成模型在实时交互、语音控制以及长视频生成中存在的延迟、质量下降和缺乏用户个性化控制的问题。

Result: 实验表明，Vidu S1在所有测试指标上均达到最佳性能，同时完全满足实时推理需求，在实时视频生成任务中实现了SOTA水平。

Insight: 创新点在于将语音控制与实时视频生成深度融合，并利用TurboDiffusion和TurboServe技术栈实现了高质量、无失真的无限长度实时生成，同时支持用户上传自定义图像和选择语音音调以增强个性化体验。

Abstract: We introduce Vidu S1, a real-time interactive video generation model supporting voice control of digital characters. Users can control video generation content at any moment through voice instructions. Vidu S1 supports infinite-length real-time video generation without blurring, drift, or visual distortion. Built with TurboDiffusion and TurboServe, Vidu S1 outputs 540p real-time videos at up to 42 FPS on regular consumer GPUs. Users can upload custom images of real people, anime, and pets, and choose different voice tones for personalized experiences. Experiments show that Vidu S1 achieves the best performance across all test metrics while fully meeting real-time inference requirements. A playable online demo is available at https://vidu.com/vidu-stream.

[92] Rethinking Brain Decoding with CLIP: The Role of Adversarial Robustness cs.CVPDF

Byeongseo Bok, Futa Waseda, Jun Liu, Isao Echizen

TL;DR: 本文探讨了在脑解码任务中使用对抗性鲁棒性改进CLIP模型的效果。研究发现，通过采用对抗性训练的CLIP变体，可以提升fMRI信号与视觉表征的对齐质量，从而在图像检索和零样本分类任务上取得更好的性能。

Details

Motivation: CLIP模型因其丰富的视觉-语言嵌入空间被广泛用于脑解码，但其表征可能包含与大脑活动不完全对齐的统计预测线索，限制了解码性能。本文旨在研究对抗性鲁棒表征是否能改善神经对齐。

Result: 在NSD和GOD数据集上的fMRI图像检索和零样本分类任务中，使用对抗性鲁棒的CLIP变体相比标准CLIP，任务性能一致提升，并在多个指标上表现出更强的对齐。

Insight: 对抗性训练通过抑制非鲁棒特征并促进更稳定、感知结构化的表征，可能更好地与大脑活动对齐；目标表征的选择对神经解码性能有重要影响，对抗性鲁棒性可作为脑解码的有用准则。

Abstract: Brain decoding aims to uncover neural mechanisms by inferring stimulus-related representations from brain signals. In fMRI studies, this is typically achieved by mapping fMRI responses to the latent representations of computational models. Recently, CLIP has become a popular choice for brain decoding due to its rich vision–language embedding space. However, aligning fMRI signals with CLIP representations remains challenging. As CLIP is not explicitly optimized for neural alignment, its representations may capture statistically predictive cues that are only partially reflected in brain activity, limiting decoding performance. In this paper, we investigate whether adversarially robust representations improve neural decoding with CLIP. Adversarial training suppresses non-robust features and promotes more stable, perceptually structured representations, which may better align with brain activity. We evaluate this by fixing the fMRI decoder and varying only the target representation (standard CLIP vs. robust variants) on fMRI-image retrieval and zero-shot classification tasks across NSD and GOD datasets. Empirical results show that this simple change consistently improves task performance and yields stronger alignment across multiple metrics. Attribution analysis further reveals consistently low agreement between standard CLIP and its robust variants, suggesting that adversarial robustness reorganizes feature importance in the visual representation. These findings suggest that the choice of target representation influences neural decoding performance and that adversarial robustness may serve as a useful criterion for brain decoding.

[93] Text as Partial Constraint: Core-Residual Alignment for Robust Vision-Language Learning cs.CV | cs.AIPDF

Chengzhen Yu, Canran Xiao, Siyuan Ma, Yang Liu

TL;DR: 本文提出了一种名为TPC（Text as Partial Constraint）的核心-残差对齐框架，旨在解决自然语言描述通常不完整（underspecified）导致视觉-语言对齐脆弱且置信度过高的问题。该方法将多视角描述视为不完全监督，提取语义核心作为对齐目标，并抑制未提及的残差信息对相似度的影响，从而学习更稳健的表示。

Details

Motivation: 自然语言描述往往不完整，使得视觉-语言对齐在释义或细节省略时变得脆弱且置信度过高，本文旨在学习一种匹配稳定且置信度能反映文本对图像约束强度的表示。

Result: 在零样本识别和对抗鲁棒性任务上，TPC在ImageNet上取得了81.42/64.05的干净/鲁棒Top-1准确率，在Avg-14迁移套件上达到76.19/52.03，并在LLaVA-1.5-7B架构下提升了LVLM迁移性能，获得85.16的POPE F1分数和59.57的OKVQA准确率。

Insight: 创新点在于将文本建模为部分约束，通过核心-残差分解和不确定性感知的对比目标，在语言监督不完整时实现更可靠的对齐；客观来看，该方法为处理不完整语言监督提供了一种原则性且实用的框架，通过显式分离核心语义与未提及残差来提升鲁棒性。

Abstract: Vision-language alignment powers open-vocabulary recognition, retrieval, and LVLM grounding, yet natural captions are often underspecified, making similarity brittle and overly confident under paraphrase and omitted details. We aim to learn representations whose matching is stable across caption views and whose confidence reflects how strongly text constrains an image. We propose Text as Partial Constraint (TPC), a core-residual alignment framework that treats multi-view captions as incomplete supervision. It distills a consensus semantic core as the alignment target, learns a single-view core predictor for standard inference with one query, and explicitly discourages vision-language similarity from depending on the orthogonal unsaid residual. An uncertainty-aware contrastive objective further softens alignment when caption views disagree, reducing overconfident updates under weak language constraints. Across zero-shot recognition and adversarial robustness, TPC achieves 81.42/64.05 Top-1 clean/robust accuracy on ImageNet and 76.19/52.03 on an Avg-14 transfer suite, while improving LVLM transfer with 85.16 POPE F1 and 59.57 OKVQA accuracy under an LLaVA-1.5-7B stack. These results suggest that modeling text as a partial constraint is a practical and principled route to more reliable vision-language representations under underspecified language supervision.

[94] FairFlow: Demystifying and Mitigating Stereotype Bias in Text-to-Image Diffusion Transformers cs.CVPDF

Chen Chen, Yuanmin Huang, Zhenfei Zhang, Mi Zhang, Xiaohan Zhang

TL;DR: 本文提出FairFlow框架，旨在揭示并缓解多模态扩散变换器（MM-DiT）在文本到图像生成中存在的刻板印象偏见。通过机制分析发现偏见由稀疏的语义绑定层介导，FairFlow通过学习属性特定的公平方向并在关键层进行稀疏注入，有效中和了性别、种族等偏见，同时保持高保真度。

Details

Motivation: 现代文本到图像生成系统广泛采用MM-DiT作为骨干，但即使面对良性提示，它们仍系统性地表现出严重的刻板印象偏见，导致算法歧视风险。现有缓解策略主要针对传统U-Net架构，对MM-DiT中偏见的精确修复仍是一个关键挑战。

Result: 在FLUX.1-dev和Stable Diffusion~3上的评估表明，FairFlow能有效中和性别、种族及交叉场景中的刻板印象漏洞，实现了最优的公平性与保真度平衡，且推理开销近乎为零，对复杂提示具有鲁棒性。

Insight: 创新点在于通过机制分析揭示了MM-DiT中偏见表征的非均匀分布特性，识别出作为内部语义绑定枢纽的稀疏层，并据此提出了一种机制引导的稀疏调控框架FairFlow，为大规模部署的MM-DiT系统提供了轻量且实用的偏见缓解方案。

Abstract: Multimodal diffusion transformers (MM-DiTs) have emerged as the prevalent backbone for modern text-to-image generation systems. However, they exhibit critical alignment vulnerabilities, systematically manifesting severe stereotype biases even under benign prompts. This poses a significant risk of algorithmic discrimination in deployed systems. Since most existing mitigation strategies were tailored for legacy U-Net architectures, the precise remediation of these vulnerabilities in MM-DiTs remains a critical open challenge. In this work, we first investigate the root cause of this vulnerability via mechanistic analysis. We reveal that bias representations in MM-DiTs are not uniformly distributed across depth, but are mediated by a sparse set of layers functioning as internal semantic binding hubs. These hubs exhibit a stage-wise propagation driving bias manifestation: early hubs establish the structural templates susceptible to bias, middle hubs actively extract core stereotypical concepts from textual conditioning, and late hubs globally solidify these biases through visual self-attention. Leveraging these architectural insights, we propose FairFlow, an intrinsic, mechanism-guided mitigation framework. FairFlow acts as an internal regulator by employing sparse steering: it learns attribute-specific fair directions and injects them exclusively at the identified semantic hubs within a constrained inference window. Evaluations on FLUX.1-dev and Stable Diffusion~3 demonstrate that FairFlow effectively neutralizes these stereotypical vulnerabilities across gender, race, and intersectional settings, achieving an optimal fairness-fidelity balance. With near-zero inference overhead and robustness to complex prompts, FairFlow provides a lightweight and practical bias mitigation for large-scale deployed MM-DiT systems. Code and datasets will be publicly released upon acceptance.

[95] BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception cs.CVPDF

Geng Li, Yuxin Peng

TL;DR: 本文提出BVS（贝叶斯视觉搜索）框架，旨在解决多模态大语言模型（MLLMs）在超高分辨率图像中细粒度感知能力不足的问题，特别是针对杂乱场景中的微小物体。BVS将感知建模为连续空间-尺度流形上的全局优化问题，通过结合先验引导与后验校正，利用MLLM的早停注意力机制构建推理感知先验，并采用尺度感知非平稳核与GP-UCB进行迭代局部观测以动态校正噪声和恢复缺失信息。

Details

Motivation: 现有方法面临两难：要么依赖低效的无先验扫描，要么依赖缺乏后验校正的静态先验启发式方法，无法纠正初始模型偏差。因此，需要一种能有效处理UHR图像中细粒度感知的框架。

Result: 大量实验表明，BVS在准确性和效率之间取得了优越的权衡，显著优于最先进的基线方法。理论分析通过次线性遗憾界限提供了保证。

Insight: 创新点在于将贝叶斯优化思想引入视觉搜索，通过推理感知先验与动态后验校正的桥接，以及尺度感知非平稳核的设计，实现了对MLLM在细粒度感知任务中偏差的有效纠正和信息恢复。

Abstract: While Multimodal Large Language Models (MLLMs) demonstrate impressive general capabilities, they struggle with fine-grained perception in ultra-high-resolution (UHR) images, particularly for tiny objects in cluttered scenes. Existing methods face a dilemma: they either rely on inefficient prior-free scanning, or depend on static prior-driven heuristics that lack posterior correction to rectify initial model biases. To address this, we propose BVS (Bayesian Visual Search), a framework that formulates perception as a global optimization problem over a continuous spatial-scale manifold. Specifically, BVS bridges prior guidance with posterior correction: it utilizes an early-stop attention rollout of MLLM to construct reasoning-aware priors, while employing a scale-aware non-stationary kernel and GP-UCB to dynamically rectify noise and recover missing information in the prior through iterative local observations. We provide theoretical guarantees via sub-linear regret bounds, and extensive experiments demonstrate that BVS significantly outperforms state-of-the-art baselines with a superior trade-off between accuracy and efficiency.

[96] A Multi-Task Deep Learning Framework for Real-Time Intelligent Video Surveillance with Temporal Event Validation cs.CV | cs.AIPDF

Estera Dumitru, Stelian Spînu

TL;DR: 本文提出了一种用于实时智能视频监控的统一多任务深度学习框架，该框架在共享GPU平台上同时执行人脸识别与区域授权、自动车牌识别、武器检测、火灾烟雾检测以及人体动作识别。该框架集成了两个针对特定场景（武器检测和破坏行为识别）优化的深度学习模型，并引入了一种基于多帧确认、置信度加权投票和级联滤波的时间事件验证架构，以提升连续视频中的鲁棒性。

Details

Motivation: 现代视频监控系统产生的视频流远超人工监控能力，因此需要自动化分析来及时检测安全事件。现有公开数据集对某些特定安全场景（如特定武器检测、破坏行为）的表征不足，且帧级预测在连续视频中容易产生误报，需要提升系统的可靠性和实时性。

Result: 在公开数据集上的独立评估显示，武器检测模型mAP@0.5达到0.947，基于SlowFast-R50的破坏行为识别模型在自建数据集上分类准确率达94.33%。集成的时间验证策略将火灾烟雾误报率从52%降至4%，视频车牌精确匹配准确率从66.7%提升至81.8%。整个框架在商用硬件上保持实时性，每帧延迟低于100毫秒。

Insight: 创新点在于将多个专门化的深度学习模型集成到一个统一框架中，并提出了一个时间事件验证架构，通过多帧信息融合将不稳定的帧级预测转化为可靠的安全事件。这种结合专门模型与时间验证的策略，为解决实时视频监控中的误报和鲁棒性问题提供了有效且实用的方案。

Abstract: Modern video surveillance systems generate far more video streams than human operators can effectively monitor, making automated analysis essential for timely detection of security events. This paper presents a unified multi-task deep learning framework that simultaneously performs face recognition with zone-based authorization, automatic license plate recognition, weapon detection, fire and smoke detection, and human action recognition on a shared GPU platform. Among the integrated modules, two task-specific deep-learning models are proposed in this work to address scenarios that are insufficiently represented in publicly available datasets: a single-class weapon detector fine-tuned on a merged and relabeled dataset, achieving a mean average precision (mAP@0.5) of 0.947, and a SlowFast-R50 action recognition model trained on a purpose-built vandalism dataset comprising 614 video clips, achieving 94.33% classification accuracy. To improve robustness in continuous video, all detection modules are integrated into a temporal event-validation architecture based on multi-frame confirmation, confidence-weighted voting, and cascaded filtering, transforming frame-level predictions into reliable security events. Each module is evaluated independently on established public datasets (LFW, D-Fire, FIRESENSE, and UCF-Crime), followed by integrated end-to-end system evaluation. The proposed temporal validation strategy reduces the fire and smoke false-alarm rate from 52% to 4% and improves video license plate exact-match accuracy from 66.7% to 81.8%, while the complete framework maintains real-time operation with a per-frame latency below 100 ms on commodity hardware. These results demonstrate that combining specialized deep-learning models with temporal event validation provides an effective and practical solution for reliable real-time intelligent video surveillance.

[97] OpenGlass: A Sensing-Computing Split Architecture for Local MLLM-Driven Real-Time Visual Assistance cs.CV | cs.AI | cs.CL | cs.HCPDF

Mengzhang Li, Yuan Yao

TL;DR: OpenGlass是一个面向盲人和低视力用户的开源、注重隐私、本地优先的低延迟多模态视觉辅助系统。它采用传感与计算分离的架构：基于ESP32的眼镜端单元捕获视觉上下文，而附近的消费级设备执行本地MLLM推理和语音输出，从而减少对云的依赖，默认将原始第一人称视觉数据保留在用户控制的设备上。

Details

Motivation: 解决云端MLLM助手需要上传第一人称视觉数据且存在多秒网络延迟，以及可穿戴眼镜因计算和功耗限制无法承载大型模型的问题，为视觉障碍用户提供低延迟、隐私保护的实时视觉辅助。

Result: 在真实的ESP32 Wi-Fi捕获下，使用调整大小的有效载荷时，OpenGlass达到中位数用户到音频延迟为993毫秒，使用原始1280x720有效载荷时为1625毫秒；分别有97.5%和93.3%的试验延迟低于2秒。

Insight: 提出了传感与计算分离的架构，将轻量级传感与本地MLLM推理解耦，在保证低延迟和隐私的同时，利用消费级设备的能力，为构建本地优先、用户可控的实时多模态辅助系统提供了参考平台。

Abstract: We present OpenGlass, an open-source, privacy-oriented, local-first system for low-latency multimodal visual assistance, with a primary focus on blind and low-vision users. Cloud MLLM assistants offer strong visual understanding, but often require uploading first-person visual data and can suffer multi-second network delays; wearable glasses are ideal for sensing, but cannot host large models under tight compute and power budgets. OpenGlass addresses this gap with a sensing-computing split: an ESP32-based glasses-side unit captures visual context, while a nearby consumer-grade device performs local MLLM inference and local speech output, reducing cloud reliance and keeping raw egocentric visual data on user-controlled devices by default. We evaluate response quality, query-ready-to-audio latency, safety-aware abstention, and auditable logs. Under real ESP32 Wi-Fi capture, OpenGlass reaches 993 ms median user-to-audio latency with resized payloads and 1625 ms with raw 1280 x 720 payloads; 97.5% and 93.3% of trials fall below 2 s, respectively. OpenGlass is a user-initiated visual-assistance reference platform for obstacle/hazard awareness, sign/object queries, and image-quality self-checking, rather than a certified navigation aid. We release source code, hardware instructions, prompts, evaluation data, and logs.

[98] OmniLayout: A Schematic-Coupled Multimodal Benchmark for Constraint-Aware Geometric Reasoning in PCB Layout cs.CVPDF

Taiting Lu, Kaiyuan Lin, Mingjia Wang, Haolin Ye, Runze Liu

TL;DR: 本文介绍了OmniLayout，这是首个用于评估大语言模型在真实几何、布线和连接约束下进行PCB布局放置推理能力的基准测试。该基准包含1,681个工业级原理图耦合的PCB布局，并设计了四项任务：几何推理、布线感知放置、电气功能保持以及工具增强的代理推理。实验结果表明，当前大语言模型在PCB布局放置方面存在显著局限性。

Details

Motivation: 尽管大语言模型在3D空间推理和几何理解方面取得了显著进展，但其在严格空间和功能约束下进行密集对象放置推理的能力尚未得到充分探索，而这正是电子设计自动化工作流中的核心挑战。

Result: 研究结果揭示了当前大语言模型在PCB布局放置任务中的重大不足，包括几何推理能力弱、布线优化能力差以及电气功能保持不一致。

Insight: 论文的创新点在于构建了首个面向PCB布局的、结合原理图的多模态基准测试，系统地评估了LLMs在复杂工程约束下的几何推理能力，为将AI应用于实际EDA流程提供了关键的评估工具和洞见。

Abstract: Recent large language models (LLMs) have demonstrated remarkable progress in 3D spatial reasoning, spatial grounding, and fine-grained geometric understanding. However, their ability to reason about densely packed object placement under strict spatial and functional constraints remains largely unexplored, despite being a fundamental challenge in practical electronic design automation (EDA) workflows. To bridge this gap, we introduce OmniLayout, the first benchmark designed to evaluate LLMs on printed-circuit-board (PCB) layout placement reasoning under real-world geometric, routing, and connectivity constraints. OmniLayout contains 1,681 industrial-grade schematic-coupled PCB layouts and includes four tasks: (1) geometric reasoning for IC physical placement, with 77.24K placement instances constrained within PCB board boundaries; (2) routability-aware placement reasoning, generating physically valid component placements; (3) electrical functionality, preserving schematic-specified connectivity and electronic functional correctness; and (4) tool-augmented agentic reasoning for invoking external tools to accomplish tasks (1)-(3). Our results reveal substantial limitations of current LLMs in PCB layout placement, including weak geometric reasoning, poor routability optimization, and inconsistent preservation of electrical functionality.

[99] Fast 3D Foundation Model Initialized Gaussian Splatting cs.CV | cs.GR | cs.ROPDF

Anurag Dalal, Daniel Hagen, Kjell G. Robbersmyr, Kristian Muri Knausgård

TL;DR: 本文提出了一种无需传统运动结构恢复（SfM）的快速高质量3D高斯溅射（3DGS）重建方法。该方法利用3D基础模型（3DFM）初始化相机位姿和点云，然后通过深度引导的损失函数联合优化相机位姿和高斯基元，即使在仅有50-60张输入视图的粗略初始化下也能快速收敛。

Details

Motivation: 解决传统3DGS重建流程依赖SfM进行初始化，导致计算成本高、速度慢的问题，旨在实现快速、高质量的3D重建，适用于机器人、VR和自主导航等近实时应用。

Result: 在Mip-NeRF 360、Tanks and Temples和RobustNeRF等基准测试上，该方法取得了具有竞争力的重建质量（PSNR 23.61 dB， LPIPS 0.19），同时将每个场景的训练时间缩短至约三分钟。

Insight: 主要创新点在于利用3D基础模型提供相机位姿和点云的初始化，从而绕过计算密集的SfM步骤；同时引入基于MLP的位姿细化模块和来自基础模型的深度引导监督，以提升稀疏视图下的重建质量，实现了速度与质量的平衡。

Abstract: This paper introduces a fast method for high-quality 3D Gaussian Splatting (3DGS) reconstruction without traditional Structure-from-Motion (SfM). The proposed approach leverages 3D Foundation Models (3DFMs) for camera pose and point-cloud initialization, then jointly optimizes both camera poses and Gaussian primitives using a depth-guided loss function. This enables fast convergence even from rough initialization with as few as 50-60 input views. To further improve reconstruction quality in sparse-view scenarios, an MLP-based pose refinement module is introduced alongside depth-guided supervision from the foundation model. Extensive experiments on Mip-NeRF 360, Tanks and Temples, and RobustNeRF demonstrate that the proposed method achieves competitive reconstruction quality (23.61 dB PSNR, 0.19 LPIPS) while reducing training time to approximately three minutes per scene. The proposed method produces ready-to-use 3DGS models at a fraction of the time required by existing pipelines, making it suitable for near real-time applications in robotics, VR, and autonomous navigation.

[100] From General Actions to Domain-Specific Monitoring: Prior-Adaptive Transfer for Skeleton-Based Action Recognition cs.CVPDF

Hao Wang, Di Yang, Jiangtao Wang

TL;DR: 本文提出了一种名为PATS（Prior-Adaptive Transfer of Skeletons）的框架，用于将基于骨骼的通用动作识别模型自适应地迁移到特定领域任务（如医疗监测）。该框架通过一个新颖的自适应先验迁移模块，在迁移过程中选择性地保留与任务相关的运动先验知识，并过滤冗余部分，从而提升特定任务的性能和效率。

Details

Motivation: 动机在于，直接将在大规模通用动作数据集上预训练的骨骼动作识别模型迁移到特定领域任务（如医疗健康监测）时效果不佳，因为这些任务范围较窄，且可能只与通用运动先验知识的一个子集相关。保留不相关的预训练组件不仅会阻碍模型适应，还会增加计算成本。

Result: 在阿尔茨海默症检测和跌倒检测这两个特定动作识别任务上的实验表明，PATS在性能和效率上均优于有竞争力的基线方法。

Insight: 主要创新点在于提出了一个自适应先验迁移模块，该模块通过迭代剪枝和精炼，将模型压缩作为一种先验选择机制，从而实现了从通用动作模型到特定领域任务的高效、选择性知识迁移。

Abstract: Skeleton-based action recognition models have recently shown strong performance on large-scale benchmarks with general actions. However, directly transferring them to domain-specific tasks e.g., healthcare monitoring, is often suboptimal, as such tasks are narrow in scope and may be relevant to only a subset of general motion priors. Moreover, not all pretrained motion patterns are equally useful for a specific task, and retaining less relevant components may hinder adaptation and increase computational cost. To address these challenges, we propose Prior-Adaptive Transfer of Skeletons (PATS), a framework that adapts general skeleton-based models by selectively retaining task-relevant motion priors while filtering redundant ones during transfer. PATS follows a standard pipeline that extracts skeleton signals from videos and employs a spatio-temporal backbone pre-trained on general actions. The key contribution lies in a novel Adaptive Prior Transfer module, which performs model compression as a prior selection mechanism through iterative pruning and refinement. Experiments on two specific action recognition tasks, Alzheimer’s detection and fall detection, show consistent improvements in both performance and efficiency over competitive baselines. The code will be released upon acceptance.

[101] Beyond Post-Quantization: Native Hash Learning with a Dedicated HASH Token cs.CV | cs.IRPDF

Xinze Liu, Ding Wang, Hengjie Zhu, Dayan Wu

TL;DR: 本文提出HashViT，一种基于Vision Transformer的原生哈希学习方法，通过引入专用的HASH令牌在Transformer内部直接学习二进制哈希码，避免了传统后量化方法中连续特征与离散汉明空间之间的差异。该方法在多个基准测试中实现了最先进或极具竞争力的检索性能。

Details

Motivation: 解决现有深度哈希方法（基于CNN和ViT）普遍采用后量化范式导致的特征空间与汉明空间不匹配问题，即连续优化表示与最终离散哈希码生成之间存在差距。

Result: 在三个广泛使用的基准测试上进行大量实验，HashViT实现了最先进（SOTA）或极具竞争力的检索性能，同时保持了紧凑汉明码的高效性。

Insight: 创新性地将哈希学习作为Transformer内部令牌演化的过程，通过结构分解的HASH令牌（包含直接生成二进制码的哈希寄存器和保留辅助连续语义的语义工作空间）以及轻量级哈希精化适配器，实现了面向二进制的表示在主干网络中的渐进形成；采用结合可学习语义中心监督、类令牌相似性蒸馏和量化正则化的统一目标进行优化。

Abstract: Efficient large-scale image retrieval requires compact representations that preserve semantic similarity under fast Hamming-space search. Deep hashing is appealing, but most existing CNN- and ViT-based methods still follow a post-quantization paradigm, where continuous visual features are first learned and binary codes are then produced by a terminal hash projection or binarization operation. This late code generation creates a feature-to-code discrepancy between the continuously optimized representation space and the discrete Hamming space used for retrieval. To address this limitation, we propose HashViT, a Vision Transformer framework for native hash token learning. Instead of treating hashing as a terminal readout, HashViT introduces a dedicated HASH token that serves as a persistent, hash-oriented retrieval state inside the transformer. The HASH token is structurally decomposed into a Hash Register for direct binary code generation and a Semantic Workspace for preserving auxiliary continuous semantics. To enable effective workspace-to-register interaction, we further design a lightweight Hash Refinement Adapter that progressively refines the Hash Register across transformer layers. As a result, binary-oriented representations are formed through token evolution within the backbone, rather than being abruptly induced by an output-level projection. HashViT is optimized with a unified objective that combines learnable semantic center supervision, class-token similarity distillation, and quantization regularization, encouraging the HASH token to encode semantically structured and compact binary representations. Extensive experiments on three widely used benchmarks demonstrate that HashViT achieves state-of-the-art or highly competitive retrieval performance while preserving the efficiency of compact Hamming codes. Code is available at https://github.com/Xinze919/HashViT.

[102] Pathways of Visual Information Flow in Vision-Language Models cs.CV | cs.CL | cs.LGPDF

Israfel Salazar, Stella Frank, Dan Oneata, Desmond Elliott, Constanza Fierro

TL;DR: 本文研究了视觉语言模型中视觉信息的流动路径。通过因果修补技术在合成与自然数据集上的实验，发现模型依赖两种不同路径处理视觉任务：直接路径（视觉信息保留在图像标记中并由后续层最终标记读取）和文本中介路径（视觉信息先转移至查询标记再被最终标记读取）。研究表明路径选择具有任务依赖性，且数据分布与提示设计可调节路径使用；模型在干预下能灵活切换路径，揭示其内部机制的可塑性。

Details

Motivation: 旨在探究视觉语言模型中视觉信息的具体流动机制，理解模型如何通过不同路径处理视觉任务，以揭示其内部工作方式。

Result: 在三个视觉任务上的实验表明，路径选择取决于任务类型，且数据分布和提示设计可调节路径使用；通过注意力敲除和输入损坏修补，发现模型在干预下能依赖文本中介路径作为备用方案。

Insight: 创新点在于识别了视觉语言模型中视觉信息流动的两种具体路径及其任务依赖性，并展示了模型在干预下的路径灵活性，这统一了先前研究中的发现，强调了基于干预的方法能揭示模型的潜在能力而非常规行为。

Abstract: We study how visual information is routed in vision-language models (VLMs). Using causal patching on controlled synthetic and natural datasets, we find that models rely on two distinct pathways to solve visual tasks: A direct pathway, where visual information is retained in image token representations and read out by the final token at later layers, and a text-mediated pathway, where visual information is first transferred to the query tokens and then read out by the final token. Across three visual tasks, we show that pathway selection is task-dependent, and that data distribution and prompt design can also modulate which pathway is used to solve the image-based query. Moreover, using attention knockouts and corrupted-input patching, we find that these pathways are flexible, under certain interventions, models can rely on the text-mediated pathway as a fallback when the usual pathway is ablated. This behavior unifies findings in prior work and shows that ablation-based interventions can reveal what models could do rather than what they normally do. Together, our results provide a mechanistic characterization of visual information flow in VLMs and highlight the flexibility of their internal mechanisms under intervention.

[103] Present but Not Remembered: Auditing How Frozen VLAs Encode, Deploy, and Steer Visual History cs.CVPDF

Chih-Ting Liao, Xin Cao

TL;DR: 本文研究了冻结视觉-语言-动作模型（VLA）如何编码、部署和利用视觉历史信息。通过层解析线性探测和因果互换干预，发现模型虽然能线性解码历史帧内容，但存储的历史信息与当前帧高度冗余，且仅在当前帧严重退化时才被因果部署。模型并未忘记过去，但未能将历史表征为与当前不同的信息。

Details

Motivation: 现有研究主要关注为VLA添加记忆模块，而忽略了探究模型内部如何表示和利用已有的时序历史信息。本文旨在通过审计冻结VLA模型，揭示其在时间轴上的信息处理机制。

Result: 在来自两个架构家族的三个VLA模型上进行实验，发现所有模型对历史的编码方式相似，但部署策略不同：在相同遮挡条件下，一种架构会越来越多地依赖历史作为后备，而另一种则减少依赖。研究进一步提出了无需训练的时间部署审计方法来区分这些机制。

Insight: 核心创新点在于揭示了VLA中历史信息的冗余性及其条件性部署机制，表明可操控性（steerability）取决于历史如何被部署，而非是否被编码。这为未来记忆增强设计提供了关键见解：应注入与当前不同的独特历史信息，而非简单地增加历史数据量。

Abstract: A frozen vision-language-action model (VLA) receives recent observations at every decision step, yet prior work has focused on adding memory rather than asking how existing history is represented and used. We study this temporal axis using layer-resolved linear probing and causal interchange interventions across three VLAs from two architecture families. We find a three-part dissociation. First, past-frame content remains linearly decodable throughout the network. Second, information unique to history beyond the current frame is nearly absent, indicating that stored history is largely a redundant copy of the present. Third, history is causally deployed only when the current frame is heavily degraded, while the action readout progressively loses dependence on history through the network. Although all models encode history similarly, their deployment strategies differ: under the same occlusion, one architecture increasingly relies on history as a fallback, whereas the other relies on it less. We further introduce a training-free temporal deployment audit that distinguishes these regimes. In the fallback regime, re-injecting history neither repairs occlusion nor disambiguates actions, confirming the redundancy of the stored representation. In the other regime, the same intervention reliably steers the predicted action toward the donor history. These results show that steerability depends on how history is deployed rather than whether it is encoded. VLAs do not forget the past; they largely fail to represent it as information distinct from the present. Our findings suggest that future memory augmentation should inject information unique to the past rather than simply more history.

[104] WorldBagel: Uncovering the Power of Unified Multimodal Models for Vision-Language-Action-World Modeling cs.CV | cs.LGPDF

Zelin Zhao, Min Shi, Bo Yuan, Haotian Xue, Jialuo Li

TL;DR: 本文提出了WorldBagel，这是一个基于现代多模态统一模型BAGEL构建的统一视觉-语言-动作-世界（VLAW）建模框架。该研究系统地探讨了统一化在世界建模中的作用，并在多个机器人操作任务和跨领域实验中验证了其有效性。

Details

Motivation: 世界模型旨在捕捉支持感知、推理和行动的环境动态，是VLAW建模的核心方向。当前统一的视觉-语言模型虽展现出强大的多模态生成能力，但其作为世界模型的潜力尚未被充分探索。

Result: 在LIBERO、Language Table和Franka等基准上的实验表明，WorldBagel在多项任务中持续优于特定任务模型，学习到的动作表示更具结构性和语义对齐性。

Insight: 创新点在于将统一的视觉-语言模型架构系统地应用于VLAW世界建模，并实证了统一化不仅是架构上的便利，更是学习有效模型的关键因素，能带来性能提升和对多模态世界建模的更深入理解。

Abstract: World models aim to capture environment dynamics in ways that support perception, reasoning, and action, and have recently become a central direction in Vision-Language-Action-World (VLAW) modeling. Meanwhile, unified vision-language models have demonstrated strong multimodal generation capabilities, yet their potential as world models remains underexplored. In this work, we introduce \texttt{WorldBagel}, a unified VLAW framework built on BAGEL, a modern multimodal unified model, and use it to systematically investigate the role of unification in world modeling. Across multi-task robotic manipulation and cross-domain experiments, \texttt{WorldBagel} consistently outperforms task-specific alternatives and learns action representations that are more structured and semantically aligned with visual and linguistic context. Experiments on LIBERO, Language Table, and Franka show that unification is not only an architectural convenience, but also a key factor in learning effective VLAW models, leading to consistent empirical gains and deeper insights into multimodal world modeling. Code and model checkpoints will be released upon acceptance.

[105] Learning to Generate Multiple Objects from Dense and Occluded Layouts cs.CVPDF

Bach-Hoang Ngo, Si-Tri Ngo, Hieu Le, Trung-Nghia Le

TL;DR: 本文针对文本到图像扩散模型在密集场景中生成物体数量不准确的问题，提出了一种布局感知注意力偏置和模态平衡损失的方法，以解决重叠物体实例所有权崩溃的问题，并引入了OverlapDepth-45K基准进行系统评估。

Details

Motivation: 解决文本到图像扩散模型在密集场景中因物体重叠和遮挡导致实例所有权崩溃，从而无法正确生成物体数量的问题。

Result: 在提出的OverlapDepth-45K基准上，该方法显著提高了计数准确性，防止了实例合并，同时保持了图像质量。

Insight: 创新点包括布局感知注意力偏置以促进区域一致的分组并抑制跨实例泄漏，以及模态平衡损失以基于遮挡程度放大被遮挡物体的梯度；从客观角度看，该方法通过结构化注意力机制和损失函数设计，有效处理了密集遮挡场景中的实例区分问题。

Abstract: Text-to-image diffusion models fail to generate correct object counts in dense scenes, where overlapping instances collapse into indistinguishable structures despite appearing visually plausible. We identify this as instance ownership collapse: tokens from overlapping objects interact freely through attention, while heavily occluded instances receive weak supervision due to their small visible areas. We address this through layout-aware attention biases that softly bias token interactions toward region-consistent grouping and suppress cross-instance leakage, paired with an amodal-balanced loss that amplifies gradients for occluded objects based on their occlusion level. To enable systematic evaluation, we introduce OverlapDepth-45K, a benchmark of densely overlapping scenes with amodal supervision. Our approach substantially improves count accuracy and prevents instance merging while preserving image quality. Project page: https://bachngoh.github.io/AIBL

[106] Brand-as-Memory: Vision-Language Models Encode Causal, Mechanistically Localizable Credibility Priors for News Sources cs.CV | cs.AIPDF

Chih-Ting Liao, Xin Cao

TL;DR: 该论文研究了视觉语言模型（VLMs）在处理新闻图像时，会基于新闻来源的品牌标识（如报头名称、Logo、域名）形成强烈的可信度先验，并可能因此忽略文章内容本身。作者提出了CueTrust跨模型诊断基准和Source-Override Index（SOI）来衡量这种‘来源覆盖’效应，并对其进行了全面的机制性解释，包括其形成位置、编码方式及因果干预方法。

Details

Motivation: 动机在于，随着VLMs越来越多地以图像形式阅读新闻和网页内容，其中包含的发布者身份信息（视觉标识）可能会让模型形成对新闻来源可信度的先验判断，从而可能覆盖对文章内容证据的评估，这是一个潜在的可靠性问题。

Result: 在七个VLMs和五种来源线索上的实验表明，模型对来源线索的‘覆盖’效应是模型和规模依赖的，且特定于媒体品牌标识。对于品牌线索，仅交换报头就能使可信度预测在约11个对数几率范围内变化，且与专业媒体评级高度相关（与Media Bias/Fact Check的rho=0.88）。该先验在模型的第19-21层因果形成，可通过干预特定方向将覆盖效应降低41%。

Insight: 创新点在于：1）提出了CueTrust基准和SOI指标来量化VLMs中‘来源覆盖内容’的现象；2）对品牌可信度先验提供了全面的机制性解释，包括其形成层、双编码（名称和Logo）、可解释的稀疏自编码器特征承载，并证明其在不同模型家族中出现在相同相对位置；3）通过因果干预证实该先验被模型主动使用，而不仅仅是可解码的关联，为理解和缓解VLMs的此类可靠性缺陷提供了工具和洞见。

Abstract: Vision-language models (VLMs) increasingly read news and web content as images, where the publisher’s identity is visually present. We show that VLMs carry a strong source-credibility prior keyed on outlet identity, and study it along three axes. (i) Cross-model benchmark. We introduce CueTrust, a cross-model diagnostic that measures which surface source cue overrides an article’s content evidence via a Source-Override Index (SOI). Across seven VLMs and five cues, the vulnerability profile is model- and scale-dependent, and the override is outlet-identity-specific and encoding-invariant, firing from the masthead name, the logo image, or the bare domain, but not from a named author, in-text authority, or page layout (clean negative controls). (ii) Mechanistic account. For the brand cue, we give a full mechanistic account: swapping only the masthead moves credibility across an approximately 11 log-odds range that tracks professional ratings (rho = 0.88 with Media Bias/Fact Check). The prior is dual-coded (name and logo), strengthens with scale, is causally formed at layers 19-21, carried by interpretable seed-stable sparse-autoencoder features, and recurs at the same relative locus in a second model family. It overrides content (about 1.8x) as a signal-magnitude effect within a shared pathway, not a privileged route. Steering the localized direction selectively reduces the override (41% reduction) and generalizes to held-out outlets, confirming the prior is causally used, not merely decodable. Deployed VLMs may thus defer to source identity over the evidence in front of them, a reliability failure we can measure across models, localize, and causally probe. We release the stimulus suite and CueTrust.

[107] Perceptual Flow Matching for Few-Step Generative Modeling cs.CVPDF

Chuyang Zhao, Yifei Song, Hongfa Wang, Jianlong Yuan, Yuan Zhang

TL;DR: 本文提出了感知流匹配（PFM）框架，用于提升流匹配模型的少步生成能力。该方法通过在预训练的感知特征空间中进行流匹配监督，而非传统的VAE潜在空间，从而显著减少了生成所需的采样步数（从35-50步降至4-8步），同时保持了生成质量。PFM无需教师模型或辅助评分网络，可轻松集成到标准流匹配训练流程中。

Details

Motivation: 解决流匹配模型在少步生成时采样效率低的问题，传统方法需要大量采样步骤才能保证质量，而现有加速或蒸馏方法又依赖额外模型或复杂架构。

Result: 在图像生成、视频生成和图像编辑任务上的广泛实验表明，PFM在4-8步内就能生成高质量结果，且比基于蒸馏的方法产生更少的伪影，达到了少步生成下的先进水平。

Insight: 创新点在于将流匹配监督从VAE潜在空间转移到感知特征空间，这使回归目标从均值寻求转向模式寻求，偏向于流形上的模式，从而在粗粒度少步积分下仍保持准确性；揭示了在适当表示空间中进行监督可使标准流匹配训练自然产生高效的少步生成器。

Abstract: We propose Perceptual Flow Matching (PFM), a simple yet effective framework for few-step generation in flow-matching models. Rather than performing velocity regression in the conventional VAE latent space, PFM supervises flow matching in a perceptual feature space using pretrained perceptual models. This simple change substantially improves the few-step generation capability of flow-matching models, reducing the number of sampling steps from 35-50 to 4-8 while preserving generation quality. Unlike existing acceleration and distillation approaches, PFM requires neither teacher models nor auxiliary score networks and can be integrated into standard flow-matching training pipelines with minimal modifications. Extensive experiments on image generation, video generation, and image editing tasks demonstrate that PFM consistently produces high-quality results while producing fewer artifacts than existing distillation-based methods. We further show that perceptual supervision shifts the regression minimizer from mean-seeking to mode-seeking, biasing predictions toward on-manifold modes that remain accurate under coarse few-step integration. Our results reveal that standard flow-matching training can naturally yield high-quality few-step generators when supervised in an appropriate representation space. We hope this insight inspires future research into representation-aware objectives for efficient generative modeling.

[108] iVISION-2DCD: A Long-Term Change Detection Dataset for Large-Scale Outdoor Construction Monitoring cs.CV | cs.ROPDF

Dayou Mao, Yuchen Lin, Ashkan Ebadi, John Zelek, Alexander Wong

TL;DR: 该论文提出了iVISION-2DCD，一个用于大规模户外施工监测的长期变化检测数据集。该数据集通过从密集LiDAR点云合成生成，包含逼真的输入图像和精确的真值标注，旨在解决施工场景中视角鲁棒的2D变化检测问题。

Details

Motivation: 动机在于自动化施工监控需要能够从任意相机视角检测变化，但目前缺乏开源的、结合2D变化检测与施工自动化的基准数据集，且在实际工地使用无人机收集数据存在安全和视角受限的问题。

Result: 使用最先进的2DCD算法进行的基准评估表明，iVISION-2DCD数据集为计算机视觉和机器人社区提出了新的研究挑战。

Insight: 创新点在于通过合成数据生成方法（包括新颖的视图合成技术）来克服双时相对齐和视角多样性的挑战，并实现了半自动的语义分割与变化标签生成，同时保留了具有挑战性的真实世界案例，从而形式化定义了施工场景中视角鲁棒的2DCD问题。

Abstract: Automation in construction is essential for reducing costs and human errors in large-scale projects. We approach the construction progress monitoring from the aspect of detecting changes in construction sites. As construction buildings continue to evolve in geometry and appearance over time, change detection need to be performed from arbitrary camera viewpoints. This necessitates developing 2D Change Detection (2DCD) algorithms that operate robustly across diverse camera perspectives at construction sites. While developing and evaluating such systems is data-intensive, no open-source benchmark dataset exists at the intersection of 2D change detection and construction automation research. Data collection using Unmanned Aerial Vehicles (UAVs) is gaining its popularity in outdoor large-scale surveying. However, in active construction sites conducting drone missions equipped with high-end sensors imposes safety concerns. Flight trajectory and collected camera viewpoints can be significantly limited. To address this critical gap, we introduce iVISION-2DCD, a large-scale synthetically generated dataset from dense LiDAR point clouds with photorealistic input images and accurate ground truth annotations. Our dataset formally defines the problem of viewpoint-robust 2DCD at construction sites and captures the inherent complexities of real-world deployment. In this paper, we present our systematic methodology for synthetic data generation, developing novel view synthesis techniques to overcome bi-temporal alignment and viewpoint diversity challenges, and implementing semi-automated semantic segmentation with change label generation while preserving challenging real-world cases. Benchmark evaluations using state-of-the-art 2DCD algorithms demonstrate that iVISION-2DCD poses novel research challenges for the computer vision and robotics communities.

[109] Flex-Forcing: Towards a Unified Autoregressive and Bidirectional Video Diffusion Model cs.CVPDF

Xinyin Ma, Julius Berner, Chao Liu, Arash Vahdat, Weili Nie

TL;DR: 本文提出Flex-Forcing，一个统一的训练和推理框架，使视频扩散模型能够无缝地在双向和自回归两种生成模式下工作。其核心是通过在时间轴和去噪步骤上联合定义的灵活分块机制，实现根据设备预算灵活分块、在块间进行双向推理以规划全局结构，同时在块内进行自回归生成以进行高效细粒度合成，并支持任意顺序、任意时间步的自回归生成。

Details

Motivation: 现有大规模生成模型在视频生成方面存在局限：双向扩散模型擅长全局一致性和视觉保真度但推理慢，自回归模型能高效流式生成但牺牲了长程一致性和存在曝光偏差。本文旨在统一这两种范式，克服各自的缺点。

Result: 在多个视频生成基准测试上的广泛实验表明，与采用固定推理调度的强基线相比，Flex-Forcing在视频质量、长视频稳定性方面持续表现更好，同时提供更快的推理速度。

Insight: 创新点在于提出了一个灵活分块机制，将时间轴和去噪步骤联合考虑，从而统一了双向和自回归生成范式，实现了设备自适应的灵活推理、全局规划与局部高效生成的结合，并突破了严格因果约束，支持更灵活的生成顺序。

Abstract: Recent progress in large-scale generative models has substantially advanced video generation, yet existing methods remain constrained by a rigid inference paradigm. Bidirectional diffusion models excel at global coherence and visual fidelity but suffer from slow inference, while autoregressive models offer efficient and streaming generation at the cost of long-range consistency and exposure bias. We introduce Flex-Forcing, a unified training and inference framework that enables a video diffusion model to seamlessly operate under both bidirectional and autoregressive generation regimes. The core idea is a flexible chunking mechanism jointly defined over the temporal axis and denoising steps. This design allows the model to (1) perform flexible chunking according to different device budgets, (2) perform bidirectional inference across chunks for global structure planning, while generating frames autoregressively within each chunk for efficient and fine-grained synthesis, and (3) perform any-order, any-timestep autoregressive generation without the strict causal constraint. Extensive experiments on multiple video generation benchmarks demonstrate that Flex-Forcing achieves consistently better video quality, long-video stability than strong baselines with a rigid inference schedule, while offering faster inference.

[110] Latent Clarity: Bridging World-Model Kinematics to Semantic Manifolds for Video Anomaly Anticipation cs.CV | cs.AI | cs.LGPDF

Abu Anas Ibn Samad

TL;DR: 本文提出了一种名为PULS（预测统一潜在空间）的连续语义世界模型流水线，用于视频异常预测。该模型包含两个模块：KSD Bridge（运动学-语义蒸馏桥）和ASP（预期状态预测器）。KSD Bridge将V-JEPA 2的物理张量映射到与文本对齐的Qwen3-VL-Embedding-2B高维空间，而ASP则用于锐化预测的未来潜在表示。该方法在UCF-Crime和XD-Violence数据集上实现了优异的异常检测性能，并通过零样本VQA验证了其语义分离能力。

Details

Motivation: 当前连续视频异常检测主要依赖于反应式的多实例学习（MIL）方法，该方法将时空特征压缩为标量分数，存在局限性。本文旨在构建一个能够桥接世界模型运动学与语义流形的预测性框架，以更早、更准确地预测视频异常。

Result: 在UCF-Crime数据集上，仅使用KSD Bridge进行映射，就实现了0.8994的chunk级AUROC；在分布外数据集XD-Violence上达到0.8162。ASP在14-way零样本VQA任务中达到44.5%的平均准确率，比观测基线高出9.6个百分点。通过三重跟踪提前时间协议，在T-0.5秒时获得了+8.9个百分点的预测优势。

Insight: 核心创新点是提出了’潜在清晰度假说’：由于JEPA的时间预测器丢弃了随机像素噪声而保留了运动学信息，预测的未来表示比观测到的当前表示在语义上更具可分离性。这通过ASP对观测张量应用时准确率崩溃至随机水平（7.3%）得到验证，表明预测和观测占据不同的子流形。该方法证明了基于牛顿不变运动学的表示具有良好的分布外泛化能力。

Abstract: Continuous video anomaly detection is dominated by reactive Multiple Instance Learning (MIL) that collapses spatiotemporal features into scalar scores. We introduce PULS (Predictive Unified Latent Space), a continuous semantic world-model pipeline comprising two modules: a 490M-parameter KSD Bridge (Kinematic-to-Semantic Distillation) and a 16.8M-parameter Anticipatory State Predictor (ASP). The KSD Bridge maps V-JEPA 2 physical tensors into the 2048-d Qwen3-VL-Embedding-2B text-aligned hypersphere, trained on a subset of UCF-Crime. This translation alone yields a chunk-level AUROC of 0.8994 for UCF-Crime and 0.8162 for out-of-distribution XD-Violence without MIL or hierarchical fusion. We introduce and validate the Latent Clarity Hypothesis: because JEPA’s temporal predictor discards aleatoric pixel noise while preserving kinematics, anticipated future representations are more semantically separable than observed presents. The ASP sharpens these anticipated future latents, achieving 44.5% mean 14-way zero-shot VQA accuracy (exceeding observation baseline by +9.6 pp). Applying the ASP to Observation Tensors collapses accuracy to 7.3% (random chance), proving Anticipation and Observation occupy distinct sub-manifolds. A Triple-Track Lead-Time protocol with an L1-surprise gate yields a peak +8.9 pp anticipatory advantage at T-0.5s (p < 0.001, N = 1,000 permutation), separating physical anticipation from static scene priors. Zero-shot transfer to XD-Violence confirms that Newtonian-invariant kinematic representations generalize out-of-distribution.

[111] Vision Non-Causal Trapezoidal Mamba: Eliminating Directional Scanning in Vision SSMs with Second-Order Dynamics cs.CVPDF

Anvitha Ramachandran, Dhruv Parikh, Haoyang Fan, Rajgopal Kannan, Viktor Prasanna

TL;DR: 本文提出了Vision Non-Causal Trapezoidal Mamba (VNCT)，一种用于视觉任务的二阶非因果状态空间模型。它通过消除传统视觉SSM中源自序列建模的方向性令牌扫描，使所有图像令牌能在单次前向传播中交互，从而降低了单图像推理延迟，并生成了对方向更鲁棒的表示。

Details

Motivation: 大多数视觉状态空间模型（SSMs）继承了因果序列建模中的方向性令牌扫描机制，这虽然对序列数据有效，但在视觉任务中会引入空间偏差和对方向敏感的表示。本文旨在消除这种不必要的方向性扫描，为视觉识别提供一个更简单、高效且鲁棒的替代方案。

Result: 在ImageNet-1K分类、COCO目标检测与实例分割以及ADE20K语义分割等多个基准测试中，VNCT一致性地超越了采用方向性扫描的视觉SSMs以及一阶非因果SSMs。具体而言，它在图像旋转和翻转下表现出更小的性能下降，并将边界IoU提升了高达3.7个百分点，实现了更准确的边界保持和目标定位。

Insight: 论文的核心创新点在于提出了一个二阶非因果视觉状态空间模型，从根本上消除了方向性扫描，实现了所有令牌的单次全局交互。从客观角度看，这证明了高性能视觉SSM无需方向性扫描，并且二阶非因果状态空间建模在提升效率、鲁棒性和边界精度方面是一个有效的设计选择。

Abstract: State Space Models (SSMs) have emerged as an alternative to Vision Transformers, yet most vision SSMs inherit directional token scanning from causal sequence modeling. While effective for sequential data, directional scanning introduces spatial bias and orientation-sensitive representations. We present Vision Non-Causal Trapezoidal Mamba (VNCT), a second-order non-causal vision SSM that enables all image tokens to interact in a single pass, eliminating direSctional scanning and achieving low single-image inference latency. VNCT exhibits more orientation-robust representations, showing reduced performance degradation under image rotations and flips, while improving Boundary IoU by up to 3.7 points, leading to more accurate boundary preservation and object localization. Across ImageNet-1K classification, COCO object detection and instance segmentation, and ADE20K semantic segmentation, VNCT consistently outperforms both directional-scanning vision SSMs and first-order non-causal SSMs. These results show that directional scanning is unnecessary for high-performance vision SSMs and that second-order non-causal state-space modeling offers a simple, efficient, and robust alternative for visual recognition.

[112] XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detection cs.CVPDF

Abhijeet Narang, Kartik Kuckreja, Shreya Ghosh, Muhammad Haris Khan, Jianfei Cai

TL;DR: 本文提出了XPlainVerse，一个包含一百万张真实与伪造图像的大规模基准数据集，旨在联合评估深度伪造检测与以人为中心的解释生成。该数据集通过多阶段过滤流程Edit-Check确保操作的可信性，并提供技术和简化两种解释风格，同时引入了EntityScore和EvidenceScore等新指标来评估解释的推理保真度。

Details

Motivation: 现有深度伪造检测模型生成的自然语言解释往往缺乏对视觉伪影的可靠依据，且现有基准主要关注分类准确性，忽略了解释是否真实反映实际篡改，这限制了可部署、可解释系统的进展。

Result: 在包含2000个解释对的人工标注验证中，数据集质量得到了人类判断的确认，证明了其可靠性和有效性。

Insight: 创新点在于构建了首个大规模、具有可靠监督的深度伪造检测与解释基准，并提出了基于实体和证据的量化指标来评估解释的推理保真度，而非仅依赖表面相似性，这为可信、可解释模型的研究提供了可扩展的基础。

Abstract: As deepfake detection models increasingly produce natural language explanations, their reasoning often remains weakly grounded in visual artifacts, limiting reliability and user trust. Existing benchmarks mainly evaluate classification accuracy, overlooking whether explanations reflect the actual manipulations. This gap hinders progress toward deployable, explainable deepfake detection systems. To this end, we introduce XPlainVerse, a large-scale benchmark designed for joint deepfake detection and human-centered explanation. XPlainVerse comprises one million real and manipulated images, pairing authentic images from five established sources with forgeries generated by twelve off-the-shelf image editing and synthesis models. We further propose a multi-stage filtering pipeline, Edit-Check, to verify if manipulations satisfy their intended edits, enabling reliable reasoning supervision at scale. Beyond dataset scale, XPlainVerse provides two complementary explanation styles: technical explanations for expert analysis and simplified explanations optimized for non-technical users. To evaluate explanation quality beyond surface similarity, we propose novel metrics, EntityScore and EvidenceScore, that measure reasoning fidelity by checking whether explanations correctly identify manipulated entities and visual evidence. Human annotations on 2,000 explanation pairs validate our dataset quality against human judgment. We believe XPlainVerse will establish grounded explanation quality as a measurable dimension of deepfake detection and support scalable research on trustworthy, interpretable models.

[113] Responsibility Distribution Estimation in Ego-View Accident Videos with Multimodal Large Language Models cs.CV | cs.AI | cs.CYPDF

Ryosei Tamura, Andrew Shin

TL;DR: 本文提出了一种基于多模态大语言模型的责任分布估计任务，用于分析驾驶员视角的交通事故视频，以预测各参与方的责任比例。研究通过构建LLM辅助的责任标注流程，并在多种输入设置下微调多模态大语言模型，实验结果表明该方法能有效执行这一基于约束的推理任务，为交通事故理解提供了新方向。

Details

Motivation: 现有交通事故理解研究多依赖基础设施摄像头、卫星图像或结构化记录，这些数据成本高且无法客观反映驾驶员事故前的实际观察视角；而驾驶员视角视频直接呈现驾驶员的视觉信息，更适合推理事故可避免性和驾驶员责任。

Result: 实验建立了强大的初始基准，表明多模态大语言模型能有效执行这一细致、基于约束的推理任务；在多种输入设置（包括原始帧、分割增强输入和文本描述）下微调的模型均表现出色。

Insight: 创新点在于首次引入驾驶员视角视频的责任分布估计任务，并利用LLM辅助标注和多模态输入微调方法；客观来看，该方法将交通事故分析从传统分类和解释扩展到更具社会和法律意义的多模态推理，为低成本、可扩展的事故评估提供了新思路。

Abstract: Recent studies on multimodal traffic accident understanding have mainly relied on infrastructure-camera footage, satellite imagery, or structured crash records. However, such data sources are costly to deploy and maintain at large scale, and they cannot objectively capture what the driver was actually able to observe before the accident. In contrast, ego-view accident videos directly represent the driver’s visual perspective, making them suitable for reasoning about avoidability and driver responsibility. In this paper, we introduce responsibility distribution estimation for ego-view traffic accident videos, a new task in which a model predicts the percentage of responsibility assigned to each involved agent. We construct an LLM-assisted responsibility annotation pipeline and fine-tune multimodal large language models under multiple input settings, including raw frames, segmentation-enhanced input, and textual descriptions. Experimental results establish a strong initial benchmark, demonstrating that multimodal LLMs can effectively perform this nuanced, constraint-based reasoning task. Our findings suggest that ego-centric accident videos provide a promising foundation for socially and legally meaningful multimodal reasoning beyond conventional accident classification and explanation tasks.

[114] When Simpler Is Better: Evaluating Translation Pipelines for Medieval Latin Manuscripts cs.CV | cs.AI | cs.CLPDF

Nguyen Kim Hai Bui, Md. Easin Arafat, Tamás Gábor Orosz, Mufti Mahmud

TL;DR: 本文系统评估了中世纪拉丁文手稿的图像到翻译流程，发现专用OCR模型结合通用视觉语言模型（VLM）的简单流程优于复杂多组件方案，并引入了首个中世纪拉丁文对齐数据集IPC。

Details

Motivation: 针对历史手稿翻译中低资源转写、古旧词汇和噪声输入等核心NLP挑战，现有视觉语言模型表现不佳，需系统评估完整翻译流程以揭示在干净文本基准中不可见的失败模式。

Result: 在CATMuS拉丁数据集上，专用OCR模型将字符错误率降低至通用VLM的4.3倍（参数量少数个数量级）；简单OCR+VLM流程在翻译质量上优于所有添加检索增强生成（RAG）或后OCR校正的复杂变体。

Insight: 创新点包括提出首个中世纪拉丁文对齐图像-转录-翻译数据集IPC，并发现‘复杂性悖论’：简单专用OCR直接输入VLM的流程因避免提示饱和和错误传播而最优，为低资源历史场景部署提供了新基准和实践指导。

Abstract: Despite remarkable progress in machine translation, Vision Language Models (VLMs) struggle on historical manuscripts, a domain that stresses core Natural Language Processing (NLP) capabilities: low-resource transliteration, archaic vocabulary, and noisy input signals. We present a systematic framework for evaluating the full image-to-translation pipeline on medieval Latin manuscripts, a setting in which scribal shorthand, ligatures, and parchment degradation expose failure modes that are invisible in clean-text benchmarks. Benchmarking on the CATMuS Latin dataset reveals a specialization gap: domain-specific Optical Character Recognition (OCR) models reduce character error rate by up to 4.3$\times$ compared to general-purpose VLMs, despite operating at orders of magnitude fewer parameters. We introduce the Interpres-Parallel-Corpus (IPC), a novel dataset comprising 1,383 aligned manuscript image lines, transcriptions, and expert translations, the first of its kind for medieval Latin. Our experiments uncover a complexity paradox: the simplest pipeline, a specialized OCR model feeding directly into a VLM, outperforms all multi-component variants. Adding retrieval-augmented generation (RAG) or post-OCR correction introduces prompt saturation and error propagation that degrade aggregate translation quality. These findings offer both a new benchmark and practical guidance for deploying translation systems in low-resource historical settings.

[115] Token-Based Affordance Grounding with Large Vision-Language Models cs.CV | cs.AI | cs.ROPDF

Seung Il Lee, Qinqian Lei, Daguang Xu, Dong Yang, Robby T. Tan

TL;DR: 本文提出TokAG，一种基于大型视觉语言模型（LVLMs）的零样本可供性接地框架，通过利用LVLM输出token级别的语义-空间信号，无需外部监督即可定位图像中与特定动作相关的区域。该方法引入空间感知的token选择机制，筛选出关注目标对象的注意力图，并将其转换为零样本可供性热图。

Details

Motivation: 现有弱监督方法在处理外中心视角图像中视觉模糊或共现动作时存在困难，且依赖简短动作短语缺乏丰富语义细节，无法区分语义相似动作；尽管LVLMs编码了丰富动作语义，但其输出不直接提供动作特定的空间定位。

Result: 在多个基准测试中，该零样本框架一致优于先前的弱监督方法：在AGD20K未见数据分割上NSS指标提升10.7%，在HICO-IIF上提升29.7%。

Insight: 创新点在于发现LVLM不同输出token的注意力图存在显著差异，并设计系统化token选择机制提取对象聚焦的注意力图，将LVLM隐含语义信号显式转化为空间定位能力，实现了零样本可供性接地。

Abstract: Affordance grounding aims to localize image regions that support a specific action, serving as a core capability for physical intelligence and embodied perception. Previous studies have primarily relied on weakly supervised learning with action labels from exocentric images. However, these methods often struggle with visually ambiguous exocentric images containing co-occurring actions; moreover, they fail to distinguish semantically similar actions because existing methods typically rely on brief action phrases that lack rich semantic details for action-specific localization. Although large vision-language models (LVLMs) encode rich action semantics and their action-conditioned textual outputs implicitly contain spatial cues, they do not directly provide action-specific spatial localization. To address these problems, we propose TokAG, a zero-shot affordance grounding framework that exploits the token-level semantic-spatial signals in LVLMs to localize action-relevant regions without external supervision. We observe that attention maps associated with different LVLM output tokens vary significantly, with many attending to irrelevant regions such as the background. Thus, we introduce a spatial-aware token-selection mechanism to systematically evaluate each output token and select the one whose attention maps exhibit dominant activation over the target object, instead of relying on arbitrary attention maps. By extracting these object-focused attention maps, we transform the LVLM’s implicit semantic signals into zero-shot affordance heatmaps. Our zero-shot framework consistently outperforms prior weakly supervised approaches across multiple benchmarks, improving NSS by 10.7% on the unseen split of AGD20K and by 29.7% on HICO-IIF. The code and models will be made publicly available.

[116] A Step Towards Robust Unsupervised Domain Adaptation via Fine-Tuning and Reinforcement Learning cs.CV | cs.AIPDF

Sushant Dagaji Desale, Rahul Mishra, Ashutosh Kumar Sinha

TL;DR: 本文提出了一种名为SFT+RL的两阶段鲁棒无监督域适应框架，旨在解决域适应模型在对抗性攻击下的鲁棒性问题。该框架首先利用有监督微调在带标签的源域上适应对抗性噪声，然后通过强化学习阶段，采用置信度引导的伪标签策略逐步标注目标域样本，并在混合的干净与对抗样本上进行训练，以提升跨域鲁棒性。

Details

Motivation: 无监督域适应中的对抗鲁棒性是一个重大挑战，主要源于噪声伪标签以及干净源域与对抗性扰动的目标域之间的固有分布偏移。现有方法难以在鲁棒性和准确性之间取得最佳权衡。

Result: 在OfficeHome、PACS和VisDA三个基准数据集上的综合评估表明，该方法有效。SFT+RL在所有三个数据集上平均提升了10.2%的干净准确率和15.8%的对抗鲁棒性，性能优于现有的最先进方法。

Insight: 创新点在于将对抗性微调与置信度引导的渐进式伪标签策略相结合，通过两阶段训练（SFT和RL）来平衡伪标签的质量与覆盖率，并利用混合批次对抗训练增强跨域鲁棒性。该方法在CLIP预训练视觉编码器基础上进行部分解冻和调整，有效利用了其丰富的语义先验。

Abstract: Adversarial robustness in Unsupervised Domain Adaptation (UDA) remains a significant challenge due to noisy pseudo labels and inherent distributional shifts between the clean source and adversarially perturbed target domains. Existing approaches often fail to achieve an optimal trade-off between robustness and accuracy, as pseudo-labels generated by domain-adapted models tend to introduce classification errors under adversarial attacks. In this work, we propose \textbf{SFT+RL}, a two-stage robust UDA framework that integrates Supervised Fine Tuning (SFT) and Reinforcement Learning (RL) on top of CLIP’s pre-trained visual encoder. In the SFT stage, we adversarially fine-tune a linear classifier using PGD-based perturbations over the labelled source domain while partially unfreezing CLIP’s projection layer. It allows adaptation to adversarial noise while preserving CLIP’s rich semantic priors. We introduce a confidence-guided pseudo-labeling strategy in the RL stage to annotate unlabeled target samples progressively. Pseudo labels are filtered using a decaying confidence threshold to balance quality and coverage, and the model is trained on a composite dataset formed by combining clean source samples with high-confidence target samples. Adversarial training is applied to mixed batches of clean and adversarial examples to enhance cross-domain robustness. Comprehensive evaluations on three benchmark datasets OfficeHome~~\cite{tomm-ude}, PACS~~\cite{pacs}, and VisDA~\cite{visda} demonstrate the effectiveness of our approach. Notably, \textbf{SFT+RL} achieves average improvements of \textbf{10.2%} in clean accuracy and \textbf{15.8%} in adversarial robustness across all three datasets, outperforming existing state-of-the-art methods.

[117] IDEAL-Bench: Indoor Dataset and Evaluation suite for Analyzing 3D Layout reasoning cs.CVPDF

Yuening Cai, Junwei Zhou, Youran Qu, Yu-Wing Tai

TL;DR: 本文提出了IDEAL-Bench，一个用于评估视觉语言模型（VLMs）在单张图像上进行结构化3D室内布局推理能力的基准测试套件。该基准基于一个包含1000个可重新渲染的逼真室内场景的数据集IDEAL-Scenes，要求模型预测每个可见物体的姿态和范围，并通过数值维度和感知渲染比较协议进行评分。

Details

Motivation: 现有的空间问答范式主要评估视觉语言模型的空间智能，但忽略了其整体3D布局推理能力这一互补维度。为了全面评估VLMs从单张图像中预测结构化3D布局的能力，需要超越CLEVR风格的简单几何基元，在逼真场景下进行评测。

Result: 对15个主流VLMs的评估表明，该任务远未解决，最强模型总体得分仅为62.1/100。所有模型在物体识别和几何回归方面表现出明显的不对称性。模型排名与基于问答和简单几何重建的基准测试结果部分不同，中游模型排名变化显著。

Insight: 创新点在于提出了一个专注于评估VLMs几何与结构能力的诊断性基准，超越了基于问答的评估局限。该基准使用程序化生成的逼真室内场景，通过可控的资产替换、光照和视角，确保图像空间的差异仅反映空间推理能力，而非语义或渲染伪影，为评估下一代VLMs是否真正理解空间而非仅进行语言近似提供了原则性诊断工具。

Abstract: Spatial question answering is the dominant paradigm for evaluating spatial intelligence in Vision-Language Models (VLMs), but it leaves a complementary axis of spatial competence under-evaluated: holistic 3D layout inference, which predicts every visible object’s pose and extent from a single image in a structured form. To this end, we introduce IDEAL-Bench, an evaluation suite that requires VLMs to predict structured 3D layouts on photorealistic indoor scenes across 10 room types, scored along five numerical dimensions and a perceptual render-and-compare protocol. By operating on semantically realistic scenes with full asset substitution under controlled lighting and viewpoint, IDEAL-Bench moves beyond CLEVR-style simple geometric primitives so that any image-space discrepancy reflects spatial reasoning alone. The benchmark is built on IDEAL-Scenes, a procedurally generated dataset of 1,000 re-renderable Blender environments with ground-truth layouts. Evaluating 15 prominent VLMs reveals three findings: the task remains substantially unsolved, with the strongest model reaching only 62.1/100 overall; all models exhibit a sharp asymmetry between object recognition and geometric regression, indicating that current VLMs are trained to describe scenes rather than to measure them; model rankings partially diverge from those on QA-based and primitive-reconstruction benchmarks: top-tier consensus holds, but mid-tier rankings shift substantially. Collectively, these findings establish IDEAL-Bench as a diagnostic suite, targeting the geometric and structural competencies that QA-based evaluation cannot surface, and paving the way towards more rigorous evaluation of spatial intelligence in next-generation VLMs. Together, these findings position IDEAL-Bench as a principled diagnostic for whether future VLMs achieve genuine spatial understanding rather than linguistic approximations of it.

[118] A Vision Based System for Guided and Collaborative Reconstruction of Fragmented Documents cs.CV | cs.HC | cs.ROPDF

Oliver Krumpek, Diana Leo

TL;DR: 本文提出了一种用于文化遗产保护的协作式实时碎片化纸质文档重建系统，该系统结合了协作机器人（cobot）和基于真空吸盘的精密定位装置，实现了0.57mm的定位重复精度。系统支持手动视觉引导或全自动定位模式，并通过评估多种局部特征匹配方法，最终选择了SE2-LoFTR作为图像解释的核心方法，以应对碎片旋转、缩放和损伤等挑战。

Details

Motivation: 解决文化遗产保护中碎片化纸质文档（如受损档案材料）的精确、无损重建难题，通过人机协作与自动化技术提高重建效率与精度。

Result: 系统实现了0.57mm的定位重复精度（针对8cm²碎片）；在图像匹配任务中，SE2-LoFTR方法在旋转、缩放鲁棒性和碎片损伤适应性方面表现最佳，被选为系统的核心算法。

Insight: 创新点包括：1）结合协作机器人与定制真空吸盘实现脆弱材料的精密无损操作；2）融合人机交互与全自动模式，提升系统灵活性；3）系统评估了局部特征匹配方法在文档重建中的适用性，为类似任务提供了方法选择依据。

Abstract: This paper presents the development and evaluation of a collaborative system for real-time reconstruction of fragmented paper documents in the context of cultural heritage preservation. The developed system includes a collaborative robot, or cobot, that can fully manage the positioning of paper fragments using a specially designed vacuum-based suction attachment. This attachment enables gentle and precise positioning, ensuring the preservation of fragile materials. With this device, we are able to achieve a positioning repeatability of 0.57mm for fragments of 8cm^2. The system offers users the flexibility to choose between manual positioning, with visual guidance, or fully automated positioning performed by the cobot. To further improve the reconstruction process, AI methods for image interpretation, specifically for segmentation and positioning tasks, were applied and evaluated for their applicability to template-based reconstruction of damaged paper fragments. Our investigation provides critical insights into the performance of different local feature matching methods under different document types, taking into account rotation, scale robustness, and the degree of damage to the fragments. With a focus on the reconstruction of damaged and optically altered archival material, SE2-LoFTR, a detector-free local feature matching method, was chosen as the preferred method for the system due to its robust performance in our experiments.

[119] Probing Identity-Specific Motion Signatures: A Controlled Diagnostic Study cs.CVPDF

Yingtie Lei, Fangxun Liu, Baicheng Wu, Colin Lee, Ziheng Zhang

TL;DR: 本文通过构建BALLER120基准数据集，系统研究了现代视频模型在身份识别任务中对身份特异性运动线索的利用程度。研究发现，模型倾向于依赖静态外观线索（如面部和球衣区域），即使存在信息丰富的运动线索；当抑制外观信息后，模型会转向学习运动微模式（如脚部位置和肘部弯曲），并展现出竞争性的准确性和更强的外观变化鲁棒性。

Details

Motivation: 传统身份识别主要依赖静态外观线索，但个体特异性运动模式可能提供更鲁棒的补充特征。本文旨在探究当身份特异性运动线索明确存在时，现代视频模型在多大程度上利用它们进行识别。

Result: 在BALLER120基准（120名职业篮球运动员罚球视频）上，现代视频模型从RGB视频中能准确预测身份，但主要依赖静态外观线索。当输入仅为轮廓或骨架时，模型转向运动微模式，取得了竞争性的准确率，并对外观变化表现出更强的鲁棒性。

Insight: 研究揭示了身份特异性运动签名是可学习且信息丰富的，但模型存在对静态捷径的偏好。通过显式抑制外观线索，可以迫使模型学习更具判别力的运动模式，这为设计更鲁棒的身份识别系统提供了新思路，即主动引导模型关注动态特征而非静态外观。

Abstract: Identity recognition (e.g., person, animal re-identification) has traditionally relied heavily on static appearance cues. Yet motion–consistent, individual-specific dynamics–can provide a complementary and potentially more robust signature, especially when appearance is weak or variable. This raises a fundamental question: when identity-specific motion cues are clearly present, to what extent do modern video models use them for recognition? To investigate this question, we conduct a systematic diagnostic study and introduce BALLER120, a controlled benchmark of 120 professional basketball players performing free-throws. By focusing on the same multi-phase action across individuals, BALLER120 reduces action-level variation and identity-correlated acquisition biases, enabling fine-grained analysis of identity-specific kinematic patterns. We find that modern video models can predict identity accurately from RGB videos, but often rely on static appearance cues such as faces and jersey regions, even when informative motion cues are available. Strikingly, when appearance is suppressed through silhouette-only or skeleton-only inputs, the same model architectures shift toward motion micro-patterns (e.g., foot placement and elbow bending). Despite containing less visual information, appearance-suppressed representations achieve competitive accuracy and stronger robustness to appearance shifts. Our qualitative analyses further show that appearance-suppressed models attend to distinctive motion patterns across individuals. Overall, our study demonstrates that identity-specific motion signatures are present, informative, and learnable, but modern video models may overlook them in favor of easier static shortcuts unless appearance cues are explicitly suppressed.

[120] RADIO1D: Elastic Representations for Condensed Vision Modeling cs.CV | cs.AI | cs.LGPDF

Greg Heinrich, Mike Ranzinger, Collin McCarthy, Natan Bagrov, Eugene Khvedchenya

TL;DR: 本文挑战了视觉语言模型（VLM）需要固定、基于图像块的2D视觉特征的假设。通过分析微调后的视觉编码器，作者发现表征在VLM训练过程中变得愈发抽象且空间一致性降低。基于此，论文提出了RADIO1D方法，它通过多教师知识蒸馏和自编码器设计，将图像压缩成紧凑、可变长度的1D令牌序列。该表征展现出强大的层次化总结能力，支持准确场景理解（即使仅用单个令牌）和改进的构图感知图像检索。在VLM中，RADIO1D通过可调节的令牌数量提供了灵活的精度-效率权衡，在多种多模态基准测试中以更低计算开销实现了有竞争力的性能。

Details

Motivation: 挑战VLM依赖固定2D图像块特征的假设，并利用VLM训练中视觉表征变得抽象且出现少数能有效总结全局图像内容的专用令牌这一观察，旨在开发更紧凑、高效的视觉表示方法。

Result: 在多种多模态基准测试中，RADIO1D以更低计算开销实现了有竞争力的性能，并支持通过调整令牌数量进行精度-效率的灵活权衡。

Insight: 核心创新在于提出了一种将图像压缩为紧凑、可变长度1D令牌序列的方法（RADIO1D），该表征具有层次化总结特性，能实现单令牌场景理解并改进图像检索，为VLM提供了新的、更高效的视觉表示范式。

Abstract: This paper challenges the assumption that vision-language models (VLMs) require fixed patch-based 2D vision features. Analyzing fine-tuned vision encoders, we find that representations become increasingly abstract and less spatially coherent during VLM training. Notably, models trained with image-text alignment (such as SigLIP2) develop a small number of specialized tokens that effectively summarize global image content. Building on this, we introduce RADIO1D, which compresses images into a compact, variable-length 1D token sequence using multi-teacher knowledge distillation and an autoencoder design. The resulting representations exhibit strong hierarchical summarization, enabling accurate scene understanding - even with a single token - and support improved composition-aware image retrieval. In VLMs, RADIO1D provides flexible accuracy-efficiency tradeoffs through adjustable token counts, delivering competitive performance on diverse multimodal benchmarks with lower computational overhead and better accuracy.

[121] Moonstone: A Multimodal Foundation Model and Benchmark for Lunar Remote Sensing cs.CV | cs.AIPDF

Ayush Prasad, Swarnalee Mazumder

TL;DR: Moonstone是首个用于月球遥感的多模态基础模型与基准测试，它整合了来自五个任务、七种仪器家族的多模态数据，构建了28通道、128像素/度的全球月球预训练数据集，并提出了MG-MAE模型进行预训练，在六个下游任务上取得了优于基线的性能。

Details

Motivation: 解决月球遥感数据分散在不同档案中且缺乏机器学习评估基准的问题，旨在为月球科学提供一个统一的多模态基础模型框架。

Result: MG-MAE预训练特征在所有六个下游任务（分类、回归、分割）上均优于从头训练的基线，并大幅超越ImageNet预训练和普通MAE基线。

Insight: 创新点包括：构建首个多模态月球遥感基准数据集；提出MG-MAE模型，采用分组卷积分词器、共享ViT编码器、缺失模态注意力掩码、覆盖自适应掩码和光谱连续性正则化等技术，以处理多模态异质数据并确保重建的物理合理性。

Abstract: Decades of orbital missions have produced multi-modal remote sensing data for the Moon, spanning optical imagery, spectroscopy, thermal emission, radar, gravity, and elemental composition. Yet these datasets remain fragmented across archives, and no benchmark exists for evaluating machine learning on lunar data. We introduce Moonstone, the first multi-modal foundation model benchmark for lunar remote sensing. Our contributions are: (1) a 28-channel, 128 pixels-per-degree (~237 m) global lunar pretraining dataset from seven instrument families across five missions, (2) MG-MAE, a modality-grouped masked autoencoder with per-group convolutional tokenizers, a shared Vision Transformer encoder, attention masking for missing modalities, coverage-adaptive masking for heterogeneous spatial coverage, and spectral continuity regularization for physically plausible reconstructions, and (3) a benchmark of six downstream tasks covering classification, regression, and segmentation. MG-MAE pretrained features outperform scratch baselines on all tasks and surpass both ImageNet-pretrained and vanilla MAE baselines by large margins. Data and code are available at https://huggingface.co/datasets/ayushprd/Moonstone and https://github.com/ayushprd/Moonstone .

[122] Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs cs.CVPDF

Anas Zafar, Leema Krishna Murali, Siddhant Bharadwaj, Ashish Vashist, Jia Wu

TL;DR: 本文针对医学视觉语言模型（VLMs）是否真正依赖视觉证据进行推理的问题，提出了一个反事实评估框架和一种名为CORAL的对比检索增强学习方法。该框架通过替换输入图像为空白、像素打乱、图像缺失和CLIP检索的困难负样本来解耦视觉和文本贡献，并引入了视觉依赖分数（VRS）和视觉幻觉率（VHR）等评估指标。CORAL模型在四个医学VQA基准测试上显著提升了准确率并降低了幻觉率。

Details

Motivation: 当前大型医学视觉语言模型在医学问答任务上报告了很高的准确率，但尚不清楚它们是基于视觉证据进行推理，还是利用了文本中的捷径。本文旨在开发一个评估框架来量化模型的视觉依赖程度，并设计训练方法来增强其视觉基础能力。

Result: 在四个医学VQA基准（PathVQA, PMC-VQA, SLAKE, VQA-RAD）的配对控制评估中，CORAL模型相比基础模型Qwen2.5-VL-7B，将宏观准确率提升了6.7个百分点（P(Delta>0)=0.988），并将视觉幻觉率（VHR）降低了8.0个百分点（P<0.001）。交叉领域诊断表明，在医学基准上图像替换导致的性能下降远小于通用领域任务，凸显了需要解决的视觉基础差距。

Insight: 论文的核心创新点在于提出了一个系统的反事实评估框架来诊断VLMs的视觉依赖问题，并设计了对比性基础目标（CGO）进行训练，通过惩罚模型在困难负样本图像交换下答案不变的行为，强制模型学习视觉基础。这为评估和提升VLMs的视觉推理可靠性提供了可复现的方法论和工具。

Abstract: Large vision language models (VLMs) report strong accuracy on medical question-answering, yet it remains unclear whether they reason from visual evidence or exploit textual shortcuts. We introduce a counterfactual evaluation framework that decouples visual and textual contributions by substituting input images with controlled surrogates blank, pixel-shuffled, image-absent, and CLIP-retrieved hard negatives and derive a suite of grounding metrics including the Visual Reliance Score (VRS) and Visual Hallucination Rate (VHR). We further introduce CORAL (COntrastive Retrieval-Augmented Learning), a 7B-parameter LoRA fine-tune of Qwen2.5-VL-7B trained with a Contrastive Grounding Objective (CGO) that penalises answer invariance under hard-negative image swaps. On a paired controlled evaluation across four closed-form medical VQA benchmarks (PathVQA, PMC-VQA, SLAKE, VQA-RAD; n=400 total), CORAL improves macro accuracy by +6.7 pp (P(Delta>0)=0.988) and reduces VHR by 8.0 pp (P<0.001) over the matched Qwen2.5-VL-7B base; neither MedVLThinker RL variant achieves a significant gain on either metric. Cross-domain diagnostics further reveal that image substitution costs only <=6.5 pp on medical benchmarks versus 48-61 pp on general-domain tasks, situating the grounding gap that CGO targets. We discuss evaluation limitations openly including train/eval benchmark overlap and underpowered secondary metrics and release our framework, training code, and model weights to support reproducible grounding audits of medical VLMs.

[123] ClinOCR-Bench: A Comprehensive Clinical Scanned Document Dataset for Optical Character Recognition Model Evaluation cs.CV | cs.AIPDF

Enshuo Hsu, Jin Zhou, Kirk Roberts

TL;DR: 本文介绍了ClinOCR-Bench，一个用于评估光学字符识别（OCR）模型的综合性临床扫描文档数据集。该数据集包含384张扫描图像，涵盖6个子集（如正常、手写、低质量等），旨在解决临床领域缺乏公开OCR评估基准的问题。

Details

Motivation: 动机在于临床电子健康记录（EHR）中，从扫描医疗文档（如外部实验室报告和手动填写的表格）提取文本信息面临挑战，且现有研究多基于私有数据，缺乏公开数据集来系统评估OCR模型，尤其是针对常见扫描伪影的影响。

Result: 基线OCR性能使用最先进的开源和专有视觉语言模型（VLMs）进行了评估，但摘要中未提及具体定量结果或与SOTA的比较水平。

Insight: 创新点在于提供了一个公开、真实且全面的临床OCR基准数据集，其特点包括多样化的文档类型和布局、全面覆盖常见EHR扫描伪影、无受保护健康信息、基于模板的训练/测试分割以及足够的样本量，填补了该领域公开评估资源的空白。

Abstract: Extracting textual information from scanned medical documents, such as external laboratory reports and manually filled forms, has been a major challenge in modern electronic health records (EHRs). Recent advancements in vision language models (VLMs) have shown great promise over traditional OCR tools. However, at this point, most clinical OCR studies were conducted on private, institutional data. To our knowledge, there are few publicly available datasets for evaluating OCR models in the clinical domain. Furthermore, common scanning artifacts that undermine OCR performance are not reflected in those datasets, leaving a systematic evaluation unfeasible. Therefore, we release a publicly available, realistic-looking OCR benchmark dataset, ClinOCR-Bench, with 384 scanned images across 6 subsets: Normal, Handwriting, Poor Quality, Rotation, Tables, and Mix-artifacts. ClinOCR-Bench features: 1) diverse document types and layouts, 2) full coverage of common EHR scan artifacts, 3) protected health information-free, 4) template-aware train/test split, and 5) adequate sample size for OCR benchmarking. Baseline OCR performance was evaluated using state-of-the-art open-weight and proprietary VLMs. The dataset and documentation are available on GitHub (https://github.com/ClinOCR-Bench/ClinOCR-Bench).

[124] ViPo-MLLM: Visual-Pose Multimodal LLM for Gloss-Free Sign Language Translation cs.CV | cs.AIPDF

Ahmed Abul Hasanaath, Bicheng Xu, Mir Rayat Imtiaz Hossain, Leonid Sigal, Hamzah Luqman

TL;DR: 本文提出ViPo-MLLM，一个视觉-姿态多模态大语言模型，用于无注释手语翻译。该框架融合了时空RGB特征和人体姿态特征，通过专用编码器建模模态内动态，并利用跨模态注意力捕获长程依赖。融合后的表示通过结构化提示进行条件化，并由一个结合对比学习和语言建模目标训练的大语言模型处理。

Details

Motivation: 解决无注释手语翻译任务中，现有方法通常使用单模态或弱融合特征，导致性能受限的问题，旨在通过细粒度建模手、身体和面部线索来提升翻译效果。

Result: 在PHOENIX14T和CSL-Daily数据集上评估，模型取得了新的最先进结果，并且与基于注释的识别方法相比也获得了有竞争力的性能。

Insight: 创新点在于将时空RGB与人体姿态特征进行深度融合，并引入跨模态注意力机制来捕获长程依赖；客观来看，其提出的结构化提示条件化和结合对比学习与语言建模的多目标训练策略，为多模态融合与序列生成任务提供了可借鉴的思路。

Abstract: Gloss-free Sign Language Translation (SLT) translates sign language videos into spoken-language sentences without gloss annotations, avoiding costly labeling but requiring fine-grained modeling of hands, body, and facial cues. Existing methods often use single-modality or weakly fused features, limiting performance. We propose ViPo-MLLM, a framework that integrates spatio-temporal RGB and human pose features. Dedicated encoders model intra-modal dynamics and cross-modal attention captures long-range dependencies. The fused representation is conditioned with a structured prompt and processed by an LLM trained with contrastive and language modeling objectives. The proposed model was evaluated on the PHOENIX14T and CSL-Daily datasets and achieved new state-of-the-art results on both datasets. Moreover, the ViPo-MLLM model attained competitive performance compared to gloss-based recognition approaches, confirming the effectiveness of the proposed pose cues and cross-modal attention mechanisms.

[125] From Geometric Labels to Semantic Understanding of Indoor Building Components Using Multimodal Large Language Models cs.CVPDF

Shuju Jing, Chao Yin

TL;DR: 本文提出了Building-MLLM，一个以点云为中心的多模态大语言模型，用于室内建筑组件的语义理解。该模型通过处理点云和指令，能够完成简单识别、复杂描述和多工程问答任务，并利用领域特定机制和合成的点云-文本数据集进行训练，在各项任务上取得了良好性能。

Details

Motivation: 现有基于点云的方法仅输出离散标签，缺乏对组件功能的解释和自然语言交互能力，无法满足设施运维中对语义理解的需求。

Result: 实验表明，Building-MLLM在简单识别、复杂描述和多工程问答三个任务类型上分别达到88.00%、65.10%和68.14%的准确率，展现了优越的室内组件语言理解能力，并在其他真实数据集上显示出初步的泛化性。

Insight: 创新点包括四个领域特定机制（点信息增强器、几何保持正则化、固定文本前缀、多维LoRA）以解决语义集中问题，以及一个多约束渐进指令生成引擎来构建合成数据集，有效平衡了识别与推理能力。

Abstract: Point cloud-based understanding has become an important enabler for facility operation and maintenance involving indoor building components. However, existing methods output only discrete labels without explaining component functions or natural language interactions. This paper proposes Building-MLLM, a point cloud-centered multimodal large language model (MLLM) for indoor components, which models point clouds and instructions to generate responses across Simple Recognition, Complex Captioning, and Multi-Engineering Question Answering tasks. Building-MLLM addresses semantic concentration through four domain-specific mechanisms: Point Information Enhancer for task-relevant semantics, Geometry-Preserving Regularization preventing geometric erosion, fixed textual prefix for domain stabilization, and multi-dimensional LoRA balancing recognition with reasoning. A multi-constraint progressive instruction-generation engine is developed to compile a synthetic point cloud-text dataset with 4198 objects, 37,782 instruction-following pairs, and 47 categories. Experiments show that Building-MLLM achieves 88.00%, 65.10%, and 68.14% on the three task types, respectively, demonstrating superior indoor component language understanding and providing initial generalizability in transfer inference on other real-world datasets.

[126] ProxyUp: Training-Free Proxy-Conditioned Video Generation for Controllable Dynamics cs.CVPDF

Zanwei Zhou, Jiazhong Cen, Jiemin Fang, Yumeng He, Chen Yang

TL;DR: 本文提出ProxyUp，一种无需训练的代理条件视频生成框架，用于精确控制视频动态。该方法利用物理模拟或真实录制的粗糙代理视频作为动态载体，通过区域潜在噪声注入和随机流松弛技术，在预训练视频生成模型上实现文本驱动的动态可控视频合成。

Details

Motivation: 现有视频生成模型仅依赖文本提示难以精确控制复杂动态（如物理合理的细粒度运动和交互），因此需要一种能够利用外部代理视频作为动态参考的生成方法。

Result: 在模拟和真实世界代理视频上的实验表明，ProxyUp在动态保真度和文本对齐方面优于现有视频编辑和运动迁移基线方法。

Insight: 创新点包括代理条件视频生成范式、区域潜在噪声注入技术以及随机流松弛方法，无需配对数据即可将外部动态先验与文本条件融合，提升生成视频的物理合理性和可控性。

Abstract: Precise control over complex dynamics remains challenging for modern video generative models, as text prompts alone often cannot specify physically plausible, fine-grained motion and interactions. We introduce $\textit{proxy-conditioned video generation}$, where a coarse proxy video from physics-based simulation or real-world recording serves as a dynamics carrier to control foreground object motion. Given a proxy video and a text prompt, the goal is to synthesize a new video that preserves the proxy dynamics while generating novel content and plausible interactions aligned with the prompt. Since paired proxy-target videos are difficult to obtain, we propose $\textbf{ProxyUp}$, a training-free framework built on pretrained video generative models. ProxyUp first inverts the proxy video into an intermediate latent representation and applies $\textbf{region-wise latent noising}$, preserving motion-critical proxy latents while injecting noise into regions intended for text-driven regeneration. To mitigate the distribution mismatch and weak foreground-background coupling introduced by this heuristic latent composition, we further propose $\textbf{Stochastic Flow Relaxation (SFR)}$, which progressively relaxes the composed latent toward the model’s learned distribution before ODE sampling. Experiments on both simulation and real-world proxies show that ProxyUp outperforms strong video editing and motion transfer baselines in dynamic fidelity and text alignment.

[127] Does It Fail to See or Fail to Know? Attributing Errors in Vision-Language Models cs.CV | cs.CLPDF

Khang Nhat Hoang Vo, Artem Vazhentsev, Artem Shelmanov, Timothy Baldwin, Yova Kementchedjhieva

TL;DR: 该论文提出了一个统一框架，用于解构视觉语言模型（VLMs）在视觉问答任务中的失败模式，并研究生成前信号能否预测这些失败来源。研究发现，VLM的错误可分为视觉/识别瓶颈导致的失败和识别后仍存在的失败，且这些失败来源可在解码前通过不同的模型内部表示进行预测，从而实现对不确定案例的针对性干预。

Details

Motivation: 解决VLM在需要超越直接可见知识的复杂问题中失败时，如何不仅量化不确定性，还能诊断失败原因（如感知、实体识别、知识检索等维度），而以往工作多孤立处理单一失败模式或将错误答案视为整体失败。

Result: 在多个数据集和模型家族上的实验表明，识别相关失败最好由视觉令牌表示捕获，而识别后仍存在的失败则由提示条件隐藏状态更好捕获，这实现了在模型生成答案前的高效失败源预测。

Insight: 创新点在于提出了一个解构VLM失败模式的统一框架，并发现生成前信号（视觉令牌表示和提示条件隐藏状态）可有效预测不同失败来源，这为后续针对性干预（如图像修复、实体识别支持或外部检索）提供了理论基础。

Abstract: Vision-language models (VLMs) perform well on visual question answering with high-quality images but struggle when questions require knowledge beyond what is clearly and directly visible. In such settings, uncertainty quantification should not only indicate whether the model is likely to fail but also diagnose why it is uncertain, across dimensions such as perception, entity recognition, and knowledge retrieval. While prior work has focused on individual failure modes in isolation or treated incorrect answers as monolithic failures, we propose a unified framework for disentangling these failure modes and investigate whether pre-generation signals can predict these failure sources. Across a range of datasets and model families, we find a consistent pattern in VLM errors: some failures arise from visual or recognition bottlenecks, while others persist after the relevant entity is identified. Our main finding is that these failure sources can be predicted before decoding: recognition-related failures are best captured by visual-token representations, while failures that remain after recognition are better captured by prompt-conditioned hidden states. This pre-generation signal enables efficient failure-source prediction before the model produces an answer, allowing uncertain cases to be routed to targeted interventions such as image repair, entity recognition support, or external retrieval.

[128] Attending to Multimodal Generation One Token at a Time cs.CV | cs.AIPDF

Varun Gupta, Vineet Gandhi, Makarand Tapaswi

TL;DR: 本文提出了一种名为’One Token at a Time’（OTaT）的分析方法，用于研究多模态大语言模型在自回归生成过程中，注意力在不同语义角色（如图像、文本、指令和已生成令牌）之间的动态转移模式。作者通过设计需要显式切换视觉和文本上下文的任务，在多个主流模型家族和不同规模的开放权重MLLMs中发现了注意力变化的规律，并利用因果注意力阻断干预验证了其功能性作用。基于这些动态规律，作者提出了一种简单的测试时干预方法，以在正确时间增强对相关模态的关注，从而显著提升多模态任务性能。

Details

Motivation: 现有的可解释性研究主要关注模型层和电路（即’在哪里’），而多模态计算在生成过程中的令牌级动态（即’何时’）尚未得到充分探索。本文旨在填补这一空白，研究注意力如何根据语义角色进行转移。

Result: 在两个主流模型家族和四个不同规模的开放权重MLLMs上，研究发现了注意力变化的一致模式：需要图像信息时对图像的注意力达到峰值，任务转换时会重新关注指令令牌，并且随着生成进行，对已生成令牌的注意力会增加。因果注意力阻断干预验证了这些趋势的功能性作用。基于分析提出的测试时干预方法，显著提升了多模态任务性能。

Insight: 创新点在于提出了OTaT分析框架，系统地揭示了MLLM生成过程中跨模态注意力的时序动态规律。客观来看，将注意力动态与语义角色和生成阶段关联起来，为理解和干预模型行为提供了新的视角，所提出的简单测试时干预方法展示了利用这些基础发现提升性能的潜力。

Abstract: Multimodal large language models (MLLMs) generate responses autoregressively, integrating visual and linguistic information in an evolving context. Prior work on interpretability has focused on individual layers and circuits (where), leaving the token-level dynamics of multimodal computation during generation (when) underexplored. We address this gap and study attention shifts as per semantic role; tracking model attention to image, text, instruction, and previously generated tokens, One Token at a Time (OTaT). We introduce multimodal tasks that require explicit switching between visual and textual context within a single response. Across two mainstream model families and four open-weight MLLMs of varying sizes, we establish consistent patterns: attention to image peaks at tokens requiring image-derived information, instruction tokens are revisited during task transitions, and attention to previously generated tokens increases as the generation progresses. Causal attention blocking interventions validate the functional role of these trends. We profile model behavior under disrupted attention and observe responses falling back to language priors, or exhibiting cross-modal leakage, denial, or recovery. Finally, informed of the attention dynamics through our novel analysis, we propose a simple test-time intervention to boost attention to the relevant modality at the right time, significantly improving multimodal task performance.

[129] Exploring SAM Supervision for Fine-Grained UAV Target Segmentation under Data Scarcity cs.CVPDF

Le-Anh Tran

TL;DR: 本文针对无人机目标分割任务中数据标注稀缺、目标尺寸小、背景杂乱等挑战，提出了一种利用SAM3作为伪标签生成器来训练轻量级分割网络的框架。该框架采用两阶段策略：首先生成粗粒度掩码进行目标定位，然后对局部图像块再次应用SAM3以生成精细掩码，最终结合粗、细掩码作为伪标签来优化名为IPS-Seg的轻量网络。

Details

Motivation: 解决无人机目标分割中因目标尺寸小、外观变化大、背景杂乱以及密集标注数据稀缺导致的轻量模型性能受限和实际部署困难的问题。

Result: 在多种监督设置下的广泛实验表明，所提框架有效，IPS-Seg在分割精度和计算效率之间取得了良好平衡，并持续受益于所提出的伪标签生成策略。

Insight: 创新点在于探索了SAM3作为伪标签生成器的两种监督范式（直接伪监督和基于局部补丁的细化策略），并构建了一个两阶段的SAM3引导伪标签生成框架，这凸显了大规模基础模型作为标注源在低标签视觉领域训练紧凑任务特定分割网络的潜力。

Abstract: Unmanned aerial vehicle (UAV) target segmentation remains challenging due to the small size of objects, appearance variations, cluttered backgrounds, and the scarcity of densely annotated data. These factors hinder the performance and practical deployment of lightweight segmentation models in real-world UAV applications. To address this problem, this paper investigates the use of SAM3 (Segment Anything Model 3) as a pseudo-label generator for training compact segmentation networks. Specifically, two supervision paradigms are explored: (i) direct pseudo-supervision using unaltered SAM3-generated masks, and (ii) a refinement strategy that re-applies SAM3 to localized image patches for improved mask quality. Based on these paradigms, a two-stage SAM3-guided pseudo-label generation framework is proposed. In the first stage, SAM3 generates coarse masks for initial object localization. The localized regions are subsequently cropped into patches and processed by SAM3 again to generate fine masks with accurate object boundaries and discard false positives. The resulting coarse and fine masks are then used as pseudo-labels to optimize a lightweight network, termed IPS-Seg, which consists of three components: an IdentityFormer backbone for feature extraction, an Atrous Spatial Pyramid Pooling module for multi-scale context aggregation, and a PixelShuffle-based decoder for spatial resolution recovery. Extensive experiments under multiple supervision settings demonstrate the effectiveness of the proposed framework. The results show that IPS-Seg achieves a favorable trade-off between segmentation accuracy and computational efficiency while benefiting consistently from the proposed pseudo-label generation strategy. These findings highlight the potential of large-scale foundation models as annotation sources for training compact task-specific segmentation networks in low-label vision domains.

[130] City-Level 3D Surface Reconstruction with Viewpoint Orientation Partitioning and Scene Completion cs.CVPDF

Liang Han, Wenyuan Zhang, Junsheng Zhou, Yu-Shen Liu, Zhizhong Han

TL;DR: 本文提出了一种基于视点方向分区和场景补全的城市级3D表面重建方法。该方法通过将具有相似方向的视点分组，实现了更准确的深度估计和并行GPU上的平衡计算，并修复了初始点云中的缺失区域，从而在大规模场景中实现了高效且高质量的表面重建。

Details

Motivation: 解决基于3D高斯泼溅（3DGS）的大规模重建方法在处理复杂几何、长优化时间和有限内存时，难以在大场景中生成高质量表面的问题。

Result: 在GauU-Scene、MatrixCity和UrbanScene3D数据集上的大量实验表明，该方法在大规模场景表面重建方面优于现有最先进方法。

Insight: 创新点在于基于视点方向的场景分区策略，确保相似方向的视图共同参与以提升深度估计精度，以及检测并修复由稀疏视点或纹理不足导致的初始点云缺失区域的策略，从而提升几何质量并实现高效并行计算。

Abstract: Multi-view 3D surface reconstruction is a longstanding challenge in computer vision. Although recent large-scale reconstruction methods based on 3D Gaussian Splatting (3DGS) achieve impressive novel-view synthesis, producing high-quality surfaces over large scenes remains difficult, due to complex geometry, long optimization, and limited memory. In this paper, we propose a novel yet simple partitioning method to efficiently and faithfully reconstruct large-scale scene surfaces. Our key insight lies in a scene partitioning method based on viewpoint orientation. This partitioning approach ensures that views with similar orientations are jointly involved for more accurate depth estimations, leading to precise surface reconstructions and balanced computation on multiple GPUs in parallel. In addition, we propose a strategy to detect and repair missing regions in the initial point cloud caused by sparse viewpoints or insufficient textures, thereby further improving the geometric quality. Extensive experiments on the GauU-Scene, MatrixCity, and UrbanScene3D datasets demonstrate that our method outperforms the state-of-the-art approaches in surface reconstruction for large-scale scenes. Project page: https://hanl2010.github.io/VOP-GS.

[131] Sparse-View Surface Reconstruction using Gaussian Splatting through High-Confidence Depth Propagation with Normal Priors cs.CVPDF

Liang Han, Bangcai Wei, Junsheng Zhou, Yu-Shen Liu, Zhizhong Han

TL;DR: 本文提出了一种基于3D高斯泼溅（3DGS）的稀疏视角表面重建新方法。该方法的核心是通过引入法线引导的深度传播策略，将高置信度区域的深度信息扩展到低置信度区域，并利用异常深度边缘感知正则化来处理高斯离散性导致的深度不连续问题。在DTU和Tanks-and-Temples数据集上的实验表明，该方法在稀疏视角表面重建任务上优于现有最先进方法。

Details

Motivation: 解决从稀疏视角进行高质量几何表面重建的挑战。现有基于3DGS的方法在稀疏视角新视图合成上表现优异，但由于几何线索有限和高斯离散性，重建高质量表面仍然困难。

Result: 在DTU和Tanks-and-Temples基准数据集上进行了广泛实验，结果表明该方法在稀疏视角表面重建任务上超越了当前最先进（SOTA）方法。

Insight: 主要创新点包括：1）提出法线引导的深度传播方法，利用高置信度区域深度约束低置信度区域；2）设计异常深度边缘感知正则化，缓解高斯离散性导致的深度不连续问题。从客观角度看，该方法将深度传播思想与3DGS框架结合，为解决稀疏视角下的几何重建难题提供了新思路。

Abstract: 3D reconstruction from sparse views is a challenging task in 3D computer vision. Recent studies on 3D Gaussian Splatting (3DGS) have achieved remarkable results with sparse views in novel view synthesis, yet reconstructing high-quality geometric surfaces from sparse views remains a challenge, due to the limited geometry clues and the discreteness of Gaussians. In this paper, we propose a novel 3DGS-based method for high-fidelity surface reconstruction from sparse views. Our key insight is to introduce a normal-guided depth propagation approach, which can extend depth information from high-confidence regions to constrain the depth in low-confidence areas. Additionally, we propose an abnormal depth edge-aware regularization to address depth discontinuities caused by the discreteness of Gaussians. Extensive experiments on DTU and Tanks-and-Temples datasets demonstrate that our method outperforms the state-of-the-art methods in sparse view surface reconstruction. Project page: https://hanl2010.github.io/DP-GS.

[132] Rethinking Depth Pruning for Vision Transformers: A Heterogeneity-Aware Perspective cs.CV | cs.AIPDF

Zhenfeng Su, Kang Zhao, Han Bao, Tao Yuan, Zhongzhe Hu

TL;DR: 本文重新审视了视觉Transformer（ViT）的深度剪枝问题，指出现有方法因忽视层间异质性而失败，并提出了HetDPT方法。该方法通过异质性感知的深度剪枝避免维度不匹配，在ImageNet-1K等多个数据集上验证了其有效性，能显著加速模型且保持精度，结合宽度剪枝后实现了极端的ViT剪枝SOTA。

Details

Motivation: 现有ViT剪枝研究多集中于宽度剪枝以减小模型尺寸，而深度剪枝（移除整个层）虽能带来更高加速，却因难以恢复精度而受限，限制了联合宽度-深度剪枝方法的加速效果。本文旨在解决深度剪枝因忽视层间异质性导致的失败问题。

Result: 在ImageNet-1K、CIFAR-100、COCO和ADE20K上的实验表明，HetDPT对DeiT-B实现了1.58倍加速且保持精度，对DeiT-S实现1.39倍加速且几乎无精度损失。结合宽度剪枝后，HetDPT+在Isomorphic-Pruning-2.6G配置下将加速比从4.24倍提升至5.19倍，保持近无损精度，创下极端ViT剪枝的新SOTA记录。

Insight: 创新点在于揭示了深度剪枝失败的关键在于层间异质性被忽视，并提出了异质性感知的深度剪枝方法HetDPT以避免维度不匹配；从客观角度看，该方法通过针对性处理层差异，有效平衡了剪枝后的加速与精度恢复，为联合剪枝提供了新思路。

Abstract: While prior studies have successfully compressed vision Transformers (ViTs) through various pruning techniques, most have concentrated on width pruning to achieve significant reductions in model size. Depth pruning, which removes entire layers from a ViT, is notoriously difficult for accuracy recovery despite its potential to deliver higher speedups, limiting the acceleration achieved by existing joint width-and-depth pruning methods. In this work, we reveal that the failure of existing depth pruning methods lies in their neglect of heterogeneity between different layers, and we introduce HetDPT, a heterogeneity-aware depth pruning method that avoids dimension mismatch. Comprehensive experiments on ImageNet-1K, CIFAR-100, COCO, and ADE20K validate our method: HetDPT achieves a 1.58$\times$ speedup for DeiT-B while maintaining accuracy and a 1.39$\times$ speedup for DeiT-S with nearly no accuracy degradation. Furthermore, when combined with width pruning, HetDPT+ sets a new state-of-the-art record in extreme ViT pruning, enhancing the acceleration ratio from 4.24$\times$ to 5.19$\times$ for the Isomorphic-Pruning-2.6G configuration while maintaining near-lossless accuracy; our code is available at https://github.com/Efficient-AI-for-All/HetDPT.

[133] G$^2$TAM: Geometry Grounded Track Anything Model cs.CVPDF

Chenming Zhu, Peizhou Cao, Jingli Lin, Wenbo Hu, Yunlong Ran

TL;DR: G^2TAM是一种基于几何的跟踪任意模型，通过结合无序RGB图像或视频，利用前馈3D重建模型提供的空间一致性，实现可提示的实例跟踪。该模型采用空间对齐的几何表示作为隐式记忆，确保跨帧和跨视角的实例身份与定位一致性。核心是一个跨模态空间编码器，将视觉和文本提示整合到共享几何空间中，支持端到端的空间重建和实例一致掩码预测。

Details

Motivation: 人类空间理解依赖于几何和语义的联合感知，而现有视频分割模型依赖显式外观记忆库进行实例跟踪，易受大视角变化和长期遮挡影响。G^2TAM旨在利用现代3D重建模型的空间一致性，提升实例跟踪的鲁棒性。

Result: 在构建的大规模数据集InsTrack上进行实验，G^2TAM在跨视角一致性、可提示实例空间跟踪、视频对象分割和空间重建方面表现强劲，为交互式几何基础空间推理奠定了基础。

Insight: 创新点包括使用几何表示作为隐式记忆替代传统外观记忆库，以及跨模态空间编码器整合视觉和文本提示到共享几何空间，这增强了跟踪的稳定性和可提示性，为3D感知任务提供了新框架。

Abstract: Human spatial understanding arises from jointly perceiving geometry and semantics, enabling consistent object identification and localization across viewpoints and time. Current video segmentation models depend on explicit object appearance memory banks for instance tracking, yet they remain vulnerable to large viewpoint changes and long-term occlusions. Leveraging the spatial consistency afforded by modern feed-forward 3D reconstruction models, we propose the Geometry Grounded Tracking Anything Model (G$^2$TAM), a unified framework for promptable instance tracking in 3D using only unordered RGB images or videos. G$^2$TAM employs spatially aligned geometric representations as implicit memory, ensuring stable instance identity and localization across frames and views. At its core is a cross-modal spatial encoder that integrates visual and textual prompts into a shared geometric space, enabling end-to-end spatial reconstruction and instance-consistent mask prediction. To support training and evaluation, we construct InsTrack, a large-scale dataset with a dedicated validation split for benchmarking. Extensive experiments show that G$^2$TAM delivers strong cross-view consistency, promptable instance spatial tracking, video object segmentation and spatial reconstruction, establishing a foundation for interactive, geometry-grounded spatial reasoning.

[134] InfraNet: Quality-Aware RGB Guidance for Efficient Infrared Object Detection cs.CVPDF

Zichao Feng, Haodong Zhu, Jingying Yang, Sheng Xu, Yangyang Ren

TL;DR: 本文提出了InfraNet，一种以红外（IR）为中心的质量感知框架，用于在恶劣视觉条件下进行高效目标检测。该框架通过不对称架构，利用红外作为主要路径进行特征提取和预测，同时使用RGB作为辅助路径提供可靠性控制的监督信号。核心模块QualGate能够学习任务导向的控制信号，在跨模态训练中抑制不可靠的RGB指导并补偿红外特征，支持灵活的RGB-IR或仅IR部署。

Details

Motivation: 现有基于融合的方法通常平等对待RGB和红外输入，当RGB模态不可靠或不可用时，其鲁棒性会受到影响。因此，需要一种能够根据RGB质量动态调节指导的框架，以提升在低光照和恶劣天气条件下的目标检测性能。

Result: 在LLVIP、FLIR-Aligned、M³FD和DroneVehicle四个基准数据集上的广泛实验表明，InfraNet在具有挑战性的低光照和恶劣天气条件下表现出强大或具有竞争力的准确性。特别是，InfraNet在仅IR推理时保持了高效率，实现了准确性和计算效率的平衡。

Insight: 创新点在于提出了质量感知融合模块QualGate，它能够动态抑制不可靠的RGB指导并补偿红外特征，从而提升模型在RGB模态不可靠时的鲁棒性。此外，框架支持灵活的部署模式（RGB-IR或仅IR），兼顾了准确性和效率，为多模态感知系统提供了可借鉴的设计思路。

Abstract: Robust object detection under adverse visual conditions remains a long-standing challenge for multi-modal perception systems. Existing fusion-based methods typically require both RGB and infrared (IR) inputs, and treat them equally during both training and inference, which compromises their robustness when the RGB modality becomes unreliable or unavailable. In this case, we propose \textbf{InfraNet}, an IR-centric quality-aware framework that regulates RGB guidance during training and supports flexible RGB–IR or IR-only deployment. InfraNet employs an asymmetric architecture where the primary IR pathway extracts multi-scale infrared features for predictions, while the auxiliary RGB pathway provides reliability-controlled supervisory signals. The core of InfraNet is \textbf{QualGate}, a quality-aware fusion module that learns a task-oriented control signal to suppress unreliable RGB guidance and compensate IR features during cross-modal training. Built upon InfraNet, we design two architectural variants: a lightweight IR-only architecture InfraNet-IR and an RGB–IR architecture InfraNet-RGB-IR. Our method is evaluated through extensive experiments on four benchmark datasets (LLVIP, FLIR-Aligned, M$^3$FD, and DroneVehicle), showing strong or competitive accuracy in challenging low-light and adverse weather conditions. Notably, InfraNet maintains high efficiency in IR-only inference, making it both accurate and computationally efficient.

[135] CineMobile: On-Device Image-to-Video Diffusion for Cinematic Camera Motion Generation cs.CV | cs.AIPDF

Xuyao Huang, Zelai Deng, Xu Wang, Xizhong Xiao, Zhijie Deng

TL;DR: 本文提出了CineMobile，一个专为移动设备设计的图像到视频扩散模型，旨在高效生成电影级相机运动效果（如子弹时间、滑动变焦、慢动作等）。该方法通过三步优化策略（蒸馏引导剪枝、扩散蒸馏与强化学习结合的4步生成器优化、混合后训练量化）显著降低了模型的计算开销和存储占用。

Details

Motivation: 解决扩散变换器（DiTs）在移动设备上因参数量大、多步迭代去噪过程导致计算开销高、难以高效生成视频的问题，以满足移动端对电影级图像到视频创作日益增长的需求。

Result: 在Wan 2.1架构的教师模型基础上，CineMobile实现了40倍的生成加速，同时保持可比的视觉质量；在NVIDIA H200 GPU上每步去噪延迟为0.6秒，在MediaTek Dimensity 8400 Ultimate 5G平台上为20秒，可生成49帧480p视频，峰值内存使用1.8 GB，模型大小压缩至1 GB以下。

Insight: 创新点在于结合蒸馏引导剪枝、扩散蒸馏与强化学习优化步数、混合后训练量化的综合轻量化策略，实现了移动端高效图像到视频生成；从客观角度看，该工作为将复杂扩散模型部署到资源受限设备提供了可借鉴的端到端优化框架。

Abstract: The growing demand for image-to-video creation on mobile devices has increasingly focused on cinematic motion effects like bullet time, dolly zoom, slow motion, etc. While Diffusion Transformers (DiTs) exhibit strong performance in video generation, their large parameter sizes and multi-step iterative denoising processes lead to substantial computational overhead, making efficient generation on mobile devices challenging. We propose CineMobile to bridge the gap. In particular, CineMobile adopts a three-fold optimization strategy: (1) leveraging a distillation-guided pruning approach to derive a compact yet efficient model that retains the essential video generation capabilities required for cinematic effects; (2) optimizing the compressed model into a 4-step generator via a combination of diffusion distillation and reinforcement learning; (3) employing a hybrid post-training quantization strategy to compress the model footprint to under 1 GB. Experimental results show that compared to the teacher model with the Wan 2.1 architecture, CineMobile achieves a 40x speedup in generation while maintaining comparable visual quality. Specifically, CineMobile generates 49-frame 480p videos with a per-step denoising latency of 0.6s on an NVIDIA H200 GPU and 20s on the MediaTek Dimensity 8400 Ultimate 5G platform, with a peak memory usage of 1.8 GB, demonstrating its practical applicability for mobile-based image-to-video creation.

[136] TestMate: Test-Time Domain Adaptation Aided by Lightweight Vision Foundation Model cs.CVPDF

Dimitrios Fotiou, Vasileios Mygdalis, Ioannis Pitas

TL;DR: TestMate是一种新颖的、实时的、无需反向传播的测试时域适应框架，用于语义分割任务。它利用轻量级视觉基础模型的泛化能力，通过零样本实例分割模型实时生成多尺度对象及其部件的未标注掩码提议，并采用启发式的大小排序竞争方案将这些提议与主模型融合，从而实现从第一帧开始的即时适应。

Details

Motivation: 解决现有测试时域适应方法在语义分割任务中的关键局限性，包括基于熵最小化的方法需要昂贵的反向传播并可能导致灾难性遗忘，以及基于记忆库的方法适应速度慢且难以处理连续域偏移的问题。

Result: 在两个基准数据集上实现了最先进的结果，证明了其在测试时域适应、源域无关域适应和在线测试时域适应三种不同任务中的有效性。

Insight: 创新点在于利用轻量级视觉基础模型的零样本能力实时生成高质量的掩码提议，并通过一种无参数、启发式的竞争融合机制实现快速、稳定的适应，避免了反向传播和灾难性遗忘，同时能作为独立模块或增强现有方法性能的插件使用。

Abstract: Test-Time Domain Adaptation (TTDA) aims to adapt Deep Neural Networks to distribution shifts using only streaming, unlabeled test data in real time. Current methods for semantic segmentation tasks suffer from critical limitations. Entropy minimization techniques require costly backpropagation, risking catastrophic forgetting and producing noisy segmentation boundaries. Memory-bank methods, while backpropagation-free, exhibit slow adaptation, requiring numerous samples to converge and struggle to handle continuous domain shifts. We introduce TestMate, a novel, real-time, and backpropagation-free TTDA framework that overcomes these issues. TestMate leverages generalization capability of a lightweight Visual Foundation Model to guide the adaptation. We use a zero-shot instance segmentation YOLOv8-seg based model to generate unlabeled mask proposals for objects and their parts at multiple scales in real time. These proposals are fused with the primary model via a heuristic, size-ordered competitive scheme, where small, high-confidence regions dominate and refine predictions in surrounding larger, less certain areas. This paremeter-free mechanism enables immediate adaptation from the first frame, inherently avoids catastrophic forgetting and effectively preserves fine object details and boundaries, even for small objects. TestMate can be used as a standalone, efficient refinement module or seamlessly integrated into existing TTDA methods to significantly boost their performance. We demonstrate state-of-the-art results across two benchmark datasets, proving TestMate’s effectiveness in three distinct adaptation tasks: TTDA, Source-Free Domain Adaptation (SFDA), and online-TTDA. Code is available.

[137] Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection cs.CVPDF

Runzhi Deng, Yundi Hu, Yiming Zhong, Zhao Wang, Xixi Liu

TL;DR: 本文提出了一种无需训练的框架GLLS，用于参考引导的多模态上下文验证，以解决工业异常检测中缺陷小、分辨率有限以及文本标准与视觉证据不匹配的问题。该方法通过全局逻辑流和局部搜索流的双流机制，结合部分感知视觉逻辑图谱和蒙特卡洛树搜索，在保持诊断决策可追溯性的同时，提升了检测性能。

Details

Motivation: 工业异常检测面临缺陷小、输入分辨率受限以及文本标准缺乏视觉证据支撑的挑战，而现有基于优化的方法通常需要大量缺陷样本，这在早期部署中难以获取。

Result: 在MMAD-QA及其他异常检测数据集上的实验表明，GLLS相比匹配和通用基线方法取得了持续的性能提升，同时保持了诊断决策在整个检查过程中对显式视觉证据的可追溯性。

Insight: 创新点在于提出了一种无需训练的双流多模态上下文学习框架，通过全局逻辑流提取可检查的视觉事实，结合局部搜索流在固定预算下选择局部证据裁剪，实现了高效且可解释的工业异常检测。

Abstract: Large Multimodal Models (LMMs) show strong few-shot generalization, but industrial anomaly detection remains difficult because defects are small, input resolution is limited, and textual standards are not always grounded in visual evidence. Recent optimization-based methods improve alignment through fine-tuning, but they often require many defective samples, which are unavailable in early deployment. We present Global Logic and Local Search (GLLS), a training-free framework for reference-guided multimodal in-context verification. GLLS uses a Part-Aware Visual-Logical Atlas to organize normal references and structured specifications in the inference context. It combines a Global & Logic Stream, where SAM 3 extracts partially checkable visual facts, with a Fine-Grained & Actions Stream, where MCTS selects local evidence crops under a fixed budget. Experiments on MMAD-QA and additional anomaly detection datasets show consistent gains over matched and general-purpose baselines, while keeping the final diagnostic decision traceable to explicit visual evidence throughout the inspection trace.

SungHun Kim, SeungJun Baek

TL;DR: 本文提出了Q-TriM模型，用于音频-视觉问答任务。该模型通过一种新颖的、浅层并行的三模态注意力机制，替代了传统的深层堆叠式跨模态融合方法，旨在减少信息损失和误差累积。

Details

Motivation: 现有AVQA系统通常依赖深层堆叠的自注意力和跨模态注意力进行融合，这种顺序堆叠可能导致细微的跨模态线索在层间丢失，并使误差在序列注意力层中累积。

Result: Q-TriM在三个AVQA基准测试上取得了最先进的性能，特别是在MUSIC-AVQA-R数据集上获得了显著提升，证明了其鲁棒性和分布外泛化能力。

Insight: 核心创新点是提出了一个以问题为引导的、查询、键、值分别来自不同模态的三模态注意力框架，并采用单阶段并行融合策略，避免了深层堆叠带来的问题。

Abstract: Audio-Visual Question Answering (AVQA) extends classical VQA by requiring joint reasoning over video and synchronized audio. However, many AVQA systems rely on deeply stacked layers of self- and cross attention across text, video, and audio. Such sequential stacking may incur loss of information such as subtle inter-modal cues over the layers, causing errors to accumulate across sequential attention layers during the fusion. We introduce Q-TriM which performs multi-modal fusion in a shallow and parallel manner instead of a deep and sequential manner. For Q-TriM, we propose a novel framework for attention operation incorporating video and audio conditioned on text. As a result, we obtain not only standard cross attention outputs but also Tri-Modal Attention representations in which Query, Key, and Value come from distinct modalities. These attention representations are combined in parallel at a single stage, thus avoiding the multi-modal fusion with deep stacks in order to mitigate error accumulation and depth-induced issues. Q-TriM achieves state-of-the-art performance on three AVQA benchmarks, including substantial gains on MUSIC-AVQA-R, which demonstrates its robustness and out-of-distribution generalization. Code is available at https://github.com/Sunghun95/Q-TriM

[139] FDR-Occ: Factorized Dense Routing for Full-Spectrum 3D Occupancy Prediction cs.CVPDF

Dubing Chen, Huan Zheng, Tianyi Yan, Yucheng Zhou, Runzhou Tao

TL;DR: 本文提出了一种名为因子化密集路由（FDR）的新方法，用于解决基于视觉的3D占据预测任务中传统显式物理投影方法存在的‘局部性瓶颈’问题。该方法将视图变换抽象为无约束的二部图路由，通过分层张量收缩实现密集的2D到3D混合，保证了全局感受野。同时，作者提出了一种分辨率-上下文解耦架构，将3D空间分解为全局宏观拓扑锚点和局部精确几何平面，使全局语义推理与精确表面定位互补。

Details

Motivation: 当前基于显式物理投影的3D占据预测方法将路由矩阵限制在稀疏的相机光线上，导致了严重的‘局部性瓶颈’，限制了网络构建整体上下文理解的能力，并且在相机外参不可靠或缺失时性能急剧下降。

Result: 在Occ3D-nuScenes和Occ3D-Waymo基准测试上，该方法取得了最先进的性能。特别是在外参未标定的设置下，其全局路由机制内化了隐式的多相机拓扑结构，相比基于物理投影的基线方法展现出显著更强的结构鲁棒性。

Insight: 核心创新点在于将视图变换抽象为无约束的密集路由问题，并提出了因子化密集路由（FDR）方法以及分辨率-上下文解耦架构。这突破了传统方法对稀疏物理投影的依赖，实现了全局感受野与可控计算复杂度的平衡，为解决外参不可靠场景下的3D感知提供了新思路。

Abstract: Vision-based 3D occupancy prediction fundamentally relies on the 2D-to-3D view transformation. Current paradigms predominantly utilize explicit physical projection, which artificially restricts the routing matrix to strict, sparse camera rays. While computationally efficient, this imposes a severe Locality Bottleneck, preventing the network from constructing holistic contextual understanding and degrading sharply when camera extrinsics are unreliable or absent. To break this bottleneck, we abstract view transformation as unconstrained bipartite routing and propose Factorized Dense Routing (FDR). By approximating dense 2D-to-3D mixing through hierarchical tensor contractions, FDR guarantees a fully-global receptive field with tractable, sub-quadratic complexity. Crucially, the mandatory spatial contraction in dense routing exposes a fundamental Resolution-Context Trade-off. To address this, we introduce a Resolution-Context Decoupled Architecture. We factorize the 3D space into a global macroscopic topological anchor (via FDR) and precise local geometric planes (via explicit projection). This decoupling enables global semantic inference and exact surface localization to complement each other without mutual compromise. Extensive experiments demonstrate that our framework achieves state-of-the-art performance on the Occ3D-nuScenes and Occ3D-Waymo benchmarks. More notably, in an uncalibrated setting where physical extrinsics are withheld, our global routing internalizes the implicit multi-camera rig topology and exhibits substantially stronger structural robustness than physical-projection baselines under the same protocol.

Octavian Gîngu, Stelian Spînu

TL;DR: 本文提出了一种基于单目视觉的完整自主室内导航框架，专为低成本微型无人机设计，用于搜索与救援任务。该系统结合了Depth Anything V2进行单目深度估计、经典计算机视觉和轻量级深度学习模型，实现了走廊探索（包括自动门检测、房间进入、OCR房间识别和受害者检查）以及基于TRISTAR三重信号融合方法的自主楼梯攀爬。

Details

Motivation: 室内搜救任务需要快速态势感知，但GPS信号不可用，且人类进入困难或危险。现有自主空中系统多依赖激光雷达、立体视觉或专用深度相机，增加了硬件复杂性和部署成本。本文旨在为低成本无人机提供一个仅依赖单目视觉的完整自主室内导航解决方案。

Result: 在真实大学建筑内的飞行评估显示，深度校准将相对深度误差从27.4%降至10%以下；门检测算法精度达0.93，F1分数为0.91。消融研究表明，多传感器融合显著提升了楼梯识别的鲁棒性。结果证明，无需专用测距硬件，在资源受限平台上即可实现可靠的室内探索与楼梯穿越。

Insight: 主要创新点在于提出了TRISTAR三重信号融合方法，将结构线索（Sobel滤波）、纹理分析（多尺度Gabor滤波）和来自单目深度估计的几何深度信息相结合，用于鲁棒的楼梯识别。整个框架展示了如何仅通过单目摄像头，结合轻量级深度学习与传统视觉方法，实现复杂的自主导航任务，为低成本、快速部署的搜救无人机提供了实用方案。

Abstract: Indoor search-and-rescue (SAR) operations often require rapid situational awareness where GNSS signals are unavailable and human access is difficult or hazardous. While most autonomous aerial systems rely on LiDAR, stereo vision, or specialized depth cameras, such solutions increase both hardware complexity and deployment costs. This paper presents a complete autonomous indoor navigation framework for low-cost unmanned aerial vehicles based exclusively on monocular vision. Implemented on a DJI Tello platform, the system combines monocular depth estimation using Depth Anything V2 with classical computer vision and lightweight deep learning models for scene understanding, victim detection, and hazard recognition. The framework consists of two independent behaviors: (i) corridor exploration with automatic door detection, room entry, OCR-based room identification, and victim inspection; and (ii) autonomous stair ascent based on TRISTAR (TRI-Signal STair Ascent Recognition), a novel triple-sensor fusion method that integrates structural cues (Sobel filtering), texture analysis (multi-scale Gabor filtering), and geometric depth from monocular depth estimation. Evaluation used real indoor flights in a university building. Depth calibration reduced relative depth error from 27.4% to below 10%, while the door detection algorithm reached a precision of 0.93 and an F1-score of 0.91. A dedicated ablation study shows that multi-sensor fusion significantly improves stair-recognition robustness compared to individual sensing modalities, and a failure-case analysis delineates the limits of monocular perception under challenging lighting and reflective surfaces. The results demonstrate that reliable indoor exploration and stair traversal are achievable on resource-constrained platforms without specialized ranging hardware, a practical, cost-effective solution for rapid SAR deployment.

[141] How Do Diffusion Classifiers Decide? A Bias-Centric Evaluation cs.CV | cs.AIPDF

Saba Fathi, Fardin Ayar, Maryam Abdolali, Ehsan Javanmardi, Manabu Tsukada

TL;DR: 本文研究了扩散分类器的决策机制，通过构建ASOB-Bench评估框架，从属性绑定、尺寸顺序偏差和背景依赖性三个维度分析了扩散分类器的偏见特性。研究发现扩散分类器在属性误绑定方面优于OpenCLIP基线，但在尺寸顺序捷径和背景依赖方面表现出更强的敏感性，其偏见模式与判别式视觉语言模型存在显著差异。

Details

Motivation: 扩散模型被重新用于零样本分类任务，但其分类决策机制尚不明确，本文旨在通过系统性偏见评估来揭示扩散分类器的决策行为。

Result: 在新建数据集上，扩散分类器比OpenCLIP基线更少出现属性误绑定；在ComCo基准测试中对尺寸顺序捷径更敏感；在ImageNet-B上准确率大幅下降，显示对背景线索的严重依赖。

Insight: 通过重构误差热图和U-Net交叉注意力可视化揭示了每种偏见背后的机制，这些单次诊断方法也可用于揭示文本到图像生成模型的类似故障模式，为构建更鲁棒的扩散模型提供指导。

Abstract: Diffusion models have recently been repurposed for zero-shot classification, giving rise to diffusion classifiers that identify the best-matching text prompt by minimizing the noise-prediction error. Despite their growing adoption, how these models make classification decisions remains poorly understood. We introduce ASOB-Bench, a bias evaluation for diffusion classifiers along three dimensions: Attribute binding, Size-Order bias, and Background dependency. These dimensions serve not as an exhaustive taxonomy but as targeted probes of how the text-conditioned reconstruction-error score reaches a decision. Such a perspective is well studied for discriminative vision-language models, yet remains overlooked for diffusion classifiers. Extending an existing framework with five new attribute categories on newly constructed datasets, we find diffusion classifiers are less prone to attribute misbinding than an OpenCLIP baseline; on the established ComCo benchmark they are substantially more susceptible to size-order shortcuts; and on ImageNet-B they suffer far larger accuracy drops, revealing heavy reliance on background over foreground cues. Reconstruction-error heatmaps and U-Net cross-attention visualizations expose the mechanism behind each bias. Because diffusion classifiers share the same denoiser as text-to-image models, these single-pass diagnostics also point toward analogous failure modes in generation. Overall, diffusion classifiers exhibit a distinct bias profile from vision-language models, offering guidance for building more robust diffusion-based models.

Gopi Raju Matta, Reddypalli Trisha, Vemunuri Divya Madhuri, Kaushik Mitra

TL;DR: PRISM3D是一个从极端运动模糊图像中进行盲3D场景重建的统一框架。它通过鲁棒的初始化策略和概率物理耦合优化来解决传统SfM方法失效的问题，并进一步提出了融合事件流的扩展版本PRISM3D-E，在自建基准和广泛实验中达到了新的SOTA性能。

Details

Motivation: 解决从极端运动模糊图像中进行盲3D重建的逆问题，传统SfM流程在此场景下完全失效，而现有方法通常依赖不切实际的清晰图像监督，因此需要一种能从严重退化输入直接进行鲁棒重建的框架。

Result: 在自建的PRISM3D-E基准和广泛实验中，无论是独立的RGB框架还是其多模态扩展版本，都建立了新的最先进（SOTA）性能。

Insight: 创新点包括：1) 首次有效利用深度密集跟踪方法（VGGSfM）从极端模糊中引导3D高斯溅射的鲁棒初始化策略；2) 通过马尔可夫链蒙特卡洛（MCMC）进行概率几何致密化与连续贝塞尔轨迹物理成像建模的耦合优化；3) 无缝集成高时间分辨率事件流作为结构先验的多模态扩展框架。

Abstract: We address the inverse problem of blind 3D scene reconstruction from extremely motion-blurred images, a scenario where traditional Structure-from-Motion (SfM) pipelines fail. Existing approaches typically circumvent this bottleneck by relying on impractical sharp-image supervision. In this work, we introduce PRISM3D, a unified framework enabling robust reconstruction directly from severely degraded inputs. To overcome the lack of a reliable starting point, we propose a Robust Initialization strategy utilizing deep dense tracking method (VGGSfM) to recover global topology where feature matching fails. To the best of our knowledge, we are the first to effectively leverage this paradigm to bootstrap 3D Gaussian Splatting from extreme motion blur. However, while robust, this initialization yields sparse and noisy geometry that causes deterministic optimization to diverge. To resolve this, we propose a coupled solution driven by probability and physics: we adopt a probabilistic formulation for geometric densification via Markov Chain Monte Carlo (MCMC) to robustly populate the sparse priors, while simultaneously modeling physical image formation via continuous Bezier Trajectories. Furthermore, while PRISM3D establishes a highly robust standalone pipeline, the availability of complementary event streams offers an opportunity to push the reconstruction fidelity further. To exploit this, we introduce PRISM3D-E, a multi-modal (RGB + Events) extension that seamlessly integrates high-temporal-resolution events as structural priors to maximize geometric recovery. Because existing datasets lack paired event streams under such severe degradation, we concurrently contribute the PRISM3D-E Benchmark to facilitate rigorous evaluation. Extensive experiments demonstrate that both our standalone RGB framework and its multi-modal extension establish new state-of-the-art performance.

[143] BAT3R: Bootstrapping Articulated 3D Reconstruction from 2D Image Collections cs.CVPDF

Jakub Zadrozny, Oisin Mac Aodha, Hakan Bilen

TL;DR: 本文提出BAT3R框架，用于从2D图像集合中自举学习铰接式3D重建。该方法仅需每个类别一个带骨骼的规范网格和未标注的2D图像，通过迭代优化预测的点云图与网格的拟合，逐步生成合成训练数据以提升预测器性能，从而减少对人工标注的铰接3D数据集的依赖。

Details

Motivation: 解决从单张图像重建铰接式3D物体的挑战，因为配对的图像-3D监督数据难以获取，且现有方法依赖人工创建的铰接3D资产和精心策划的姿态分布，成本高昂。

Result: 在性能上，该方法仅使用显著更弱的3D监督，达到了与需要手动策划铰接训练数据集的DualPM相当的水平。

Insight: 创新点在于通过自举学习框架，利用未标注2D图像和单个规范网格迭代生成合成数据，减少了对大规模标注铰接3D数据的需求，为数据高效3D重建提供了新思路。

Abstract: 3D reconstruction of articulated objects from a single image is challenging because large training datasets with paired image and 3D supervision are difficult to obtain. Recent point map-based methods achieve strong performance but rely on synthetic datasets rendered from manually created articulated 3D assets with carefully curated pose distributions. While camera viewpoints can be easily sampled, generating realistic object articulations remains costly and labor-intensive. We propose a training framework that reduces this requirement by leveraging unannotated 2D images collections with only a single rigged canonical mesh per category. Starting from a weak 3D shape predictor trained on canonical-pose renders, we iteratively estimate object articulation and camera pose by fitting the mesh to predicted point maps. The recovered articulations and viewpoints are then used to render updated synthetic training data, progressively improving the predictor. Despite using substantially weaker 3D supervision, our models achieve performance comparable with DualPM, which requires manually curated articulated training datasets.

[144] GeoSelect: Spatial-Program Execution for Training-Free Referring Remote Sensing Image Segmentation cs.CV | cs.AIPDF

Yuhang Jiang, Guohui Deng, Miaozhong Xu, Chao Ruan, Jinling Zhao

TL;DR: 本文提出GeoSelect，一种无需训练的方法，用于遥感图像指代分割任务。该方法将自然语言描述重新定义为空间程序的执行过程，通过冻结的纯文本语言模型将描述合成为领域特定语言，并利用确定性执行器运行程序，从而实现对空间、比较和序数关系的显式控制。

Details

Motivation: 现有无需训练的方法通过隐式的视觉-语言激活或区域-文本相似性来处理指代描述，难以有效控制遥感图像中占主导地位的空间、比较和序数关系（如“最大的船”或“从左数第二个球场”）。

Result: 在RRSIS-D测试集上达到58.86 mIoU，在RISBench测试集上达到55.27 mIoU，这比之前最佳的无需训练方法在RRSIS-D上的性能提高了一倍以上，且无需指代监督，仅需单个GPU。

Insight: 创新点在于将指代分割重构为空间程序的显式执行，引入了可组合的候选集类型和几何场等抽象，使中间程序、场和排名均可检查，并通过可靠性阶梯机制确保任何表达式都能返回答案，提升了可解释性和鲁棒性。

Abstract: Referring remote sensing image segmentation isolates the object named by a natural-language expression in an aerial image. Existing training-free methods resolve the expression through implicit vision-language activations or region-text similarity, which gives weak control over the spatial, comparative, and ordinal relations that dominate aerial referring: they cannot represent constructions such as the largest ship or the second court from the left. We propose GeoSelect, a training-free pipeline that reframes referring as the execution of a typed spatial program. A frozen, text-only language model synthesises the expression into a small domain-specific language, a well-formedness checker accepts the program, and a deterministic executor runs it. The central abstraction is a single scored candidate set type under which every operator composes: continuous geometric fields realise position and proximity as dense pixel-level maps, while discrete set and order operators add the extremum, ordinal, counted-union, and relational constructions that fields alone cannot express. Because execution is explicit, every intermediate program, field, and ranking is inspectable, and a reliability ladder degrades any failing program to a field-only special case, so every expression returns an answer. GeoSelect attains 58.86 mIoU on RRSIS-D test and 55.27 mIoU on RISBench test, more than twice the best prior training-free method on RRSIS-D, with no referring supervision and on a single GPU. A controlled comparison with candidates and segmenter fixed attributes the gain to explicit execution, not the backbone; an oracle decomposition localises the residual gap to detection recall on RRSIS-D and selection on RISBench, and an exposure audit confirms robustness to pretraining leakage. Code will be released upon acceptance at the project page https://avalon-s.github.io/GeoSelect/.

[145] SGF-CDNet: A Consistency-Discrepancy Graph Network over Semantic-Geometric Fused Nodes for Face Forgery Detection cs.CVPDF

Jiayao Jiang, Bin Liu, Nenghai Yu

TL;DR: 本文提出了一种名为SGF-CDNet的新型人脸伪造检测方法，该方法通过构建语义-几何融合节点，并利用一致性-差异图神经网络进行双路径关系推理，以捕捉面部区域间的细微不和谐关系，从而有效检测深度伪造人脸。

Details

Motivation: 针对深度伪造技术快速发展带来的挑战，现有伪造人脸可能缺乏明显伪影，但不同面部区域间常存在细微的不协调，本文旨在通过建模这种不和谐关系来提升检测鲁棒性。

Result: 在多个公开数据集上的大量实验表明，SGF-CDNet取得了优越的性能，确立了其作为人脸伪造检测可靠解决方案的地位。

Insight: 创新点在于深度融合面部解析的语义区域与面部关键点的几何信息来构建节点，并设计双路径图网络分别评估面部组件的自然生物模式一致性以及挖掘伪造引入的结构冲突，这种语义-几何融合与一致性-差异双路径推理的结合是核心贡献。

Abstract: The rapid advancement of deepfakes necessitates robust face forgery detection. Although forged faces may lack obvious artifacts, they often contain subtle disharmony among different facial regions. We propose SGF-CDNet, a Consistency-Discrepancy Graph Network (CD-GNN) over Semantic-Geometric Fused (SGF) nodes. First, SGF-CDNet constructs SGF nodes by deeply fusing semantic regions from face parsing with geometric information from facial landmarks, allowing nodes to capture both high-level concepts and precise geometric constraints. Next, a dual-path CD-GNN performs parallel relational reasoning on these nodes across two dimensions: consistency and discrepancy. The consistency path evaluates if facial components follow natural biological patterns, while the discrepancy path mines for structural tensions and feature conflicts introduced by forgeries. By integrating these processes, our model effectively identifies disharmonious relationships between facial components. Extensive experiments on public datasets demonstrate that SGF-CDNet achieves superior performance, establishing it as a reliable solution for face forgery detection.

[146] NavEYE: Vision-Centered Multi-Sensor Fusion-Based Situational Awareness System for Intelligent Surface Vehicles cs.CVPDF

Ryan Wen Liua, Junxiong Lianga, Haoyu Wanga, Mengwei Baoa

TL;DR: 本文提出了一个名为NavEYE的以视觉为中心的多传感器融合态势感知系统，用于智能水面载具。该系统融合了自动识别系统、雷达和RGB相机数据，通过多约束门控数据关联和自适应加权融合等方法，提升了复杂导航环境下对船舶目标的感知准确性和鲁棒性，旨在增强态势感知并降低碰撞风险。

Details

Motivation: 智能水面载具的智能性、可靠性和安全性高度依赖于复杂导航环境下的态势感知能力，现有单一传感器存在局限性，需要融合互补传感器以实现高质量感知。

Result: 在由AIS、雷达和相机采集的真实世界感知数据集上进行的大量实验证明了该方法的可行性、鲁棒性和有效性，定量和定性评估均显示出优越性能。

Insight: 创新点在于提出了多约束门控数据关联方法以匹配不同时间分辨率的AIS与雷达数据，并设计了距离感知自适应加权融合和基于时效性衰减的拼接融合策略来处理数据丢失问题，最终通过归一化方位和距离特征的联合约束实现多模态数据的关联与融合。

Abstract: With the rapid development of sensor and artificial intelligence (AI) technologies, intelligent surface vehicles (ISVs) have gained increasing attention from academia and industry. Their intelligence, reliability, and safety depend heavily on situational awareness in complex navigational environments. To achieve high-quality perception, we develop a vision-centered multi-sensor fusion system, named NavEYE, by exploiting complementary sensors, including the automatic identification system (AIS), radar, and RGB camera. Specifically, we first propose a multi-constrained gated data association method (MCGA) to accurately match low-temporal-resolution AIS data with high-temporal-resolution radar data. Their fusion result is then obtained by selectively implementing distance-aware adaptively weighted fusion (DAWF) and timeliness decay-based stitching fusion (TDSF), which reduce the uncertainty caused by AIS or radar data loss in real-world sensing scenarios. Based on accurate and robust visual object detection, we further associate and fuse AIS, radar, and visual data through joint constraints of normalized bearing and distance features. According to the fusion results, comprehensive information related to ships of interest can be automatically obtained, helping enhance situational awareness and reduce collision risk for ISVs. The feasibility, robustness, usability, and effectiveness of the proposed multi-sensor fusion method and situational awareness system are demonstrated through extensive experiments on a real-world sensing dataset collected from AIS, radar, and camera. The experimental results show the superior performance of our fusion method in both quantitative and qualitative evaluations. In addition, the shipboard NavEYE system can promote navigational safety for ISVs in complex and dynamic environments.

[147] USE: A Unified Self-Ensembling Framework for Test-Time Prompt Tuning cs.CV | cs.LGPDF

Siru Jiang, Jian Liang, Ran He, Tieniu Tan

TL;DR: 本文提出了一种统一的自集成框架（USE），用于改进基于CLIP的测试时提示调优（TPT）方法。该框架通过在优化和推理阶段都采用自集成策略，自适应地强调测试图像本身而非其增强视图，以获得更可靠的伪标签，从而提升模型在下游任务上的性能。

Details

Motivation: 现有TPT方法通过多视图增强优化文本提示，但其优化过程可解释为隐式地从自生成的伪标签中学习，这可能导致优化与推理阶段的目标不一致。本文旨在通过统一框架确保两个阶段的一致性，并提高伪标签的可靠性。

Result: 在多个数据集上的广泛实验表明，所提出的自集成（SE）策略和USE框架均优于现有方法，其中SE作为轻量级无优化TTA方法也能带来一致的性能提升，并且与现有TTA方法结合时能进一步提高效果。

Insight: 创新点在于将TPT重新解释为伪标签学习，并引入统一的自集成框架来对齐优化与推理目标，其中自集成策略自适应地加权测试图像与增强视图，以生成更可靠的伪标签，同时该策略本身可作为独立的轻量级TTA方法使用。

Abstract: Test-time adaptation (TTA) has emerged as a popular paradigm for improving the performance of vision-language models (e.g., CLIP) on downstream tasks. Among existing CLIP-based TTA methods, Test-Time Prompt Tuning (TPT) is a pioneering work that optimizes textual prompts using multiple test-time augmentations and remains a strong baseline to date. In this work, we revisit TPT and reveal that its optimization can be interpreted as implicitly learning from self-generated pseudo labels. Building on this perspective, we propose a unified self-ensembling framework (USE) that ensures consistency between the optimization and inference stages. During optimization, we introduce a simple yet effective self-ensembling (SE) strategy that emphasizes the test image itself over its augmented views adaptively to obtain more reliable pseudo labels. To fully exploit the potential of augmentations, we further apply the same strategy at inference time, unifying the objectives of both stages. Notably, SE can also act as a lightweight optimization-free TTA method. Extensive experiments across multiple datasets demonstrate that SE and USE outperform their counterparts, respectively. Furthermore, SE yields consistent performance gains when integrated with existing TTA methods. The code is available at https://github.com/sirujiang/USE.

[148] EgoInertia-MI: A Multimodal Egocentric Vision and IMU Benchmark for Motor Impairment Assessment cs.CVPDF

Fatemah Alhamdoosh, Pietro Pala, Abduallah Mohamed, DK Arvind

TL;DR: 本文介绍了EgoInertia-MI，一个结合了同步的第一人称视角视频和可穿戴IMU信号的多模态基准数据集，用于运动障碍分析。该数据集包含健康志愿者模拟不同严重程度运动障碍（无、轻度、重度）时执行的19种上下肢活动。研究建立了动作识别和运动障碍严重程度估计两个基准任务，并评估了多种单模态和多模态基线方法。

Details

Motivation: 传统临床评估通常是间歇性的，可能无法捕捉运动行为的细微时间变化。虽然可穿戴IMU和第三人称视频在客观运动评估方面显示出潜力，但第三人称记录存在隐私问题且需要受限的采集设置。第一人称视角则提供了一种更自然且注重隐私的替代方案。

Result: 实验结果表明，第一人称视角视频为运动障碍评估提供了强有力的线索，而多模态融合实现了最佳整体性能，在严重程度估计上达到0.78 Macro-F1，在动作识别上达到0.93 Macro-F1。

Insight: 创新点在于提出了首个结合同步第一人称视频和IMU的多模态基准数据集EgoInertia-MI，用于模拟运动障碍的评估。这凸显了结合第一人称视觉和可穿戴传感技术，在生态效度和隐私保护方面进行运动评估的潜力。

Abstract: Motor impairments, including tremor, bradykinesia, gait abnormalities, and postural instability, are common across many neurological and movement-related conditions. Conventional clinical assessments are often intermittent and may fail to capture subtle temporal variations in motor behavior. While wearable IMUs and third-person video have shown promise for objective motor assessment, third-person recordings raise privacy concerns and require constrained acquisition setups. In contrast, egocentric vision provides a more naturalistic and privacyaware alternative. In this work, we introduce EgoInertia-MI, a multimodal benchmark dataset combining synchronized egocentric video and wearable IMU signals for motor impairment analysis. The dataset contains 19 upper- and lower-body activities performed by healthy volunteers simulating varying levels of motor impairment severity levels: no impairment, mild impairment, and severe impairment. We establish two benchmark tasks: action recognition and motor impairment severity estimation, and evaluate multiple unimodal and multimodal baselines. Experimental results show that egocentric video provides strong cues for motor impairment assessment, while multimodal fusion achieves the best overall performance, reaching 0.78 Macro-F1 for severity estimation and 0.93 Macro-F1 for action recognition. These findings highlight the potential of combining egocentric vision and wearable sensing for ecologically valid and privacy-aware motor assessment. Code and data are available at:https://fatemah-alh.github.io/EgoInertia-MI-Page/.

[149] Reward Lightning: Fast Video Generation via Homologous Preference Distillation cs.CVPDF

Jiaxiang Cheng, Bing Ma, Xuhua Ren, Kai Yu, Peng Zhang

TL;DR: 本文提出了Reward Lightning框架，通过同源偏好蒸馏（HPD）在视频扩散模型中同时实现偏好对齐和蒸馏加速。核心是引入潜在奖励模型（LRM）直接在潜在空间评估视频质量，并利用同源表示避免梯度冲突，从而在1-4步内生成高质量视频。

Details

Motivation: 现有方法在视频扩散模型中优化偏好对齐和蒸馏加速时，因表示空间不匹配而导致目标冲突，一个目标的提升常损害另一个。本文旨在解决这一矛盾，实现两者的统一优化。

Result: 实验表明，LRM在偏好准确性上超越像素级和潜在级基线11.0%和14.7%；Reward Lightning在仅1-4步内生成高保真视频，在VBench基准上平均得分提升2.1%，并在文本对齐、运动质量和视觉质量上领先。

Insight: 创新点包括：提出同源原则，在共享潜在表示中联合优化对齐与加速，避免梯度冲突；引入LRM直接在潜在空间评分，省去解码开销；HPD方法重用共享主干进行对抗蒸馏和偏好对齐，实现快速且对齐良好的生成。

Abstract: Achieving simultaneous preference alignment and distillation acceleration in video diffusion models remains an open challenge. Existing methods optimize the two objectives over mismatched representation spaces, where improving one objective often compromises the other. To overcome this, we propose Reward Lightning, a unified framework that aligns and accelerates a video diffusion model within a single shared representation. Its central principle is homology: both objectives are evaluated on identical latent features, which mitigates the gradient conflicts that arise when they are optimized over disjoint representations. As a foundational component, we first introduce a latent reward model (LRM) that scores videos directly in the latent space, without decoding back to the pixel space. Building on the LRM, homologous preference distillation (HPD) reuses this shared backbone to perform adversarial distillation and preference alignment jointly, yielding few-step generators that remain faithful and well aligned. Extensive experiments demonstrate that the LRM surpasses pixel-level and latent-level reward baselines by $11.0%$ and $14.7%$ in preference accuracy, and that Reward Lightning generates high-fidelity videos in merely $1$ to $4$ steps, improving the average VBench score by $2.1%$ while leading in text alignment, motion quality, and visual quality. Project page: https://reward-lightning.github.io.

[150] DS-SAC: Density Search for Sample Consensus cs.CVPDF

Suraj Thapa, Muhammad Aminul Islam

TL;DR: 本文提出了一种名为DS-SAC（密度搜索样本一致性）的确定性鲁棒估计框架，用于解决计算机视觉中的几何模型估计问题。该方法通过搜索密集区域来避免传统RANSAC及其变体依赖的随机最小采样，并采用局部探索和递归分区策略来高效寻找高一致性模型。

Details

Motivation: 传统RANSAC及其变体依赖随机最小采样，可能导致效率低下和结果不稳定；本文旨在开发一种确定性方法，通过密度搜索来更可靠地估计几何模型。

Result: 在大规模真实数据集上进行单应性矩阵、基础矩阵和本质矩阵估计的实验表明，DS-SAC在AUC分数上更高，中位姿态误差具有竞争力或更低，且运行时间快于RANSAC、MAGSAC、LO-RANSAC和GC-RANSAC等广泛使用的鲁棒估计器。

Insight: 创新点在于将随机采样替换为确定性密度搜索，结合局部探索和基于符号残差的递归分区，实现了多项式复杂度的高效全局探索；这为鲁棒估计提供了一种更稳定、可预测的替代方案。

Abstract: Robust geometric model estimation is a fundamental problem in computer vision. RANSAC and its variants remain widely used for this task; however, they rely on stochastic minimal sampling. In this article, we propose Density Search Sample Consensus (DS-SAC), a deterministic robust estimation framework, that avoids repeated random sampling by searching dense regions. Starting from an initial model estimated from the available points, the method performs local exploration via forward and backward search. To facilitate global exploration, DS-SAC recursively partitions the point set using signed residuals and searches each valid partition for high-consensus models. We show that DS-SAC has polynomial complexity with respect to the number of points, making it an efficient alternative to stochastic consensus-based methods. Experiments on large-scale real-world datasets for homography, fundamental matrix, and essential matrix estimation show that DS-SAC achieves higher AUC scores, competitive or lower median pose errors, and faster runtime compared with widely used robust estimators, including RANSAC, MAGSAC, LO-RANSAC, and GC-RANSAC.

[151] SAGE: Synchronized Action-Gaze Recognition and Anticipation for Human Behavior Understanding cs.CVPDF

Chenyi Kuang, Nakul Agarwal

TL;DR: 本文提出了一种名为SAGE的统一框架，用于同步识别和预测人类-物体交互（HOI）与人类注视行为。该框架基于Transformer架构，将注视数据整合到时空注意力机制中，以端到端的方式同时处理当前和未来的动作与注视。论文在VidHOI、EGTEA Gaze+和自建Exo-Cook三个基准数据集上验证了其有效性。

Details

Motivation: 现有模型通常将注视模式与人类动作分开处理，忽略了它们之间的相互依赖关系以及统一解决方案的优势。本文旨在通过一个统一的模型来捕捉和利用这种双向关联，以更深入地理解人类行为。

Result: 在VidHOI、EGTEA Gaze+和Exo-Cook三个基准数据集上的实验表明，该联合建模方法取得了持续强劲的性能，通常超过了为单个任务定制的先进模型（SOTA）。

Insight: 主要创新点在于提出了一个统一的端到端框架，将动作与注视的同步识别与预测整合到一个模型中，并利用时空注意力机制融合注视数据。此外，为缺乏数据的第三人称（exocentric）视频场景建立了新的基准数据集Exo-Cook，推动了该领域的研究。

Abstract: Human object interaction (HOI), gaze pattern, and their anticipation are intricately linked, providing valuable insights into cognitive processes, intentions, and behavior. However, most existing models handle gaze and actions separately, missing both their interdependence and the advantages of a unified solution. This paper presents a novel unified framework, SAGE (Synchronized Action-GazE), which integrates simultaneous recognition and anticipation of both HOI and human gaze into a single unified end-to-end trainable model. Our approach leverages a transformer-based architecture and incorporates gaze data into spatiotemporal attention mechanisms to simultaneously predict current and future human actions and gaze behavior. We explore this bidirectional relationship between gaze and actions under different scenarios, whether requiring a close-up, detailed view (egocentric) or a wider, more contextual view (exocentric), making our framework versatile for various applications. Additionally, due to lack of datasets for comprehensive analysis of both HOI and gaze in exocentric videos, we establish a new benchmark Exo-Cook to facilitate further research in this domain. Experiments on three benchmark datasets: VidHOI, EGTEA Gaze+, and Exo-Cook show that jointly modeling gaze and actions across current and future frames achieves consistently strong results, often surpassing specialized state-of-the-art models tailored to individual tasks. By unifying actions and attention in a comprehensive way, our work lays the groundwork for more intuitive human-machine interaction.

[152] Full Glyph Images Beat Token Embeddings: A Controlled Study for Transformers cs.CV | cs.AIPDF

Shuyang Xiang, Hao Guan

TL;DR: 这篇论文挑战了语言模型中基于离散词嵌入的文本表示范式，特别针对中文，提出使用字符序列的完整字形图像作为输入，由视觉编码器处理。通过构建一个严格控制的双分支框架，作者发现基于视觉的模型在多个解码器骨干上均优于基于索引的基线模型，收敛更快且对字符损坏更鲁棒。

Details

Motivation: 动机是质疑当前语言模型普遍采用的离散词嵌入表示假设，尤其是对于中文这种表意文字，探索是否可以用视觉图像直接表示文本以获得更好的性能。

Result: 在严格控制变量的实验中，基于视觉的模型在所有测试的解码器骨干上均优于基线，最高准确率达到0.429（基线为0.355），相对提升21%，且收敛所需训练轮数减半。在字符损坏情况下，损坏的视觉模型性能与干净的基线相当。

Insight: 创新点在于用完整的字形图像完全替代离散词嵌入作为Transformer的输入，并构建了严格的对照实验框架来隔离输入表征的影响。研究发现，这种视觉方法的优势依赖于空间连贯的输入和具有2D位置编码的ViT编码器，且主要适用于中文等具有均匀视觉密度和部首结构的文字系统，表明Transformer对模态的假设比通常认为的更灵活，离散词化并非中文语言建模的根本要求。

Abstract: Modern language models generally represent text as sequences of discrete token embeddings, an assumption deeply rooted in current practice but rarely questioned. We challenge this representation, especially for Chinese, by replacing index-based token embeddings entirely with a single rasterized image of the character sequence, processed by a vision encoder composed of a shared ResNet and a shallow Vision Transformer. To isolate the role of input representation, we construct a dual-branch controlled framework in which both a Vision-based model and an index-based baseline share an identical decoder backbone, training objective, optimizer, and data curriculum. Any performance difference is therefore attributable to the input modality only. Across all tested decoder backbones, the Vision-based model consistently outperforms the baseline, reaching a peak accuracy of 0.429 versus 0.355 for the index-based baseline,that is, a 21% relative improvement, while converging in about half the number of training epochs. The advantage emerges especially within the first five epochs (under 21% of total data) and persists under moderate character corruption: the corrupted Vision model matches the clean index-based baseline. Ablation studies reveal that the advantage requires both spatially coherent input and a ViT encoder with 2D positional encodings. A cross-script comparison on English shows the advantage does not transfer directly to alphabetic writing systems, suggesting that the uniform visual density and radical structure of Chinese characters are enabling conditions. These findings suggest that transformers are more modality-agnostic than commonly assumed, and that discrete tokenization is not a fundamental requirement for Chinese language modeling.

[153] Paired Uterine Whole-Slide Images and Pathology Reports for Multimodal Computational Pathology cs.CVPDF

Han Li, Jingsong Liu, Ayako Ura, Junlin Hou, Zhengyang Xu

TL;DR: 本文介绍了TUM-Uteria数据集，这是一个包含455个全切片图像与病理报告配对样本的子宫病理学数据集，旨在支持计算病理学中的多模态学习、全切片图像分析和自动报告生成研究。

Details

Motivation: 当前计算病理学中多模态模型的发展受限于缺乏将全切片图像与临床意义丰富的病理报告配对的数据集，现有数据集通常只关注单一终点的标注，无法充分捕捉完整临床诊断工作流程中的丰富信息。

Result: 该数据集包含216个临床病例，共计455个切片级别的WSI-报告对，并经过了由认证病理学家参与的结构化多阶段验证程序以确保标注可靠性。

Insight: 论文的创新点在于构建了一个在病例和切片级别均配对了WSI与诊断报告的子宫病理数据集，填补了该领域高质量多模态数据资源的空白，为开发联合分析图像与文本的AI系统提供了关键基础。

Abstract: Uterine diseases represent an important category of gynecologic pathology and require accurate histopathological assessment for diagnosis and treatment planning. Whole-slide images (WSI) have enabled the digital transformation of pathology workflows and provided new opportunities for artificial intelligence (AI) in computational pathology. In particular, multimodal models that jointly analyze histopathology images and pathology reports have shown promising potential for automated pathology report generation and AI-assisted diagnosis. However, the development of such systems remains limited by the scarcity of datasets that pair whole-slide images with clinically meaningful pathology reports. Instead, existing pathology datasets focus on patch- or slide-level annotations of a single endpoint (e.g., disease class), which do not fully capture the rich information in full clinical diagnostic workflow reports. Here, we introduce TUM-Uteria, a uterine pathology dataset comprising WSIs paired with diagnostic pathology reports at both the case and slide levels, collected from a tertiary medical center. The dataset contains 216 clinical cases, comprising 455 slide-level WSI-report pairs. The dataset underwent a structured multi-stage validation procedure involving board-certified pathologists to ensure reliable annotations. TUM-Uteria supports research in computational pathology, including whole-slide image analysis, multimodal learning, and automated pathology report generation.

[154] PreSIST: Vision-Language-Informed Object Persistence Prediction in Open-World Scenes cs.CV | cs.ROPDF

Amanda Adkins, Tarunvidyut Ravisankar, Joydeep Biswas

TL;DR: 本文提出了PreSIST方法，用于预测开放世界场景中已观测物体在未来任意时刻是否仍会停留在其最后被观测到的位置。该方法通过结合物体属性和场景上下文来估计实例级别的持久性先验，并利用概率持久性滤波器整合观测数据。其核心是利用视觉语言模型（VLM）的推理能力，将场景上下文与物体使用和人类活动联系起来，从而在长期观测数据缺失时进行预测。

Details

Motivation: 解决机器人长期部署中环境动态变化的问题。现有系统通常在重访场景并观测到物体移动后才被动更新地图，而本文旨在让机器人能够主动地、基于物体出现的上下文来推理其可能持续存在的时间。

Result: 在一个新的、包含真实世界物体持久性标注的数据集上进行实验，结果表明，PreSIST-Lang和PreSIST-Vis在开放世界持久性预测任务上超越了基线方法。

Insight: 主要创新点在于利用视觉语言模型（VLM）的推理能力，将场景上下文（如物体用途、人类活动）与物体持久性联系起来，实现了无需长期观测的先验预测。同时，提出了PreSIST-Vis这一仅使用视觉模态的高效部署模型，通过PreSIST-Lang生成的伪标签进行训练，实现了性能与效率的平衡。

Abstract: Robots deployed over long periods must reason about environments that change over time. Existing long-term perception systems often address object change reactively, updating their maps only after revisiting a scene and observing that an object has moved. Instead, robots should reason proactively about how long objects are likely to persist using the context in which they appear. For example, a car at a traffic light and a car in a parking spot share the same semantic class, but their contexts imply different persistence durations. We propose PreSIST (Predictive Scene-conditioned Instance Survival over Time), a method for predicting whether an observed object will remain in its last seen pose at arbitrary future times. PreSIST estimates instance-level persistence priors from object properties and scene context, then integrates these priors with a probabilistic persistence filter as observations become available. Its key insight is that the reasoning capabilities of vision-language models (VLMs) can relate scene context to likely object use and human activity, enabling persistence prediction before long-term observations are available. We develop two interchangeable variants: PreSIST-Lang, which estimates persistence priors using a VLM, and PreSIST-Vis, a novel vision-only model trained using PreSIST-Lang pseudo-labels for efficient deployment. Experiments on a new dataset of in-the-wild object persistence annotations show that PreSIST-Lang and PreSIST-Vis outperform baselines on open-world persistence prediction.

[155] Seeing Once is Enough? Online Geometry-Aware Token Pruning for 3D Question Answering cs.CV | cs.AI | cs.LGPDF

Ruei-Chi Lai, Bolivar Solarte, Chin-Hsuan Wu, Yi-Hsuan Tsai, Min Sun

TL;DR: 本文提出了一种用于3D视觉问答任务的在线几何感知令牌剪枝方法。该方法通过深度信息和相机姿态将输入帧投影到共享体素空间，识别跨帧的空间重叠区域，并在令牌进入语言模型前选择性剪枝冗余图像令牌，从而降低计算成本。

Details

Motivation: 现有MLLM扩展到3D视觉问答时通常需要多视角场景输入，导致推理时计算成本高昂；而现有解决方案依赖离线预处理，无法实现高效在线处理。

Result: 该方法在ScanQA、SQA3D和OpenEQA-HM3D基准测试上应用Qwen2.5-VL-7B和Qwen3-VL-8B模型，实现了性能提升，同时令牌使用量减少高达50%。

Insight: 创新点在于首次提出可无缝集成到现有MLLM的在线令牌剪枝方法，无需额外训练；核心洞察是利用几何信息（深度和姿态）在体素空间识别并剪枝跨帧冗余令牌，实现内存使用和计算效率的优化。

Abstract: Recent Multi-modal Large Language Models (MLLMs) have demonstrated remarkable performance on 2D question answering tasks. However, extending these models to the 3D question answering remains challenging, as they typically require multiple views of the scene, which incurs substantial computational cost at inference. To mitigate this issue, existing solutions rely on strategic frame selection or token-merging algorithms that require preprocessing in advance all frames of the scene, i.e., an offline fashion. In contrast, we propose the first online token-pruning method that can be integrated seamlessly with current MLLM models for 3D question answering tasks, without additional training and with lower memory usage.Our key insight is to project each input frame into a shared voxel space using depth information and camera pose, identifying spatially-overlapped regions across frames and selectively pruning redundant image tokens before they enter the language model. Our method enables efficient online processing while reducing up to 50% of token usage. We apply this approach to Qwen2.5-VL-7B and Qwen3-VL-8B, demonstrating improved performance on the ScanQA, SQA3D, and OpenEQA-HM3D benchmarks.

[156] Sparse4D-Radar: An Efficient and Robust Framework for Surround-View 3D Object Detection via 4D Radar-Camera Fusion cs.CVPDF

Fuyuan Ai, Yuchen Tan, Jiehui Chen, Zhiwei Xu, Chunyi Song

TL;DR: 本文提出了Sparse4D-Radar，一个高效且鲁棒的环视3D目标检测框架，通过4D雷达-相机融合实现。该框架包含一个轻量级基础版本和两个专用模块，以提升定位精度和模态稳定性，在复杂驾驶场景下实现了SOTA性能，并在检测精度、环境鲁棒性和推理效率之间取得了良好平衡。

Details

Motivation: 解决主流4D雷达-相机融合方法仅支持前视感知、缺乏成熟的环视解决方案，以及直接扩展到360°覆盖会引入过高计算成本的问题。

Result: 在OmniHD-Scenes数据集上的综合实验表明，该方法在环视3D检测上达到了最先进的性能，在复杂驾驶场景下比先前方法提升了超过7%的mAP和10%的ODS，同时以近10 FPS的速度运行。

Insight: 创新点包括：1）设计了可变形融合模块，将雷达-相机特征嵌入稀疏查询以构建轻量级基础框架；2）引入了速度一致性采样模块，利用雷达速度线索细化特征以增强运动感知；3）提出了自适应模态门控模块，根据特征置信度动态调整跨模态融合权重。这些设计共同提升了定位精度和模态鲁棒性。

Abstract: In recent years, 4D imaging radar has gained wide attention in autonomous driving for its robustness against harsh weather and ability to output target velocity. Nevertheless, mainstream 4D radar-camera fusion methods only support front-view perception, lacking mature solutions for surround-view sensing. Directly expanding these pipelines to full 360° coverage introduces excessive computation cost and limits real-world deployment. To tackle these limitations, this work proposes Sparse4D-Radar, an efficient robust surround-view multi-modal fusion framework. We first design a Deformable Fusion module to embed radar-camera features into sparse queries, constructing the lightweight base version Sparse4D-Radar-Base. Two dedicated modules are further introduced to boost localization accuracy and modality stability: Velocity-Consistency Sampling (VCS) refines features via radar velocity cues for motion awareness, and Adaptive Modality Gating (AMG) dynamically adjusts cross-modal fusion weights according to feature confidence. Combining all components, we build Sparse4D-Radar-Acc for high-precision detection demands. Comprehensive experiments on OmniHD-Scenes verify that our approach achieves state-of-the-art surround-view 3D detection performance. Compared with prior arts, our method obtains over 7% mAP and 10% ODS improvements under complex driving scenes while running at nearly 10 FPS, striking a favorable trade-off among detection accuracy, environmental robustness and inference efficiency. Our open-source code is available at https://github.com/Aiuan/Sparse4D-Radar.

[157] SiamJEPA: On the Role of Siamese Student Encoders in JEPA cs.CV | stat.MLPDF

Makoto Yamada

TL;DR: 本文提出了SiamJEPA，一种在联合嵌入预测架构（JEPA）中使用孪生学生编码器的自监督表示学习方法。该方法通过引入孪生编码器作为正则化器，旨在提升表示的可分离性和早期训练效率。实验表明，在有限训练预算下，SiamJEPA优于单编码器JEPA变体及需要更长训练的掩码自编码器（MAE）。

Details

Motivation: 现有JEPA方法（如I-JEPA和V-JEPA）通常使用单一学生编码器，而受大脑启发的表示学习框架更自然地采用孪生编码器。本文旨在探索孪生学生编码器在JEPA模型中的作用，以改进自监督表示学习。

Result: 在ImageNet线性探测任务上的广泛实验表明，SiamJEPA在有限训练预算下持续优于可比的单编码器JEPA变体，并且比需要更长训练的掩码自编码器（MAE）获得了更高的线性探测准确率。

Insight: 论文的创新点在于将孪生学生编码器引入JEPA框架，并证明其不仅是架构选择，更是预测性表示学习的重要归纳偏置。这为JEPA模型设计提供了新见解，表明孪生学生架构是改进自监督表示学习的简单有效途径。

Abstract: Recently, Joint Embedding Predictive Architectures (JEPAs) have attracted significant attention in the computer vision and machine learning communities as a promising framework for self-supervised representation learning. Unlike masked autoencoders that reconstruct pixels, JEPA models learn representations by predicting latent embeddings of masked regions. Existing JEPA-based methods, such as I-JEPA and V-JEPA, typically employ a single encoder in the student network. In contrast, using Siamese encoders for student network is more naturally aligned with brain-inspired representation learning frameworks, yet their role in JEPA models remains largely unexplored. In this paper, we investigate the effect of Siamese student encoders in JEPA-based representation learning. To this end, we propose SiamJEPA, masked Siamese student encoders equipped with an exponential moving average (EMA) teacher network. SiamJEPA can also be viewed as a JEPA formulation of the brain-inspired representation learning model PhiNet. Through extensive experiments on ImageNet linear probing, we demonstrate that Siamese encoders act as an effective regularizer for the JEPA objective, improving representation separability and accelerating learning during the early stages of training. Furthermore, SiamJEPA consistently outperforms comparable single-encoder JEPA variants under limited training budgets and achieves higher linear probing accuracy than Masked Autoencoders (MAE) which requires longer training. Our findings reveal that Siamese student encoders are not merely an architectural choice but constitute an important inductive bias for predictive representation learning. These results provide new insights into the design of JEPA-based models and suggest that incorporating Siamese student architectures offers a simple yet effective approach for improving self-supervised representation learning.

[158] SOV-CAD: Stepwise Orthographic Views Guided CAD Modeling Sequence Reconstruction cs.CV | cs.AIPDF

Zhaopeng Feng, Chen Zhi, Xuhong Zhang, Zhengwen Feng, Xinkui Zhao

TL;DR: 本文提出SOV-CAD框架，通过逐步正交视图引导，将CAD建模序列重建任务重新定义为顺序决策问题。该方法利用离线强化学习与决策变换器架构，结合几何对齐奖励驱动的连续视觉反馈，以模拟人类迭代、反馈驱动的设计工作流程，从而生成更准确、更符合设计意图的建模序列。

Details

Motivation: 现有方法通常整体生成完整的CAD建模序列，忽略了人类设计工作流程中迭代、反馈驱动的本质，导致无法有效保留设计意图和支持参数化编辑。本文旨在通过引入逐步视觉监督来解决这一局限性。

Result: 大量实验表明，SOV-CAD在CAD序列重建任务上超越了现有最先进（SOTA）方法，同时展现出强大的数据效率。

Insight: 主要创新点在于引入了逐步正交视图作为丰富的视觉监督信号，并将CAD重建任务形式化为顺序决策问题，利用离线强化学习结合几何对齐奖励来整合连续视觉反馈，从而模拟了更符合人类习惯的、基于反馈的增量建模过程。

Abstract: Reconstructing Computer-Aided Design (CAD) modeling sequences from images is crucial for preserving design intent and supporting parametric editing. However, existing methods typically generate full CAD sequences holistically, overlooking the iterative, feedback-driven nature of human design workflows. We address this limitation by introducing the rich stepwise visual supervision: at each modeling step, the system observes the target’s orthographic projections, the projections of the incrementally constructed model, and the active sketch, enabling informed action selection. To effectively leverage this on-the-fly feedback, we propose SOV-CAD, a framework that formulates CAD reconstruction as a sequential decision-making task and employs offline reinforcement learning with a Decision Transformer architecture. This design incorporates continuous visual feedback guided by geometric alignment rewards, resulting in a more accurate and human-like modeling process. Extensive experiments show that SOV-CAD surpasses state-of-the-art methods in CAD sequence reconstruction while exhibiting strong data efficiency. Code of SOV-CAD is available at: https://github.com/LukePhong/SOV-CAD

[159] HCSU: A Dataset and Benchmark for Fine-Grained Historical Calligraphy Style Understanding cs.CV | cs.AIPDF

Yinsheng Yao, Yan Liu, Chen Ye

TL;DR: 本文提出了HCSU数据集，这是首个专为细粒度历史书法风格理解设计的综合性数据集，包含来自10个朝代49位著名书法家的39,307张精心整理的字符图像，并系统地将墨迹真迹与石刻拓本解耦以解决模态混合问题。该数据集提供了层次化的专家撰写美学描述，支持细粒度风格判别和可解释的美学推理两种评估协议。

Details

Motivation: 解决现有数据集存在的模态混合和标签扁平化问题，以促进大视觉语言模型在文化遗产保护中实现细粒度书法风格感知这一关键挑战。

Result: 广泛评估揭示了书法相关知识与视觉风格感知之间的持续差距：最先进的大视觉语言模型表现出非平凡的性能，但对字体级别、文本和来源特定线索敏感，且难以将美学判断基于细粒度笔触证据。

Insight: 创新点在于构建了首个解耦墨迹与拓本模态的细粒度书法风格数据集，并引入层次化美学描述以支持更严格的评估；客观上，该工作暴露了当前多模态架构在专家级视觉推理方面的根本局限性，为文化遗产领域的AI研究提供了新的基准。

Abstract: Automated fine-grained perception of calligraphy styles–a task vital to cultural heritage preservation–remains a critical challenge for Large Vision-Language Models (LVLMs), largely constrained by existing datasets that suffer from modal mixture and flattened labels. To bridge this gap, we introduce HCSU, the first comprehensive dataset tailored for fine-grained Historical Calligraphy Style Understanding. HCSU comprises 39,307 meticulously curated character images from 49 historically prominent calligraphers across 10 dynasties, systematically decoupling authentic ink manuscripts (Tie) from stone rubbings (Bei) to resolve the long-standing modal mixture problem. Moving beyond conventional flattened labels, HCSU provides hierarchical expert-written aesthetic descriptions, enabling two rigorous evaluation protocols: fine-grained style discrimination and interpretable aesthetic reasoning. Extensive evaluations reveal a persistent gap between calligraphy-related knowledge and visually grounded style perception: state-of-the-art LVLMs show non-trivial performance but remain sensitive to script-level, textual, and source-specific cues, and often struggle to ground aesthetic judgments in fine-grained brushwork evidence. Ultimately, the HCSU benchmark exposes fundamental limitations in current multimodal architectures, aiming to inspire the evolution of expert-level visual reasoning for cultural heritage preservation. The dataset is available at https://huggingface.co/datasets/Tongji209/HCSU.

[160] Perceiving Better Moments: Cover Frame Reselection and Enhancement for Live Photos with the Live2K Dataset cs.CVPDF

Junyu Lou, Kai Chen, Weiyi You, Hui Zeng, Lei Zhang

TL;DR: 该论文针对智能手机Live Photos功能中，用户重选封面帧时因视频流质量差导致的视觉退化问题，提出了Live Photo封面帧重选与增强任务，并构建了首个真实世界数据集Live2K，开发了一个集成多帧融合、引导色彩增强和超分辨率的一阶段基线模型。

Details

Motivation: 解决Live Photos中因照片流和视频流成像管线分离导致的封面照片与视频帧在分辨率、色彩保真度和动态范围上的巨大质量差距，使得用户重选的低质量封面帧能够被有效增强。

Result: 论文构建了包含2042个Live Photos的真实世界数据集Live2K，并提出了一个统一的基线模型，为Live Photo增强研究建立了首个基准。

Insight: 创新性地定义了LPRE任务，并利用Live Photos内部固有的高质量封面图像作为结构和色彩参考、用户重选的低质量帧作为重建目标、相邻视频帧提供时序线索，通过一个一阶段模型统一处理多帧融合、引导色彩增强和超分辨率这些挑战性子任务。

Abstract: Modern smartphones capture Live Photos, short video bursts surrounding a still image, offering a dynamic and engaging photographic experience. However, the cover photo and video components are generated by two distinct imaging pipelines: the photo stream undergoes full computational photography processing, while the video stream is constrained by real-time efficiency and heavy compression. This intrinsic separation produces a substantial quality gap in resolution, color fidelity, and dynamic range between the cover photo and video frames. When users reselect an alternative frame from the video to replace an imperfect cover, the chosen frame often suffers from severe degradation, making direct replacement visually unsatisfactory. Restoring such frames requires simultaneous enhancement of spatial detail and color appearance, a task considerably more challenging than ordinary super-resolution or color enhancement. To address this, we define the Live Photo Cover Frame Reselection and Enhancement (LPRE) task, which leverages the intrinsic cues available within each Live Photo: the high-quality cover image as a structural and color reference, the user-reselected low-quality frame as the reconstruction target and several adjacent video frames providing temporal cues. Building upon this formulation, we construct Live2K, a real-world dataset of 2,042 Live Photos, and develop a unified one-stage baseline that integrates multi-frame fusion, guided color enhancement and super-resolution, establishing the first benchmark for Live Photo enhancement research.

[161] SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering cs.CV | cs.AIPDF

Kai Tang, Jinhao You, Bohua Zhang, Yichen Guo, Yiding Sun

TL;DR: 本文提出了SeeMe框架，旨在缓解大型视觉语言模型（LVLM）中的幻觉问题。该框架借鉴传统机器学习中的特征工程思想，通过三阶段的视觉令牌工程来重构视觉令牌，从而抑制无关或噪声令牌对解码过程的误导，提升输出与视觉输入的一致性。

Details

Motivation: 现有方法主要关注解码阶段的干预，而忽略了幻觉的一个关键来源：无关或噪声的视觉令牌会误导解码过程，导致生成与视觉输入不一致的内容。

Result: 在MME、POPE和AMBER基准测试上对四种LVLM进行的实验表明，SeeMe能持续减少幻觉并提高输出一致性。

Insight: 创新点在于将传统机器学习中的特征工程概念引入LVLM，通过训练无关的令牌工程过程直接处理视觉令牌，为缓解幻觉问题提供了新视角；客观来看，该方法从输入表征层面而非仅解码阶段进行干预，是一种新颖且有效的思路。

Abstract: Large Vision-Language Models (LVLMs) have achieved remarkable progress in visual understanding tasks such as image captioning and visual question answering. However, they remain susceptible to hallucinations, generating content that is inconsistent with the actual visual input. Existing methods primarily intervene at the decoding stage, while overlooking a critical source of hallucinations: irrelevant or noisy visual tokens that mislead the decoding process. To address this issue, we propose SeeMe, a training-free framework that introduces the concept of feature engineering from traditional machine learning into LVLMs. SeeMe restructures visual tokens through a three-stage token engineering process to suppress hallucination sources while preserving informative visual evidence. Experiments on MME, POPE, and AMBER benchmarks across four LVLMs demonstrate that SeeMe consistently reduces hallucinations and improves output consistency, providing a novel perspective for mitigating hallucinations in LVLMs.

[162] Beyond Scene Priors: Fine-Grained Traffic Scene Reasoning with Benchmarking and Query-Guided Small-Object Focus cs.CVPDF

Waikit Xiu, Qiang Lu, Zian Wang, Xinjie Yang, Zhiwei Chen

TL;DR: 本文针对安全关键交通场景下多模态大语言模型（MLLMs）存在的‘关键证据稀释’问题，即模型过度关注背景而忽略关键小物体，提出了一个细粒度交通推理基准（FGTR-Bench）和一个文本引导的小物体推理MLLM（TSR-MLLM）。TSR-MLLM通过查询引导的小物体聚焦图（TG-SOF）在解码边界对最相关的视觉特征进行稀疏门控残差增强，从而在不引入外部检测器或图像重编码的情况下，提升了模型对局部细微证据的推理能力。

Details

Motivation: 标准MLLMs在视觉-语言对齐时倾向于过度关注背景，导致对安全关键交通场景中至关重要的微小物体（如交通标志）的注意力被稀释，即‘关键证据稀释’问题。现有VQA数据集缺乏大规模、包含大量干扰项且需要精确定位局部证据的评估，未能充分暴露此缺陷。

Result: 在提出的FGTR-Bench基准上，TSR-MLLM（基于Qwen3-VL-4B）在匹配设置下以74.1%的总体准确率超越了最强的4B基线模型2.1个百分点，在需要证据定位的测试子集上提升更显著。此外，在DriveQA-V（CARLA Signs）基准上，仅使用贪心解码且无需任务特定微调，TSR-MLLM仍保持竞争力。

Insight: 论文的核心创新点在于：1）构建了一个大规模、包含干扰项的细粒度交通推理基准（FGTR-Bench），用于评估模型对局部微小证据的推理能力；2）提出了一种轻量级的文本引导小物体聚焦机制（TG-SOF），通过稀疏门控残差在单次推理中增强与问题最相关的视觉特征，有效缓解了证据稀释问题，且无需外部模块或图像重编码，保持了推理效率。

Abstract: In safety-critical traffic scenarios, answering complex questions relies on minute, localized visual cues. However, standard Multimodal Large Language Models (MLLMs) tend to over-attend to backgrounds, overwhelming crucial small objects during visual-language alignment, a failure mode we term ‘critical evidence dilution.’ Furthermore, existing visual question answering (VQA) datasets rarely expose this flaw, as they lack large-scale, distractor-heavy evaluations that require pinpointing local evidence. To bridge this evaluation and architecture gap, we introduce the Fine-Grained Traffic Reasoning Benchmark (FGTR-Bench) and the Text-Guided Small-Object Reasoning MLLM (TSR-MLLM). FGTR-Bench comprises 40,236 single-image Multiple-Choice Questions (MCQs) created via multi-agent generation, consistency checks, and expert audits, alongside a disjoint 4,947-sample blind test split. To resolve evidence dilution, TSR-MLLM, built on Qwen3-VL-4B, uses a query-conditioned Text-Guided Small-Object Focus (TG-SOF) map. Applied once at the decoder boundary, the map adds sparse Top-K gated residuals to the most question-relevant vision slots while leaving text tokens unchanged. Together with lightweight decoder adaptation, TSR-MLLM preserves single-pass inference without external detectors or image re-encoding. Under matched settings, TSR-MLLM outperforms the strongest 4B baseline by 2.1 points on FGTR-Bench (74.1% overall), with larger gains on evidence-local tracks. Furthermore, it remains competitive on DriveQA-V (CARLA Signs) under greedy decoding without task-specific fine-tuning.

[163] CritiqueDriveVLM: From Verifier-Guided Reinforcement Learning to Latent Thought Distillation for Autonomous Driving cs.CV | cs.AIPDF

Zhaohong Liu, Hao Ye, Xianlin Zhang, Mengshi Qi

TL;DR: 本文提出了CritiqueDriveVLM，一个用于自动驾驶的三阶段端到端视觉语言模型框架。该框架通过验证器引导的多轮强化学习训练一个高精度的‘系统2教师模型’，再通过潜在思维蒸馏将其推理能力压缩到一个快速、无需思维链的‘系统1学生模型’中，从而在保持高可靠性的同时，大幅降低推理延迟。

Details

Motivation: 标准的监督微调方法在自动驾驶的视觉语言模型中存在推理幻觉和保守偏差问题，而传统的工具增强或思维链方法虽然能缓解这些问题，但会导致极高的计算开销和延迟，无法满足实时部署需求。本文旨在解决这一可靠性与效率之间的权衡难题。

Result: 在DriveLMM-01基准测试上，无工具的教师模型将多项选择质量从55.54%显著提升至76.54%，达到SOTA水平。通过蒸馏得到的学生模型在保持竞争力的推理深度的同时，将平均生成长度大幅压缩至仅28个token，推理延迟降低了88%（从3482毫秒降至416毫秒）。

Insight: 核心创新点在于将验证器引导的多轮强化学习与潜在思维蒸馏相结合的统一框架。该方法通过多维验证器提供细粒度反馈，迫使模型内化逻辑推理，并通过对齐师生模型的潜在表征，将深度逻辑能力压缩到快速的前馈网络中，为低延迟自动驾驶提供了一条新路径。

Abstract: End-to-end Vision-Language Models (VLMs) show immense potential in autonomous driving. However, standard Supervised Fine-Tuning (SFT) often suffers from reasoning hallucinations and conservative biases. While traditional tool-augmented frameworks and Chain-of-Thought (CoT) approaches mitigate these issues, they incur exorbitant token consumption and unacceptable latency, rendering real-time deployment impractical. To resolve this reliability-efficiency trade-off, we propose CritiqueDriveVLM, a novel unified three-stage framework internalizing reasoning directly into the VLM. First, we introduce Critique-Driven Multi-Turn Reinforcement Learning (RL) guided by a multi-dimensional verifier. By providing granular scalar feedback and a multi-turn penalty, we force the policy to internalize logical deduction, cultivating a robust System-2 Teacher that achieves high accuracy without fragile external tools. Subsequently, we propose Latent Thought Distillation to overcome the latency bottleneck. By aligning the Student’s latent representations with the Teacher’s fully converged reasoning states, we compress deep logical capabilities into a fast, CoT-free System-1 Student. Extensive experiments on the widely-used DriveLMM-01 benchmark demonstrate remarkable improvements. Compared to the base model, our tool-free Teacher significantly boosts Multiple Choice Quality (MCQ) from 55.54% to a state-of-the-art 76.54%. Crucially, our distilled Student preserves competitive reasoning depth while drastically minimizing generation length to an average of merely 28 tokens. This slashes inference latency by 88% (from 3482 ms to 416 ms), paving a highly robust pathway for low-latency autonomous driving.Our source code is available at https://github.com/MICLAB-BUPT/CritiqueDriveVLM.

[164] Beyond Random Sampling: Distribution-Aware Alignment for Semi-Supervised Medical Image Segmentation cs.CVPDF

Weihao Yan, Yeqiang Qian, Yi Dong, Ming Yang

TL;DR: 本文提出了一种分布感知的半监督医学图像分割框架，旨在解决低标注数据场景下随机采样导致的表示偏差问题。该框架包含离线分布感知样本选择策略和内存引导的复制粘贴模块，通过选择代表性结构锚点和跨域对齐来提升数据效率。

Details

Motivation: 半监督医学图像分割通常假设数据独立同分布并采用随机采样，但在低数据场景下存在严重的表示偏差，无法捕捉异构的医学数据流形。

Result: 在六个不同的2D和3D数据集上进行广泛实验，结果显示在极低标注比例（如1/16）下仍能实现强大的分割性能。

Insight: 创新点包括利用视觉基础模型和密度K中心算法进行分布感知样本选择，以及针对医学图像类别不平衡设计的内存引导复制粘贴模块，通过语义内存机制实现跨域对齐和语义一致性。

Abstract: Precise medical image segmentation is crucial for clinical diagnosis and treatment planning, yet relies heavily on expensive expert annotations. Semi-supervised medical image segmentation (SSMIS) offers a cost-effective solution but typically operates under the assumption of independent and identically distributed (i.i.d.) data, defaulting to random sampling. While statistically valid at scale, this strategy suffers from severe representation bias in low-data regimes, failing to capture the heterogeneous medical data manifold. To address this, we propose a highly data-efficient framework driven by distribution alignment. First, we introduce an offline Distribution-Aware Sample Selection strategy. By leveraging Vision Foundation Models (VFMs) and our designed Density-K-Center algorithm, we explicitly identify representative structural anchors, establishing a more representative labeled domain. Second, to bridge the remaining distribution gap, we propose the Memory-guided Copy-Paste (MCP) module. Tailored for the inherent class imbalance in medical scans, MCP leverages a semantic memory mechanism to retrieve historically consistent priors for cross-domain alignment, encouraging semantic consistency. Coupled with an easy-to-hard progressive schedule, this framework effectively mitigates early-stage pseudo-label noise. Extensive experiments on six diverse 2D and 3D datasets demonstrate strong segmentation performance, particularly in extremely low-labeled scenarios (\eg, 1/16 ratio).

Qiang Chen, Xiao Wang, Hao Si, Qingquan Yang, Meiwen Chen

TL;DR: 本文提出了一种用于EAST托卡马克装置中破裂预测的分层多模态到单模态知识蒸馏框架。该框架利用可见光图像和时间序列信号训练多模态教师模型，然后通过图结构级、表示级和决策级蒸馏将知识转移到仅使用时间序列信号的学生模型，旨在保持多模态学习判别优势的同时显著降低推理成本。

Details

Motivation: 现有数据驱动的破裂预测器主要依赖时间序列诊断信号，而可见光图像能提供等离子体变形、局部增亮和辐射结构演化等互补的空间线索，但图像模态显著增加了推理时的计算成本。为了解决这一问题，本文旨在开发一个高效的蒸馏框架。

Result: 在包含640次放电的EAST数据集上的实验结果表明，所提出的框架能够保留多模态学习的判别优势，同时大幅降低推理成本，为EAST中的高效破裂预测提供了有效途径。

Insight: 创新点在于提出了一个分层的多模态到单模态知识蒸馏框架，通过Transformer编码器和原型引导的时空超图模块学习破裂前兆表征，并设计了图结构级、表示级和决策级的多层次蒸馏策略，以实现知识从计算复杂的多模态模型向轻量单模态模型的有效迁移。

Abstract: Plasma disruption is a critical threat to tokamak safety. Existing data-driven predictors mainly rely on time-series diagnostic signals, while visible images provide complementary spatial cues including plasma deformation, local brightening, and radiation-structure evolution. Although the image modality improves the model’s discriminative capability, it also substantially increases the computational cost during inference. To address this issue, we propose a hierarchical multi-to-single-modal knowledge distillation framework for disruption prediction on a synchronized EAST multimodal dataset. During training, visible images and time-series signals are used to train a multimodal teacher, which learns disruption precursor representations through Transformer-based encoders and a prototype-guided spatiotemporal hypergraph module. During inference, only the time-series student is retained, with multimodal knowledge transferred through graph-structure-level, representation-level, and decision-level distillation. On the 640-discharge EAST dataset, the results demonstrate that the proposed framework can preserve the discriminative advantages of multimodal learning while substantially reducing inference cost, and providing an effective route for efficient disruption prediction in EAST. The source code of this paper will be released on https://github.com/Event-AHU/OpenFusion.

[166] Topology-Driven Transferability Estimation for 3D Medical Vision Foundation Models cs.CVPDF

Jiaqi Tang, Shaoyang Zhang, Fandong Zhang, Shu Zhang, Yang Liu

TL;DR: 本文提出了一种非参数、基于拓扑的框架，用于评估3D医学视觉基础模型在分割任务中的可迁移性。该方法通过最小生成树（MST）将密集特征与语义标签的对齐解耦为局部边界感知拓扑一致性（LBTC）和全局表示拓扑差异（GRTD）两个互补的几何尺度，并利用任务自适应门控机制融合它们。在大规模基准测试中，该方法在加速56倍的同时，实现了最先进的可迁移性估计性能。

Details

Motivation: 现有可迁移性估计（TE）指标主要针对图像级分类任务设计，无法保留对3D医学图像分割至关重要的空间关系和细粒度边界细节，且主流模型选择方法依赖计算成本高昂的穷举微调。

Result: 在包含114,000个3D医学体积、涵盖多种解剖任务的大规模基准测试中，该拓扑框架实现了最先进（SOTA）的可迁移性估计，平均加权Kendall系数超出基线0.36，同时将评估速度提升了56倍。

Insight: 创新点在于提出了一种非参数的、基于拓扑的评估框架，将特征-标签对齐解耦为局部边界和全局布局两个互补的几何尺度进行量化；一个反直觉的机制是，即使未经微调，随机初始化的分割解码器也能作为拓扑保持的空间投影器，稳定全局对齐评估；通过任务自适应门控融合双指标，能适应不同的临床复杂性。

Abstract: The growing number of medical vision foundation models highlights the need for effective model selection. However, mainstream selection methods rely on exhaustive fine-tuning, which is computationally expensive. Most of the existing Transferability Estimation (TE) metrics are primarily designed for image-level classification. They fail to preserve spatial relationships and fine-grained boundary details, which are crucial for the segmentation task. Additionally, while image-level tasks typically process a single feature vector per input, dense prediction tasks in 3D medical imaging require voxel-wise evaluation against dense annotations. To bridge these gaps, we propose a \textit{non-parametric, topology-driven} framework that estimates transferability directly from the alignment between the sparse 1-skeleton graph of dense features and semantic labels via Minimum Spanning Trees (MST). We decouple the alignment into two complementary geometric scales: Local Boundary-Aware Topological Consistency (LBTC) to assess boundary separability, where we prove that the MST leakage rate serves as a finite-sample lower bound on the Bayes error; and Global Representation Topology Divergence (GRTD) to evaluate the overall anatomical layout. Crucially, we formally justify a counterintuitive mechanism: Although without fine-tuning, the randomly initialized segmentation decoder acts as a topology-preserving spatial projector, reducing the variance of pairwise distance estimates and stabilizing global alignment evaluation. Fused via a task-adaptive gating mechanism, these dual metrics adapt to diverse clinical complexities. Evaluated on a large-scale benchmark of 114,000 3D medical volumes across diverse anatomical tasks, our topological framework achieves state-of-the-art transferability estimation with an average weighted Kendall (outperforming by 0.36) while accelerating evaluation by 56 times.

[167] AquaStereo: Enabling Underwater Stereo Matching via Depth-Conditioned Diffusion and Geometry Self-Distillation cs.CVPDF

Qizhe Wei, Yingping Liang, Shaodi You, Ying Fu

TL;DR: 本文提出AquaStereo框架，通过深度条件扩散生成水下立体图像对以解决数据稀缺问题，并利用几何自蒸馏策略提升特征鲁棒性，从而有效改善基于学习的水下立体匹配模型在退化图像中的性能。

Details

Motivation: 解决基于学习的立体匹配模型在水下环境中因领域数据稀缺和退化图像难以提取判别性对应关系而性能不佳的问题。

Result: 在具有挑战性的水下场景中，AquaStereo显著提高了鲁棒性和零样本泛化能力，通过广泛的实验验证了其有效性。

Insight: 创新点包括：深度条件扩散管道生成几何对齐的水下立体数据；基于冻结双目教师的几何自蒸馏策略；以及融合时序与语义特征的可学习感知帧以增强特征稳定性。

Abstract: Learning-based stereo matching models struggle in underwater environments due to scarce in-domain data and the difficulty of extracting discriminative correspondences from degraded imagery. In this work, we present $\textbf{AquaStereo}$, a perception-enhanced framework with a data simulation pipeline and a self-distillation strategy that jointly address data scarcity and feature degradation in underwater stereo matching. First, a depth-conditioned diffusion pipeline renders underwater stereo pairs while preserving binocular geometry, with a lightweight left-right consistency module ensuring geometric alignment. Training on this synthetic corpus effectively narrows the terrestrial-underwater gap and improves zero-shot robustness. Second, a frozen binocular teacher trained on clean terrestrial pairs guides a student exposed to rendered underwater pairs with perturbations. A stage-weighted sequence loss is performed to align the student’s disparities with the teacher’s geometry, while a clean-branch supervision with shared pseudo targets prevents scale drift. To further enhance feature stability under turbidity and low texture, we introduce learnable perception frames, a perception-enhanced feature formulation that constructs robust matching descriptors by fusing temporal cues from two auxiliary views encoded by a video backbone with semantic features extracted by a strong image encoder. Extensive experiments demonstrate that $\textbf{AquaStereo}$ substantially improves robustness and zero-shot generalization in challenging underwater scenarios. The code is available at https://github.com/qz-wei/AquaStereo.

[168] Road-Aware Anomaly Segmentation with Query-Guided Polygons and CLIP in Autonomous Driving cs.CVPDF

Zhiran Yan, Gordon Elger

TL;DR: 本文提出了一种轻量级、无需重新训练、无需OOD数据、无需额外监督的道路感知异常分割框架，用于自动驾驶中的开放世界异常检测。该方法基于掩码Transformer分割网络，利用查询级掩码置信度和多边形道路先验来检测可能对应异常的间隙区域，并引入基于CLIP的零样本语义过滤模块以抑制误报。

Details

Motivation: 传统语义分割模型在封闭集假设下运行，难以识别未知或意外物体，这在自动驾驶的开放世界环境中至关重要。现有模型常误分类或忽略分布外（OOD）道路异常，带来安全风险。

Result: 在三个公共基准测试（Fishyscapes、SMIYC和RoadAnomaly）上评估，该方法表现稳健。特别是在Fishyscapes LostAndFound上实现了最高的平均精度（AP），并在大多数指标上优于无需训练的基线方法Maskomaly。

Insight: 创新点包括结合空间先验（多边形道路先验）和语义验证（CLIP零样本过滤）进行异常检测，无需OOD数据或重新训练，提高了方法的实用性和可部署性。从客观角度看，该方法有效利用了现有分割模型的置信度信息和通用视觉语言模型的语义理解能力，实现了轻量级后处理框架。

Abstract: Traditional semantic segmentation models operate under a closed-set assumption and struggle to recognize unknown or unexpected objects-an essential capability for autonomous driving. As a result, such models often misclassify or overlook out-of-distribution (OOD) road anomalies, posing safety risks in open-world environments. We present a lightweight, postprocessing, road-aware anomaly segmentation framework that requires no retraining, no OOD data, and no auxiliary supervision. Our approach builds on a mask transformer-based segmentation network by exploiting query-level mask confidence and deriving a polygonal road prior to detect gap regions that may correspond to anomalies. To further suppress false positives, we introduce a CLIP-based zero-shot semantic filtering module using in-distribution prompts, with optional generalized OOD prompts. By jointly leveraging spatial priors and semantic verification, our framework produces robust and interpretable anomaly predictions. Evaluation on three public benchmarks-Fishyscapes, SMIYC, and RoadAnomaly-shows consistently strong performance. In particular, our method outperforms the training-free baseline Maskomaly on most metrics and achieves the highest AP on Fishyscapes LostAndFound. These results demonstrate the practicality and deployability of our approach for real-world autonomous driving systems.

[169] Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment cs.CVPDF

Zixiang Zhou, Zhentao Yu, Yifeng Ma, Hongmei Wang, Wenqing Yu

TL;DR: 本文提出Aura框架，用于解决多主体视频生成中的身份一致性和复杂关系建模问题。该方法通过VLM提取多模态语义特征，并设计两阶段对齐策略将特征映射到DiT空间，结合主题感知机制和渐进式提示引导，实现了高保真且身份一致的视频生成。

Details

Motivation: 现有方法在保持身份一致性和建模多主体复杂关系方面存在困难，因此需要一种统一的框架来提升可控视频合成的质量。

Result: 在单主体和更具挑战性的多元素场景中，该方法在实验中达到了最先进的性能水平。

Insight: 创新点包括引入AI导演级描述、利用VLM提取多模态特征、设计两阶段对齐策略、主题感知RoPE-Shift机制、主题感知可学习令牌、记忆令牌以及渐进式自适应提示引导，这些技术共同提升了生成视频的语义对齐和视觉质量。

Abstract: Subject-driven and multi-element video generation are central to controllable video synthesis, but existing methods still struggle to preserve identity consistency and model complex relationships among multiple subjects. In this paper, we propose Aura, a unified framework for high-fidelity and identity-consistent video generation. To better capture scene dynamics and subject interactions, we introduce AI director-level captions that provide dense and structured descriptions of video content. We further leverage a vision-language model (VLM) with learnable queries to extract multimodal semantic features from textual and visual references, covering both global semantics and fine-grained visual cues. To bridge the representational gap between the VLM and the Diffusion Transformer (DiT), we design a two-stage alignment strategy that progressively maps VLM features into the DiT feature space. For visual conditioning, we adopt token concatenation to inject reference information directly into the generation process. To distinguish heterogeneous subject types and reduce common copy-paste artifacts, we develop a subject-aware RoPE-Shift mechanism. To further differentiate reference images of different categories, we introduce subject-aware learnable tokens. In addition, we introduce Memory Tokens to balance the training signal across examples with different numbers of reference subjects. During inference, Progressive-APG (Adaptive Prompt Guidance) further alleviates oversaturation and improves semantic alignment with user prompts. Finally, we build a high-quality video-subject image dataset through a dedicated data construction pipeline. Extensive experiments show that our method achieves state-of-the-art performance on both single-subject generation and more challenging multi-element scenarios.

Zhiran Yan, Yutong Xin, S Shyam Shenoi, Rui Song, Gordon Elger

TL;DR: 本文提出了一个用于道路施工区域检测与地理定位（RZDG）的多模态数据集和基准框架。该数据集包含模拟和真实世界数据，支持图像语义分割、3D目标检测和地理定位等任务。同时，论文扩展了AB3DMOT跟踪器，构建了一个用于道路施工区域检测与地理定位的流程，并在新数据集上验证了其有效性。

Details

Motivation: 自动驾驶车辆依赖的高清地图更新不及时，且缺乏临时道路施工区域等半静态信息，这影响了导航的可靠性。目前缺乏公开数据集来评估道路施工区域的检测与地理定位模型，阻碍了可靠自动驾驶系统的发展。

Result: 在提出的RZDG数据集上进行了基准测试，评估标准为预测位置与真实位置在一米内视为正确。实验结果显示，该方法在真实和模拟数据上均取得了较高准确率：真实数据上精确率0.565、召回率0.898、F1分数0.597；模拟数据上精确率0.615、召回率0.809、F1分数0.665。

Insight: 主要创新点在于创建了首个公开的、包含多模态传感器数据与全面标注的道路施工区域检测与地理定位数据集。方法上的创新是将AB3DMOT跟踪器扩展用于目标地理定位，实现了从局部坐标系到全球坐标系的准确转换，为解决地图更新滞后问题提供了数据和技术基础。

Abstract: Autonomous vehicles often rely on high-definition (HD) maps for navigation; however, these maps are not frequently updated and often lack semi-static information, such as temporary roadwork zones, which can significantly alter the road network. This limitation underscores the urgent need for an accurate global position of roadwork zones. However, the absence of publicly available datasets for evaluating roadwork zone detection and geo-localization models has hindered the development of reliable autonomous driving systems. To address this challenge, we propose the Roadwork Zone Detection and Geo-localization (RZDG) dataset, which includes both simulated and real-world data, providing multimodal sensor inputs along with comprehensive annotations. The dataset supports multiple perception tasks, including image semantic segmentation, 3D object detection, and object geo-localization. In addition, we introduce a tracker-based roadwork zone detection and geo-localization (RZDG) pipeline, an extension of AB3DMOT, for accurate object geo-localization in roadwork zones. We benchmark our approach on the RZDG dataset, demonstrating its effectiveness in detecting roadwork zones and transforming object positions from the local coordinate system to the global coordinate system. A prediction is considered a true positive (TP) if its estimated position falls within one meter of the ground truth. Our experimental results show that our approach achieves high accuracy on both real and simulated data. Specifically, we report: Precision: 0.565 (real) / 0.615 (simulated) Recall: 0.898 (real) / 0.809 (simulated) F1-score: 0.597 (real) / 0.665 (simulated).

[171] IRIS: An Intelligent Vision-Language System for Ocular Surface Diseases via Topic Tree and Scene-Driven VQA Generation cs.CV | cs.AIPDF

Hao Wei, Wenjin Qi, Dasen Dai, Minqing Zhang, Wu Yuan

TL;DR: 本文提出了IRIS系统，这是一个针对眼表疾病（OSD）的智能视觉语言系统，通过构建大规模、高质量的视觉问答数据集IRIS-120K，并采用主题发现树和场景驱动的数据生成范式来注入临床先验知识，从而显著提升了紧凑型视觉语言模型在OSD领域的细粒度理解和推理能力。

Details

Motivation: 当前通用视觉语言模型在眼表疾病等专业领域的临床推理能力，受限于缺乏高质量、多模态的指令微调数据，因此需要打破这一数据瓶颈。

Result: 通过在IRIS-120K数据集上对齐一个紧凑的40亿参数视觉语言模型，IRIS在眼表疾病理解任务上取得了最先进的性能，全面超越了参数规模高达340亿的通用和专用医学视觉语言模型。

Insight: 核心创新在于提出了一个协同数据生成范式，通过主题发现树（TFT）将视觉特征层次化锚定到精确的解剖和病理概念以强化医学演绎逻辑，并结合场景驱动策略合成角色自适应的临床对话以确保实用性泛化。这证明了结构化知识注入比单纯的参数缩放更有效，为在移动边缘设备上部署资源高效、专家级AI进行可扩展的OSD筛查提供了可能。

Abstract: While Large Vision-Language Models (VLMs) demonstrate remarkable generic capabilities, their clinical reasoning in specialized domains like ocular surface diseases (OSDs) is severely hindered by a paucity of high-fidelity, multimodal instruction-tuning data. To dismantle this data bottleneck, we introduce IRIS, an Intelligent Recognition and Interaction System tailored for fine-grained OSD understanding via external eye photography. First, we curate IRIS-120K, the largest and most comprehensive OSD visual question-answering (VQA) dataset to date. Crucially, to overcome the semantic shallowness of conventional image-caption pairs, we propose a synergistic data generation paradigm to explicitly inject clinical priors. Our data engine operates via a dual-branch framework: 1) a Topic Finding Tree (TFT) that hierarchically anchors visual features to precise anatomical and pathological concepts, enforcing rigorous medical deduction logic; and 2) a Scene-driven strategy that synthesizes role-adaptive clinical dialogues to ensure pragmatic generalization. By explicitly aligning a compact 4B-parameter VLM on this structurally enriched corpus, IRIS achieves state-of-the-art performance, comprehensively outperforming both generalist and specialized medical VLMs with up to 34B parameters. Our findings underscore that structured knowledge injection profoundly prevails over sheer parameter scaling, unlocking the potential for resource-efficient, expert-level AI deployment on mobile edge devices for scalable OSD screening. Code, datasets, and model weights will be publicly released by this repo.

[172] Event Detection in Videos: A Framework for the Development of New Methods cs.CVPDF

Anastasia Zakharova, Thierry Bouwmans, Anthony Cioppa, Adrien Deliège, Antonio Greco

TL;DR: 本文提出一个用于视频事件检测方法开发的严谨框架，旨在解决该领域因缺乏大规模数据集和标准化评估方法而导致的比较偏差问题。该框架基于数据集、性能评估和方法部署场景三大支柱，帮助研究人员在多样化的现有方法中定位自己的工作。

Details

Motivation: 视频事件检测领域存在方法多样但缺乏大规模数据集和严格性能评估标准的问题，导致方法比较和发展存在偏差，因此需要建立一个系统化框架来规范新方法的开发。

Result: 论文未在摘要中提及具体的定量实验结果或基准测试，但提出了一个基于数据集、性能评估和部署场景的框架，旨在为未来方法提供标准化开发路径。

Insight: 创新点在于首次构建了一个系统化的视频事件检测方法开发框架，强调数据集标准化、评估严谨性和实际部署场景的结合，为领域提供了可重复比较的基础结构。

Abstract: Event detection tasks in videos, the most important aspect of video surveillance, aim to detect events either at the pixel-level, frame-level, or clip-level. Plenty of methods intended for event detection in different environments, for various applications, and within different acquisition techniques were introduced. Naturally, the attempts were made as well to classify these algorithms in terms of detection of performance or in terms of real-time abilities. Nevertheless, the lack of a large-scale dataset as well as rigorous performance evaluation methods have biased such comparisons as well as the development of the methods. Given the diversity of existing approaches, we believe it is essential for researchers to position their work within such a rich landscape. Thus, we propose a rigorous framework for developing new methods in event detection for videos. Specifically, this framework is based on three main pillars: datasets, performance evaluation, and scenarios for deploying methods.

Yaxuan Li, Jiarui Zeng, Shaofei Huang, Zhedong Zheng

TL;DR: 本文提出DreamNav，一种由粗到精的扩散细化空中视觉伺服框架，用于解决无人机在最后10米范围内的精确导航问题。该方法通过两阶段策略：首先使用三角参数化进行鲁棒旋转估计以缓解角度周期性带来的优化不稳定，然后利用预训练的世界模型通过视觉想象选择最小化与目标视觉差异的轨迹。

Details

Motivation: 解决无人机在仅使用单目视觉进行最后10米精确自主导航时面临的尺度模糊、旋转不连续和细粒度空间推理等挑战，现有方法在大视角变化下容易失败或缺乏对未见环境的泛化能力。

Result: 在基于University-1652数据集构建的大规模基准PairUAV（包含72个场景的480万图像对）上进行广泛实验，DreamNav在准确性和泛化性上超越了强大的视觉伺服和基础模型基线，并实现了对未见场景的零样本迁移。

Insight: 创新点在于将视觉伺服任务分解为粗估计和扩散细化的两阶段流程，结合三角参数化解决旋转估计的周期性问题，并利用预训练世界模型进行基于视觉想象的轨迹优化，提升了在复杂环境下的鲁棒性和泛化能力。

Abstract: In this work, we study the last-meter precision navigation for UAVs, e.g., autonomously reaching a target within the final 10 meters using monocular vision. This task is challenging due to scale ambiguity, rotation discontinuities, and the need for fine-grained spatial reasoning. Existing methods often fail under large viewpoint changes or lack generalization to unseen environments. To this end, we propose DreamNav, a coarse-to-fine diffusion-refined aerial visual servoing framework. In the first coarse-estimation stage, a robust regression policy employs a trigonometric parameterization to predict rotation by jointly modeling sine and cosine components, effectively mitigating optimization instabilities caused by angular periodicity. Given this coarse estimate, the second diffusion-refined stage utilizes a pre-trained world model to simulate future visual observations for candidate actions, selecting the trajectory that minimizes visual discrepancy with the target through a process of visual imagination. To support rigorous evaluation, we contribute PairUAV, a large-scale benchmark comprising 4.8 million image pairs across 72 scenes, curated from the University-1652 dataset. Extensive experiments show DreamNav outperforms strong visual servoing and foundation model baselines in accuracy and generalization, with zero-shot transfer to unseen scenes.

[174] The Good, the Bad, and the Brittle: Benchmarking Robustness and Generalisation of Histopathology Foundation Models cs.CV | cs.LGPDF

Dhyey Yajnik, Amina Asif, Fayyaz Minhas

TL;DR: 该研究评估了12种病理学基础模型（PFMs）在临床相关扰动和分布偏移下的鲁棒性与泛化能力，发现PFMs普遍优于传统CNN，但模型规模扩大带来的性能提升存在边际递减效应，中等规模模型（如UNI2/Virchow-2）展现出与大型模型相当甚至更优的鲁棒性。

Details

Motivation: 旨在探究病理学基础模型的鲁棒性和泛化性是否已达到极限，并评估其在临床实际扰动和分布偏移下的可靠性。

Result: 使用鲁棒性评估与增强工具箱（REET）和基于非冗余K折验证（NR-Kfold）协议进行测试，结果显示PFMs在准确性和鲁棒性上均优于CNN基线，但模型参数增加带来的鲁棒性提升有限；NR-Kfold分析表明训练-测试分布差异会导致系统性准确率下降和变异性增加。

Insight: 创新点包括提出扰动性能指数（PPI）来量化模型在扰动下的性能趋势，以及采用非冗余K折验证来评估分布偏移；研究指出下一代病理学基础模型应更注重数据质量、多模态信息和领域对齐，而非单纯增加参数规模，以实现真正的临床可靠性。

Abstract: How robust and generalisable are pathology foundation models and have their scaling limites been reached? We benchmarked twelve pathology foundation models (PFMs) and ResNet baselines using our Robustness Evaluation and Enhancement Toolbox (REET) across eleven clinically realistic perturbations and a dissimilarity-driven Non-Redundant K-fold validation (NR-Kfold) protocol. We introduce a Perturbation Performance Index (PPI) to summarise accuracy trends under controlled perturbation sweeps and analyse robustness scaling with parameter count. We show that PFMs consistently outperform CNNs in both robustness and domain generalisation, yet model scaling shows diminishing returns: mid-sized models such (UNI2/Virchow-2 etc.) achieve comparable or greater resilience than larger systems. NR-Kfold analysis further reveals systematic accuracy loss and increased variability when training-test similarity is broken, underscoring the need for explicit distribution-shift evaluation. These findings suggest that the next generation of pathology foundation models must prioritise data quality, multimodality information and domain alignment over parameter count to achieve genuine clinical reliability.

[175] ResearchStudio-Reel: Automate the Last Mile of Research from Paper to Poster, Video, and Blog cs.CV | cs.AI | cs.HC | cs.MA | cs.MMPDF

Lingao Xiao, Yalun Dai, Yangyu Huang, Qihao Zhao, Wenshan Wu

TL;DR: 本文提出了ResearchStudio-Reel系统，旨在自动化研究传播的‘最后一英里’，即从论文自动生成可编辑的海报、演讲视频和博客文章。系统通过一个共享的上游提取器和多个技能组合，确保生成内容的事实一致性，并支持在PowerPoint或Word中编辑。

Details

Motivation: 解决研究传播中手动制作海报、视频和博客的低效问题，现有自动化方法通常孤立处理每个输出、无法编辑、且依赖软性VLM偏好评分导致质量瓶颈。

Result: 在Paper2Poster基准测试中，系统在海报美观性和信息性上均优于现有自动化系统和前沿LLM，在84%至93%的论文中总体胜出，且是唯一能同时生成三种可编辑产物的流程。

Insight: 创新点在于将‘最后一英里’构建为技能组合，采用共享提取器减少重复工作，并通过确定性原语和硬性渲染门控确保质量；系统还实现了视频、幻灯片、字幕和博客的交互式同步浏览。

Abstract: Research dissemination, turning a paper into a poster, a talk video, and a blog post, is still a manual last mile. Prior automation treats each artifact in isolation that each re-extract the paper from scratch, usually ship one-way renders the author cannot reopen in PowerPoint or Word, and gates quality on soft VLM-preference scores that plateau while load-bearing sections still read as empty. We argue this last mile is best built as a composition of skills: thin agent-readable contracts that share one upstream extractor and wrap deterministic primitives in a measured-fill loop whose exits are hard pass/fail render gates. We instantiate this as ResearchStudio-Reel, five Claude Code and Codex skills organized into one shared extractor (Paper2Assets), three editable generators (Paper2Poster, Paper2Video, Paper2Blog), and one interactive convergence layer (Paper2Reel). Paper2Assets extracts each paper once into a shared bundle that can be reused by every downstream skill; The three generators produce a print-ready poster, a synchronized talk video, and a bilingual blog that stay factually consistent and round-trip through PowerPoint or Word; Paper2Reel then binds all three into a self-contained HTML viewer whose section-level clicks jump the video, slides, captions, and blog to matching content. On the Paper2Poster benchmark, our posters lead every aesthetic and information sub-criterion against both prior automated systems and single-shot frontier LLMs, surpassing the authors’ own on aesthetics under two held-out VLM judges and winning overall on 84% to 93% of papers; capability audits further show that, by uniquely pairing narration-aligned on-slide highlights with a bilingual blog gated by layout-aware DOCX repair, ResearchStudio-Reel is the only pipeline to ship all three editable artifacts. Project is available at https://aka.ms/ResearchStudio

[176] Transferability Between Understanding and Generation in Unified Multimodal Models cs.CV | cs.AIPDF

Jiwon Kang, Heeji Yoon, Jaewoo Jung, Jaewon Min, Minkyeong Jeon

TL;DR: 本文研究了统一多模态模型（UMMs）中图像理解与生成任务之间的可迁移性，即训练一个任务上的能力是否能无监督地提升另一任务上的相同能力。通过实验发现，可迁移性取决于模型架构：完全共享Transformer主干和统一视觉编码器的模型表现出稳定的跨任务迁移，而松散耦合的设计则几乎没有。基于此，作者提出了一种实用的训练策略：通过训练理解任务并让其迁移到生成任务，来提升特定生成能力（如计数），同时最小化分布偏移。

Details

Motivation: 尽管统一多模态模型（UMMs）将图像理解和生成集成在单一架构中，但这两个任务如何相互作用仍缺乏深入研究。本文旨在探索UMMs中的可迁移性，以理解任务间的交互机制，并寻求更有效的训练方法。

Result: 在三个具体能力（计数、空间关系、文本识别/生成）上验证了所提策略的有效性，表明跨任务可迁移性可以在UMMs中被系统性地利用，从而提升特定生成性能。

Insight: 核心创新点在于揭示了UMMs中理解与生成任务间可迁移性的存在及其对架构的依赖性，并据此提出了一种通过训练理解任务来间接优化生成任务的新颖训练策略，这有助于在提升特定能力时避免直接微调生成任务可能带来的分布偏移问题。

Abstract: Unified Multimodal Models (UMMs) integrate image understanding and generation within a single architecture, yet how the two tasks interact remains understudied. We investigate $\boldsymbol{\mathsf{transferability}}$ in UMMs: whether training a capability on one task improves the same capability on the other without explicit supervision. Through controlled experiments, we empirically find that transferability depends on architecture-models with fully shared transformer backbone and a unified visual encoder exhibit consistent cross-task transfer, while loosely coupled designs show little or none. Leveraging this transferability, we propose a practical training strategy. The most straightforward way to improve a target generative capability (e.g., counting) is to fine-tune generation directly, but this can degrade visual quality due to distribution shift. Instead, we train the corresponding understanding task and let it transfer into generation, which improves capability-specific generative performance while minimizing distribution shift. We validate this across three capabilities-counting, spatial relation, and text recognition/generation-showing that cross-task transferability can be systematically exploited in UMMs.

[177] EVAS: Efficient Multimodal Temporal Forgery Localization via Audio-Visual Synergy and Steered Boundary Calibration cs.CVPDF

Shen Shen, Quan Zhang, Dan Jiang, Ke Zhang

TL;DR: 本文提出EVAS，一种用于长视频中稀疏伪造片段定位的端到端多模态框架。该框架通过多阶段音视频协同机制学习深度多模态取证表示，并引入边界感知细化策略进行边界校准，采用解耦训练范式提升泛化能力，同时结合轻量级HourglassFFN降低计算开销。

Details

Motivation: 针对长视频中稀疏分布的伪造片段定位难题，尤其是在篡改嵌入细微、跨模态信号弱且时间分散的情况下，需要可靠的多模态取证方法。

Result: 在三个基准数据集上的实验表明，EVAS在平均定位精度和平均召回率上达到了最先进水平（SOTA）。

Insight: 创新点包括多阶段音视频协同机制以捕获稀疏篡改的高阶语义痕迹，边界感知细化策略通过无效帧掩码抑制模糊区域并锐化过渡预测，以及解耦训练范式分离表示学习与推理目标以增强泛化稳定性。

Abstract: The rapid proliferation of artificial intelligence-generated content necessitates reliable multimodal forensics. Beyond video-level binary classification, precisely localizing sparsely distributed forged segments in long-form videos remains a critical challenge. This task is particularly difficult when manipulations are subtly embedded and cross-modal signals are weak and temporally diffuse. To address these challenges, we propose EVAS, an end-to-end multimodal framework for temporal forgery localization. At its core, a Multi-Stage Audio-Visual Synergy mechanism facilitates progressive cross-modal interaction to learn deep multimodal forensic representations and capture high-order semantic traces of sparse manipulations. Furthermore, we introduce a Boundary-Aware Refinement strategy to achieve steered boundary calibration. By incorporating invalid-frame masking, this strategy suppresses ambiguous regions and sharpens transition predictions. We adopt a decoupled training paradigm with auxiliary heads to disentangle representation learning from inference objectives, enhancing model generalization and stability. Additionally, a lightweight HourglassFFN is incorporated to reduce computational overhead. Extensive experiments demonstrate that EVAS achieves state-of-the-art average localization accuracy and average recall across three benchmark datasets, validating its effectiveness for fine-grained temporal forgery localization.

[178] Wan-Streamer v0.2: Higher Resolution, Same Latency cs.CV | cs.AI | cs.GR | cs.LGPDF

Lianghua Huang, Zhi-Fan Wu, Yupeng Shi, Wei Wang, Mengyang Feng

TL;DR: 本文介绍了Wan-Streamer v0.2，这是一个保持低延迟的端到端音视频交互模型的升级版本。该版本在保持约200毫秒模型端信号延迟和25 FPS的前提下，将交互输出流的分辨率从192x336提升至640x368，从而支持更清晰的中景智能体视觉呈现。为实现这一目标，模型采用了Thinker-Performer架构，其中Thinker作为单GPU低延迟路径处理流式感知和语言状态，而Performer则利用多GPU Ulysses风格的上下文并行组来处理高分辨率视频潜变量的生成。

Details

Motivation: 动机在于升级现有音视频交互模型，在保持原有低延迟特性的同时，显著提升输出视频流的分辨率，以支持更清晰、细节更丰富（如姿态、视线、手部、物体和局部场景布局）的实时对话智能体。

Result: 模型在保持约200毫秒模型端信号延迟（包含350毫秒双向网络预算后总远程交互延迟约为550毫秒）和25 FPS的条件下，成功将输出分辨率从192x336提升至640x368。

Insight: 创新点在于通过Thinker-Performer架构分离计算密集型任务：Thinker负责低延迟的流式感知和语言状态缓存，而Performer则利用多GPU和Ulysses通信模式并行处理高分辨率视频潜变量生成，从而在不增加用户可见延迟的前提下实现了分辨率的显著提升。这种将语言状态仅作为K/V条件传递给Performer的设计，避免了在Performer组内传输单独的语言序列，优化了通信效率。

Abstract: We present Wan-Streamer v0.2, a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model. v0.2 keeps the v0.1 modeling formulation, but raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS. The higher-resolution stream supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation. To support the larger visual stream without adding user-visible delay, v0.2 keeps the thinker as a single-GPU low-latency path for streaming perception, the short language/state Transformer pass that builds the generation cache, and final decoding. The performer becomes a multi-GPU Ulysses-style context-parallel group for the expensive next-unit latent generation. Each performer rank writes incoming K/V into a pre-sharded local cache. The long high-resolution latent video sequence is split across ranks for denoising and gathered through Ulysses communication, while the much shorter audio latent sequence is generated without sequence sharding. In this split, the thinker’s language/state computation reaches the performer only as K/V conditioning, so no separate language sequence has to be communicated inside the performer group. This concentrates additional hardware on visual generation while preserving the compact thinker-performer boundary, keeping total remote interaction latency at approximately 550 ms when a 350 ms bidirectional network budget is included.

[179] TrustCLIP: Learning Private Visual Features via Adversarial Reconstruction cs.CVPDF

Nikos Athanasiou, Ilya A. Petrov, Angela Yao, Shugao Ma, Eric Sauser

TL;DR: 论文提出TrustCLIP，一个对抗重建驱动的隐私保护框架，旨在防止视觉或视觉语言模型中的高级特征被生成模型逆向重建，从而泄露原始图像隐私。该方法通过优化特征投影，在保持下游任务性能的同时，显式地降低生成攻击者的重建质量。

Details

Motivation: 动机在于当前视觉和视觉语言模型提取的高级特征容易被生成模型逆向重建，导致原始图像隐私泄露，现有基于判别性隐私度量的防御方法未能有效应对这种生成式重建威胁。

Result: 在传统分类和多模态大语言模型（MLLM）管道中，TrustCLIP均能有效降低生成式逆向重建的保真度，同时保持下游任务性能。

Insight: 创新点在于将特征条件生成器作为显式的隐私对抗者，并直接针对生成式重建攻击进行优化，这提供了一种更直接、更有效的防御生成模型特征逆向攻击的新思路。

Abstract: Vision and vision-language models rely on high-level visual representations that are increasingly used across recognition, retrieval, and multimodal reasoning pipelines. However, recent advances in generative modeling have shown that such features can often be inverted, enabling realistic reconstructions of the underlying image and raising significant privacy risks. We revisit this problem through the lens of reconstruction and propose TrustCLIP, a reconstruction-driven framework that treats a feature-conditioned generator as an explicit privacy adversary. TrustCLIP learns a projection between encoder features and downstream modules that is explicitly optimized to degrade the reconstructions produced by generative attackers while retaining the necessary signals for downstream tasks. Unlike prior defenses that rely on discriminative privacy metrics, TrustCLIP directly optimizes against a generative reconstruction attacker, targeting a threat not captured by standard evaluation protocols. We demonstrate its effectiveness in both conventional classification and multimodal large language model pipelines. Across these settings, TrustCLIP consistently reduces the fidelity of generative inversions while maintaining downstream task performance. Project page: https://atnikos.github.io/trustclip/

[180] UniSkip-Mamba: A Frequency-Aware State Space Model for Audio-Visual Temporal Forgery Localization cs.CV | cs.SDPDF

Cangjin Qiu, Quan Zhang, Dan Jiang, Ke Zhang

TL;DR: 本文提出UniSkip-Mamba，一种用于音频-视频时序伪造定位（AV-TFL）的频率感知状态空间模型框架。该框架通过统一的跨模态序列融合保留相位关系，并利用创新的Skip-Scanning Mamba块实现频率感知正则化，引导模型专注于低/中频的判别性伪造模式，从而在精度、鲁棒性和推理速度上实现显著提升。

Details

Motivation: 随着AI生成内容的泛滥，复杂的多媒体篡改引发了恶意应用的严重关切，使得AV-TFL成为紧迫的研究前沿。现有方法在处理所有频率分量时缺乏区分，导致对高频噪声过拟合，且在真实世界数据退化下鲁棒性有限。

Result: 该方法在LAV-DF数据集上达到63.4%的AP@0.95（提升9.8%），在AV-Deepfake1M数据集上达到63.58%的mAP（提升14.32%），实现了SOTA性能，且推理速度提升6倍。

Insight: 核心创新点在于通过系统的频域分析发现判别性伪造模式集中于低/中频（0-0.15），并据此设计了频率感知的Skip-Scanning Mamba块（采用Group-Scan-Merge机制）进行正则化，从信号处理角度为模型在精度和鲁棒性上的提升提供了理论依据。

Abstract: With the proliferation of AI-generated content, sophisticated multimedia manipulation has raised critical concerns about malicious applications such as opinion manipulation and evidence fabrication, making Audio-Visual Temporal Forgery Localization (AV-TFL) an urgent research frontier. Existing TFL methods have progressed along two main paradigms: Transformer-based temporal modeling and channel-wise multimodal fusion. While these approaches capture temporal dependencies and cross-modal correlations, they process all frequency components indiscriminately, leading to overfitting on high-frequency noise and limited robustness under real-world data degradation. Through systematic frequency domain analysis, we find that forgery-discriminative patterns concentrate in the low/mid-frequency range (normalized frequency 0-0.15), while high-frequency components primarily introduce noise, removing them even improves detection performance by +1.4%. Based on this phenomenon, we propose UniSkip-Mamba, a frequency-aware State Space Model framework that incorporates Unified Multimodal Sequence Fusion to preserve cross-modal phase relationships, and Skip-Scanning Mamba Blocks that implement frequency-aware regularization through a novel Group-Scan-Merge mechanism, naturally biasing learning toward discriminative low/mid-frequency patterns (0-0.15) while maintaining representational completeness. We achieve state-of-the-art (SOTA) performance: 63.4% AP@0.95 on LAV-DF (+9.8% improvement) and 63.58% mAP on AV-Deepfake1M (+14.32% improvement), with 6x faster inference. Our frequency-domain analysis provides theoretical justification from a signal processing perspective for why skip-scanning inherently improves both accuracy and robustness.

[181] A non-invasive video-based method for individual identification of wildlife using gait dynamics cs.CVPDF

Muhammad Aamir, Matthew Wijers, Sangyun Shin, Andrew Loveridge, Andrew Markham

TL;DR: 本文提出了一种基于视频的非侵入式野生动物个体识别方法，利用步态动力学作为独特的行为特征。该方法采用全自动流程，结合SAM3进行动物分割，ResNet18提取空间特征，VideoPrism建模时序运动，并通过余弦相似度进行个体聚类。实验表明，该方法在多物种视频数据上实现了良好的个体内一致性和个体间区分度。

Details

Motivation: 解决野生动物个体识别中传统侵入式方法（如物理标记）的局限性，以及现有步态分析在野生动物领域应用受限的问题，旨在开发一种可扩展、非侵入的视频识别方案。

Result: 在多源野生动物视频数据上的实验显示，该方法具有强个体内一致性和清晰的个体间分离；余弦相似度分布和轮廓分数等定量结果证实了其有效性。

Insight: 创新点在于将SAM3分割、深度时空表示学习（ResNet18+VideoPrism）与余弦相似度聚类结合，构建了首个全自动视频步态识别流程，为可扩展生态监测提供了新思路。

Abstract: Gait is a distinctive behavioral characteristic that enables non-invasive individual identification without requiring physical interaction with an animal. While gait-based analysis has been extensively studied in humans, its application to wildlife remains limited due to environmental variability and the lack of scalable identification methods. This paper presents a fully automated, video-based pipeline for wildlife gait analysis and individual identification using deep spatiotemporal representation learning. The proposed pipeline uses the Segment Anything Model 3 (SAM3) to generate high-quality RGB and binary silhouette masks, robustly isolating animals from complex natural backgrounds. Segmented video sequences are processed using a convolutional neural network (ResNet18) for spatial feature extraction and a transformer-based video model (VideoPrism) for temporal motion modeling. Both models are fine-tuned using a classification objective and subsequently used as feature extractors to generate discriminative gait representations. Cosine similarity is then used to compare gait signatures, enabling similarity-based clustering of individuals without reliance on physical markings or invasive tagging. Experiments conducted on multi-source wildlife video data across multiple species demonstrate strong intra-individual consistency and clear inter-individual separation. Quantitative results using cosine similarity distributions and silhouette scores confirm the effectiveness of the proposed method. These findings demonstrate that gait dynamics provide a viable, non-invasive approach for individual identification in wildlife and highlight the potential of video-based deep learning pipelines for scalable ecological monitoring.

[182] CRISP: A Spatiotemporal Camera-Radar Backbone for Driving via Forecasting-Based World-Model Pretraining cs.CV | cs.AI | cs.LG | cs.ROPDF

Jingyu Song, Yi Liu, Katherine A. Skinner

TL;DR: CRISP是一种基于预测世界模型预训练的时空相机-雷达融合骨干网络，通过历史多视角图像和雷达扫描数据学习统一的鸟瞰图表示，以预测未来激光雷达点云作为监督信号。该模型仅在预训练阶段使用激光雷达作为特权监督，部署时仅需相机和雷达传感器。

Details

Motivation: 现有相机-雷达融合模型通常依赖任务特定监督进行训练，限制了可重用表示学习的发展。本文旨在通过基于预测的表示学习，构建一个可迁移到多种下游任务的通用时空融合骨干网络。

Result: 在nuScenes数据集上的实验表明，CRISP显著提升了长时域点云预测性能，并能有效迁移到3D检测、跟踪、在线建图、运动预测、未来占据预测和规划等下游任务，展现出优越的泛化能力。

Insight: 创新点包括增强的雷达编码器、雷达增强的时序自注意力机制以及带模态创新门控的多模态特征渲染，这些设计将雷达距离和多普勒线索注入鸟瞰图时序传播，并允许BEV令牌选择性融合相机和雷达证据，为实用传感器配置下的可扩展驾驶表示学习提供了新路径。

Abstract: Camera-radar (CR) fusion is a practical sensing configuration for autonomous driving, but existing models are typically trained with task-specific supervision, limiting reusable representation learning. We present CRISP, a spatiotemporal CR backbone pretrained through forecasting-based representation learning. Given historical multi-view images and radar sweeps, CRISP learns a unified bird’s-eye-view (BEV) representation by predicting future LiDAR point clouds. LiDAR is used only as privileged supervision during pretraining; the deployed model requires only camera and radar. To make forecasting-based pretraining effective for CR fusion, CRISP introduces an enhanced radar encoder, radar-enhanced temporal self-attention, and multimodal feature rendering with modality innovation gating. These components inject radar range and Doppler cues into BEV temporal propagation and allow BEV tokens to selectively incorporate camera and radar evidence. Experiments on nuScenes show that CRISP improves long-horizon point cloud forecasting and transfers effectively to downstream tasks, including 3D detection, tracking, online mapping, motion forecasting, future occupancy prediction, and planning, suggesting that predictive CR pretraining is a promising path toward scalable driving representations under practical sensor configurations. The project website is https://umfieldrobotics.github.io/CRISP.

[183] Explainable Novel Category Discovery in Semantic Concept Space cs.CV | cs.AIPDF

Ifrat Ikhtear Uddin, Yang Zhou, KC Santosh, Longwei Wang

TL;DR: 本文提出了xNCD，一种可解释的新类别发现框架。该框架通过在结构化的语义概念空间中直接执行基于表示的学习和伪标签分配，来解决现有方法在潜在特征空间中操作导致结果不透明、缺乏语义解释的问题。

Details

Motivation: 现有新类别发现方法通常在潜在特征空间中操作，虽然能准确分离新类别，但无法提供定义每个发现类别的语义证据，缺乏可解释性。

Result: 在CIFAR-10、CIFAR-100和CUB-200数据集上的实验表明，xNCD在保持强大发现性能的同时提供了内在解释。在任务无关评估下，xNCD在CIFAR-10上达到92.63%的总准确率（接近UNO的93.4%），并将CIFAR-100的总准确率从73.2%提升至76.45%，且是唯一能提供人类可读的簇级和实例级解释的方法。

Insight: 核心创新在于将新类别发现过程路由至语义概念瓶颈空间，通过将视觉特征与预训练多模态模型的视觉-语言相似性先验对齐来学习无标签概念表示，并使用统一的自标签目标。这从理论上限制了假设空间，使发现的类别天然具有稳定的概念签名和实例级概念证据，从而具备可解释性。

Abstract: Novel category discovery aims to identify unseen classes from unlabeled data by transferring knowledge from labeled categories, but most existing methods perform discovery in opaque latent feature spaces. As a result, they may separate novel categories accurately while providing little insight into what semantic evidence defines each discovered group. We propose xNCD, an explainable novel category discovery framework that performs both representation-based discovery and pseudo-label assignment directly in a structured semantic concept space. Instead of clustering arbitrary deep features, xNCD learns a label-free concept representation by aligning visual features with vision-language similarity priors from pretrained multimodal models, and then applies a unified labeled-and-unlabeled self-labeling objective over concept-space logits. This design makes each discovered category explainable by construction through stable concept signatures and instance-level concept evidence. Theoretically, we show that routing discovery through a semantic concept bottleneck induces a strict restriction of the feature-space hypothesis class, excluding a large family of unconstrained decision rules and biasing induced partitions toward semantically interpretable concept coordinates. Experiments on CIFAR-10, CIFAR-100, and CUB-200 demonstrate that xNCD preserves strong discovery performance while providing intrinsic explanations. Under task-agnostic evaluation, xNCD achieves 92.63% overall accuracy on CIFAR-10, close to UNO’s 93.4%, and improves CIFAR-100 overall accuracy from 73.2% to 76.45%, while being the only compared method that provides human-readable cluster- and instance-level explanations.

[184] RAF: Reliability-Aware Fusion of Camera, LiDAR, and 4D RADAR for Robust 3D Object Detection in Adverse Weather cs.CVPDF

Heejun Park, Jaeseok Jeong, Kuk-Jin Yoon

TL;DR: 本文提出了一种可靠性感知融合（RAF）方法，用于在恶劣天气条件下实现鲁棒的3D目标检测。该方法通过显式监督像素级可靠性估计，有效融合相机、激光雷达和4D雷达等多模态传感器数据，以弥补单一传感器在恶劣天气下的性能下降。

Details

Motivation: 恶劣天气条件下，激光雷达和4D雷达的点云会因反射减弱而变得稀疏，而相机图像也可能因雨雪遮挡镜头而变得不可靠。现有方法中，相机融合的置信度图缺乏直接监督，仅通过检测损失学习，导致不可靠视觉线索的抑制效果有限。

Result: 在K-Radar和VoD数据集上的大量实验表明，集成RAF方法能持续提升基于激光雷达-雷达基线的检测精度，最高实现了+6.5 $AP_{BEV}$ 和 +7.4 $AP_{3D}$ 的性能增益。

Insight: 创新点在于引入了对像素级可靠性估计的显式监督，为识别和抑制不可靠视觉线索提供了直接的学习信号。同时，框架利用了预训练的激光雷达-雷达网络，仅训练新增的相机分支、BEV融合编码器和检测头，实现了高效的多模态融合。

Abstract: Robust 3D object detection in adverse weather conditions is challenging due to sensor limitations. Although combining complementary modalities such as LiDAR and 4D RADAR has shown promise, the sparsity of these sensors becomes apparent in adverse weather with reduced reflections, leading to objects with few or no point cloud returns. To address this limitation, camera sensors provide visual cues even when LiDAR and RADAR signals are weakened. However, cameras themselves are also vulnerable to adverse weather, where some regions become unreliable due to snow or rain occluding the camera lens. While some camera-fusion methods designed for adverse weather learn to weigh image regions via confidence maps, these maps receive no direct supervision and are learned solely through the detection loss. We introduce Reliability-Aware Fusion (RAF), which explicitly supervises per-pixel reliability estimation and provides a direct learning signal for identifying and suppressing unreliable visual cues. Our framework leverages pretrained LiDAR-RADAR networks, keeping their backbones frozen while only training the added camera branch, BEV fusion encoder, and detection head. Extensive experiments on the K-Radar and VoD datasets demonstrate that integrating RAF consistently improves detection accuracy over LiDAR-RADAR baselines, achieving up to +6.5 $AP_{BEV}$ and +7.4 $AP_{3D}$ gains. Code is available at https://github.com/parkie0517/RAF.

[185] QSVideo: Query-Conditioned Semantic Temporal Retrieval for Video Understanding cs.CVPDF

Wei Ao, Lan Wang, Vishnu Naresh Boddeti

TL;DR: 本文提出QSVideo，一个用于视频理解的查询条件语义时序检索统一框架，旨在解决现有多模态检索方法在相关性估计偏差、多样性不足和时序崩溃方面的局限性。该框架通过查询条件语义排序器（QSRanker）将任意问题转化为检索友好查询，并沿对象、动作和位置维度估计结构化相关性，进而设计QSRetrieval联合优化相关性和多样性以选择信息更丰富的帧，并针对长视频和流视频提出时序对齐策略以提高证据召回率。

Details

Motivation: 现有视觉语言模型（VLMs）在视频理解中性能随视频时长增加而下降，因为与查询无关的视频片段会干扰其语言组件；现有多模态检索方法存在相关性估计偏差、多样性有限和时序崩溃的问题，需要系统性地解决相关性、多样性和时序建模的挑战。

Result: 在长视频和流视频基准测试上的广泛实验表明，QSVideo在严格的帧数限制约束下显著提升了视频VLM的性能。

Insight: 创新点包括：将任意问题重构为检索友好查询的查询条件语义排序器（QSRanker），沿对象、动作和位置维度进行结构化相关性估计；联合优化相关性和多样性的QSRetrieval框架，以选择信息更丰富的帧；以及针对长视频和流视频的定制时序对齐策略，以提高证据召回率。从客观角度看，该工作系统性地整合了相关性、多样性和时序建模，为视频理解中的检索问题提供了统一的解决方案。

Abstract: The performance of vision-language models (VLMs) in video understanding declines with increasing video duration, as video moments unrelated to the query confuse their language components. Multimodal retrieval has emerged as a critical component of video understanding, addressing this challenge by localizing key visual evidence. However, existing multimodal retrieval methods suffer from biased relevance estimation, limited diversity, and temporal collapse. In this paper, we propose QSVideo, a unified framework that systematically addresses relevance, diversity, and temporal modeling in video retrieval. We first introduce a query-conditioned semantic ranker, QSRanker, which reformulates arbitrary questions into retrieval-friendly queries and estimates structured relevance along object, action, and location dimensions. Building upon this, we design QSRetrieval to jointly optimize relevance and diversity for more informative frame selection. Moreover, we propose temporal alignment strategies tailored for both long and streaming videos to improve evidence recall. Extensive experiments on long and streaming video benchmarks demonstrate that QSVideo greatly enhances video VLM performance under strict frame limit constraints. The code is available at https://github.com/human-analysis/QSVideo.

[186] TORINO: Token Reduction via Interpretable Concept Overlap in Vision-Language Models cs.CV | cs.AIPDF

Riccardo Renzulli, Gabriele Spadaro, Shruthi Gowda, Alaa Eddine Mazouz, Van-Tam Nguyen

TL;DR: 本文提出了TORINO，一种无需微调、即插即用的视觉语言模型（VLM）自适应视觉令牌缩减框架。它利用稀疏自编码器（SAE）将视觉令牌投影到可解释的隐空间，通过分析令牌间共享概念激活的“概念重叠”程度来对令牌进行分组，并在组内通过剪枝或合并来移除冗余，从而动态适应输入图像的复杂性。

Details

Motivation: 现有视觉语言模型的巨大计算成本主要源于输入语言模型的视觉令牌数量过多，而现有的令牌缩减方法依赖于注意力分数或成对相似性，缺乏对每个令牌的显式语义表示。

Result: 在多个视觉语言基准测试上的实验表明，TORINO在显著减少视觉令牌数量的同时，实现了有利的效率-精度权衡，性能损失最小。

Insight: 核心创新在于利用稀疏自编码器构建可解释的语义概念空间，并基于“概念重叠”这一新度量来指导令牌缩减，这为理解令牌语义关系提供了新视角；同时，其自适应动态缩减机制优于固定预算方法，提供了统一的剪枝与合并框架。

Abstract: Vision-Language Models (VLMs) have demonstrated impressive capabilities across different tasks, but their computational cost is dominated by the large number of visual tokens fed to the language model. Existing token reduction methods rely on attention-based scores or pairwise similarity, without an explicit semantic representation of each token. We introduce TORINO (TOken Reduction via Interpretable coNcept Overlap), a plug-and-play framework for adaptive visual token reduction in VLMs that requires no fine-tuning of the underlying model. TORINO leverages Sparse Autoencoders (SAEs) to project visual tokens into an interpretable latent space where token relationships can be analyzed through shared concept activations. Specifically, we define concept overlap as the degree of agreement between active SAE latents and use it to group tokens that share semantic content. Reduction within each group is then performed by either pruning or merging, providing a unified framework that preserves semantically important visual information while removing redundancy. Unlike fixed-budget approaches, TORINO dynamically adapts the reduction rate to input complexity, allowing different images to retain different numbers of tokens. Experiments across multiple vision-language benchmarks show that TORINO achieves favorable efficiency-accuracy trade-offs, reducing the number of visual tokens with minimal performance loss.

[187] G2VD: Generalizable AI-Generated Video Detection via Counterfactual Intervention and Causal Disentanglement cs.CV | cs.AIPDF

Meng Du, Hongchang Chen, Ran Li, Junjie Zhang, Qi Ouyang

TL;DR: G2VD是一个通过反事实干预和因果解耦实现可泛化AI生成视频检测的框架。它利用变分自编码器生成受控的反事实样本，并通过频域和像素域对齐来引导检测器关注生成器固有的伪造痕迹，同时设计了一个因果解耦分类器来分离任务相关线索与领域特定偏差。

Details

Motivation: 现有AI生成视频检测方法在域内评估表现良好，但在面对未见生成器时性能显著下降，主要原因是检测器依赖于领域特定的虚假线索（如生成器指纹和风格）而非内在伪造痕迹，存在捷径学习问题。

Result: 在四个公共数据集上，G2VD展现出强大的平均跨域性能，并在匹配骨干网络上获得一致提升；在具有挑战性的GenVidBench跨域设置中，其准确率超过90%，AUC接近0.95，且仅使用10%的原始训练数据就达到了这一性能。

Insight: 创新点在于通过反事实干预管道（CFIPipeline）生成对齐的样本以聚焦内在伪造特征，并结合因果解耦分类器与基于HSIC的独立性约束来分离任务相关线索和领域偏差，从而提升跨域泛化能力，为解决检测器对虚假线索的依赖提供了因果视角的方法。

Abstract: The rapid advancement of AI-generated videos poses increasing security risks and calls for robust detectors with strong cross-domain generalization. Although existing methods achieve promising results under in-domain evaluation, their performance often degrades substantially when tested on unseen generators. A key reason is shortcut learning, where detectors rely on domain-specific spurious cues, such as generator-dependent fingerprints and generation styles, instead of intrinsic forgery traces. To address this issue, we propose G2VD, a Generalizable AI-Generated Video Detection framework based on counterfactual intervention and causal disentanglement. First, G2VD introduces a counterfactual intervention pipeline (CFIPipeline) that generates controlled counterfactual samples via variational autoencoders (VAEs), followed by frequency-domain and pixel-domain alignment, thereby encouraging the detector to focus on generator-intrinsic cues. Building on this intervention process, we further design a causal disentanglement classifier consisting of two domain-anchored branches with distinct classification objectives, combined with an HSIC-based independence constraint to encourage the separation of task-relevant cues from domain-specific bias. Across four public datasets, G2VD shows strong average cross-domain performance and consistent gains over matched backbones. On the challenging GenVidBench cross-domain setting, it exceeds 90% accuracy and reaches an AUC close to 0.95. Notably, this performance is obtained using only 10% of the original training data. The code is available at https://github.com/dumeng98/G2VD.

[188] Aperture-aware Dispersion 5-D Light-field Imaging Spectrometer cs.CV | cs.CGPDF

Chenglong Huang, Tao Lv, Jianing Yang, Chongde Zi, Linsen Chen

TL;DR: 本文提出了一种孔径感知色散光场成像光谱仪（ADLIS），旨在解决高维视觉感知中系统小型化与感知维度提升之间的矛盾。该方法利用双折射晶体的光谱滤波特性，通过仅使用一个由石英晶体制成的相位板，实现了紧凑的角-光谱编码，从而能够从单个探测器重建完整的5维（空间、角度、光谱）光谱光场数据立方体。

Details

Motivation: 传统获取5维光谱光场（5D-SLF）数据立方体依赖于庞大昂贵的相机阵列，不便于广泛应用；而现有的单探测器系统因编码能力不足，在不同维度分辨率之间存在根本性的权衡限制。本文旨在设计一种兼具紧凑性和高分辨率的成像系统。

Result: 通过仿真数据训练和真实实验验证，该系统实现了鲁棒的高性能5D-SLF成像，同时保持了全空间分辨率。

Insight: 核心创新在于利用双折射材料的固有光谱滤波特性进行孔径复用调制，将设计范式从微透镜阵列的空间分割转变为编码集成，通过孔径编码将所有视点叠加到每个传感器像素上，并采用端到端（E2E）的孔径设计与重建优化框架，以实现全分辨率光场恢复。

Abstract: Enhancing perceptual dimensions while miniaturizing imaging systems presents significant challenges for high-dimensional visual sensing. Conventionally, the acquisition of the 5D (x,y,u,v,λ) spectral light field (5D-SLF) data cube relies on bulky and expensive camera arrays, which are impractical for widespread application. Existing single-detector systems are fundamentally limited by a trade-off between the resolutions of different dimensions owing to insufficient coding capabilities. Here we introduce an Aperture-aware Dispersion Light-field Imaging Spectrometer (ADLIS), that targets a synergy between compactness and resolution through aperture-multiplexed modulation, leveraging the inherent spectral-filtering properties of birefringent material. Using only a manufacturing-friendly and cost-effective phase plate made of birefringent quartz crystal, the aperture of the proposed ADLIS enables compact angular-spectral encoding that is highly sensitive to both the incident angle and spectrum of incoming light. In contrast to the viewpoint-separation approach of microlens arrays, ADLIS employs aperture encoding to superimpose all viewpoints onto each sensor pixel. This shifts the design paradigm from spatial division to encoding integration, aiming to achieve full-resolution light field recovery. Thus, we develop the Aperture-aware Dispersion Light-field Imaging (ADLI) framework, which optimizes the aperture design and 5D-SLF reconstruction in an end-to-end (E2E) manner. Trained by simulation data and validated through real-world experiments, our system achieves robust high-performance 5D-SLF imaging while maintaining full spatial resolution.

[189] Hierarchical Evidence-Driven Reasoning for Long Document Understanding cs.CV | cs.AIPDF

Junyu Xiong, Yonghui Wang, Rongjian Gu, Chenyu Liu, Bing Yin

TL;DR: 本文提出HIEVI-RAG，一个用于封闭域文档理解的分层、证据驱动的多模态RAG框架。它通过将复杂查询分解为原子子问题、粗粒度页面检索、细粒度页面验证以及基于记忆的迭代生成四阶段流程，解决了现有多模态RAG中检索干扰页面和单次检索依赖导致的级联错误问题。在四个基准测试上的评估表明，该框架显著优于现有开源基线，平均准确率超出最强基线8.05%。

Details

Motivation: 解决现有多模态检索增强生成（RAG）在长文档理解中面临的两个关键挑战：一是标准语义相似性检索器经常获取主题重叠但无答案的干扰页面，误导下游生成；二是僵化的单次处理流程严重依赖初始检索成功，任何核心证据的遗漏都会导致级联错误。

Result: 在四个基准测试上进行了广泛评估，结果表明该框架具有强大的效能和协同作用，显著优于现有开源基线，平均准确率超出最强报告基线8.05%。

Insight: 创新点在于提出了一个分层、证据驱动的四阶段协同处理流程，特别是引入了使用GRPO训练的专用多页面验证器EVIAGENT进行跨页面细粒度验证，以及利用累积子问题上下文进行多轮动态推理的记忆引导迭代生成机制。这从系统设计层面提升了RAG对长文档、多模态、多跳推理任务的鲁棒性和准确性。

Abstract: Retrieval-Augmented Generation (RAG) streamlines long-document understanding by leveraging retrieval mechanisms to restrict input images to a highly curated subset. However, existing multimodal RAG pipelines primarily face two critical challenges: first, standard semantic similarity retrievers frequently fetch topically overlapping yet answer-void distractor pages that mislead downstream generation; second, rigid single-pass pipelines heavily depend on initial retrieval success, where any omission of core evidence inevitably causes cascading errors. To address these challenges, we introduce HIEVI-RAG, a hierarchical, evidence-driven multimodal RAG framework for closed-domain document understanding. HIEVI-RAG systematically factorizes complex queries into a cooperative four-stage pipeline: (1) hierarchical question decomposition to break multi-hop root queries into atomic child questions; (2) coarse visual page retrieval leveraging a multimodal retriever to fetch candidate pages based on semantic similarity; (3) fine-grained page verification via EVIAGENT, a specialized multi-page verifier trained with GRPO to execute cross-page reasoning over multi-image blocks; and (4) memory-guided iterative generation that leverages accumulated sub-question context to execute multi-round, dynamic reasoning over the prioritized sequence. Extensive evaluations across four benchmarks demonstrate the robust efficacy and synergy of our framework, which significantly outperforms existing open-source baselines and exceeds the strongest reported baseline by an average of 8.05% in accuracy.

[190] PixelPilot: Scalable Vision-Language-Action Models for End-to-End Autonomous Driving cs.CVPDF

Pin Tang, Guoqing Wang, Xiangxuan Ren, Zhongdao Wang, Guodongfang Zhao

TL;DR: 本文提出PixelPilot，一种用于端到端自动驾驶的新型视觉-语言-动作模型（VLA）。它采用解耦的规划与提升范式，将场景理解和轨迹预测重新定义为与传感器无关的图像平面2D到2D任务，以促进跨异构数据集的可扩展训练，并在推理时将2D轨迹确定性地提升到3D空间。

Details

Motivation: 现有VLA通常从2D图像预测和优化3D轨迹，但这种方法与相机参数固有地纠缠，导致跨异构驾驶数据集的数据可扩展性有限，并且直接在3D空间中优化会导致模型严重收敛于依赖自车状态而非视觉场景理解的平凡解。

Result: 大量实验表明，PixelPilot在开环和闭环设置下均达到了最先进的性能，验证了其卓越的可扩展性和视觉推理能力。

Insight: 核心创新在于解耦的规划与提升范式，将规划任务与相机参数解耦，提升了数据可扩展性。同时，提出了知识注入的策略学习方法，通过Group Relative Policy Optimization（GRPO）应用密集的中间奖励，以强制实现从视觉感知到空间规划的严格因果链，确保充分利用视觉线索。

Abstract: Vision-Language-Action Models (VLAs), which leverage the advanced reasoning capabilities of Vision-Language Models (VLMs), show promising generalization in complex autonomous driving scenarios. Existing VLAs typically predict and optimize 3D trajectories from 2D images. While intuitive, this 2D-to-3D prediction is inherently entangled with camera parameters, leading to limited data scalability across heterogeneous driving datasets. Moreover, directly optimizing in 3D space induces severe convergence to trivial solutions, where VLAs rely on ego-status rather than visual scene understanding. To address these issues, we propose PixelPilot, a novel VLA featuring a decoupled planning and lifting paradigm. In the planning phase, PixelPilot reformulates scene understanding and trajectory prediction as sensor-agnostic 2D-to-2D tasks in the image plane, thereby facilitating scalable training across diverse datasets. The planned 2D trajectories are then deterministically lifted to 3D only during inference, ensuring the full exploitation of visual cues and generalization across different vehicles. To realize this paradigm, we propose a knowledge-instilled policy learning strategy that applies dense, intermediate rewards via Group Relative Policy Optimization (GRPO) to enforce a rigorous causal chain from visual perception to spatial planning. Extensive experiments demonstrate that PixelPilot achieves state-of-the-art performance in both open-loop and closed-loop settings, validating its superior scalability and visual reasoning capabilities.

[191] Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis cs.CVPDF

Zhipeng Xu, Zulong Chen, Qing Liu, Junhao Ji, Jinxin Hu

TL;DR: 本文提出了SAYRE框架，一种场景感知的文档合成方法，旨在为关键信息提取任务生成可扩展的训练数据，以解决实际部署中大型多模态模型成本高、本地可部署模型监督数据不足的问题。该方法通过分析少量示例文档，捕获类别特定的内容模式和布局惯例，合成文档-模式-标注三元组，并引入错误驱动生成机制来创建具有挑战性的训练样本。

Details

Motivation: 关键信息提取的实际部署面临挑战：高性能通常依赖于成本高昂的服务器端大型多模态模型，而紧凑的本地可部署模型则缺乏足够的KIE监督数据。

Result: 在约束类别和开放类别的KIE实验中，SAYRE持续提升了Qwen3-VL骨干模型的性能，并在设备端LMM中实现了最强的整体性能。数据扩展实验显示，随着合成数据的增加，性能整体呈上升趋势，特别是对于较小模型和开放类别提取。错误分析表明，合成训练通过改进对密集表格、业务标识符和合同条款的模式感知提取，减少了字段级错误。

Insight: 创新点在于提出了无需手工模板设计的场景感知文档合成框架，以及错误驱动生成机制来扩展真实世界的失败案例为困难训练样本。从客观角度看，这是一种以数据为中心的有效方法，通过合成数据增强来提升多模态KIE的实用性，尤其有利于资源受限的本地部署场景。

Abstract: Key Information Extraction (KIE) converts visually rich documents into structured data, but practical deployment remains challenging: strong performance often relies on costly on-server Large Multimodal Models (LMMs), while compact locally deployable models lack sufficient KIE supervision. We present SAYRE, a scene-aware document synthesis framework for generating scalable KIE training data without hand-crafted template design. Given a few exemplar documents, SAYRE captures category-specific content patterns and layout conventions to synthesize document-schema-annotation triples. It further introduces error-driven generation, which expands real-world failure cases into hard training examples while preserving their structural patterns. Experiments on constrained- and open-category KIE show that SAYRE consistently improves Qwen3-VL backbones and achieves the strongest overall performance among on-device LMMs. Data scaling experiments show an overall upward trend as more synthesized data is introduced, especially for smaller models and open-category extraction. Error analysis further shows that synthesized training reduces field-level errors by improving schema-aware extraction over dense tables, business identifiers, and contract clauses. These results establish scene-aware synthesis as an effective data-centric approach for improving practical multimodal KIE.

[192] Learning Structured Visual Compositional Representations for Weakly Supervised Referring Expression Comprehension cs.CVPDF

Lian Xu, Mohammed Bennamoun, Farid Boussaid, Hamid Laga, Yulan Guo

TL;DR: 本文提出了一种结构化视觉组合表示学习框架SVCR，用于弱监督指称表达理解任务。该方法显式建模一元对象嵌入和二元关系嵌入，形成结构化视觉表示空间，并通过组合对齐机制统一匹配视觉与文本嵌入。在RefCOCO系列数据集上的实验表明，SVCR达到了最先进的性能。

Details

Motivation: 现有弱监督指称表达理解方法主要基于锚点级视觉表示，即使引入辅助线索，关系交互仍隐式编码在单个锚点特征中，导致视觉表示扁平且仅为一元形式，难以与语言的结构化特性对齐。

Result: 在RefCOCO、RefCOCO+和RefCOCOg数据集上进行广泛实验，SVCR实现了最先进的性能。

Insight: 创新点在于显式建模结构化视觉表示（包含一元对象和二元关系嵌入）以及统一的组合对齐机制，这提升了视觉与文本在弱监督下的组合匹配能力，为弱监督视觉语言对齐提供了新思路。

Abstract: Referring expression comprehension (REC) aims to localize the object in an image described by natural language. In Weakly supervised REC (WREC), existing approaches primarily operate on anchor-level visual representations. Even when enriched with auxiliary cues, relational interactions remain implicitly encoded within individual anchor features. The resulting visual representation remains flat and unary-only, limiting its ability to align with the structured nature of language. In this work, we propose a Structured Visual Compositional Representation (SVCR) learning framework for WREC. Rather than implicitly encoding relations within unary anchors, the proposed SVCR explicitly models both unary object embeddings and pairwise relational embeddings, forming a structured visual representation space. We further introduce a compositional alignment mechanism that matches unary and pairwise visual representations with their corresponding textual embeddings in a unified manner, enabling compositional visual-textual matching under weak supervision. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg show that the proposed SVCR achieves state-of-the-art performance. These results demonstrate the effectiveness of explicit structured visual representations and visual-textual alignment for WREC.

[193] Learning Flexible Generalization in Video Quality Assessment by Bringing Device and Viewing Condition Distributions cs.CVPDF

Nikolay Safonov, Dmitriy S. Vatolin

TL;DR: 该论文针对移动设备多屏视频质量评估（VQA）问题，提出了首个大规模主观数据集，包含超过300种不同安卓设备的观看条件和显示属性元数据。作者提出了一种聚合分数提取策略，并适配VQA模型以实现设备特定的质量估计。结果表明，结合设备和上下文信息能实现更准确、灵活的质量预测，为流媒体服务的细粒度优化提供了新机会。

Details

Motivation: 当前视频质量评估（VQA）指标在模拟人类感知时，往往忽略了观看条件（如环境光照）和显示特性（如亮度、分辨率）对失真可见度的显著影响，特别是在移动设备多屏评估领域研究覆盖不足。

Result: 论文在收集的大规模跨设备主观数据集上验证了所提方法，结果表明，结合设备和上下文信息能实现更准确、灵活的质量预测，有助于弥合实验室评估与真实世界多样化媒体消费条件之间的差距。

Insight: 主要创新点在于构建了首个大规模、跨多种移动设备并包含观看条件与显示属性元数据的主观视频质量数据集，并提出了一种聚合分数提取与设备特定模型适配策略，推动了感知质量模型向更贴近真实消费场景的灵活泛化发展。

Abstract: Video quality assessment (VQA) plays a critical role in optimizing video delivery systems. While numerous objective metrics have been proposed to approximate human perception, the perceived quality strongly depends on viewing conditions and display characteristics. Factors such as ambient lighting, display brightness, and resolution significantly influence the visibility of distortions. In this work, we address the question of the multi-screen quality assessment on mobile devices, as this area still tends to be under-covered. We introduce a first large-scale subjective dataset collected across more than different 300 Android devices, accompanied by metadata on viewing conditions and display properties. We propose a strategy for aggregated score extraction and adaptation of VQA models to device-specific quality estimation. Our results demonstrate that incorporating device and context information enables more accurate and flexible quality prediction, offering new opportunities for fine-grained optimization in streaming services. Ultimately, this work advances the development of perceptual quality models that bridge the gap between laboratory evaluations and the diverse conditions of real-world media consumption. We made the dataset and the code available at https://videoprocessing.github.io/device-viewing-conditions.

[194] DiCE-CIR: Direct Composition Learning for Efficient Zero-Shot Composed Image Retrieval cs.CVPDF

Gwang-Ho Na, Ho-Joong Kim, Seong-Whan Lee

TL;DR: 本文提出DiCE-CIR方法，用于零样本组合图像检索（ZS-CIR），该方法通过直接组合参考图像和编辑文本来预测组合查询表示，避免了现有投影方法所需的额外步骤和训练-推理差异。

Details

Motivation: 解决现有ZS-CIR方法依赖投影映射导致的训练复杂、效率低下以及训练与推理不一致的问题。

Result: 在CIRCO基准上达到SOTA性能，在CIRR基准上取得有竞争力的结果，同时保持了较高的计算效率。

Insight: 创新点在于直接组合学习框架和利用大语言模型自动构建大规模训练样本，避免了手动标注三元组的需求，并设计了促进目标对齐、编辑一致语义转换和检索判别性的训练目标。

Abstract: Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image from a multimodal query consisting of a reference image and an edit text describing the desired modification. Recent ZS-CIR studies have relied on projection-based methods that map a reference image into pseudo-word tokens in the text embedding space. However, such methods require additional projection and re-encoding steps, increasing training complexity, reducing efficiency, and introducing a discrepancy between training and inference. In this paper, we propose DiCE-CIR, a direct composition learning method that predicts composed query representations by directly composing a reference image and an edit text. To enable scalable training without manually annotated triplets, we automatically construct compositional training samples from large-scale image-caption pairs using a large language model. Based on these samples, we train a lightweight composition module with objectives that promote alignment with the target, edit-consistent semantic transformation, and retrieval discriminability. We conduct extensive experiments on ZS-CIR benchmarks and show that DiCE-CIR achieves state-of-the-art performance on CIRCO and competitive performance on CIRR while maintaining high computational efficiency.

[195] GlaKG: A Biomarker-Centric Fundus Knowledge Graph for Explainable Glaucoma Diagnosis and Risk Assessment cs.CV | cs.LGPDF

Cheng Huang, Jia Zhang, Yi Jiang, Yang Liu, Karanjit Kooner

TL;DR: 本文提出了GlaKG，一个以生物标志物为中心的眼底知识图谱，用于青光眼的可解释诊断和风险评估。该图谱整合了结构性生物标志物、临床规则和图像特征，通过可追溯的推理链进行诊断和风险分层。

Details

Motivation: 青光眼是全球不可逆失明的主要原因，而现有的自动化诊断系统大多依赖不透明的深度学习模型，缺乏临床可解释性。因此，需要一种能够提供明确推理过程的诊断方法。

Result: 在一个公开的、AI标注的眼底数据集上，GlaKG在二元青光眼分类中达到F1=0.9953，在四类风险分层中达到0.930的准确率和0.922的加权F1。作者指出，由于数据集生物标志物注释与标签高度相关，这些结果应被视为使用干净结构化生物标志物可达到的上限。

Insight: GlaKG的核心创新在于提供了一个临床可审计的推理框架，通过明确的生物标志物证据和规则激活来支持每个决策，增强了模型的可解释性。其采用的后处理融合框架将基于知识的推理与标签信息严格分离，并通过推理链分数暴露边界案例，提高了诊断的透明度和可靠性。

Abstract: Glaucoma is a leading cause of irreversible blindness worldwide, yet most automated diagnosis systems rely on opaque deep-learning models that offer little clinical interpretability. We present GlaKG, a biomarker-centric fundus knowledge graph that integrates structural biomarkers, clinically grounded rules, and image features to produce traceable reasoning for glaucoma diagnosis and risk stratification. GlaKG encodes six entity types (Fundus Image, Optic Disc, Neural Rim, Pathology, Diagnosis, Risk Level), eight relation types, and 11 clinically validated rules into a unified graph, so that every prediction is accompanied by an explicit reasoning chain linking biomarker evidence to activated clinical rules. To keep knowledge-based reasoning strictly separate from label information, we adopt a post-processing fusion framework that combines ResNet50 image embeddings with a normalized KG reasoning-chain score via a tunable weight alpha, with all fitting confined to the training split. On a publicly available, AI-annotated fundus dataset, GlaKG reaches F1 = 0.9953 for binary glaucoma classification and 0.930 accuracy with 0.922 weighted F1 for four-class risk stratification; we report openly that the dataset’s biomarker annotations are highly label-correlated, and therefore frame these figures as an upper bound attainable with clean structured biomarkers rather than as leakage-free image-only performance. Feature-importance analysis shows KG-derived and biomarker features contributing near-equally (51.1% vs. 48.9%), and the reasoning chain flags borderline cases by exposing low chain scores rather than failing silently. GlaKG’s central contribution is therefore a clinically auditable reasoning framework that complements raw predictive performance by explicitly exposing the biomarker evidence and rule activations behind each decision.

[196] Enhancing Video Physical Consistency via Role-aware Joint Training and Modality-decoupled Denoising cs.CVPDF

Guangting Zheng, Haojing Chen, Hao Li, Jingtao Zhang, Zhen Yang

TL;DR: 本文提出了一种名为VPT的微调框架，旨在提升视频扩散模型的物理一致性。该框架通过角色感知信号对场景实体进行分类，并采用模态解耦去噪策略，将视觉与辅助模态（如光流）独立处理，以减少推理时的累积误差。实验表明，VPT在保持视觉质量的同时，显著提高了物理一致性。

Details

Motivation: 现有视频扩散模型在视觉保真度上表现优异，但难以维持长距离的物理一致性；传统像素重建目标主要关注外观细节，而忽略了场景的底层动态。现有方法整合辅助模态引入物理先验时，存在未区分不同实体类型的运动模式、联合建模导致容量冲突与预训练视觉先验削弱，以及辅助模态在推理时可能累积误差等问题。

Result: 在VideoPhy基准测试中，VPT相比Wan2.1-T2V-1.3B模型，在SA指标上相对提升39.4%，在PC指标上相对提升17.9%；在VideoPhy-2基准上也取得了一致的改进，实现了物理一致性的显著提升。

Insight: 创新点包括：引入角色感知信号对实体进行物理角色分类（如智能体、受控对象、被动对象和背景），以更清晰地建模不同运动模式；提出模态解耦去噪策略，为视觉和辅助通道分配独立噪声水平，结合损失权重衰减，使辅助模态作为软约束而非强依赖，减轻推理时的递归预测误差；此外，还引入了跨步自动引导机制以进一步增强物理动态。这些设计有效平衡了物理约束与视觉质量，减少了辅助模态的误差传播。

Abstract: While modern video diffusion models excel in visual fidelity, maintaining long-range physical consistency remains a formidable challenge. Conventional pixel-reconstruction objectives mainly focus on appearance details and often fail to capture the underlying dynamics of a scene. To mitigate this, recent efforts have integrated auxiliary modalities (e.g., optical flow) to introduce physics priors via joint training with video appearance. However, these methods have three main limitations: (1) they do not distinguish the different motion patterns of different entity types; (2) joint modeling of visual and auxiliary modalities can cause capacity conflicts and weaken the pretrained visual prior; and (3) auxiliary modalities may accumulate errors during inference. To address these issues, we propose \textbf{VPT}, a fine-tuning framework for improving physical consistency in video diffusion models. VPT introduces a role-aware signal that groups entities into agents, controlled objects, passive objects, and background, so that different physical roles can be modeled more clearly. We further propose a modality-decoupled denoising strategy, where the visual and auxiliary channels are assigned independent noise levels. Together with a loss-weight decay strategy, this design makes auxiliary modalities serve as soft constraints rather than strong dependencies, mitigating recursive prediction errors during inference. We also introduce cross-step auto-guidance to further strengthen physical dynamics. Experiments show that VPT improves physical consistency while preserving visual quality, achieving relative gains of 39.4% in SA and 17.9% in PC on VideoPhy benchmark over Wan2.1-T2V-1.3B, and consistent improvements on VideoPhy-2 benchmark. The project page is available at https://tom-zgt.github.io/VPT.

[197] Video Generation Models Are Inherent Lighting Estimators cs.CVPDF

Ziqi Cai, Shuchen Weng, Kaiqi Liu, Zifeng Wang, Zhiquan Zhang

TL;DR: 本文提出V-LITE框架，通过将光照估计重构为引导视频修复任务，利用视频生成模型内在的照明理解能力，从单段野外视频中恢复动态高动态范围环境贴图。该方法在合成铬球插入的启发下，结合HDR感知VAE和LoRA微调策略，在混合数据集上实现了时间连贯的HDR环境贴图生成。

Details

Motivation: 从单段野外视频中恢复动态环境贴图对于真实感渲染至关重要，但现有方法仍面临挑战。视频生成模型能生成具有复杂光照的真实感场景，蕴含对光照的内在理解，本文旨在解锁这种内部知识。

Result: 大量实验表明，V-LITE能生成时间连贯的HDR环境贴图，证明现代视频扩散模型不仅是合成器，也是强大的、天生具备物理场景光照估计能力的估计器。

Insight: 创新点在于将光照估计重构为引导视频修复任务，并插入合成铬球以利用模型的时空上下文生成物理可信反射；技术上设计了HDR感知VAE和高效的LoRA微调策略，并构建了混合数据集来提供HDR先验和动态时空上下文。

Abstract: Recovering dynamic environment maps from a single in-the-wild video is crucial for photorealistic rendering, yet remains a challenge. Recent video generation models can produce photorealistic scenes with complex lighting, possessing an inherent understanding of lighting. In this paper, we introduce V-LITE (Video generation models are inherent lighting estimators), a framework that unlocks this internal knowledge by reframing lighting estimation as a guided video inpainting task. Inspired by VFX industry practices, we insert a synthetic chrome ball into the scene to compel the model to generate physically plausible reflections from the surrounding spatio-temporal context. To bridge the gap from LDR-native models to the HDR domain, we design an HDR-aware VAE and employ an efficient LoRA-based fine-tuning strategy. We then construct a mixed dataset comprising high-fidelity HDR images to provide realistic HDR priors, and in-the-wild HDR videos to provide dynamic spatio-temporal context. Extensive experiments demonstrate that V-LITE produces temporally coherent HDR environment maps, revealing that modern video diffusion models are not merely synthesizers but also powerful, inherently capable estimators of physical scene lighting.

[198] TubeLite: Lightweight Multi-Actor Spatio-Temporal Action Detection cs.CVPDF

Ali Soltaninezhad, Melissa Cote, Alejandro Rico Espinosa, Tunai Porto Marques, Alexandra Branzan Albu

TL;DR: 本文提出TubeLite，一种轻量化的时空动作检测框架，旨在解决视频中动作检测的时空定位问题。该方法通过低抖动演员检测、高斯加权特征提取、高效短期时序传播和边界感知时序预测头，构建稳定的动作管，避免使用光流和重型时空Transformer。

Details

Motivation: 现有方法在构建时序稳定的动作管时面临抖动、碎片化和时序定位不精确的挑战，且常依赖计算成本高的时空Transformer或光流，导致可扩展性受限。

Result: 在MultiSports和UCF101-24数据集上，TubeLite将Video-mAP@0.5分别提升了4.5和7.1个百分点，优于对比方法，且参数量和浮点运算量远低于基于Transformer的替代方案。

Insight: 创新点在于通过轻量化设计实现时空一致性，包括在空间和语义层面显式强制时序一致性，以及结合边界感知建模来提升定位精度，证明了有效的时空动作检测可通过原则性轻量时序建模获得。

Abstract: Spatio-temporal action detection in videos requires jointly localizing actors in space and identifying action boundaries over time. A common challenge is constructing temporally stable action tubes, as frame-level detectors often suffer from jitter, fragmentation, and imprecise temporal localization. Many recent approaches address this by introducing heavy spatio-temporal transformers or optical-flow-based pipelines, leading to high computational cost and limited scalability. We propose TubeLite, a lightweight framework for spatio-temporal action detection that focuses on stable tube construction and boundary-aware temporal modeling. TubeLite represents each actor as a tube, defined as a sequence of bounding boxes associated with a single actor over time, and explicitly enforces temporal consistency at both the spatial and semantic levels. The method combines low-jitter actor detection, Gaussian-weighted actor feature extraction, efficient short-term temporal propagation, and a boundary-focused temporal prediction head, while avoiding optical flow and large-scale temporal attention. Despite its compact design, TubeLite achieves strong video-level localization performance. It improves Video-mAP@0.5 by 4.5 and 7.1 percentage points over the best compared method on the MultiSports and UCF101-24 datasets, respectively, with substantially fewer parameters and floating-point operations than transformer-based alternatives, demonstrating that effective spatio-temporal action detection can be obtained through principled, lightweight temporal modeling.

[199] Solve the Missing First Step: Can VLMs Standardize Raw Heterogeneous Medical Data? cs.CVPDF

Xin Chen, Dongliang Xu, Cunhao Zhu, Xudong Luo, Haoyang Lyu

TL;DR: 这篇论文研究了医学AI应用中常被忽视的原始异构医疗数据标准化问题，提出了医学数据标准化基准（MDS-Bench），用于评估视觉语言模型（VLMs）处理原始医疗数据的能力。研究发现，即使当前性能最佳的VLM（Gemini 3 Flash）在端到端任务上的成功率也仅为48.6%，揭示了该问题是实际医疗AI诊断的关键瓶颈。

Details

Motivation: 现有医学AI基准主要评估模型在给定标准化医疗图像和文本上的诊断能力，但忽视了实际临床实践中医疗数据原始、异构且分散的特性。论文旨在解决这一缺失步骤——原始医疗数据标准化，以弥合模型能力与真实应用场景之间的差距。

Result: 在构建的MDS-Bench基准（包含1,939个手动标注的原始医疗数据标准化任务）上，实验表明最佳VLM（Gemini 3 Flash）的端到端成功率仅为48.6%，凸显了模型在处理原始异构医疗数据方面的显著不足。

Insight: 创新点在于首次系统性地将原始医疗数据标准化作为独立研究问题提出，并构建了专门基准来评估VLM在此任务上的能力。这揭示了医疗AI实际部署中一个关键且未被充分探索的预处理瓶颈，为未来研究指明了方向。

Abstract: As vision-language models (VLMs) are increasingly applied to medical AI, existing benchmarks mainly focus on evaluating their diagnosis ability over given medical images and texts, implicitly assuming that standardized medical images, texts or question-answer pairs are already prepared. However, this assumption does not hold when we apply VLMs in real clinical practice, where medical data is often raw, heterogeneous, and fragmented across different sources. In this paper, we study this missing step, i.e., raw medical data standardization. Specifically, models are given raw dataset folders and evaluated on their ability to identify source formats, convert raw medical images into VLM-compatible visual inputs, extract relevant textual information, and organize the results into structured image-text pairs. To construct this Medical Data Standardization Benchmark (MDS-Bench), we manually annotate 1,939 raw medical data standardization tasks covering diverse clinical practice, radiology modalities, annotation formats, and directory layouts. Extensive experiments show that even the best performing VLMs, i.e., Gemini 3 Flash, achieve only 48.6% end-to-end success rate. Our research highlights raw medical data standardization as a critical bottleneck for medical AI diagnosis in real practice.

[200] MergeSurv: Merging-Based Continual Learning for Survival Analysis on Whole-Slide Images cs.CVPDF

Vu Minh Tran, Doanh C. Bui, Maï K. Nguyen, Khang Nguyen

TL;DR: 本文提出MergeSurv，一种基于模型合并的持续学习框架，用于全切片图像（WSI）的生存分析。该方法独立微调每个任务的病理视觉-语言基础模型，然后将学习到的参数顺序合并为一个统一模型，无需存储历史训练数据。实验在四个TCGA队列上进行，结果表明MergeSurv在性能和减轻灾难性遗忘方面优于多种基线方法。

Details

Motivation: 现有WSI生存分析模型通常针对每个癌症队列独立训练，对于千兆像素级的WSI，持续的适应计算成本高昂。本文旨在开发一种可扩展且保护隐私的持续学习方法。

Result: 在四个TCGA队列上的实验表明，MergeSurv在性能上优于朴素的微调方法以及代表性的基于正则化和基于重演的持续学习方法，同时有效减少了灾难性遗忘。

Insight: 创新点在于将模型合并技术引入计算病理学的持续学习场景，提出了一种无需存储历史数据、可扩展且保护隐私的参数合并框架。其探索的两种推理策略（OFA和VEA）也为模型应用提供了灵活性。

Abstract: Survival analysis on Whole Slide Images (WSIs) is important in computational pathology for prognosis estimation and treatment planning. However, existing survival models are typically trained independently for each cancer cohort, making continual adaptation computationally expensive for gigapixel-scale WSIs. In this study, we propose MergeSurv, a merging-based continual learning framework for WSI survival analysis. A pathology vision-language foundation model is independently fine-tuned on each task, and the learned parameters are sequentially merged into a unified model without storing previous training data. We further investigate two inference strategies: One-for-All (OFA) and Voting-Expert Aggregation (VEA). Experiments on four TCGA cohorts demonstrate that MergeSurv outperforms naive fine-tuning as well as representative regularization-based and rehearsal-based continual learning methods, while effectively reducing catastrophic forgetting. The results suggest that model merging is a promising direction for scalable and privacy-preserving continual learning in computational pathology.

[201] SparseOcc++: Geometry-Aware Sparse Latent Representation for Semantic Occupancy Prediction cs.CVPDF

Pin Tang, Zhongdao Wang, Guoqing Wang, Xiangxuan Ren, Chao Ma

TL;DR: SparseOcc++是一个用于自动驾驶中基于视觉的3D语义占据预测的几何感知稀疏框架。它通过场景补全场显式地将场景补全与语义分割解耦，将补全任务重新定义为稀疏锚体素上的有符号距离回归，并利用几何引导的传播模块生成完整的体素场景，从而在保持细粒度3D结构的同时显著提升效率和精度。

Details

Motivation: 现有密集体素表示在大部分空区域浪费计算，而BEV和TPV投影牺牲了细粒度3D结构；完全稀疏表示（如SparseOcc）将场景补全与语义预测纠缠，导致过度激活、计算开销和几何模糊。

Result: 在nuScenes数据集上，SparseOcc++比SparseOcc的IoU提升了2.3个百分点，速度加快了3.9倍；在SemanticKITTI数据集上，比OccFormer实现了5.9倍的加速，达到了新的最先进水平（SOTA）。

Insight: 核心创新在于将场景补全与语义分割显式解耦，通过场景补全场进行几何感知的稀疏距离回归，并结合正交分解与离散化距离学习来鲁棒建模复杂室外几何，从而在稀疏表示中高效且精确地恢复3D结构。

Abstract: Vision-based 3D semantic occupancy prediction is essential for autonomous driving, yet dense voxel representations waste computation on largely empty space, while BEV and TPV projections compromise fine-grained 3D structure. Fully sparse representations offer an attractive alternative, but existing methods, including SparseOcc, entangle scene completion with semantic prediction by indiscriminately propagating high-dimensional features into empty regions and applying voxel-wise classification. This creates excessive activations, computational overhead, and geometric ambiguity. We present SparseOcc++, a geometry-aware sparse framework that explicitly decouples scene completion from semantic segmentation. SparseOcc++ reformulates completion as signed-distance regression on sparse anchor voxels through a scene completion field (SCF). To model complex outdoor geometry robustly, it combines orthogonal decomposition with discretized distance learning. A geometry-guided propagation module then converts the SCF into a complete volumetric scene and restricts semantic segmentation to geometrically verified regions. Experiments establish new state of the art: SparseOcc++ improves IoU by 2.3 points and is 3.9x faster than SparseOcc on nuScenes, while achieving a 5.9x speedup over OccFormer on SemanticKITTI.

[202] DeGenseGS: Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting cs.CVPDF

Yimo Wang, Bin Kang, Shuojue Yang, Yueming Jin

TL;DR: 本文提出DeGenseGS，一个用于4D手术场景理解的新框架。它基于4D高斯泼溅技术，通过解耦语义演化和几何变形来实现实时、文本可提示的重建。该方法包含一个基于HexPlane的时空纠缠模块、一个栅格化原生语义提取机制和一个角度对齐优化策略，以提升在组织变形和拓扑变化下的语义-解剖对齐鲁棒性。

Details

Motivation: 现有解决方案通过刚性耦合方案将视觉语言模型集成到可变形场中，导致语义含义与物理解剖结构严重错位。本文旨在解决手术场景中语义与几何紧密绑定所导致的语义-解剖对齐问题。

Result: 在CholecSeg8k和EndoVis18数据集上的广泛评估表明，DeGenseGS实现了最先进的性能，具有增强的几何完整性和鲁棒的语义-解剖对齐。

Insight: 核心创新在于独立建模语义演化和几何变形，通过解耦方案避免语义特征与几何扭曲的刚性绑定。具体技术包括使用共享运动学潜在变量同步语义突变与场景动态的时空纠缠模块，以及从拓扑连续特征图推断语义的栅格化原生提取机制，这有助于处理剧烈的组织变形。

Abstract: Real-time, text-promptable 4D reconstruction is indispensable for autonomous surgical interaction. Severe misalignment between semantic meaning and physical anatomy still persists, largely because existing solutions integrate Vision-Language Models into deformable fields via a rigid coupling scheme that tightly binds semantic features to geometric warping. In this paper, we propose DeGenseGS, Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting, a novel framework that independently models semantic evolution and geometric deformation. Specifically, we propose a HexPlane-based spatiotemporal entanglement module that uses shared kinematic latents to synchronize semantic mutations with scene dynamics, while explicitly disentangling semantic updates from geometric deformation. To further ensure robustness against reconstruction artifacts, we devise a Rasterization-Native Semantic Extraction mechanism that infers semantics from topologically continuous feature maps. Additionally, we incorporate an angular-aligned optimization strategy that conforms to the native hyperspherical latent space, thereby preventing semantic distortion. Extensive evaluations on the CholecSeg8k and EndoVis18 datasets demonstrate that DeGenseGS achieves state-of-the-art performance. Our framework yields enhanced geometric completeness and robust semantic-anatomic alignment, enabling spatially continuous segmentation despite drastic tissue deformation and topological transitions.

[203] DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation cs.CVPDF

Ruizhe Zeng, Siyu Cao, Lu Zhang, Zhiyong Liu

TL;DR: 本文提出了DGSeg，一个用于推理分割的动态门控框架。该框架利用多模态大语言模型（MLLM）生成互补的语义和空间线索，并分别输入到独立的分割分支中。通过一个轻量级的动态门控模块，自适应地融合两个分支的预测结果，以抑制噪声和冲突区域，从而更准确地保留推理意图。

Details

Motivation: 现有的推理分割方法将丰富的多模态推理压缩为稀疏的线索（如点或框），这常常引入模糊性和噪声，导致线索无法准确保留推理意图。同时，现有方法通常将多个互补线索联合输入到单一分割过程中，使得模糊或错误的线索影响整个预测。

Result: 在多个基准测试中，DGSeg持续优于强基线模型。在具有挑战性的ReasonSeg验证集和测试集上，分别达到了69.6%和67.3%的gIoU（广义交并比）。

Insight: 核心创新在于将语义和空间线索解耦，通过独立的分支进行处理，并设计了一个基于相对分支质量监督的动态门控模块进行自适应融合。这种设计允许模型在融合时抑制不可靠的线索，从而更鲁棒地整合多模态信息，而非简单地将所有线索联合输入。

Abstract: Reasoning segmentation aims to predict pixel-wise masks for targets given complex language queries. Existing approaches leverage Multimodal Large Language Models (MLLMs) for vision-language reasoning and generate intermediate target cues (e.g., points or boxes) to guide a segmentation model. However, compressing rich reasoning into sparse cues often introduces ambiguity and noise, preventing these cues from accurately preserving the reasoning intent. While multiple complementary cues can enrich target information, existing methods typically feed them jointly into a single segmentation process, allowing ambiguous or erroneous cues to affect the entire prediction. Therefore, we propose DGSeg, a reasoning segmentation framework that learns to fuse predictions guided by semantic and spatial cues. Specifically, the MLLM jointly reasons about both target identity and spatial location, producing complementary semantic and spatial cues that are fed into separate segmentation branches. Their predictions are adaptively integrated by a lightweight dynamic gating module trained with relative branch-quality supervision to suppress noisy or conflicting regions. Extensive experiments demonstrate that DGSeg consistently outperforms strong baselines on multiple benchmarks and achieves 69.6% and 67.3% gIoU on the challenging ReasonSeg validation and test splits. Code is available at https://github.com/RZZeng/DGSeg.

[204] TGRIP: A Text-Guided Approach to Vehicle Instance Prediction in Autonomous Driving cs.CVPDF

Miguel Antunes-García, Santiago Montiel-Marín, Fabio Sánchez-García, Rodrigo Gutiérrez-Moreno, Rafael Barea

TL;DR: 本文提出了一种名为TGRIP的文本引导方法，用于自动驾驶中的车辆实例预测。该方法通过将丰富的语义先验知识注入到实例预测循环中，利用视觉-语言基础模型从多摄像头图像生成密集的、语义增强的鸟瞰图作为辅助监督，从而学习兼具几何一致性和语义判别性的时空表示。

Details

Motivation: 当前最先进的鸟瞰图端到端实例预测方法主要依赖几何监督，将场景中的智能体视为通用的移动障碍物，缺乏明确的语义感知能力，这限制了模型在复杂场景（如超车、交叉路口）中解决歧义的能力。

Result: 实验结果表明，TGRIP在nuScenes数据集上超越了现有的最先进模型，验证了语义增强是鲁棒、端到端运动预测的基本要素。

Insight: 创新点在于首次尝试将语义引导与未来实例预测这一时序任务统一起来，通过师生框架利用基础模型生成语义增强的BEV地图作为辅助监督，为模型提供了超越几何信息的语义判别能力。

Abstract: Bird’s-Eye View (BEV) end-to-end instance prediction has emerged as a robust paradigm for autonomous driving perception, effectively mitigating the error propagation inherent in traditional modular pipelines. However, current state-of-the-art approaches rely predominantly on geometric supervision, such as occupancy regression and optical flow, effectively treating scene agents as generic moving obstacles. This absence of explicit semantic awareness imposes limitations on the capacity of the model to solve ambiguities in complex scenarios, particularly those where object-specific behavior is essential for accurate forecasting (e.g. overtaking, intersections). In this paper, we introduce Text-Guided Representation for Instance Prediction (TGRIP), a novel framework that bridges this gap by injecting rich semantic priors into the instance prediction loop. The proposed teacher-student pipeline employs Vision-Language Foundation Models to generate dense, semantic-enhanced BEV maps from multi-camera images. These maps serve as auxiliary supervision during training, guiding the network to learn spatio-temporal representations that are not only geometrically consistent but also semantically discriminative. To the best of our knowledge, this represents the first attempt to unify semantic guidance with the temporal task of future instance prediction. The experimental results demonstrate that TGRIP surpasses existing state-of-the-art models in nuScenes, validating the hypothesis that semantic enrichment is a fundamental element for robust, end-to-end motion prediction. Code is available on https://github.com/miguelag99/TGRIP.

[205] EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization cs.CV | cs.AIPDF

Youngkil Song, Yoonjae Baek, Dongwon Kim, Inho Kim, Dongkeun Kim

TL;DR: 本文提出EventCoT，一种事件中心视频思维链框架，用于解决推理时序定位任务。该方法首先将输入视频转换为紧凑的事件令牌以高效识别问题相关事件，然后通过嵌入匹配对齐占位符令牌与视觉嵌入来生成答案并定位时间区间。

Details

Motivation: 解决推理时序定位任务中需要同时进行高级推理和精确时间定位的挑战，现有方法在联合生成答案和时间区间方面存在效率与精度不足的问题。

Result: 在ActivityNet-RTL数据集上达到了最先进水平，同时使用的视觉令牌数量显著少于先前工作；在ReXTime基准测试中获得了强大的零样本性能。

Insight: 创新点在于事件中心视频思维链框架，通过事件令牌化实现高效事件识别，并利用嵌入匹配机制进行时间区间定位，提升了推理效率与准确性。

Abstract: Reasoning temporal localization (RTL) requires a model to generate an answer that itself contains the time interval supporting it, so high-level reasoning and precise temporal grounding must be produced jointly in a single response. To tackle this challenging task, we propose the first event-centric video chain-of-thought framework, dubbed EventCoT. EventCoT first performs event-centric tokenization of the input video to convert it into compact event tokens, enabling efficient identification of question-relevant events. It then reasons within the identified events to generate the answer, grounding the time interval via embedding matching that aligns placeholder tokens with visual embeddings. EventCoT achieves state-of-the-art results on ActivityNet-RTL for reasoning temporal localization while using substantially fewer visual tokens than previous work. To verify its general performance, we further evaluate EventCoT on the grounded video question answering benchmark ReXTime, where it attains strong zero-shot results.

[206] PAGE: Towards Practical Human-level Gaze Target Estimation cs.CV | cs.HCPDF

Zhoutong Ye, Chengwen Zhang, Zhaibin Cui, Mingze Sun, Jiaqi Liu

TL;DR: 本文提出了PaGE（Practical Gaze Estimator）模型，旨在实现实用化的人眼注视目标估计。该模型通过显式建模场景与头部特征间的复杂交互，并采用大型ViT-H+骨干网络作为教师模型，在更大规模的无标签数据集上蒸馏出轻量级学生模型。该方法在多个注视估计任务上取得了SOTA性能，并在多数指标上超越了人类水平。

Details

Motivation: 现有注视目标估计模型难以达到人类水平，限制了其实际应用。论文旨在缩小这一人机差距，实现更实用、高性能的注视估计。

Result: PaGE在多个注视估计任务上取得了SOTA性能，在9个评估指标中的7个上超越了人类表现，并在剩余2个指标上将人机差距缩小了至少60%。蒸馏出的学生模型在保持大部分性能的同时，足够轻量，可部署于机器人和消费设备。

Insight: 核心创新在于显式建模场景与头部特征的交互，以及利用大规模无标签数据和知识蒸馏来同时提升性能与实用性。这为将高性能计算机视觉模型轻量化部署提供了可借鉴的路径。

Abstract: Gaze target estimation, the task of predicting where a person is looking in a scene, is crucial to understanding human attention and intent. It is a challenging task that combines high-level understanding of global scene semantics and precise spatial reasoning using human appearance (e.g. pose, eye orientation). As a result, human-level performance remains elusive for existing models, limiting their practical application. To this end, we propose PaGE (Practical Gaze Estimator), a gaze estimation model that explicitly models the complex interaction between scene and head features. Using a PaGE model with a large ViT-H+ backbone as the teacher, we further distill student models with lighter backbones on a much larger and more diverse unlabeled dataset. The architectural improvements and novel training recipe allow PaGE to achieve state-of-the-art performance on several gaze estimation tasks, outperforming humans in 7 out of 9 metrics while reducing the human-AI gap by at least 60% in the remaining 2. The distilled student models retain most of the teacher’s performance while being lightweight enough for practical deployment on robots and consumer devices. The code and model checkpoints are available at our project page.

[207] ProCon: Projection-Consistency Memory for Training-Free Anomaly Detection cs.CVPDF

Joongwon Chae, Lihui Luo, Yang Liu, Dongmei Yu, Peiwu Qin

TL;DR: ProCon是一种无需训练的无监督异常检测框架，通过将内存检索转化为无解码器的重建过程，利用软投影残差作为异常证据，在多个数据集上实现了优异的图像和像素级检测性能。

Details

Motivation: 现有基于内存的异常检测方法通常将内存库作为最近邻查找表，这种硬检索方式容易产生假正常匹配，且无法测试图像块是否被局部正常邻域一致支持。

Result: 在MVTec-AD、VisA和Real-IAD数据集上，ProCon在七种标准指标下均取得强劲性能，图像AUROC分别达到99.8%、99.2%和93.2%，展示了优越的异常检测能力。

Insight: 创新点在于用软正常投影替代硬检索，并通过构建种子扰动分层内存、中值聚合残差和层共识融合深度特定残差图来稳定残差，无需解码器训练、骨干网络微调或伪异常监督。

Abstract: Memory-based anomaly detection is attractive because it localizes defects from normal images without training a decoder or synthesizing pseudo anomalies. However, most memory methods still use the memory bank as a nearest-neighbor lookup table: a test patch is treated as normal if it has one nearby normal anchor. This hard retrieval view is vulnerable to false-normal matches and does not test whether the patch is consistently supported by a local normal neighborhood. We propose ProCon, a training-free framework that turns memory retrieval into decoder-free reconstruction. ProCon softly projects each test patch onto nearby normal memory vectors and uses the projection residual as anomaly evidence. To stabilize this residual, it constructs seed-perturbed layer-wise memories, aggregates bank residuals by a median, and fuses depth-specific residual maps by layer consensus. ProCon requires no decoder training, backbone fine-tuning, learned fusion weights, or pseudo-anomaly supervision. Across MVTec-AD, VisA, and Real-IAD under the single-category evaluation protocol, ProCon achieves strong image- and pixel-level performance under seven standard metrics, including image AUROC scores of 99.8%, 99.2%, and 93.2%, respectively. Ablations show that the gains come from replacing hard retrieval with soft normal projection and stabilizing the residuals through memory and depth consensus. The code is available at https://github.com/jw-chae/Procon

[208] Graph Representation Learning of Longitudinal Medical Imaging Trajectories for Treatment Response Prediction cs.CV | cs.AI | cs.LGPDF

Johannes Kiechle, Richard Osuala, Daniel M. Lang, Stefan M. Fischer, Ivana Janíčková

TL;DR: 该论文提出了一种基于成像的3D时空框架，用于预测乳腺癌患者对新辅助化疗（NACT）的病理完全缓解（pCR）反应。该方法将先进的图神经网络与跨时间点的时间交互关系建模相结合，并引入了三种新颖的互补自监督治疗轨迹表示学习目标。

Details

Motivation: 在乳腺癌治疗中，病理完全缓解（pCR）是长期预后的重要替代标志物。由于患者对NACT的治疗反应差异很大，需要能够准确估计个体化治疗反应的预测模型，以支持有效的治疗决策。

Result: 在来自公共ISPY-2数据集的585名患者队列上的实验表明，该方法在多项分类指标上显著优于视觉和自监督学习的基线模型，并建立了乳腺癌pCR预测的基准。

Insight: 创新点在于提出了一个整合图神经网络和时序关系建模的3D时空框架，并设计了三种新颖的自监督学习目标来学习治疗轨迹的表示。此外，研究还系统地评估了每个患者轨迹中可用DCE-MRI时间点的数量以及扫描间时间差的影响，为纵向医学影像建模提供了有价值的见解。

Abstract: In patients with breast cancer, pathological complete response (pCR) has been established as a clinically meaningful surrogate marker for long-term outcomes. While commonly treated with neoadjuvant chemotherapy (NACT), effective treatment decision-making remains challenging, as therapeutic response can vary substantially across patients, calling for predictive models capable of accurately estimating individualized treatment response. To address this, we propose an imaging-based 3D spatio-temporal framework for treatment response prediction that integrates a state-of-the-art graph neural network with relational modeling of temporal interactions across timepoints alongside three novel complementary self-supervised treatment trajectory representation learning objectives. Experiments across a cohort of 585 patients from the public ISPY-2 dataset demonstrate that our method substantially outperforms both vision and self-supervised learning baselines across several classification metrics. Alongside establishing a breast cancer pCR prediction benchmark, we include a principled ablation of our method and further introduce and empirically assess the impact of the available number of DCE-MRI timepoints per patient trajectory and the inclusion of inter-scan time-differences. Overall, our study substantiates the utility of clinically meaningful longitudinal medical imagaging modeling for predicting NACT-induced pCR. We will publicly share our code repository and a user-friendly PyPI library for dataset curation upon publication, effectively promoting reproducible open-source research.

[209] HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better cs.CVPDF

Gengluo Li, Xingyu Wan, Shangpin Peng, Weinong Wang, Hao Feng

TL;DR: 本文提出了HunyuanOCR-1.5，一个轻量级的端到端OCR专用视觉语言模型。它在HunyuanOCR-1.0的基础上，通过引入DFlash解码技术显著提升了长结构化输出的推理速度，并利用Agentic Data Flow数据构建系统增强了模型在古文识别、细粒度图表解析、多图像问答等长尾任务上的能力。该模型在OmniDocBench v1.6基准测试中达到顶级水平，并实现了轻量级OCR VLM中最快的推理速度。

Details

Motivation: 为了解决现有OCR模型在处理长结构化文档（如密集文档、表格、公式）时推理速度慢，以及在古文、细粒度图表、多图像理解等长尾任务上能力不足的问题，旨在构建一个既高效又能力全面的轻量级端到端OCR VLM。

Result: 在OmniDocBench v1.6基准测试中达到顶级（top-tier）水平。在推理效率上，使用DFlash技术实现了Transformer推理速度6.37倍的提升，在vLLM下也有2.14倍的加速，成为轻量级OCR VLM中最快的模型。在多个长尾任务（如古文OCR、细粒度图表表格解析等）上创造了新的性能里程碑。

Insight: 主要创新点包括：1）将DFlash技术适配于OCR解码，在不改变输出分布的前提下，大幅加速了长结构化文本的生成，这是解码效率上的重要创新；2）提出了Agentic Data Flow，一种由智能体驱动的数据构建系统，能够自动化地将模型弱点转化为可执行的数据需求并进行数据构建，这是一种系统性的数据工程创新，能有效提升模型在长尾场景下的能力。

Abstract: We present HunyuanOCR-1.5, a lightweight end-to-end OCR-specialized vision-language model. HunyuanOCR unifies document parsing, text spotting, information extraction, text-image translation, and multi-image document understanding within a single end-to-end VLM. Building upon the lightweight architecture of HunyuanOCR-1.0, HunyuanOCR-1.5 does not redesign the backbone, but systematically improves both efficiency and capability. For efficiency, we adapt DFlash to OCR decoding, significantly reducing the latency of long structured outputs such as dense documents, tables, and formulas while preserving output distribution. Powered by DFlash, HunyuanOCR-1.5 achieves a 6.37x Transformer inference speedup and a 2.14x speedup under vLLM, delivering the fastest inference among lightweight OCR VLMs. For capability, we propose Agentic Data Flow, an agent-driven data construction system that transforms model weaknesses into executable data requirements and autonomously performs material search, quality verification, and pipeline development. It substantially improves long-tail capabilities in ancient-script OCR, fine-grained chart and table parsing, multi-image text-centric QA, low-resource multilingual parsing, and document hallucination evaluation. HunyuanOCR-1.5 ranks among the top-tier end-to-end OCR solutions on OmniDocBench v1.6 while achieving new performance milestones across these long-tail tasks. Combined with an upgraded pretraining and post-training recipe, HunyuanOCR-1.5 further extends its capability in high-resolution, long-context, and multi-task scenarios. Experiments demonstrate faster inference, broader OCR capability coverage, and the deployment advantages of a lightweight end-to-end model. We will release the model weights and training code to support future research and real-world OCR applications.

[210] TimeThink: Reasoning with Time for Video LLMs cs.CVPDF

Handong Li, Longteng Guo, Zikang Liu, Dongze Hao, Yepeng Tang

TL;DR: 本文提出TimeThink，一个用于视频大语言模型的强化学习框架，旨在通过逐步的时间线索奖励机制，引导模型在长视频序列中发现和验证时间局部证据，以提升视频推理能力。

Details

Motivation: 现有视频大语言模型通常依赖基于最终预测结果的奖励进行对齐，这为模型在中间推理过程中如何发现相关时间证据提供的指导有限，因此需要一种能显式引导时间证据发现的监督方法。

Result: 在视频推理、时间定位和通用视频理解基准测试上的广泛实验表明，TimeThink持续提升了时间定位和推理性能，在开源视频强化学习模型中达到了最先进水平。

Insight: 创新点在于将时间线索步骤作为视频推理的基本优化单元，并引入了逐步时间过程奖励与联合过程-结果优化目标，以平衡推理保真度与任务正确性；同时构建了自动生成时间证据片段的训练数据集TimeThink-RFT-20K，支持可扩展训练。

Abstract: Video reasoning requires models to identify and verify temporally localized evidence within long video sequences. Recent Video Large Language Models (Video-LLMs) have shown promising reasoning abilities when aligned with reinforcement learning, yet existing approaches typically rely on outcome-based rewards that supervise only the final prediction. Such supervision provides limited guidance on how models should discover the relevant temporal evidence during intermediate reasoning. In this work, we propose TimeThink, a reinforcement learning framework that explicitly guides temporal evidence discovery in Video-LLMs. Our key idea is to treat temporal clue steps as the fundamental optimization primitive of video reasoning, where each reasoning step references a candidate time interval in the video. We introduce a step-wise temporal process reward that provides localized credit assignment for these clues and a joint process–outcome optimization objective that balances reasoning fidelity with task correctness. To enable scalable training, we construct TimeThink-RFT-20K, a dataset with automatically derived temporal evidence segments. Extensive experiments across video reasoning, temporal grounding, and general video understanding benchmarks show that TimeThink consistently improves both temporal localization and reasoning performance, achieving state-of-the-art results among open-source video RL models.

[211] Geometry-aware Depth-guided Representation Learning for Structure-preserving Low-light Image Enhancement cs.CVPDF

Fang Gao, Jiongkai Qin, Jiabao Wang, Jingfeng Tang, Ming Cheng

TL;DR: 本文提出了一种深度引导的多尺度注意力网络（DMSA-Net），用于几何感知的低光照图像增强。该方法通过反射率-几何交互将深度结构先验引入低光照表示学习，利用基于Retinex的分解模块获取光照不变的反射率表示并推断深度线索，再通过多尺度深度引导融合策略在分层编码器-解码器架构中自适应整合几何与外观特征。

Details

Motivation: 现有低光照图像增强方法主要关注外观恢复，未能充分利用场景几何信息来保持结构一致性，导致低光照退化削弱了图像可见性和对视觉表示至关重要的结构线索。

Result: 在多个基准数据集上的实验表明，DMSA-Net在实现有效低光照恢复的同时，改善了结构保持性能。

Insight: 创新点在于将深度结构先验显式引入低光照表示学习，通过反射率-几何交互和多尺度深度引导注意力机制实现几何感知的增强；此外，构建了深度增强的低光照数据集LOL-D以促进相关研究。

Abstract: Low-light degradation reduces image visibility and weakens structural cues that are important for visual representation and scene understanding. Existing low-light image enhancement methods mainly focus on appearance restoration, while insufficiently exploiting scene geometry to preserve structural consistency. To address this limitation, this paper proposes a Depth-guided Multi-scale Attention Network (DMSA-Net) for geometry-aware low-light image enhancement. DMSA-Net introduces depth-related structural priors into low-light representation learning through reflectance-geometry interaction. A Retinex-based decomposition module is first used to obtain illumination-invariant reflectance representations, from which depth cues are inferred to characterize scene structure under degraded illumination. A multi-scale depth-guided fusion strategy is then embedded into a hierarchical encoder-decoder architecture, where depth-aware attention adaptively integrates geometric and appearance features. Experiments on several benchmark datasets show that DMSA-Net achieves effective low-light restoration while improving structural preservation. Moreover, we construct LOL-D, a depth-augmented low-light dataset, to facilitate research on geometry-aware low-light vision.

[212] Semantic Video Communication via Multi-Scale Convolution and Dynamic Routing for Next-Generation Networks cs.CVPDF

Gengtian Shi, Jinze Yu, Chenhao Wu, Shaofei Wang, Eiji Fukuzawa

TL;DR: 本文提出了一种面向下一代网络的生成式AI驱动的语义视频通信框架，旨在通过传输语义而非原始比特来应对视频流量的指数级增长。该框架通过多尺度时间卷积编码器高效建模不同时间粒度的运动模式，并引入基于胶囊的动态路由机制来迭代优化视频片段与自然语言查询之间的语义对齐。

Details

Motivation: 解决带宽受限传输下的高效分层时间建模，以及在网络边缘设备上实现视频内容与自然语言查询之间鲁棒的语义对齐这两个关键挑战。

Result: 在ActivityNet Captions基准测试上取得了显著提升，达到了42.9%的Recall@0.5和41.1%的平均IoU，同时保持了对于边缘部署至关重要的计算效率。

Insight: 创新点在于将多尺度时间卷积（O(T)复杂度，适合资源受限的IoT部署）与胶囊动态路由机制（用于建模非单调的语义对齐）相结合，并通过多任务学习目标进行统一优化，为面向目标的语义通信提供了灵活且高效的解决方案。

Abstract: The exponential growth of video traffic demands novel semantic communication paradigms that transmit meaning rather than raw bits. We present a generative AI-enabled framework for semantic video communication addressing two critical challenges: efficient hierarchical temporal modeling for bandwidth-constrained transmission and robust semantic alignment between video content and natural language queries at network edge devices. Our approach introduces a multi-scale temporal convolutional encoder that captures motion patterns across different temporal granularities with O(T) complexity suitable for resource-constrained IoT deployments. We further propose a capsule-based dynamic routing mechanism that iteratively refines segment-query associations, enabling flexible modeling of non-monotonic semantic alignments essential for goal-oriented communication. These components are unified through a multi-task learning objective optimizing temporal boundary regression, cross-modal alignment, and capsule diversity. Experiments on ActivityNet Captions demonstrate significant improvements, achieving 42.9% Recall@0.5 and 41.1% mean IoU while maintaining computational efficiency critical for edge deployment.

[213] Consistent and Editable: A Balanced Framework for Text-Guided Video Editing cs.CVPDF

Tao Jin, Li Xiao

TL;DR: 本文提出了一种名为EquiEdit的文本引导视频编辑框架，旨在平衡视频编辑中的时间一致性和可编辑性。该框架通过设计的时间Mamba模块增强帧间一致性，并利用基于频谱变换的噪声注入策略提高编辑灵活性。实验表明，该方法在保持输入视频保真度的同时，有效提升了编辑视频的一致性和可编辑性。

Details

Motivation: 现有基于扩散模型的文本引导视频编辑方法往往难以平衡时间一致性与可编辑性，两者通常呈负相关关系。本文旨在解决这一权衡问题，提出一个协同提升一致性和可编辑性的高质量视频编辑框架。

Result: 大量定性和定量实验表明，该方法在时间一致性和可编辑性方面均表现出色，同时对输入视频具有很高的保真度。

Insight: 创新点包括：1) 设计具有时间感知扫描机制的时间Mamba模块，通过四个方向的扫描有效增强帧间一致性；2) 提出基于傅里叶变换的频谱噪声注入策略，在保留初始潜在噪声隐藏结构的同时提高编辑灵活性。客观来看，该工作通过协同优化两个通常冲突的目标，为视频编辑提供了新的平衡框架。

Abstract: Recently, diffusion models have achieved considerable success in the text-guided video editing domain. However, existing works often struggle to balance the trade-off between temporal consistency and editability in video editing, with consistency and editability typically being inversely related. To address this, we propose a high-quality video editing framework enhanced for consistency and editability, named EquiEdit, which improves coordinatively the temporal consistency and editability of the edited videos while achieving a balance between the two. In terms of temporal consistency, the proposed temporal Mamba module with a tailored temporal-aware scanning scans fused video sequences following four designed directions, effectively enhancing the inter-frame consistency of edited videos. For editability, we design a noise injection strategy based on the spectral transformation to increase editing flexibility, where the Fourier transform is used to preserve the hidden structure in the initial latent noise used for editing, ensuring inter-frame consistency of the edited video and fidelity to the input video. Extensive qualitative and quantitative experiments demonstrate the effectiveness of our method in terms of temporal consistency and editability, as well as its great fidelity to the input video itself.

[214] An event-driven framework for fly-inspired visual motion detection cs.CV | cs.NEPDF

Qinbing Fu, Jingyu Huang, Yan Xie, Jigen Peng, Yuchao Tang

TL;DR: 本文提出了一种受果蝇视觉系统启发的、基于事件相机的运动检测计算框架。该框架结合了时间表面编码的前端事件表示和仿生神经网络，用于估计前景运动方向，并引入了自底向上的注意力机制来抑制背景运动。

Details

Motivation: 解决在动态环境中，机器视觉和自主系统对快速、可靠运动检测的需求，并应对事件相机存在的噪声问题以及事件视觉表示与仿生神经处理有效结合的挑战。

Result: 在真实世界的地面车辆数据集上进行了评估，与基于帧的基线模型和基于优化的方法进行了比较。实验结果表明，该框架有效地结合了事件驱动视觉的时间优势和仿生神经处理的高效性与可解释性。

Insight: 创新点在于将事件相机的时间表面编码与受果蝇视叶启发的、无需训练的前馈神经网络相结合，并引入注意力机制来增强前景目标显著性。这为低延迟、低功耗的实时嵌入式运动检测提供了一个高效且可解释的仿生计算范式。

Abstract: Fast and reliable motion detection is essential for machine vision and autonomous systems operating in dynamic environments. This work integrates emerging event-based sensing with biologically structured neural computation to establish an efficient computational paradigm for visual motion detection. The proposed framework is built upon a recently developed fly-inspired neural network that emulates motion-processing circuits in the optic lobe. Owing to its feed-forward and training-free architecture, the neural model requires only a small number of interpretable parameters and is well suited for real-time embedded implementation. Event cameras provide low-latency, low-power, and high-dynamic-range visual sensing by asynchronously transmitting brightness-change events. However, their performance can be degraded by event noise, including temporal noise and junction-leakage-induced activity, particularly under low-light conditions. Moreover, effective integration between event-based visual representations and biologically inspired neural processing remains under-explored. To address these challenges, we propose an event-driven computational framework that combines time-surface encoding for front-end event representation with a fly optic-lobe-inspired neural network for foreground motion-direction estimation. A bottom-up attention mechanism is further incorporated to suppress background motion and enhance the saliency of foreground targets. The proposed method is evaluated on real-world ground-vehicle datasets and compared with a baseline frame-based model and an optimization-based approach. Experimental results demonstrate that the framework effectively combines the temporal advantages of event-driven vision with the efficiency and interpretability of bio-inspired neural processing.

[215] Claim-Level Rubric Rewards for Video Caption Reinforcement Learning cs.CVPDF

Mingqi Gao, Hongyuan Dong, Yifei Chen, Zhisheng Zhong, Zheng Ruan

TL;DR: 本文提出了Claim-Level Rubric Rewards (CuRe)，一个用于密集视频描述强化学习的结构化奖励框架。该框架通过将描述分解为基于类别的原子主张，将整体评估转化为更细粒度、更可靠的主张级验证，以解决现有奖励设计在事实准确性和文本对齐方面的局限性。

Details

Motivation: 现有密集视频描述强化学习的奖励设计存在瓶颈，要么是跨异构标准的整体响应级评判，容易导致事实不准确和风格奖励攻击；要么是基于参考描述的文本对齐评估，过于依赖僵硬的文本匹配，无法保持开放式生成任务的完整性和多样性。

Result: 摘要中未提及具体的定量实验结果、基准测试或与SOTA的比较。

Insight: 核心创新在于将奖励建模重新定义为细粒度的主张级验证，通过结构化评分标准将描述分解为原子主张，从而将复杂的整体评估转化为更简单可靠的单元验证，有望提升奖励信号的精确性和鲁棒性。

Abstract: In this paper, we introduce Claim-Level Rubric Rewards (CuRe), a structured reward framework designed to address the reward-design bottleneck in reinforcement learning for dense video captioning. Existing reward designs generally fall into two categories: holistic response-level judgment across heterogeneous criteria, or alignment-based evaluation against reference captions. However, both paradigms suffer from fundamental limitations. Holistic rewards struggle to ensure factual accuracy and are prone to stylistic reward hacking, while reference-based rewards overly rely on rigid textual alignment, failing to preserve the completeness and diversity inherent to open-ended generation tasks. To address these challenges, CuRe reformulates reward modeling as fine-grained claim-level verification. Specifically, CuRe decomposes captions into category-aware atomic claims through a structured rubric, converting holistic evaluation into simpler and more reliable claim-level verification.

[216] UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation cs.CVPDF

Mengmeng Liu, Diankun Zhang, Jiuming Liu, Jianfeng Cui, Hongwei Xie

TL;DR: UNIVERSE是一个用于自动驾驶的统一视频-动作模型，它基于单个掩码调制扩散变换器（DiT）构建，通过联合训练未来视频潜在表示和自车轨迹token，使密集的视频监督能直接塑造轨迹去噪过程，从而提升跨域动作泛化能力。模型引入了模态解耦可见性掩码，在共享历史上下文的同时阻止未来视频与轨迹token间的相互注意力，确保了因果有效性并实现了高效的纯轨迹推理。

Details

Motivation: 现有级联或双DiT设计将视频想象与动作预测分离，削弱了视频学习到的世界动态向轨迹分支的传递，导致动作模型可能过拟合数据集特定的驾驶先验，而视频模型仅间接正则化规划。本文旨在通过统一的架构，使密集视频监督能更直接地提升轨迹生成与泛化能力。

Result: 在NAVSIM基准测试上达到91.0 PDMS（优于双DiT变体的89.6），并在nuScenes和Bench2Drive上展示了强大的零样本迁移能力，无需微调。纯轨迹推理相比联合视频-动作推演实现了4.3倍的加速，同时保持了可比的规划精度。

Insight: 核心创新在于使用单个DiT统一建模视频与轨迹，通过联合训练和共享参数使视频监督直接正则化轨迹生成。提出的模态解耦可见性掩码机制巧妙地在训练时共享历史信息并隔离未来模态，既保证了因果性，又支持了高效的单模态（轨迹）推理部署。

Abstract: World Action Models (WAMs) have shown strong potential for improving action generalization in autonomous driving by using future video prediction as dense supervision for scene dynamics and temporal causality. However, it remains unclear which architecture better transfers video-modeling benefits to trajectory generation. Existing cascaded or dual-DiT designs separate video imagination from action prediction, weakening the transfer of video-learned world dynamics to the trajectory branch: the action model may still overfit dataset-specific driving priors, while the video model only indirectly regularizes planning. We propose UNIVERSE, a unified video-action model built upon a single mask-modulated Diffusion Transformer. By co-training future video latents and ego-trajectory tokens within shared generative parameters, UNIVERSE allows dense video supervision to directly shape trajectory denoising, leading to stronger cross-domain action generalization. To ensure causal validity and efficient deployment, we introduce a Modality-Decoupling Visibility Mask, which shares historical context across modalities while blocking mutual attention between future video and trajectory tokens. This prevents future-target leakage and enables trajectory-only inference by removing future-video denoising at test time, achieving a $4.3\times$ speedup over joint video-action rollout while maintaining comparable planning accuracy. The same model also supports video-only and joint video-action rollouts. Experiments show that UNIVERSE achieves 91.0 PDMS on NAVSIM (vs. 89.6 for the Two-DiT variant), and demonstrates strong zero-shot transfer to nuScenes and Bench2Drive without fine-tuning, while ablations confirm the importance of single-DiT unification, video co-training, and mask-based modality decoupling.

Adrian Szvoren, Dimitrios Kanoulas, Nilufer Tuptuk

TL;DR: 本文首次对视觉-语言-动作（VLA）导航策略中的视觉基础进行了实证评估。提出了一种基于实时语义分割的基础方法，使用SegFormer将可通行区域标记为绿色，不可通行区域标记为红色，并评估了仅观察分割和联合观察-目标增强两种变体。在Grand Tour数据集上使用OmniVLA模型进行实验，结果表明视觉基础显著降低了远距离航点的平均误差，尤其对长指令效果更佳，但对图像目标指令改善有限。分析表明，基础主要起到轨迹长度正则化的作用。

Details

Motivation: VLA模型虽然能实现基于自然语言和视觉目标的机器人导航，但仍易受感知干扰和场景模糊性影响。本文旨在评估视觉基础（visual grounding）对VLA导航策略的改进效果，解决模型在复杂指令下的鲁棒性问题。

Result: 在Grand Tour数据集上，使用OmniVLA模型，视觉基础方法在最远航点处将平均航点误差降低了27-44%（具体取决于指令长度）。对于长指令的改进效果优于短指令，但对图像目标指令几乎没有提升。归一化误差分析表明，该方法将预测路径长度减少了30%，但并未改善单位距离的推理能力。

Insight: 创新点在于首次对VLA导航策略进行视觉基础的系统性实证研究，并提出了一种简单、计算成本低的实时分割基础方法，无需模型重新训练即可提升导航性能。客观来看，该方法的核心洞察是视觉基础主要作为一种轨迹长度正则器，通过抑制路径过度延伸来降低误差，这为理解VLA模型的行为和设计轻量级改进策略提供了新视角。

Abstract: Vision-language-action (VLA) models enable robot navigation from natural language and visual goals, but remain susceptible to perceptual distractions and ambiguous scene interpretations. This paper presents the first empirical evaluation of visual grounding for VLA navigation policies. We propose a real-time segmentation-based grounding method that highlights traversable areas in green and non-traversable areas in red using SegFormer. Two variants are evaluated: observation-only segmentation and joint observation-goal augmentation. Using OmniVLA on the Grand Tour dataset, we show that visual grounding reduces the mean waypoint error by 27-44% at the farthest waypoint, depending on the instruction length. The benefits are greater for long instructions than for short instructions, and grounding provides little improvement for image goals. Normalized error analysis indicates that grounding primarily acts as a trajectory length regularizer, reducing the predicted path length by 30% without improving per-unit-distance reasoning. Our results indicate that visual grounding offers a simple, computationally inexpensive method to improve VLA navigation without model retraining, although it cannot compensate for missing training signals in out-of-distribution instructions.

[218] A Multimodal Reasoning Typology for Grounding Chart-Image Coherence in Science Communication cs.CVPDF

Avina Nakarmi, Sohom Sen, Xun Song, Sreyashi Samaddar, Aritra Dasgupta

TL;DR: 本文提出了一种多模态推理类型学，用于刻画科学出版物中图表与图像之间的连贯性。该类型学定义了从R1到R5的五种推理鸿沟，描述了图表、图像和文本如何共同传达科学主张以及读者所需的解释工作。研究基于79篇创伤性脑损伤论文中的32个图表-图像对构建，并通过专家与非专家对视觉语言模型描述的评估，验证了该类型学能系统识别信息传递成功或失败的关键点。

Details

Motivation: 科学出版物中图表与图像常同时出现，但现有计算工作未能系统刻画它们之间的连贯性。论文旨在解决如何表征这种多模态单元（图表、图像及链接它们的标题）所构成的科学主张，以及读者理解所需的推理差异。

Result: 研究基于79篇创伤性脑损伤论文的32个图表-图像对构建了类型学，并通过领域专家和三位非专家对25个对的视觉语言模型描述进行评判。结果表明，该类型学能预测评判一致与分歧之处，并识别出连贯性依赖于上下文知识而非图表本身的节点。

Insight: 创新点在于提出了一个基于沟通基础理论、自底向上构建的多模态推理类型学，系统化了图表-图像对连贯性的分析框架。该框架为图表设计者提供了平衡文本与多模态内容的工具，有助于弥合专家与非专家在解读科学信息时的认知鸿沟。

Abstract: Charts and images appear together throughout scientific publications, yet most computational work does not characterize their coherence. We argue that a chart, its accompanying image, and the caption that links them form a multimodal unit, and that the inferential work required to read it varies systematically. To capture this variation, we develop a typology of reasoning gaps, R1 through R5, that characterizes how chart, image, and text jointly convey a scientific claim, and the interpretive work this demands of the reader. Some pairs restate the same data, while in other pairs, charts are used to quantify a structure the image localizes, project image content onto an external variable, audit an image-based claim, or jointly construct a frame that neither panel can establish alone. The typology is anchored in the grounding theory of communication and was derived bottom-up, with a neuroscience expert, from a corpus of 79 traumatic brain injury papers and 32 chart-image pairs. Crucially, the levels provide a systematic mechanism for identifying where grounding succeeds or breaks down, rather than leaving it to subjective inference. We show this in a study in which a domain expert and three non-experts judge vision-language model (VLM) descriptions of 25 pairs: the level predicts where their judgments align and where they diverge, isolating the points at which contextual knowledge, not the figure, carries coherence. This typology thus offers figure designers a systematic way to balance text against chart-image pairs, bridging the expert-to-non-expert divide in reading a scientific takeaway.

[219] Repurposing CLIP to Localize at Pixel Level cs.CVPDF

Jiaxiang Fang, Shiqiang Ma, Jing Wang, Siyu Chen, Fei Guo

TL;DR: 本文提出CLIPix框架，通过重新利用CLIP模型实现像素级定位。该方法通过追溯CLIP的分类过程识别物体特定注意区域，并采用噪声抵抗校正策略和定位嵌入策略来提升分割精度，在PASCAL和COCO数据集上达到SOTA性能。

Details

Motivation: 解决CLIP等大规模视觉语言模型在图像级开放集定位表现优异，但难以适应像素级密集预测的问题，主要挑战在于全局特征偏差。

Result: 在PASCAL和COCO数据集上的大量实验表明，CLIPix实现了最先进的性能，验证了其有效性。

Insight: 创新点包括通过追溯分类过程获取像素级定位线索、噪声抵抗校正策略减少全局偏差影响，以及定位嵌入策略融合定位与细节信息，保持了CLIP的泛化能力并扩展至任意物体分割。

Abstract: Large-scale Vision-Language Models like CLIP have demonstrated impressive open-set localization capabilities at the image level. However, adapting this capability to pixel-level dense prediction poses challenges due to global feature biases. In this paper, we introduce CLIPix, a simple yet effective framework that repurposes CLIP to perform pixel-level localization. By tracing back CLIP’s classification process, CLIPix identifies object-specific attentive regions and repurposes them as pixel-level localization cues. To address noise introduced by global biases, we propose a Noise-Resistant Correction strategy, refining these cues for more precise segmentation. Additionally, we introduce a Localization Embedding strategy to integrate both localization and enriched detail information, enabling accurate, high-resolution segmentation. Our approach preserves CLIP’s generalization strength and unlocks its potential for segmenting arbitrary objects. Extensive experiments on the PASCAL and COCO datasets demonstrate that CLIPix achieves state-of-the-art performance, underscoring its effectiveness.

[220] GUSH3R: Everyone Everywhere All at Once as Gaussians cs.CVPDF

Keito Abe, Kaede Shiohara, Takashi Otonari, Toshihiko Yamasaki

TL;DR: GUSH3R是一个前馈框架，用于从单目视频中在线重建动态人-场景环境。该方法将动态人体（everyone）和静态场景（everywhere）统一建模为3D高斯泼溅（3DGS）图元，在一次前向传播中（all at once）实现几何一致且支持新视角合成的重建。

Details

Motivation: 解决从单目视频中联合建模场景几何、相机运动和非刚性人体动态以实现照片级真实感渲染的挑战性问题。现有前馈方法在效率上有优势，但通常局限于非真实感表示（如点云、网格）或无法有效处理非刚性物体（尤其是动态人体）。

Result: 在单目人-场景数据集上的实验表明，该方法在新视角合成质量上具有竞争力，同时与基于优化的方法相比，显著提高了推理效率。

Insight: 创新点在于提出了一个统一的前馈框架，将动态人体和静态场景统一表示为3D高斯泼溅图元，实现了高效、几何一致且支持高质量渲染的重建。从客观角度看，将3DGS这种新兴的高效渲染表示与动态人体-场景重建任务结合，并设计前馈网络进行在线预测，是一个有前景的方向。

Abstract: Reconstructing dynamic human-scene environments from monocular videos is a challenging problem that requires jointly modeling scene geometry, camera motion, and non-rigid human dynamics while enabling photorealistic rendering. Recent feed-forward methods can efficiently predict geometry, but they are often limited to non-photorealistic representations such as point clouds and meshes, or they fail to handle non-rigid objects, particularly dynamic humans. To fill this gap, we present GUSH3R (Gaussian-Unified Scene Human 3D Reconstruction), a feed-forward framework for online dynamic human-scene reconstruction. From a monocular human-scene video, our method reconstructs dynamic humans (everyone) and static scenes (everywhere) in a single forward pass (all at once) as 3D Gaussian Splatting (3DGS) primitives (as gaussians), which are geometrically consistent and capable of novel view synthesis. Experiments on monocular human-scene datasets demonstrate that our approach achieves competitive novel view synthesis quality while significantly improving inference efficiency compared to optimization-based methods.

[221] Vision Pretraining for Dense Spatial Perception cs.CVPDF

Zelin Fu, Bin Tan, Changjiang Sun, Shaohui Liu, Kecheng Zheng

TL;DR: 本文提出了一种名为掩码边界建模的自监督视觉预训练方法，通过动态学习亚像素级边界表示，并利用这些边界标记作为掩码目标来促进密集视觉标记的学习。该方法被扩展为LingBot-Vision模型，并在多个下游视觉任务中验证了其有效性，特别是在深度补全任务上推动了从LingBot-Depth 1.0到2.0的进步。

Details

Motivation: 密集空间感知对于物理智能至关重要，但现代视觉基础模型往往优先考虑语义不变性，牺牲了详细的空间理解。本文的动机是基于边界和形状不连续性为感知几何属性提供关键线索这一前提，从边界中心视角研究视觉预训练。

Result: 以DINOv3为强基线，LingBot-Vision在多种下游视觉任务中表现出色，特别是在深度补全任务上实现了从LingBot-Depth 1.0到2.0的显著提升，从而增强了深度估计性能。

Insight: 论文宣称的创新点在于将边界建模视为一种可扩展的预训练原则，超越了简单的线段检测，用于学习具有空间结构的视觉表示。从客观角度看，该方法通过自监督方式动态学习精细的边界表示，并将其整合到密集标记学习中，为提升模型的空间感知能力提供了一种新颖的范式。

Abstract: Dense spatial perception is essential for physical intelligence, where visual systems are expected to recover structured, metric, and actionable representations from pixel observations. Modern visual foundation models tend to prioritize semantic invariance, often at the expense of detailed spatial understanding. In this work, we study vision pretraining through a boundary-centric lens, motivated by the premise that boundaries and shape discontinuities offer essential cues for perceiving geometric properties. Concretely, we propose masked boundary modeling, a self-supervised paradigm that dynamically learns sub-pixel boundary representations and subsequently leverages the discovered boundary-bearing tokens as masked targets to facilitate dense visual token learning. By scaling this framework, we develop LingBot-Vision and demonstrate its efficacy across a diverse set of downstream vision tasks with DINOv3 as a strong baseline. Remarkably, LingBot-Vision drives the progression from LingBot-Depth 1.0 to LingBot-Depth 2.0 for depth completion, and thereby yields enhanced depth estimation, a key pillar for embodied artificial intelligence. Our findings reveal that boundary modeling goes beyond simple line segments and instead serves as a scalable pretraining principle for learning spatially structured visual representations.

[222] FlowMark: Mask-Guided Video Watermarking cs.CVPDF

Vishal Asnani, Shruti Agarwal, John Collomosse

TL;DR: FlowMark是一种基于自动预测物体掩码引导的视频水印框架，通过端到端可训练架构结合区域感知编码和噪声增强训练，在保证高感知质量的同时，对压缩、几何变换和内容变化具有鲁棒性。

Details

Motivation: 解决现有基于区域的水印方法需要用户提供掩码指导的问题，通过学习自动识别最佳水印嵌入区域，实现内容自适应掩码以消除感知闪烁。

Result: 在图像和视频数据集上，FlowMark能可靠嵌入128位消息，PSNR高达50.08 dB，在视频原生时间编辑（如帧交换、插入、删除）和社交媒体分发管道（如YouTube和Facebook重编码）下保持鲁棒水印恢复。

Insight: 创新点包括通过专用掩码预测器网络自动学习最优嵌入区域，以及结合区域感知编码与噪声增强训练实现内容自适应掩码，有效提升水印的时空一致性和抗攻击能力。

Abstract: We present FlowMark, a video watermarking framework guided by automatically predicted object masks. In contrast to prior region-based approaches that require user-supplied mask guidance, FlowMark learns to identify optimal regions for watermark embedding through a dedicated Mask Predictor network. Our end-to-end trainable architecture combines region-aware encoding with noise-augmented training to ensure robustness against compression, geometric transformations, and content variation, while preserving high perceptual quality. Our content-adaptive masking keeps watermark signals coherent with natural video dynamics, effectively eliminating perceptual flicker. Beyond compression robustness, FlowMark maintains reliable watermark recovery under video-native temporal edits (e.g., frame swap, insertion, deletion, resampling, and interpolation) and real-world social media distribution pipelines (e.g., YouTube and Facebook re-encoding). Experimental results on both image and video datasets show that FlowMark reliably embeds $128$-bit messages with up to $50.08$ dB PSNR, offering strong performance for content provenance, temporal authenticity verification, and video integrity protection.

[223] Learning Probabilistic Embeddings for Unsupervised Action Segmentation cs.CVPDF

Shuai Li, Duc Manh Vu, Juergen Gall

TL;DR: 本文提出了一种用于无监督时序动作分割的概率嵌入学习方法，以解决现有方法因学习确定性嵌入而容易陷入局部最优的问题。该方法将帧表示建模为高斯分布，通过从分布中采样来估计伪标签，从而在多个具有挑战性的数据集上取得了与当前最优方法相当甚至更好的性能。

Details

Motivation: 现有无监督时序动作分割方法通常采用联合表示学习和聚类范式，并使用最优传输（OT）生成伪标签，但它们学习的是确定性嵌入，导致基于伪标签学习嵌入和从嵌入估计伪标签的迭代过程容易快速陷入局部最优。

Result: 在多个具有挑战性的时序动作分割数据集上评估，该方法取得了与当前最优方法（SOTA）相当甚至更好的结果；与使用确定性嵌入的基线方法相比，将MoF（帧级准确率）提升了最高20.7%，F1分数提升了最高19.0%。

Insight: 核心创新在于将帧表示从确定性嵌入改为概率嵌入（具体为高斯分布），并通过采样来估计伪标签，这有助于避免优化过程陷入局部最优，为无监督表示学习与聚类结合提供了新的优化思路。

Abstract: This paper concerns the problem of unsupervised temporal action segmentation for long, untrimmed videos. Recent successful approaches follow a joint representation learning and clustering paradigm, where optimal transport (OT) is adopted to produce pseudo labels for learning frame representations. These approaches alternate between estimating pseudo labels using OT and optimizing the parameters with gradient descent during training, where OT is used for obtaining the final temporal action segmentation. A major limitation of these works is that they learn a deterministic embedding for frame representations. The iterative procedure between learning deterministic embeddings based on pseudo labels and estimating pseudo labels from the learned embedding can thus get quickly stuck in a local optimum. As an alternative, we thus propose to learn a probabilistic embedding for frame representations. The embeddings are modeled by Gaussian distributions and we sample from the distributions before estimating the pseudo labels. We evaluate our approach on several challenging temporal action segmentation datasets and achieve results comparable to, and in some cases, better than the state of the art. Compared to baselines with deterministic embeddings, our approach improves MoF up to 20.7% and F1-score up to 19.0%. Our code is available at https://github.com/derkbreeze/PEOT.

[224] Is the Geometry Doing the Work? An Operating-Point Audit of Hierarchy in Hyperbolic Vision-Language Models cs.CV | cs.LGPDF

Jaeyoung Kim, Eunseok Kim, Dongsuk Jang

TL;DR: 本文对三种已发布的双曲视觉语言模型（MERU、HyCoCLIP和PHyCLIP）进行几何机制审计，发现其声称的双曲几何结构在实际操作中并未有效激活。通过诊断指标分析，揭示出三种失效模式：曲率参数未成为有效资源、锥形和遍历机制不工作，以及层次结构评估存在不确定性。

Details

Motivation: 动机在于探究双曲表示模型是否真正利用了其几何结构，而非仅依赖曲率参数；通过开发诊断工具，审计现有模型以验证其几何机制的有效性。

Result: 审计结果表明，所有检查点均保持近欧几里得操作点（√cρ ≤ 1），锥形机制不活跃或饱和，层次检索增益与曲率无关；模型未实现预期的双曲几何机制。

Insight: 创新点在于提出了一套针对双曲模型几何机制的必要条件诊断方法，并揭示了蕴含目标存在低曲率宽锥捷径，导致模型未能激活双曲结构；审计结果可提炼为五参数几何报告，为未来层次声称提供评估框架。

Abstract: Whether a hyperbolic representation model uses its geometry cannot be read off its curvature parameter: what matters is the dimensionless operating point $\sqrt{c}ρ$ and whether the radial and cone machinery is active there. We develop a battery of necessary-condition diagnostics and audit three published hyperbolic vision-language families – MERU, HyCoCLIP, and PHyCLIP – across released checkpoints and controlled interventions on a fixed GRIT snapshot, identifying three failure modes. First, curvature is not an active resource: the operating point stays near-Euclidean ($H(u)\approx 1$; no audited converged checkpoint reaches $\sqrt{c}ρ>1$), and releasing the curvature floor moves curvature and norms but keeps the operating point near-Euclidean, without substantial downstream degradation. Second, the cone and traversal machinery is measured inoperative: entailment cones are inactive, saturated, or misaligned, and graded traversal fails under controlled readouts, while directed radial depth is a bounded non-detection above shuffle-null controls at quantified sensitivity; the one surviving native-relation residual remains non-operative. Third, hierarchy-looking evaluations are underdetermined: taxonomy correlations are carried by angular distance, and coarse-retrieval gains track box/compositional supervision, not curvature. A mechanistic account explains why: the entailment objective admits a low-curvature, wide-cone shortcut, and a parameter-free aperture identity (cones saturate iff $\sqrt{c}ρ\le 2K$) locates the edge where every entailment-trained unclamped run settles; entailment-off runs show no arrest there. The shortcut is the dominant accelerator of collapse, not its sole cause. These formulations, as released, do not instantiate the radial/cone mechanism their geometry motivates; we distill the audit into a five-number geometry report for future hierarchy claims.

[225] SteelBench: Evaluating Vision-Language Models in Real-World Industrial Environments cs.CVPDF

Suryanarayana Reddy Yarrabothula, Manisha Chawla, Kunal Sinha, Gagan Raj Gupta, Sashank Lekkala

TL;DR: 本文提出了SteelBench，一个用于评估视觉语言模型在真实工业监控环境中性能的诊断性基准。该基准包含1,345个密集标注的视频片段，旨在联合评估模型在工人活动识别、安全规则推理和标注溯源方面的能力。研究发现，未经审计的模型生成标注会显著高估同族模型的性能，且现有模型在工业场景下的表现远低于人类水平，尤其在安全推理方面存在严重缺陷。

Details

Motivation: 现有视频基准主要评估消费视频、第一人称视角或模拟工业环境中的动作识别，无法测试模型在真实工业CCTV（存在灰尘、蒸汽、低光照、遮挡和活动重叠等复杂视觉与流程条件）下的性能。

Result: 在来自四个架构家族的九个视觉语言模型上，最佳模型的动作识别准确率仅为42.6%，远低于人类基准的84.6%。未经审计的VLM生成标注会使同族模型准确率虚高达17个百分点。模型在识别、鲁棒性、校准和安全推理方面表现割裂，即使预测出正确动作，仍有37-58%的情况做出错误安全判断，且没有模型能通过5项诊断检查中的2项以上。

Insight: 创新点在于构建了首个针对真实工业监控环境的诊断性基准，并引入了标注溯源审计协议，以量化标注来源对模型评估的影响。这揭示了在专业领域评估中，依赖模型辅助标注可能带来的评估偏差风险，强调了构建高质量、可溯源的基准数据集的重要性。

Abstract: Existing video benchmarks evaluate action recognition on consumer videos, egocentric recordings, or simulated industrial environments. They do not test vision-language models under the visual and procedural conditions of real industrial CCTV, where workers appear as distant figures amid dust, steam, low light, glare, occlusion, and overlapping activities. We introduce STEELBENCH, a diagnostic benchmark for industrial surveillance that jointly evaluates per-worker activity recognition, safety-rule reasoning, and annotation provenance. SteelBench contains 1,345 densely annotated clips, curated from 149 hours of operational plant footage and 10,024 candidate clips using temporal deduplication, class balancing, and visibility-aware stratified sampling. Each clip includes dense per-worker action labels, PPE attributes, spatial context, and safety-rule annotations. Because model-assisted annotation can shape the labels later used for model evaluation, SteelBench includes a provenance-aware audit protocol. The protocol measures label influence, evaluates sensitivity to ground-truth provenance, and reports a human reference from expert-reviewed labels. Applying this audit, we find that unaudited VLM-sourced ground truth can inflate same-family model accuracy by up to 17 percentage points. Across nine VLMs from four architectural families, the best model reaches only 42.6% action accuracy, compared with an 84.6% human benchmark. Performance also fragments across recognition, robustness, calibration, and safety reasoning. Even when models predict the correct action, 37-58% of cases still yield incorrect safety judgments, and no model passes more than 2 of 5 diagnostic checks. The dataset is publicly available on Hugging Face.

[226] ChatImage: Navigating Long-Form LLM Answers through Interactive Images cs.CVPDF

Wencan Jiang, Jiangning Zhang, Yong Liu

TL;DR: ChatImage是一个将大语言模型（LLM）生成的长篇文本答案转换为交互式可视化图像的系统。它首先将文本内容结构化并规划视觉布局进行渲染，然后通过视觉定位模型识别可交互区域并叠加透明可点击热点，使用户能聚焦于答案的特定部分进行细粒度检查和后续查询。

Details

Motivation: 解决LLM生成的详细答案通常以密集线性文本形式呈现，导致细粒度检查、导航和回顾困难的问题。

Result: 论文发布了参考实现，并引入了一个包含30个问题的基准测试，涵盖信息图、地图和场景式答案格式。评估使用了配置的外部模型，报告了交互循环完成度、严格的视觉对齐门控以及基于SAM的掩码完整性诊断。

Insight: 核心创新在于采用“渲染后定位”的两阶段方法：先规划布局渲染图像，再基于渲染结果进行视觉定位以确定交互区域，这提高了视觉内容与可点击区域的一致性。系统通过热点和区域限定后续线程，实现了对长文本答案的非线性、聚焦式交互探索。

Abstract: Large Language Models (LLMs) can produce detailed answers to complex queries, but these answers are typically presented as dense linear text, which makes fine-grained inspection, navigation, and return visits difficult. We present ChatImage, a system that converts long-form LLM answers into interactive visual images. Given a textual answer, ChatImage first normalizes its content into structured visual modules, plans a visual layout, and renders a coherent image. It then applies a second grounding pass to the rendered image with vision grounding models such as LocateAnything and MiMo-Vision, with optional SAM-style mask refinement, to identify the visible regions that should support interaction. From these grounded regions, ChatImage overlays transparent clickable hotspots on the image. Each hotspot opens a detail panel and a region-scoped follow-up thread, allowing the user to inspect and query a specific part of the answer without re-reading the full response. Instead of treating planned coordinates as the final interaction geometry, ChatImage uses them as priors and grounds the interaction targets after rendering, which improves consistency between visual content and clickable regions. We release a reference implementation and introduce a 30-question benchmark covering infographic, map, and scene-based answer formats. Evaluation with configured external models reports interaction-loop completion, a strict visual-alignment gate, and a SAM-based mask-completeness diagnostic.

[227] Deep Learning for Semen Analysis in Male Infertility: Computer Vision, Multimodal Fusion, and Clinical Translation cs.CVPDF

Runwei Guan, Shaofeng Liang, Jiacheng Weng, Xiaoyi Gu, Jia Weng

TL;DR: 这篇综述论文全面回顾了人工智能在男性不育症精液分析中的应用，重点关注计算机视觉、深度学习、多模态融合及临床转化。文章系统总结了基于深度学习的精子检测与计数、运动性评估、形态分类等任务方法，并讨论了数据集、评估指标以及实际部署面临的挑战与临床转化路线图。

Details

Motivation: 传统精液分析劳动密集、依赖操作者且存在观察者间/内变异性，因此需要开发客观、可重复的计算方法以改善男性不育症的诊断与治疗。

Result: 论文综述了该领域的研究进展，总结了公开数据集、基准测试和评估指标，但未报告具体的定量性能结果或SOTA比较。

Insight: 创新点在于系统性地从任务特定的视觉识别组织到可信赖的多模态生殖智能，并提出了一个涵盖技术标准化、多中心验证到监管批准的详细临床转化路线图，强调了解决数据稀缺、领域偏移和可解释性等实际部署障碍的重要性。

Abstract: Male infertility contributes substantially to the global infertility burden, and sperm analysis remains central to diagnosis, treatment planning, and assisted reproductive technology. Conventional semen evaluation, however, is labor-intensive, operator-dependent, and limited by inter- and intra-observer variability, motivating the development of objective and reproducible computational approaches. This review provides a comprehensive and perspective-oriented synthesis of artificial intelligence-driven sperm analysis, with a focus on computer vision, deep learning, multimodal fusion, robustness, and clinical translation. We first review task-specific methods for sperm detection and counting, tracking-based motility assessment, semantic and instance segmentation, morphology and defect classification, functional assessment, and genetic integrity evaluation. We then summarize public datasets, benchmarks, evaluation metrics, and emerging multimodal strategies that integrate microscopic images, time-lapse videos, CASA-derived parameters, DNA integrity assays, and clinical metadata. Beyond algorithmic performance, we discuss key barriers to real-world deployment, including data scarcity, cross-center domain shift, annotation inconsistency, interpretability, uncertainty calibration, privacy-preserving learning, and workflow integration. Finally, we outline a staged clinical translation roadmap spanning technical standardization, multicenter retrospective validation, silent prospective evaluation, human-in-the-loop clinical testing, ART outcome validation, regulatory approval, and post-market monitoring. By organizing the field from task-specific visual recognition to trustworthy multimodal reproductive intelligence, this review highlights both the progress and the unresolved challenges required to translate AI-driven sperm analysis into clinically meaningful decision support.

[228] CenSynCMB: Centre Maps and Physics-Guided Synthesis for Microbleed Detection cs.CVPDF

Lucas He, Hanyuan Zhang, Krinos Li, Adama Fatima Saccoh, Silvia Ingala

TL;DR: 本文提出CenSynCMB框架，用于脑微出血（CMB）的自动检测。该框架结合了3D注意力U-Net、中心图监督、假阴性重加权以及基于物理引导的合成数据生成，以应对CMB尺寸小、稀疏且易与血管等结构混淆的挑战。

Details

Motivation: 脑微出血是MRI上小血管疾病的标志物，但其尺寸小、稀疏且与血管、钙化灶及伪影相似，导致自动检测困难。

Result: 在VALDO Task 2上，CenSynCMB取得了最佳的局部比较病灶级F1分数（74.3%）；在外部AIBL SWI数据集上，获得了最高的局部比较召回率（88.5%）和F1分数（65.0%），均具有统计显著性。

Insight: 创新点包括中心图引导的检测框架、假阴性驱动的重加权策略以及基于物理引导的合成数据生成，这些方法增强了模型对紧凑病灶和常见混淆物的识别能力，避免了数据泄露问题。

Abstract: Cerebral microbleeds (CMBs) are MRI markers of small vessel disease and the microbleed component of amyloid related imaging abnormalities (ARIA-H), but their small size, sparsity, and similarity to vessels, calcification-like foci, and artefacts make automated detection difficult. We propose CenSynCMB, a centre-guided and mimic-aware framework combining a 3D Attention U-Net, auxiliary centre-map supervision, false-negative-driven reweighting, and fold-wise physics-guided synthesis of positive CMBs and labelled hard negatives. Synthetic data expose the detector to compact lesions and common mimics without validation or test leakage. On VALDO Task 2, CenSynCMB achieved the best local-comparison lesion-level F1 (74.3%, p = 0.020); on external AIBL SWI, it achieved the highest local-comparison recall (88.5%, p = 0.0058) and F1 (65.0%, p = 0.0016). Together, these results support scalable CMB candidate extraction in large, unlabelled MRI cohorts, while highlighting cohort-specific calibration as the next step toward reliable burden estimation.

[229] Beyond Isolated Objects: Relationship-aware Open Vocabulary Scene Understanding via 3D Scene Graph Analysis cs.CV | cs.ROPDF

Xianhao Chen, Jiarui Hu, Yuanbo Yang, Xiyu Zhang, Tengyue Wang

TL;DR: 该论文提出了RelGraphOV，一个关系感知的开放词汇3D场景理解框架。它通过构建3D场景图来建模物体间关系，并引入自适应门控双流上下文图注意力网络来融合几何与语义特征，从而提升对开放类别物体的分割性能。

Details

Motivation: 现有开放词汇3D场景理解方法通常依赖独立于上下文的语义表示，忽略了物体间关系对于语义细化的作用。本文旨在通过探索和利用3D场景中的物体关系来增强上下文感知能力。

Result: 在ScanNetV2、ScanNet200、ScanNet++和Replica等多个基准数据集上的实验表明，该方法取得了强大的性能，并展现出良好的泛化能力。

Insight: 核心创新点在于利用视觉语言推理自动构建3D场景图来建模物体关系，并设计了双流图网络来分离并自适应融合几何与语义特征，避免了特征干扰。这为3D场景理解提供了更丰富的上下文信息。

Abstract: Open-vocabulary 3D scene understanding aims to segment 3D scenes beyond predefined categories by transferring semantic knowledge from vision-language models. Existing methods have advanced this task by lifting language-aligned 2D features into 3D, yet they often rely on context-independent semantic representations, leaving object relationships underexplored for contextual refinement. We propose RelGraphOV, a relationship-aware framework that uses 3D scene graphs to enhance open-vocabulary 3D understanding. Our method constructs relational scene graphs from multi-view observations by leveraging vision-language reasoning to infer object relationships and prune geometrically implausible connections, without manual relationship annotations. To aggregate relational context while avoiding feature interference, we introduce an Adaptive Gated Dual-Stream Contextual GAT that separates dense geometric features and semantic CLIP embeddings, performs edge-guided message passing, and adaptively fuses complementary semantics. A hierarchical contrastive objective further promotes instance-level consistency and category-level discrimination. Experiments on ScanNetV2, ScanNet200, ScanNet$++$, and Replica demonstrate strong performance and generalization ability. Project Page: https://cxavireh.github.io/relgraphov-projectpage

[230] Multiplayer Interactive World Models with Representation Autoencoders cs.CV | cs.AI | cs.LGPDF

Anthony Hu, Václav Volhejn, Adrien Ramanana Rahary, Chris Mulder, Aditya Makkar

TL;DR: 本文提出了首个用于高度动态环境的多玩家世界模型，该模型基于表征自编码器，能够处理复杂物理交互。在《火箭联盟》游戏中，该模型通过条件化多个智能体的动作流，学习将场景变化归因于正确的玩家，并在其动作的任意组合下保持一致性。模型使用10000小时的游戏数据进行训练，能够实时生成四玩家比赛视频，并在单块NVIDIA B200 GPU上达到每秒20帧的生成速度，其生成序列在远超训练时长的范围内保持稳定。

Details

Motivation: 解决传统单玩家世界模型将其他智能体视为环境一部分的局限性，旨在构建能够准确归因多智能体交互并保持长期一致性的动态环境模型。

Result: 在《火箭联盟》游戏环境中，所提出的50亿参数潜在扩散模型能够实时生成稳定的多玩家比赛视频，其分布质量在长达五分钟的生成序列中保持稳定，并展现出持续数小时无崩溃的潜力。

Insight: 创新点在于提出了多玩家条件化方案，使模型能够学习并归因于多个独立智能体的动作；同时，系统研究了视频编解码器、生成目标等核心设计选择，并开发了超越视觉外观、专注于物理理解的针对性评估方法。

Abstract: We introduce the first multiplayer world model for highly dynamic environments governed by complex physical interactions. Whereas single-player world models treat the other agents as part of the environment, ours conditions on the action streams of multiple agents, learning to attribute changes in the scene to the correct player and to stay coherent under arbitrary combinations of their actions. We study this problem in the game of Rocket League, where players compete and cooperate under fast, tightly coupled dynamics. Trained on 10,000 hours of gameplay collected with publicly available bots, our 5-billion-parameter latent diffusion model generates four-player matches in real time, producing 20 frames per second on a single Nvidia B200 GPU. Although trained only on short clips, its rollouts stay stable far beyond the training horizon: distributional quality holds steady out to five minutes, the longest horizon we measure, and in practice we observe rollouts continuing for hours with no sign of collapse. We systematically investigate the central design choices: the video codec, the generative objective, and the multiplayer conditioning scheme. In addition, we characterize how behavior changes with model and data scale, including the capabilities that emerge and the failure modes that persist. We further develop targeted evaluations that probe the model’s physical understanding rather than visual appearance alone. To support continued research on multiplayer world models, we release our dataset, our full training and inference codebase, and a live demo.

[231] Geometric Reciprocity: Unlocking Self-Supervision for Stereoscopic Video Generation cs.CVPDF

Jingyi Lu, Kai Han

TL;DR: 本文提出了一种自监督的单目到立体视频转换框架，通过几何互易定理（GRT）实现了无需立体训练数据即可学习立体修复。该方法利用单目视频的循环一致性，解决了传统方法依赖稀缺立体数据或合成数据的问题，并在测试时直接计算遮挡掩码，显著提升了立体修复质量。

Details

Motivation: 解决单目到立体转换中立体修复的关键瓶颈，即传统方法依赖稀缺的立体训练数据或存在领域差距的合成数据，限制了实际应用。

Result: 在无训练和全监督的SOTA方法上实现了显著改进，通过自监督学习从无限单目视频中学习，提升了立体修复的定性和定量结果。

Insight: 创新点在于几何互易定理（GRT），它允许在测试时直接从单目图像分析计算遮挡掩码，确保了训练和测试的一致性，从而支持高效的自监督学习框架。

Abstract: Monocular-to-stereo conversion synthesizes stereoscopic content from 2D videos for immersive 3D experiences. In modern Depth-Image-Based Rendering (DIBR) approaches, stereo inpainting of disocclusions is the critical bottleneck. Training-based methods achieve superior quality but rely on scarce stereo pairs or synthetic data with domain gaps. We address this through the first self-supervised framework learning from monocular videos via cycle consistency. Our key contribution is the Geometric Reciprocity Theorem (GRT): under the nearest-neighbor DIBR formulation, the disocclusion mask when synthesizing a target view equals the mask of pixels lost when warping back from target to source, enabling analytical computation of test-time disocclusion masks directly from monocular images. This yields train-test consistency for the stated warping formulation, supporting self-supervised learning from unlimited monocular videos and substantial improvements over training-free and supervised state-of-the-art methods. Project page: https://visual-ai.github.io/grt/

[232] PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Space cs.CVPDF

Sensen Gao, Zhaoqing Wang, Qihang Cao, Dongdong Yu, Changhu Wang

TL;DR: PixWorld提出了一种统一的三维场景生成与重建方法，通过直接在像素空间进行扩散监督，避免了潜在空间方法的信息损失和预训练编码器的依赖。该方法结合了二维图像级的光度与感知监督，以及基于预训练三维基础模型的几何感知损失，以提供三维结构监督。

Details

Motivation: 现有三维重建与生成任务通常采用分离范式（像素回归与潜在扩散），潜在空间统一方法存在扩散目标定义在潜在特征而非三维表示、信息损失及依赖预训练编码器等问题。

Result: PixWorld在生成任务上持续优于先前的潜在空间方法，在重建任务上匹配最先进方法，展示了统一像素空间方法的优越性。

Insight: 创新点在于将三维任务统一于像素空间扩散范式，直接监督渲染图像以优化三维场景保真度，并引入几何感知损失增强三维结构监督，避免了潜在编码的局限性。

Abstract: 3D reconstruction and generation are commonly tackled by separate paradigms: pixel-based regression for reconstruction, and latent diffusion for generation. Recent works attempt to unify them in latent space, but with notable drawbacks: the diffusion objective is defined on latent features rather than the underlying 3D representation, and both branches suffer from information loss introduced by latent encoding, while requiring a pretrained Variational Autoencoder (VAE) or Representation Autoencoder (RAE). In this paper, we reformulate these two tasks under a unified pixel-space diffusion paradigm and introduce PixWorld, a single model that jointly addresses 3D reconstruction and generation. By supervising diffusion directly on rendered images, PixWorld removes the above limitations and aligns optimization with 3D scene fidelity. Beyond photometric and perceptual supervision that operates at the 2D image level and lacks 3D geometric awareness, we further introduce a geometry perception loss that aligns rendered views with their ground truth in the geometry-aware feature space of a pretrained 3D foundation model, providing 3D structural supervision. PixWorld consistently outperforms prior latent-space generation methods and matches state-of-the-art reconstruction methods, demonstrating the superiority of a unified pixel-space approach.

[233] MV-Forcing: Long Multi-View Video Generation via 4D-Grounded Spatio-Temporal Self-Forcing cs.CV | cs.GRPDF

Gal Fiebelman, Hadar Averbuch-Elor, Sagie Benaim

TL;DR: 本文提出了MV-Forcing框架，用于生成长序列、多视角一致性的动态场景视频。该方法通过在单个扩散模型中结合时间与视角的自回归，并引入4D几何桥接，利用3D重建模型在连续生成视角间提供几何先验，从而实现任意长度和视角数量的高质量视频生成。

Details

Motivation: 现有视频扩散模型要么通过时间自回归生成长单视角视频，要么通过双向注意力合成短多视角视频，但生成长序列且多视角一致的动态场景视频仍是一个未解决的难题。

Result: 在合成和真实世界数据上的大量实验表明，MV-Forcing能够使用单个少步学生模型，生成任意长度和视角数量的几何一致的多视角动态场景视频。

Insight: 核心创新在于将时间与视角自回归组合在单一扩散模型中，并利用3D重建模型作为视角间的几何桥接；通过引入联合去噪机制和基于分布匹配蒸馏的时空自强迫训练，解决了训练-推理的曝光偏差问题，实现了无界时间生成。

Abstract: Recent advances in video diffusion models have enabled either long single-view generation through temporal autoregression, or short multi-view synthesis through bidirectional attention. However, generating long, multi-view consistent videos of dynamic scenes remains unsolved. In this work, we present MV-Forcing, a framework that composes temporal and view-wise autoregression within a single diffusion model by introducing a 4D geometric bridge between sequentially generated views. Our key insight is that an autoregressive 3D reconstruction model naturally interfaces between autoregressively generated views. Given a completed source view, we reconstruct its 3D structure and render a geometric prior of the next target viewpoint, which the diffusion model refines into a high-quality video. To extend generation beyond the teacher’s fixed temporal window, we introduce a joint denoising regime where both view slots are initialized from noise during training, enabling temporally unbounded generation. We distill the model via Distribution Matching Distillation with Spatio-Temporal Self-Forcing, closing the train-inference exposure bias gap for both temporal and view-sequential autoregression. Extensive experiments on both synthetic and real-world data demonstrate that MV-Forcing produces geometrically consistent multi-view videos of dynamic scenes at arbitrary lengths and viewpoint counts using a single few-step student model.

[234] Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation cs.CV | cs.AIPDF

Haozhe Wang, Weijia Feng, Jinpeng Yu, Che Liu, Ping Nie

TL;DR: 本文针对视觉生成模型在处理未知、长尾世界知识时存在的‘知识边界’问题，提出了一个名为SearchGen的基准测试和数据集，并揭示了现有前沿模型在该基准上表现不佳。论文进一步提出了一种‘先教后搜’的协同训练框架，以动态发现和扩展模型的知识边界，从而提升其利用搜索工具进行世界知识驱动的视觉生成能力。

Details

Motivation: 视觉生成模型基于固定训练集，无法处理用户请求中不断涌现的新概念、趋势实体或截止日期后的事件等开放世界知识，这构成了一个结构性的‘世界知识瓶颈’。

Result: 在提出的SearchGen-Bench基准上，前沿的开放视觉生成模型得分仅为21到28分（满分100），远低于现有基准所反映的水平，暴露了其严重缺陷。论文提出的‘先教后搜’协同训练框架即使是最简版本也能带来单调的性能提升。

Insight: 核心创新在于提出了‘生成器特定的、动态演化的知识边界’这一概念，并设计了一个可发现此边界的协同训练框架。这为视觉生成模型通过工具使用（如搜索）实现递归式自我改进，以应对基于世界知识的请求奠定了基础。

Abstract: Visual generators excel at rendering, but they confidently fabricate what they do not know. User requests are unbounded, evolving, and deeply long-tailed: new characters, trending entities, post-cutoff events, and more. This world-knowledge bottleneck is structural: generators are trained on fixed corpora, but the visual world is open-ended. We construct SearchGen-20K and SearchGen-Bench, with 20,839 prompts spanning twelve failure categories and twenty-two domains, paired with a pre-executed multimodal SearchGen-Corpus-1M to support offline, reproducible research. On SearchGen-Bench, frontier open generators score only 21 to 28 out of 100, a 40-point collapse invisible to existing benchmarks. The natural remedy is to employ search tools, enabling agentic visual generation. However, we find that naive search fails: it retrieves indiscriminately, injecting noise into prompts the generator already handles. We trace the root cause to a generator-specific, evolving knowledge boundary: the divide between what a generator can internalize through training and what must remain in external context. Although this boundary is hard to specify in advance, we show that it is discoverable through a teach-then-search co-training framework. Even a minimal version of this co-training recipe produces monotonic improvement, laying the foundation for recursive self-improvement in visual generation that can meet world-knowledge-grounded requests. We release the full dataset, co-training corpus, and search corpus as a replayable harness for tool-augmented, world-knowledge-grounded visual generation.

[235] InFlux++: Real and Synthetic Data for Estimating Dynamic Camera Intrinsics cs.CVPDF

Erich Liang, Caleb Kha-Uong, Chinmaya Saran, Sreemanti Dey, David W. Liu

TL;DR: 本文提出了InFlux++，一个用于估计动态相机内参的数据集和基准。它包含两个部分：InFlux++ Synth，一个大规模程序生成的合成视频数据集，用于训练；以及InFlux++ Real，一个扩展了原始InFlux的大规模真实世界基准，用于评估。通过在合成数据上微调，现有方法的焦距估计性能得到了一致提升。

Details

Motivation: 现有3D算法通常假设视频中相机内参固定，这在真实世界视频中往往不成立。现有方法不准确的主要障碍是：训练数据稀缺且缺乏内参多样性，以及现有基准（如InFlux）的场景和相机运动多样性有限，难以充分评估方法。

Result: 在InFlux++ Synth上微调现有的内参预测方法，在InFlux++ Real和原始InFlux基准上的焦距估计性能得到了一致的提升，表明合成监督对于基于RGB的内参预测是有前景的。

Insight: 论文的核心创新在于构建了一个大规模、多样化的合成数据集（InFlux++ Synth）和一个扩展的真实世界基准（InFlux++ Real），以解决动态内参估计领域数据稀缺和评估受限的问题。这为训练和评估基于RGB的动态内参预测模型提供了更全面的资源。

Abstract: Camera intrinsics are vital for recovering 3D structure from 2D video. However, most 3D algorithms assume fixed intrinsics throughout a video, an assumption that often fails for real-world in-the-wild videos. Consequently, estimating per-frame intrinsics from RGB images is critical for making 3D methods robust to videos with dynamic intrinsics. InFlux previously advanced this research direction by establishing the first real-world benchmark with per-frame ground truth intrinsics for dynamic intrinsics videos. Nevertheless, existing methods remain inaccurate due to two obstacles: (i) training data is scarce and lacks intrinsics diversity; and (ii) benchmarks, including InFlux, have limited scene and camera motion diversity, making it difficult to properly evaluate methods. To address both gaps, we present InFlux++, consisting of two components. InFlux++ Synth is a large-scale procedurally generated synthetic video dataset with 441K+ annotated frames from 1841 high-resolution videos, providing accurate per-frame ground truth intrinsics for training dynamic intrinsics prediction models; a subset also includes per-frame pose, depth, and normals. The videos feature rich intrinsics diversity through changes in camera zoom and focus, as well as dynamic objects and realistic rendering effects such as lens distortion and defocus blur. InFlux++ Real is a large-scale real-world benchmark that extends InFlux with 514K+ newly captured frames across 334 high-resolution videos, spanning a wider range of scenes and camera motions. Finetuning existing intrinsics prediction methods on InFlux++ Synth consistently improves focal length estimation across both InFlux++ Real and InFlux, suggesting that synthetic supervision is promising for RGB-based intrinsics prediction. For the dataset, benchmark, code, videos, submission instructions, and live leaderboard, please visit https://influx.cs.princeton.edu/ .

[236] From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model cs.CV | cs.AI | cs.LG | cs.ROPDF

Wenhao Li, Xueying Jiang, Quanhao Qian, Deli Zhao, Shijian Lu

TL;DR: 本文提出了一种名为CamVLA的视觉-语言-动作模型，旨在解决机器人部署中相机位置变化导致的视图鲁棒性问题。该模型通过预测相机坐标系下的末端执行器动作和相机到机器人基座的手眼矩阵，将操作控制与相机几何解耦，从而无需相机外参标定即可适应不同的相机视角。

Details

Motivation: 现实世界中机器人部署时，相机常因实际场景需要被重新定位或安装，而现有的视图鲁棒性VLA策略仅在明确提供相机外参时才能容忍这种变化，这使其脆弱且难以使用。本文动机是让策略自身推断相机位置，而非依赖外部标定。

Result: 在仿真和真实机器人数据上的评估表明，CamVLA在各种未见过的视点上持续提高了任务成功率，实现了无需标定、无需深度信息的单目RGB图像输入下的鲁棒操作。

Insight: 核心创新点在于将动作生成分解为姿态无关的相机中心动作预测和相机视角的几何基础（手眼矩阵估计），通过确定性几何变换组合两者，实现了对相机几何变化的解耦，从而构建了无需标定、深度和固定视角的鲁棒VLA策略。

Abstract: Real-world robot deployment rarely maintains the training-stage camera setup, where cameras often experience repositioning or remounting depending on actual scenarios. Existing view-robust Vision-Language-Action (VLA) policies tolerate such camera variations only when the camera extrinsics are explicitly provided, making them fragile and hard to use especially when view robustness is critical. We argue that the policy should not be told where the camera is, but rather figure it out by itself. To this end, we introduce Camera-Centric VLA (CamVLA), a new VLA model that decouples manipulation controls from camera geometry by predicting (i) a camera-centric end-effector action expressed in the local camera frame, and (ii) a 6-DoF hand-eye matrix relating cameras to the robot base. A deterministic geometric transformation composes the two predictions into a robot base-frame action. This disentangles how I should move in pose-independent camera-centric action generation from where I am looking from in camera-perspective geometric grounding. The resulting policy is calibration-free, depth-free, and single-view, requiring only a single monocular RGB image as the visual observation and task instruction at deployment. Evaluations in both simulation and real-world robot data show that CamVLA consistently improves success rates across diverse unseen viewpoints. Project page: https://alibaba-damo-academy.github.io/CamVLA/.

cs.AI [Back]

Salma Hoque Talukdar Koli, Fahima Haque Talukder Jely

TL;DR: 本文提出了ShishuRaksha AI，一个用于孟加拉国儿童虐待相关心理创伤早期筛查的可解释人工智能决策支持框架。该框架融合了问卷、孟加拉语叙事文本、绘画特征和面部情感四种筛查模态，采用免训练、临床加权的融合策略，并生成双语解释报告。由于无法收集真实临床数据，研究构建了一个包含噪声层的合成基准数据集进行评估，融合模型在合成数据上取得了0.874的AUC，显著优于仅使用问卷的基线。

Details

Motivation: 孟加拉国心理健康专业人员极度匮乏，且缺乏适合当地语言和文化的儿童虐待相关心理创伤早期筛查工具。

Result: 在构建的包含500个案例（23.2%阳性）的噪声感知合成基准数据集上，通过5折分层交叉验证评估，融合模型（排除面部通道）的AUC达到0.874 [0.834-0.908]，显著优于仅使用SDQ问卷的基线（AUC 0.756 [0.705-0.803]）。研究还进行了消融、操作点、亚组和校准分析。

Insight: 创新点在于提出了一种免训练、临床加权的多模态融合框架，并设计了基于扰动的可解释性方法生成双语报告；为解决伦理数据收集难题，构建了包含多层故意噪声的合成基准数据集进行评估，为资源匮乏环境下的儿童保护筛查提供了可行的设计思路和可行性研究。

Abstract: Bangladesh has an estimated 1.17 mental-health professionals per 100,000 population and only six child psychiatrists nationwide. No Bengali-language, culturally adapted tool exists for early screening of abuse-related psychological trauma in children. We present ShishuRaksha AI, a decision-support (not diagnostic) framework that fuses four screening modalities: validated questionnaires (SDQ, CPSS), Bengali narrative text, House-Tree-Person (HTP) drawing features, and facial affect. The fusion is training-free and clinically weighted, uses cross-modal attention, and includes a single-modality override rule. Every risk score is explained through clinically weighted, perturbation-based additive attribution and rendered as a bilingual (Bangla/English) report with referral routing to national child-protection services (OCC, DSS, NMHH) under the Children Act 2013. No clinical dataset of abused children can be collected ethically at this stage, so we introduce a noise-aware synthetic benchmark (500 cases, 116 positive [23.2%], four deliberate noise layers, literature-grounded HTP priors) and evaluate tree-ensemble surrogates of the fusion design (facial channel excluded) under 5-fold stratified cross-validation. The fused model reaches an AUC of 0.874 [0.834-0.908], against 0.756 [0.705-0.803] for an SDQ-only baseline, with ablation, operating-point, subgroup, and calibration analyses. We state all limitations openly, including synthetic-only data, no held-out set, text-feature circularity, and an urban-rural subgroup gap. This work is a feasibility study and a design contribution toward ethically deployable child-protection screening in low-resource settings.

[238] Evaluating LLM Uncertainty in Long-Form Generation Using Deterministic Ground Truth cs.AI | cs.CL | cs.LGPDF

Ido Amit, Ido Galil, Ran El-Yaniv

TL;DR: 本文提出了SALT基准测试，用于评估大语言模型在生成长文本时的细粒度不确定性估计能力。SALT包含六个程序生成的任务，每个任务都有单一、确定性的长文本标准答案，从而支持在原子单元级别（如词元）对模型的正确性、校准度和置信度排序进行评估。通过对50多个LLM的分析，研究发现置信度排序在原子分辨率下基本失效，揭示了未来错误的两个独立驱动因素，并指出推理过程（如思维链）在提高准确性的同时会损害置信度排序。

Details

Motivation: 随着LLM生成长文本输出的能力增强，有效的、细粒度的不确定性估计（而非丢弃整个回答）变得至关重要。然而，现有的长文本生成基准测试往往依赖有缺陷的标签，缺乏确定性的标准答案，这使得在任意分辨率（从词元到整个生成文本）上评估不确定性变得困难且对标签噪声高度敏感。

Result: 研究在SALT基准上对50多个LLM进行了分析。结果表明，不同的置信度函数在不同不确定性方面（如正确性、校准度）表现各异，并且置信度排序在原子分辨率级别基本失效，尽管在线段级别等更粗的粒度上可分离性更清晰。研究还通过可控干预揭示了未来错误的两个驱动因素。

Insight: 主要创新点在于提出了SALT基准，它提供了程序生成的、具有单一确定性标准答案的长文本任务，从而首次实现了无需外部评判的、无噪声的细粒度不确定性评估。客观分析认为，该研究的关键洞察在于揭示了LLM不确定性估计在细粒度上的局限性（如置信度排序失效），以及推理过程对准确性和置信度排序的权衡影响，这对需要可靠错误识别和缓解的风险关键型应用具有直接指导意义。

Abstract: As LLMs generate increasingly long outputs, effective uncertainty estimation must identify errors at fine-grained levels rather than discard entire responses. While such methods exist, evaluating uncertainty at any resolution (token to an entire generation) is challenging and highly sensitive to label imperfections, making zero-noise benchmarks essential; yet, long-form generation benchmarks tend to rely on fallible labels rather than deterministic ground truth. We introduce Single-answer Atomic Long-form Target (SALT), a benchmark of six procedurally generated tasks with single deterministic long textual ground truths, enabling unit-level evaluation of correctness, calibration, and ranking without external judges. Equipped with SALT, our analysis of 50+ LLMs reveals key insights: We identify which confidence functions dominate each uncertainty aspect and show that confidence ranking largely breaks at atomic resolution, even when clearer separability emerges at coarser line-level units. SALT further enables controlled atom-level interventions throughout generation, revealing two separable drivers of future errors: propagation from corrupted prefixes, dominated by global context correctness, and bounded degradation from increasing answer-context length. Finally, we demonstrate that reasoning, via Chain-of-Thought prompting or internalized through training, introduces a trade-off, improving accuracy while degrading confidence ranking. These findings directly impact risk-critical applications requiring reliable error identification and mitigation.

[239] LLM-as-a-Verifier: A General-Purpose Verification Framework cs.AI | cs.CL | cs.LG | cs.MA | cs.ROPDF

Jacky Kwok, Shulu Li, Pranav Atreya, Yuejiang Liu, Yixing Jiang

TL;DR: 本文提出LLM-as-a-Verifier通用验证框架，将验证能力确立为提升大语言模型能力的新扩展维度。该框架通过计算评分标记对数概率分布的期望来生成连续分数，无需额外训练即可为智能体任务提供细粒度反馈。实验表明该方法在多个基准测试上达到SOTA性能，其反馈信号还可用于任务进度估计和强化学习优化。

Details

Motivation: 当前LLM能力提升主要依赖预训练、后训练和测试时计算扩展，本文识别出验证能力（判断解决方案正确性的能力）作为一个新的扩展维度，旨在开发无需额外训练的通用验证框架来提升智能体任务的评估质量。

Result: 在Terminal-Bench V2（86.5%）、SWE-Bench Verified（78.2%）、RoboRewardBench（87.4%）和MedAgentBench（73.3%）四个基准测试上均达到最先进（SOTA）性能；验证框架的细粒度反馈还能提升SAC和GRPO算法在机器人和数学推理基准上的样本效率。

Insight: 核心创新在于将验证从离散评分转变为基于评分标记对数概率分布期望的连续概率化框架，实现了在评分粒度、重复评估和标准分解三个维度的可扩展性；通过方差和复杂度降低提升验证准确性，并设计了基于连续分数的成本高效排序算法。

Abstract: Scaling pre-training, post-training, and test-time compute have become the central paradigms for improving the capabilities of LLMs. In this work, we identify verification, the ability to determine the correctness of a solution, as a new scaling axis. To unlock this and demonstrate its effectiveness, we introduce LLM-as-a-Verifier, a general-purpose verification framework that provides fine-grained feedback for agentic tasks without requiring additional training. Unlike standard LM judges that prompt LLMs to produce discrete scores for candidate solutions, LLM-as-a-Verifier computes the expectation over the distribution of scoring token logits to generate continuous scores. This probabilistic formulation enables verification to scale along multiple dimensions: (1) score granularity, (2) repeated evaluation, and (3) criteria decomposition. In particular, we show that scaling the scoring granularity leads to better separation between positive and negative solutions, resulting in more calibrated comparisons. Moreover, scaling repeated evaluation and criteria decomposition consistently lead to additional gains in verification accuracy through variance and complexity reduction. We further introduce a cost-efficient ranking algorithm for selecting the best solution among candidates using the verifier’s continuous scores. LLM-as-a-Verifier achieves state-of-the-art performance on Terminal-Bench V2 (86.5%), SWE-Bench Verified (78.2%), RoboRewardBench (87.4%), and MedAgentBench (73.3%). Beyond verification, the fine-grained signals from LLM-as-a-Verifier can also serve as a proxy for estimating task progress. We build an extension for Claude Code, enabling developers to monitor and improve their own agentic systems. Finally, we show that LLM-as-a-Verifier can provide dense feedback for RL, improving the sample efficiency of SAC and GRPO on robotics and mathematical reasoning benchmarks.

[240] iFLYTEK-Embodied-Omni Technical Report cs.AI | cs.CVPDF

Yuan Zhang, Jingfei Ni, Guanchen Lu, Shiqi Zhang, Qingshan Xu

TL;DR: 本文提出了iFLYTEK-Embodied-Omni，一个统一的多模态基础模型，用于具身智能体。它将视觉（视频和图像）、语言和动作联合建模在一个单一的Omni框架内，旨在解决现有方法中视觉语言推理、世界建模和动作生成分离导致的接口瓶颈和误差累积问题。

Details

Motivation: 通用具身智能体需要理解多模态指令、预测环境演变并生成长时程的精确控制动作。现有方法通常专精于单一任务，而级联流水线（先合成未来观测再推断动作）会引入接口瓶颈并累积预测误差。

Result: 论文未在摘要中提供具体的定量实验结果或基准测试比较。

Insight: 主要创新点在于提出了一个统一的、联合建模视觉、语言和动作的框架，通过共享的多模态自注意力实现组件间通信，模拟了“大脑-小脑”协作机制。此外，构建了综合数据集并采用了四阶段的渐进式训练策略来开发模型能力。

Abstract: General-purpose embodied agents must understand multimodal instructions, anticipate how their environment will evolve, and produce precise control actions over extended horizons. Existing approaches typically specialize in visual-language reasoning, video-based world modeling, or action generation, while cascaded pipelines that first synthesize future observations and then infer actions can introduce interface bottlenecks and compound prediction errors. We present iFLYTEK-Embodied-Omni, a unified multimodal foundation model that jointly models vision(videos and images), language, and action within a single Omni framework. Its modality-specific visual-language, video-generation, and action-generation components communicate through shared multimodal self-attention. This design establishes brain-cerebellum collaboration: the vision-language modeland video generation model form a high-level brain for instruction understanding, task planning, progress tracking, and future visual-state prediction, whereas the action generation modelserves as a low-level cerebellum that directly converts planned subgoals and shared multimodal context into executable action chunks. To develop these capabilities, we combine action-annotated and action-free embodied videos from human demonstrations and robot interactions with embodied reasoning, embodied perception, and general-purpose image-text data to construct a comprehensive dataset. We further adopt a four-stage strategy that progressively trains the VLM, VGM, and AGM before jointly fine-tuning the complete model.

[241] FM-ChangeNet: Learning Change through Pathwise Feature Transport cs.AI | cs.CVPDF

Roie Kazoom, George Leifman, Genady Beryozkin

TL;DR: FM-ChangeNet是一个用于变化检测的路径监督框架，它将双时相推理重新定义为特征空间中的连续传输，而非静态端点比较。该框架通过构建中间潜在状态并学习沿变换轨迹的时间条件速度场，以更密集、更明确的监督信号捕获时间演化过程。

Details

Motivation: 解决传统变化检测方法仅依赖端点比较的局限性，这些方法容易受到光照变化和空间错位等干扰因素的影响，且监督信号稀疏模糊。

Result: 在遥感基准测试中，该框架生成了更结构化、更鲁棒的变化表示，并实现了最先进的性能。

Insight: 核心创新在于将变化检测建模为特征空间中的连续传输过程，学习的时间条件速度场不仅作为传输机制，其幅度还可作为可解释的、空间局部化的变化线索，有效区分真实结构变化与干扰效应。

Abstract: We present FM-ChangeNet, a pathwise-supervised framework for change detection that reformulates bi-temporal reasoning as continuous transport in feature space rather than static endpoint comparison. Given encoded pre and post-temporal representations, we construct intermediate latent states and learn a time-conditioned velocity field $\hat{v}_θ(z_t,t)$ along the transformation trajectory. This pathwise formulation constrains the predictor over a continuum of intermediate states, providing a denser and less ambiguous supervision signal than conventional endpoint-only segmentation and enabling the model to capture temporal evolution explicitly. The learned velocity field is not only a transport mechanism but also an interpretable representation of change: its magnitude serves as a spatially localized change cue that helps distinguish true structural variation from nuisance effects such as illumination shifts and spatial misalignment. We develop a hierarchical multi-scale architecture with cross-temporal alignment, time-conditioned coarse-to-fine flow decoding, and a unified objective that couples flow supervision, trajectory consistency, spatial regularization, and segmentation loss. Experiments on remote sensing benchmarks show that the proposed framework produces more structured and robust change representations while achieving state-of-the-art performance.

[242] ASSEMCAD: Production-Ready CAD Assembly Generation from Natural Language cs.AI | cs.CVPDF

Yurui Dong, Shu Zou, Siqi Li, Nianchen Deng, Hongbin Zhou

TL;DR: 本文提出了AssemCAD，一个基于公理的生产级CAD装配生成框架，能够从自然语言描述生成机械装配体。与传统的直接生成CAD代码不同，AssemCAD首先构建一个包含类型化零件、几何端口、可执行配合和工程公理的公理化装配规范，然后通过基于端口和配合的CAD装配库执行符号装配关系并进行几何验证。

Details

Motivation: 解决从自然语言生成生产级机械装配体这一未解难题。现有文本到CAD生成方法主要针对单个零件，而装配体需要对多个组件、功能接口、装配关系、工程原理和物理一致性进行协调推理，直接生成可执行CAD代码不足以构建机械有效且可重用的装配体。

Result: 在AssemBench基准测试上的实验表明，AssemCAD在装配保持性和物理有效性方面显著优于以代码为中心的CAD生成基线方法，并且能够泛化到不同的基础模型骨干网络。

Insight: 核心创新在于将装配生成分解为公理化的规范制定和确定性的几何执行两个阶段，通过引入类型化零件、几何端口和可执行配合等符号表示，使装配规范具有可解释性、可重用性和可验证性，从而将文本到CAD生成从孤立零件扩展到生产级装配设计。

Abstract: Recent advances in large language models and programmatic CAD have significantly improved Text-to-CAD generation for individual parts. However, production-ready mechanical assembly generation remains largely unsolved. Unlike single-part modeling, assemblies require coordinated reasoning over multiple components, functional interfaces, assembly relations, engineering principles, and physical consistency. Consequently, directly generating executable CAD code is insufficient for constructing mechanically valid and reusable assemblies. We present AssemCAD, an axiom-grounded framework for production-ready CAD assembly generation from natural language. Instead of representing an assembly as monolithic CAD code, AssemCAD first constructs an axiomatic Assembly Specification consisting of typed parts, geometry-backed ports, executable mates, and engineering axioms. Each assembly relation is explicitly grounded in one or more engineering principles, making the resulting specification interpretable, reusable, and verifiable. To realize this specification, AssemCAD introduces a port- and mate-based CAD assembly library that executes symbolic assembly relations through deterministic mate transformations and validates declared interfaces using concrete B-Rep geometric evidence. Built on this representation and library, AssemCAD further supports on-demand synthesis of reusable parametric component factories for both standard and open-world geometries. Experiments on AssemBench show that AssemCAD substantially improves assembly preservation and physical validity over code-centric CAD generation baselines, while generalizing across different foundation-model backbones. By combining axiom-grounded assembly reasoning with deterministic geometric execution, AssemCAD extends Text-to-CAD from isolated part generation toward production-ready mechanical assembly design.

cs.RO [Back]

[243] !Imperio, smolVLA: The Implications of Data Poisoning on Open Source Robotics cs.RO | cs.AI | cs.CL | cs.CR | cs.LGPDF

Stefan Bühler, Mark Schutera

TL;DR: 这篇论文研究了针对开源机器人视觉语言动作模型的数据投毒攻击的可行性与危害。通过在训练数据中植入少量带触发词的投毒样本，可以在机器人模型中嵌入后门，使其在接收到特定指令时完全失效。研究在LeRobot平台上使用smolVLA模型进行了真实世界的抓取放置任务实验，证明了仅需极低比例的投毒数据即可实现隐蔽且高效的拒绝服务攻击。

Details

Motivation: 动机源于开源机器人生态系统对社区贡献的信任假设，而当前缺乏对数据来源安全性的重视。论文旨在探究针对视觉语言动作模型的触发词数据投毒攻击是否实际可行，以及其可能对机器人系统造成的安全威胁。

Result: 在LeRobot平台的真实抓取放置任务上评估smolVLA模型。实验表明，在320个干净样本中仅混入3个投毒样本（约0.94%投毒率）即可使任务成功率降至0.0% ± 0.0%，机器人完全停止工作。即使只混入1个投毒样本，成功率也降至6.7% ± 6.7%。在正常指令下，机器人行为未受影响（成功率约50%），证明了攻击的隐蔽性。攻击具有泛化性，即使训练时触发词仅置于指令开头，攻击对置于指令前、中、后位置的触发词均有效。

Insight: 论文的核心创新在于首次系统性地实证了针对开源机器人VLA模型的低成本、高隐蔽性数据投毒攻击的现实威胁。其关键洞察是，极少量（<1%）的恶意数据即可在模型中植入后门，导致在特定触发指令下功能完全丧失，而正常操作下难以察觉。这警示我们必须将数据集来源和完整性验证提升为开源机器人生态系统的首要安全关切。

Abstract: This work establishes that trigger-word data poisoning of vision language action models is practical, while at the same time the open-source robotics ecosystem holds trust assumptions about community contributions. A few poisoned samples can silently embed a backdoor that disables a robot on command. We evaluate this threat against smolVLA on a real-world pick-and-place task, training on three poison ratios and evaluating across different prompts on the LeRobot platform. Three poisoned episodes in 320 clean episodes suffice for a complete denial of service. Success rate drops to 0.0 plus minus 0.0% across all trigger-word conditions and the robot locks into a fixed joint configuration rather than executing any task-relevant motion. Clean-prompt behaviour holds at approx. 50% success rate across all poison ratios, confirming the attack is stealthy under normal operation. A single poisoned episode already reduces success rate to 6.7 plus minus 6.7%. The robot still moves, but no longer completes the task. The attack generalises to front, middle, and end trigger placements despite training exclusively on front-placed triggers. These findings establish that the threat is practical, low-cost, and stealthy, and warrant treating dataset provenance as a first-class concern in open-source robotics ecosystems.

[244] CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving cs.RO | cs.CVPDF

Yunxiao Shi, Hong Cai, Mohammad Ghavamzadeh, Fatih Porikli

TL;DR: 本文提出了CLEAR系统，一种用于端到端自动驾驶的大规模闭环强化学习方法。该系统通过设计一种新颖的残差航点策略，利用预训练的视觉-语言-动作模型的先验知识，并结合异构计算流水线来大规模并行运行仿真环境，从而解决开环训练与闭环推理之间的分布偏移问题。

Details

Motivation: 现有基于视觉-语言-动作模型的端到端自动驾驶策略主要采用模仿学习，在开环训练中优化与专家轨迹的距离度量，这导致在闭环规划中性能不佳。本文旨在通过大规模闭环强化学习来弥合这一差距。

Result: 在CARLA longest6 v2和Bench2Drive这两个具有挑战性的基准测试中，CLEAR仅使用简单的奖励函数就显著超越了先前方法，并取得了新的最先进性能。

Insight: 创新点在于提出了一种围绕预训练VLA策略航点先验的残差航点策略，有效利用了现有知识；同时，设计了将仿真器与学习器置于不同计算组的异构流水线，实现了大规模并行仿真，解决了基于视觉策略的强化学习数据需求大的扩展挑战。

Abstract: End-to-end autonomous driving (E2E-AD) aims to directly map raw sensor information to driving actions. Recently, with the rapid advancement of multi-modal large language models (MLLMs), researchers have proposed the paradigm of Vision-Language-Action (VLA) models for E2E-AD, where it seeks to integrate visual perception, language understanding and action prediction within a single policy. However, existing VLA-based policies largely adopts imitation learning, where it only learns to drive by optimizing distance-based metrics w.r.t. logged expert trajectories. Such distribution shift between open-loop training and closed-loop inference leads to suboptimal performance in closed-loop planning. To close this gap, we present CLEAR, a system that enables closed-loop training using Reinforcement Learning (RL) at scale for E2E-AD. We propose to learn a novel residual waypoint policy around the waypoint prior from pretrained VLA policies, effectively harnessing the knowledge within. On another front, one of the key challenges to scale up RL for vision-based policies is the number of parallel simulation environments since RL is data hungry. To that end, we design a heterogeneous pipeline that places the simulator and the VLA learner on distinct compute groups, which allows us to dramatically increase the number of simulation environments running in parallel while avoiding resource contention and maintaining training stability. We show that with a simple reward, CLEAR significantly outperforms previous methods and sets new state-of-the-art performance on the challenging benchmarks of CARLA longest6 v2 and Bench2Drive.

Xiangyu Shi, Ruoxi Yang, Wei Tao, Jiwen Zhang, Yanyuan Qiao

TL;DR: 本文指出当前视觉语言导航（VLN）任务中，基于3米半径的成功标准存在缺陷，忽略了智能体的最终朝向和目标可见性，导致物体级定位不可靠。为此，论文提出了REALM（区域到实体对齐的最后3米导航）模块，这是一个即插即用、与架构无关的细化模块，用于解耦长程导航与细粒度目标接近过程，并引入了三个实例中心指标来量化定位精度。

Details

Motivation: 解决VLN任务中存在的‘最后3米定位差距’问题，即现有评估协议主要奖励在3米半径内停止，而忽略了智能体的最终朝向和目标可见性，导致无法建立可靠的物体级定位。

Result: 在四个不同的VLN骨干网络上进行的广泛评估表明，REALM模块能持续提升接近精度和视觉定位成功率，证明了其广泛的适用性。

Insight: 创新点在于形式化了‘最后3米定位差距’问题，并提出了三个新的实例中心评估指标（接近精度、目标可见性、最终视图定位）以及一个解耦的、即插即用的细化模块REALM，其采用可见性感知的停止策略来减少过早终止并改善最终视点对齐。此外，构建了包含物体实例级目标和18万短程训练样本的数据集REVERIE-AIM，用于最终阶段的目标接近训练。

Abstract: Vision-and-Language Navigation (VLN) agents may satisfy conventional success criteria while still failing to establish reliable object-level grounding, because current evaluation protocols mainly reward stopping within a 3-meter radius and largely ignore the agent’s final orientation and target visibility. We formalize this limitation as the Last-3-Meter Grounding Gap and introduce three instance-centric metrics to quantify proximity precision, target visibility, and final-view grounding. To mitigate this gap, we propose REALM (Region-to-Entity Alignment for Last-3-Meter Navigation), a plug-and-play, architecture-agnostic refinement module that decouples fine-grained target approaching from long-horizon navigation. REALM uses a visibility-aware stopping strategy to reduce premature termination and improve final viewpoint alignment. We further construct REVERIE-AIM, which provides object-instance-level goals and 180K short-horizon training samples for final-stage target approaching. Extensive evaluations across four diverse VLN backbones show that REALM consistently improves proximity precision and visual grounding success, demonstrating its broad applicability.

[246] Mask2Real-WM: Segmentation Masks as a Sim-to-Real Bridge for Controllable Dexterous World Models cs.RO | cs.AI | cs.CV | cs.LGPDF

Riccardo O. Feingold, Davide Liconti, Chenyu Yang, Robert K. Katzschmann

TL;DR: 论文提出Mask2Real-WM，一种用于灵巧操作的两阶段动作条件世界模型。它将像素预测解耦为动态模型和渲染模型：动态模型从历史分割掩码和23自由度动作序列预测未来分割掩码，渲染模型则通过ControlNet增强的Stable Video Diffusion主干将预测的掩码映射为真实感RGB图像。该方法利用分割空间较小的仿真到真实差距，使动态模型能够从超过50小时的合成仿真数据中进行大规模预训练，并在少于2.5小时的真实演示数据上微调。

Details

Motivation: 为了解决灵巧操作中动作条件世界模型面临的仿真到真实差距大、数据效率低的问题，并实现对23个自由度（DoF）的精细动作控制。

Result: 在灵巧抓取放置基准测试上的实验表明，掩码条件化和仿真预训练对于在所有23个自由度上实现每自由度动作可控性是必需的。相比之下，单体基线模型只能捕捉粗略的手部和末端执行器轨迹，无法可靠反映细粒度的、每个关节的动作效果。

Insight: 创新点在于将像素预测解耦为分割掩码动态预测和掩码到RGB的渲染两个阶段，利用分割空间较小的仿真到真实差距来桥接仿真与真实数据。这允许动态模型从大规模合成数据中预训练，再以少量真实数据微调，实现了对高自由度灵巧操作的精细、可控的未来预测。

Abstract: Action-conditioned world models allow robots to predict the future consequences of candidate actions without additional physical interaction, supporting policy evaluation, planning, and data augmentation. We present Mask2Real-WM, a two-stage action-conditioned world model for dexterous manipulation that decouples pixel prediction into a dynamics model and a rendering model. The dynamics model predicts future segmentation masks from past masks and 23-DoF action sequences. The rendering model maps the predicted masks to photorealistic RGB using a ControlNet-augmented Stable Video Diffusion backbone. The smaller sim-to-real gap in segmentation space enables the dynamics model to benefit from large-scale pretraining on over 50 h of synthetic simulation data, followed by fine-tuning on fewer than 2.5 h of real demonstrations. Experiments on a dexterous pick-and-place benchmark show that mask conditioning and simulation pretraining are both required for per-DoF action controllability across all 23 degrees of freedom. In contrast, monolithic baselines capture broad hand and end-effector trajectories but do not reliably reflect fine-grained, per-joint action effects.

[247] VLM-CASE: Vision-Language Model Enabled Context-Adaptive Safety Envelopes for Anticipatory Safe Autonomous Driving cs.RO | cs.CV | eess.SYPDF

Tianjia Yang, Ke Li, Ruwen Qin, Xianbiao Hu

TL;DR: 本文提出了VLM-CASE框架，旨在赋予自动驾驶车辆预见性安全能力。该框架利用经过LoRA微调的视觉语言模型（VLM）从图像中推理路面和能见度条件，并据此参数化一个结合物理极限和责任敏感安全（RSS）保证的上下文自适应安全包络（CASE）。一个模型预测控制器（MPC）在该包络内自由驾驶，而VLM异步运行，不阻塞实时控制回路。

Details

Motivation: 解决恶劣驾驶条件（如恶劣天气）对自动驾驶的双重挑战：同时降低车辆感知能力和物理操控能力。现有系统多为事后反应，而人类驾驶员能通过预判场景并重新规划速度、跟车距离和转向来应对。

Result: 在CARLA闭环仿真中，在需要横向和纵向控制的任务上，跨多种天气、路面和光照条件下进行验证。所提出的VLM-CASE-MPC控制器完成了所有试验，优于传统的MPC基线和最先进的VLM集成控制器。消融实验证实了性能提升来自上下文自适应，其中摩擦力和能见度自适应被证明是互补的。

Insight: 主要创新点在于将VLM的语义场景理解与形式化的安全模型（CASE）相结合，实现了基于物理和场景推理的预见性安全包络。框架设计巧妙，VLM异步运行不阻塞控制回路，且控制器无关，可与多种底层控制器配对，为安全自动驾驶提供了一个有前景的方向。

Abstract: Adverse driving conditions, such as bad weather, remain a principal barrier to autonomous driving because they degrade two things at once: what the vehicle can perceive and what it can physically do. Human drivers cope by anticipation, reasoning about the scene and re-budgeting speed, following distance, and steering before grip or sight is lost, whereas current autonomous driving systems at best react after the fact. This paper proposes VLM-CASE, a framework that gives an autonomous vehicle this anticipatory capacity while keeping its motion bounded by a formal safety model at all times. A vision-language model (VLM), fine-tuned with low-rank adaptation (LoRA), reasons about the scene from the front-camera image and reports the road surface and visibility conditions. This output parametrizes a context-adaptive safety envelope (CASE), derived from physical limits and the guarantees of responsibility-sensitive safety, that couples braking and steering through a shared friction budget. A model predictive controller then drives freely within the envelope, while the VLM runs asynchronously so it never blocks the real-time control loop. We validate the framework in closed-loop CARLA simulation on tasks that demand both lateral and longitudinal control, across a range of weather, road-surface, and lighting conditions. The resulting controller, VLM-CASE-MPC, completes all trials, outperforming a conventional MPC baseline and a state-of-the-art VLM-integrated controller. Ablations confirm that the gains come from context adaptation, with the friction and visibility adaptations proving complementary. Furthermore, the framework is controller-agnostic and pairs with almost any low-level controller, offering a promising direction for safe autonomous driving. The dataset and supplementary materials for VLM-CASE are available at https://github.com/ytj254/VLM-CASE.

[248] Cortex: A Bidirectionally Aligned Embodied Agent Framework for Long-horizon Manipulation cs.RO | cs.AI | cs.CVPDF

Jiaqi Peng, Xiqian Yu, Delin Feng, Yuqiang Yang, Wenzhe Cai

TL;DR: 本文提出了Cortex框架，这是一个双向对齐的具身智能体框架，旨在解决长时程操作任务中高层语义规划与低层运动执行之间的鸿沟。该框架通过定制化的规划接口，将可执行的子任务计划从高层视觉语言模型传递至低层视觉语言动作模型，并利用标准化的技能基元与可追踪性原则自动标注大量开源视频与仿真数据。

Details

Motivation: 现有视觉-语言-动作模型在长时程任务中因马尔可夫性（仅依赖当前观测）而表现不佳，而分层双系统方法又面临高层规划语义与低层执行运动学之间的不匹配问题。

Result: 在Libero-long和RoboTwin基准测试中，Cortex分别比单体基线模型提升了3.1%和4.1%。其通用视觉语言模型能够零样本完成未见过的真实世界长时程任务（如多阶段化学实验），仅需与微调后的视觉语言动作模型结合即可实现，而仅靠视觉语言动作模型微调则无法达成。

Insight: 创新点包括将操作子任务标准化为32个规范技能基元，在数据生成流程中注入可追踪性原则（如代表性物体属性和改进的轨迹可达性），以及采用事件平衡采样策略来构建训练数据以减少子任务转换时的规划模糊性。推理阶段通过从任务上下文到技能约束的精心设计约束工程进一步增强性能。

Abstract: While recent Vision-Language-Action (VLA) models show promise toward generalist manipulation policies, they struggle with long-horizon tasks due to their Markovian nature-relying solely on current observations. Hierarchical dual-system methods address this but suffer from a gap between high-level planning semantics and low-level execution kinematics. We introduce Cortex, a bidirectionally aligned embodied agent framework with a customized planning interface that conveys executable and tractable subtask plans from high-level VLM to low-level VLA. Specifically, we standardize manipulation subtasks into 32 canonical skill primitives and inject tractability principles, such as representative object attributes and improved trajectory reachability, into the data generation pipeline. This enables automatic annotation of over 4k hours of open-source video data and generation of 30 hours of simulation data. We further devise an event-balanced sampling strategy to construct training data for fine-tuning the framework to better handle planning ambiguity during subtask transitions, enhanced by carefully designed harness engineering from task contexts to skill constraints during inference. Both open-loop VLM and closed-loop system evaluations demonstrate Cortex’s efficacy, e.g., it outperforms monolithic baselines by 3.1% on Libero-long and 4.1% on RoboTwin. Notably, Cortex’s generalist VLM enables zero-shot completion of unseen real-world long-horizon tasks, such as multi-stage chemistry experiments, by simply combining with a fine-tuned VLA-a capability infeasible through VLA fine-tuning alone.

[249] Deform360: A Massive Multi-view Visuotactile Dataset for Deformable World Models cs.RO | cs.CVPDF

Hongyu Li, Wanjia Fu, Xiaoyan Cong, Zekun Li, Binghao Huang

TL;DR: 该论文提出了Deform360，一个大规模的多视角视觉触觉数据集，用于研究可变形物体的世界建模。数据集包含198个日常物体、1980个交互序列和超过215小时的观测数据，使用41个环绕摄像头和双手触觉夹爪捕获全局运动和接触引起的局部变形。论文还开发了一种无标记的视觉触觉3D跟踪流程来提取密集几何和运动，并系统评估了当前最先进的2D视频模型和3D粒子模型，揭示了结构先验与可扩展性之间的权衡。

Details

Motivation: 预测物体动态（即世界建模）是机器人操作的基本挑战，而建模可变形物体尤其困难，因其高维状态空间和复杂材料特性。当前世界模型通过2D像素空间或3D几何空间两种范式处理，但由于缺乏多样化、大规模的真实世界数据，对其相对优势和局限性的系统理解仍然不足。

Result: 论文利用Deform360数据集系统评估了当前最先进的世界模型，比较了2D视频模型与3D粒子模型。分析揭示了结构先验与可扩展性之间的关键权衡，为未来可泛化的以可变形物体为中心的世界建模研究提供了坚实的基准。此外，通过在可变形物体上执行机器人规划任务，初步展示了数据集的现实世界适用性。

Insight: 主要创新点在于创建了首个大规模、多视角、结合视觉与触觉的可变形物体交互数据集，并开发了无标记的3D跟踪流程来提取密集几何信息。这为系统比较不同建模范式（2D vs. 3D）提供了基础，揭示了模型设计中的关键权衡，推动了可变形物体世界建模的基准测试和实际应用。

Abstract: Predicting object dynamics (i.e., world modeling) is a fundamental challenge for robotic manipulation, and modeling deformable objects presents a particularly difficult case due to their high-dimensional state spaces and complex material properties. While current world models approach this through two distinct paradigms: learning the dynamics over the 2D pixel space or more explicit 3D geometric space. A systematic understanding of their relative strengths and limitations remains elusive due to the lack of diverse, large-scale real-world data. To address this, we present Deform360, a large-scale visuotactile dataset featuring 198 daily-life objects, 1,980 interaction sequences, and over 215 hours of observations from 41 surround-view cameras and bimanual tactile grippers to capture both global motion and contact-induced local deformations. Leveraging a novel markerless visuotactile 3D tracking pipeline to extract dense geometry and motion, we systematically evaluate current state-of-the-art world models, comparing 2D video models against 3D particle models. Finally, we provide a preliminary demonstration indicating the real-world applicability of our dataset by performing robot planning tasks on deformable objects. Our analysis reveals key insights into the trade-offs between structural priors and scalability, providing a solid benchmark for future research in generalizable deformable object-centric world modeling. Project website: https://deform360.lhy.xyz

eess.IV [Back]

[250] Motion Estimation Techniques for Volumetric Video Attribute Compression eess.IV | cs.CVPDF

Haoran Hong, Eduardo Pavez, Antonio Ortega, Ryosuke Watanabe, Keisuke Nonaka

TL;DR: 本文针对动态实体点云的属性压缩问题，提出了三种基于运动的编码技术：一种基于几何的帧间编码方案、一种基于图的运动估计方案，以及一种无需插值的分数体素运动估计方法。实验表明，该方法在MPEG点云数据集上显著优于现有主流压缩标准，特别是在有损几何条件下实现了显著的码率节省。

Details

Motivation: 当前点云压缩技术中，基于运动的动态实体点云几何压缩已取得显著进展，但属性压缩中的运动估计技术仍未被充分探索，难以有效减少属性数据的时间冗余。

Result: 在MPEG点云数据集上的实验结果表明，所提方案在无损和有损几何条件下均优于G-PCC、GeS-TM和V-PCC。在有损几何条件下，相比G-PCC、GeS-TM和V-PCC分别实现了平均55.3%、42.3%和16.5%的码率节省。

Insight: 创新点包括将运动估计技术系统性地引入点云属性压缩领域，提出了基于图的运动估计和无需插值的分数体素运动估计方法，有效提升了运动精度和压缩效率，为减少属性时间冗余提供了新思路。

Abstract: Point cloud compression relies on techniques to compress both geometry and attributes. Motion-based approaches for dynamic solid point cloud geometry compression within the geometry-based point cloud compression (G-PCC) framework have achieved significant reductions in geometry rate. However, motion-based techniques for attribute compression remain underexplored, making it challenging to achieve significant reductions in the temporal redundancy of attributes. Firstly, this paper proposes a geometry-based inter-coding scheme to compress the attributes of dynamic solid point clouds. Secondly, a graph-based motion-estimation scheme for point-cloud attribute compression is proposed. Thirdly, an interpolation-free fractional-voxel motion estimation method is proposed to refine motion accuracy to fractional-voxel precision. Our experimental results on the MPEG point cloud dataset show that the proposed scheme outperforms G-PCC, GeS-TM, and V-PCC in lossless and lossy geometry conditions. We achieve average bitrate savings of $55.3%$, $42.3%$, and $16.5%$ over G-PCC, GeS-TM, and V-PCC, respectively, under lossy-geometry conditions.

[251] Deep Learning-Based Characterization of Detonation-Cell Size Distributions in Soot-Foil Records eess.IV | cs.CVPDF

Mingyang Bu, Robson A. Schneider, Karl P. Chatelain, Mhedine Alicherif, Yingchen Shi

TL;DR: 本文提出了一种基于深度学习实例分割（Mask R-CNN）的新方法，用于自动识别和提取爆炸胞格的高阶特征。该方法通过构建自定义异构数据集（数值模拟和物理实验）并结合迁移学习，实现了在高噪声流场中精确的像素级掩码预测。

Details

Motivation: 传统的手动测量爆炸胞格（soot foils）方法耗时且主观，而现有的计算机视觉技术在处理具有高噪声、边界模糊和严重重叠的真实实验图像时泛化能力差。

Result: 在基准验证中，模型在像素级别上表现出高度一致性，并对复杂真实烟熏图具有强鲁棒性。预测的平均胞格尺寸与手动测量结果吻合良好，在规则和不规则条件下的相对误差分别低于2%和3.5%。

Insight: 创新点在于利用实例分割模型和异构数据集训练，克服了仅提取全局平均尺寸的限制，实现了对胞格尺寸沿传播方向瞬态空间演化的自动跟踪，并能定量提取不规则性指数（RI）和胞格偏转角标准差等高阶规律性特征，提高了统计分析的效率和客观性。

Abstract: The geometric size and regularity of detonation cells are key physical parameters for characterizing detonation waves. Traditional manual measurement of soot foils is time-consuming and subjective, while existing computer vision techniques often exhibit poor generalization on real experimental images with high noise, blurred boundaries, and severe overlapping. To address this, we propose a novel method for automated recognition and high-order feature extraction of detonation cells based on deep learning instance segmentation (Mask R-CNN). By constructing a custom heterogeneous dataset (numerical simulations and physical experiments) and integrating transfer learning, the model achieves accurate pixel-level mask prediction within highly noisy flow fields. Results indicate high pixel-level agreement in benchmark validations and strong robustness against noise in complex real-world soot foils. Predicted average cell sizes agree well with manual measurements, yielding relative errors under 2% and 3.5% for regular and irregular conditions, respectively. Sensitivity ablation experiments confirm the model’s scale adaptability and guided the establishment of a standardized preprocessing paradigm for appropriate image patching. Overcoming the limitation of extracting only global average sizes, this model achieves automated tracking of the transient spatial evolution of cell sizes along the propagation direction. Furthermore, it quantitatively extracts high-order regularity features, such as the irregularity index (RI) and standard deviation of cell deflection angles, demonstrating consistency with theoretical expectations. The proposed method enhances the efficiency and objectivity of statistical analysis, providing a powerful data extraction tool for experimental and numerical soot foils.

Rashadul Hasan Badhon, Atalie Carina Thompson, Jennifer I. Lim, Theodore Leng, Minhaj Nur Alam

TL;DR: 本研究提出了一种基于双向跨模态注意力网络的OCT B扫描与单通道en face OCTA图像融合方法，用于糖尿病视网膜病变的自动分类。该方法在多个数据集和评估场景下均优于单模态OCT分类，并探索了从OCT扫描生成的翻译OCTA作为硬件OCTA的低成本替代方案。

Details

Motivation: 糖尿病视网膜病变是全球视力损害的主要原因，需要准确且易于获取的筛查工具。OCT提供视网膜高分辨率结构信息，而OCTA提供互补的血管信息，两者融合可改善诊断。

Result: 跨模态融合在所有评估场景（包括数据集内、组合数据集和跨数据集泛化）中均一致优于单模态OCT基线。使用真实OCTA融合提高了分类准确性和判别性能，而翻译OCTA在大多数设置中取得了相当或更优的结果，并提高了敏感性和跨数据集泛化能力。

Insight: 创新点在于提出基于注意力的OCT-OCTA en face融合网络，并验证了计算生成的OCTA可作为硬件OCTA的实用低成本替代品，增强了模型对领域偏移的鲁棒性，有利于在资源有限环境中部署高性能视网膜筛查系统。

Abstract: Diabetic retinopathy (DR) is a leading cause of vision impairment worldwide, highlighting the need for accurate and accessible screening tools. Optical Coherence Tomography (OCT) provides high-resolution structural information of the retina, whereas OCT angiography (OCTA) offers complementary vascular information that is highly relevant for DR diagnosis. In this study, we propose a cross-modal fusion of OCT B-scans with single-channel en face OCTA using a bidirectional cross-modal attention network for automated DR classification. Two independent datasets, OCT500 and UIC, comprising 730 subjects in total, were utilized to evaluate performance under within-dataset, combined-dataset, and cross-dataset generalization settings. A ConvNeXt V2 model trained solely on OCT images served as the unimodal baseline. In addition to ground-truth (GT) OCTA, we explored the use of translated (TR) OCTA generated from OCT scans, eliminating the requirement for dedicated OCTA hardware. Experimental results demonstrate that cross-modal fusion consistently outperforms unimodal OCT classification across all evaluation scenarios. Fusion with GT OCTA improved classification accuracy and discriminative performance, while TR OCTA achieved comparable or superior results in most settings. Furthermore, TR OCTA improved sensitivity and cross-dataset generalization, indicating enhanced robustness to domain shifts. These findings demonstrate that attention-based OCT-OCTA en face fusion provides clinically meaningful improvements for DR detection and suggest that computationally generated OCTA can serve as a practical, low-cost alternative to hardware-acquired OCTA, enabling broader deployment of high-performance retinal screening systems in resource-limited clinical environments.

Dineth Perera, Thaariq Firdous, Oshadha Samarakoon, Roshan Godaliyadda, Parakrama Ekanayake

TL;DR: 本文提出MambaRefine-CD，一个基于共享MambaVision编码器的区域-边界时序精化框架，用于遥感图像二元变化检测。该方法通过D-RBI模块构建包含配对特征、绝对差值和符号差值的时序证据，并将其分离为区域流和Sobel条件边界流。区域特征通过CRAM-lite增强并由自适应感受野FPN解码，而精细边界流则指导粗预测的有界残差精化。

Details

Motivation: 解决遥感变化检测中既需要完整定位变化区域，又需要精确勾勒变化边界的双重挑战。

Result: 在DSIFN-CD和WHU-CD数据集上的实验表明，该方法在验证评估设置下取得了优异的变化类F1和IoU分数；消融实验支持了符号时序证据和完整区域-边界精化流程的有效性。

Insight: 创新点在于将时序证据明确分解为区域和边界分量进行并行处理，并引入符号差值信息以及由精细边界流引导的有界残差精化机制，实现了对变化区域内部和边缘的协同优化。

Abstract: Binary change detection in remote sensing requires both complete changed-region localization and accurate boundary delineation. We present MambaRefine-CD, a region-boundary temporal refinement framework built on a shared MambaVision encoder. The proposed D-RBI module constructs temporal evidence from paired features, absolute differences, and signed differences, then separates it into region and Sobel-conditioned boundary streams. Region features are enhanced with CRAM-lite and decoded by an adaptive receptive-field FPN, while the finest boundary stream guides a bounded residual refinement of the coarse prediction. Experiments on DSIFN-CD and WHU-CD show strong changed-class F1 and IoU under verified evaluation settings, and ablations support the contribution of signed temporal evidence and the full region-boundary refinement pipeline.

[254] CompressedVQA-AEV: Full-Reference and No-Reference Quality Assessment Models for Asymmetric Encoded Videos eess.IV | cs.CV | cs.MMPDF

Wei Sun, Xingwei Liu, Dandan Zhu, Xiangyang Zhu, Weixia Zhang

TL;DR: 本文提出了针对非对称编码视频质量评估的两种模型：全参考模型CompressedVQA-AEV-FR和无参考模型CompressedVQA-AEV-NR。FR模型使用Swin-B骨干网络提取参考视频与失真视频间的多阶段相似性统计特征进行质量预测；NR模型则结合基于SigLIP2和Swin-B的互补帧级编码器，通过时序平均池化和交叉折叠集成在无参考数据下估计感知质量。

Details

Motivation: 解决QoMEX 2026大挑战中非对称编码视频的质量评估问题，分别针对有参考视频和无参考视频两种场景设计有效的质量评估模型。

Result: 在QoMEX 2026大挑战中，CompressedVQA-AEV-FR在FR赛道获得第一名，CompressedVQA-AEV-NR在NR赛道获得第四名，证明了模型的有效性。

Insight: 创新点包括：在FR模型中采用Swin-B提取多阶段相似性统计特征；在NR模型中设计基于SigLIP2和Swin-B的互补帧级编码器架构，并结合时序平均池化与交叉折叠集成策略提升无参考评估性能。

Abstract: This report presents our solutions to the QoMEX 2026 Grand Challenge on Video Quality Assessment for Asymmetric Encoded Videos, comprising a full-reference (FR) model, CompressedVQA-AEV-FR, and a no-reference (NR) model, CompressedVQA-AEV-NR. The FR approach leverages a Swin-B backbone to extract multi-stage similarity statistics between reference and distorted videos for quality prediction. For the NR setting, our model employs complementary frame-level encoders based on SigLIP2 and Swin-B, followed by temporal mean pooling and cross-fold ensembling to estimate perceptual quality without reference data. Our CompressedVQA-AEV-FR achieves first place in the FR track of QoMEX 2026 Grand Challenge, while CompressedVQA-AEV-NR secures fourth place in the NR track, demonstrating the effectiveness of our proposed models. The code is available at https://github.com/sunwei925/CompressedVQA-AEV.

q-bio.QM [Back]

[255] Triple-Phase Multimodal Knowledge Aggregation Framework for Microbial Keratitis Subtype Diagnosis on Slit-Lamp Photography q-bio.QM | cs.CV | eess.IVPDF

Yiqing Wang, Maria A. Woodward, Ziyun Yang, N. Venkatesh Prajna, Chunming He

TL;DR: 该论文提出了一种用于微生物性角膜炎亚型诊断的三阶段多模态知识聚合框架，利用裂隙灯照片（蓝光、硬化散射和白光照明）和临床元数据进行细菌性与真菌性角膜炎分类。该模型结合了跨模态对比学习、模态特定微调和特征级多模态集成学习，以实现患者级别的预测。

Details

Motivation: 微生物性角膜炎需要快速病原体识别以指导治疗，但基于培养和PCR的诊断方法速度慢且资源密集。因此，研究旨在开发一种基于裂隙灯摄影和临床数据的快速、非侵入性诊断框架。

Result: 在来自印度和美国的1,645名患者和17,158张图像的多中心数据集上评估，模型达到85.84%的准确率、84.46%的平均F1分数和0.885的AUC。在所有设置下，该框架均保持最佳性能。

Insight: 创新点包括三阶段多模态聚合（对比学习、微调、集成学习）和针对跨站点泛化的重新评估方法（如重采样和平衡评估），强调了在真实临床场景中模型评估的重要性。

Abstract: Microbial keratitis requires rapid pathogen identification to guide treatment, but culture- and PCR-based diagnostics are slow and resource-intensive. We developed a triple-phase multimodal framework for bacterial-versus-fungal keratitis classification using slit-lamp photographs acquired under blue-light, sclerotic-scatter, and white-light illumination, together with clinical metadata. The model combines cross-modality contrastive learning, modality-specific fine-tuning, and feature-level multimodal ensemble learning for patient-level prediction. We evaluated the framework on a multicenter dataset of 1,645 patients and 17,158 images from India and the United States. The model achieved 85.84% accuracy, 84.46% average F1-score, and 0.885 AUC. Site-specific evaluation showed that pooled results were overly optimistic, whereas resampling- and balance-based re-evaluation provided a more realistic assessment of cross-site generalization. Under all settings, our framework remained the top-performing approach. Upon acceptance, the code will be released and dataset access will be provided subject to University of Michigan data-sharing clearance.

cs.LG [Back]

[256] Aligning Language Models with Selective Prediction cs.LG | cs.AI | cs.CLPDF

Gaoxiang Luo, Yifan Wu, Sinian Zhang, Aryan Deshwal, Ju Sun

TL;DR: 该论文提出了一种名为RLSR（强化学习选择奖励）的新颖对齐框架，旨在通过选择性预测（SP）来增强大语言模型（LLM）的可靠性。该方法在LLM的后训练对齐阶段，将SP性能指标（风险-覆盖率曲线下面积，AURC）作为对齐目标，使模型仅对可能正确的输入进行预测，从而降低错误率并实现人机协作。

Details

Motivation: 动机在于LLM越来越多地被部署在高风险的真实AI系统中作为关键决策组件，其可靠性成为首要的实际关切。现有LLM对齐方法主要关注正确性或校准指标，而本文旨在通过选择性预测来提升LLM的可靠性，即在模型不确定时拒绝预测，交由人类处理。

Result: 在领域内和领域外任务上，所提出的RLSR框架相比多个对齐基线，在风险-覆盖率权衡方面取得了显著更好的性能，即实现了更优的选择性预测效果。

Insight: 创新点在于将选择性预测性能指标（AURC）直接作为LLM对齐的优化目标，而不仅仅是传统的正确性指标，这为提升LLM在实际高风险应用中的可靠性提供了一种新的对齐范式。从客观角度看，该方法强调了模型在不确定性下的决策能力，是实现可靠人机协作的重要一步。

Abstract: Large language models (LLMs) are increasingly deployed as critical decision-making components in high-stakes real-world AI systems, rendering LLM reliability a foremost practical concern. In this paper, we focus on enhancing LLM reliability through selective prediction (SP), a strategy that allows an LLM to only predict for inputs where it is likely to be correct (i.e., coverage) and hence reduce the error rate (i.e., risk) on that portion of inputs – flagging the remaining inputs for future human discretion. In other words, SP improves LLM reliability by balancing the risk-coverage trade-off and enabling seamless human-AI collaboration. To integrate SP into LLMs, we focus on the LLM post-training alignment stage and propose to align LLMs with SP performance metrics, in contrast with existing LLM alignment methods that focus primarily on correctness or calibration metrics. Specifically, we propose a novel alignment framework, Reinforcement Learning for Selection Reward (RLSR), which targets the area under the risk-coverage curve (AURC) – a popular SP performance metric – as its alignment objective. RLSR achieves substantially better risk-coverage trade-off compared to multiple alignment baselines on both in-domain and out-of-domain tasks.

[257] DynaVieW: Schema-Guided World Modeling for Understanding Hierarchical Visual Dynamics cs.LG | cs.AI | cs.CL | cs.CVPDF

Silin Gao, Hao Zhao, Zeming Chen, Sepideh Mamooler, Antara Raaghavi Bhattacharya

TL;DR: 论文提出了DynaVieW，一种基于模式引导的动态世界模型，旨在解决多模态大语言模型在理解和预测视频或多图像序列中视觉场景动态演化方面的困难。该模型通过分层模式学习交错的状态-转移序列，联合建模转移预测和状态模拟，以增强对视觉动态的深度理解。

Details

Motivation: 多模态大语言模型难以系统地对视频或多图像序列中视觉场景的时间演化进行建模，这需要模型预测或模拟多个层次的动态构成要素，如视觉序列中的动作及其导致的视觉环境变化。

Result: DynaVieW在视觉动态理解方面的提升，促进了其在视觉叙事生成和世界模拟等下游任务中的性能，表现出更好的一致性、可控性和指令遵循能力。

Insight: 创新点在于提出了一个分层模式引导的动态世界建模框架，通过交错的状态-转移序列来结构化地理解视觉动态，并采用专家混合架构、跨专家选择性注意力以及模式令牌重加权损失来确保学习的有效性和鲁棒性。

Abstract: Multimodal LLMs struggle to systematically model the temporal evolution of visual scenes in videos or multi-image sequences. Such inputs require models to predict or simulate multiple levels of dynamic constituents, such as actions taken in the visual sequence, and the associated changes to the visual environment that result. To address this challenge, we propose a dynamic schema-guided world model, DynaVieW, optimized for visual dynamic prediction and simulation. DynaVieW achieves an in-depth understanding of visual dynamics by learning interleaved state-transition sequences, where states cover broad visual scenes from video keyframes, and transitions capture comprehensive dynamic constituents within a hierarchical schema. DynaVieW jointly models transition prediction and state simulation under a mixture-of-experts architecture, with a cross-expert selective attention and a schema token re-weighted loss, to ensure effective and robust learning. DynaVieW’s understanding of visual dynamics boosts its downstream performance in visual narrative creation and world simulation, showing improved consistency, controllability, and instruction-following.

[258] Multi-Turn On-Policy Distillation with Prefix Replay cs.LG | cs.AI | cs.CL | stat.MLPDF

Baohao Liao, Hanze Dong, Christof Monz, Xinxing Xu, Li Dong

TL;DR: 本文提出了一种名为Replayed-Prefix On-Policy Distillation (ReOPD)的方法，用于解决多轮交互任务中的在线策略蒸馏(OPD)成本高昂的问题。该方法通过重用预收集的教师轨迹作为重放前缀，让学生在选定步骤行动，同时教师提供密集的每步监督，无需执行新的环境交互。

Details

Motivation: 动机在于解决完全在线策略蒸馏(OPD)在智能体任务中的高成本问题，因为每次更新都需要学生重新在环境中进行轨迹采样并查询教师模型，计算开销大。

Result: 在数学推理（使用Python）和搜索环境中，ReOPD保持或提升了OPD级别的准确性，学生训练期间工具调用次数为零，且每个训练步骤的速度至少比OPD快4倍。

Insight: 创新点在于将多轮OPD重新定义为一种可靠性感知的前缀分布设计问题，并提出了一个简单的步长衰减采样计划来强调早期、分布偏移较小的前缀，从而有效解决了学生占用分布与教师可靠性之间的双向分布偏移问题（前缀陷阱）。

Abstract: We study on-policy distillation (OPD) for agentic tasks, where an LLM agent interacts with an environment over multiple turns and a student imitates a teacher over these multi-turn interaction histories. Fully online OPD is costly because each update requires fresh student rollouts through the environment and teacher queries at visited histories. We propose Replayed-Prefix On-Policy Distillation (ReOPD), an off-environment alternative that reuses pre-collected teacher trajectories as replayed prefixes: the student acts at selected steps, while the teacher provides dense per-step supervision without executing new environment interactions. We show that multi-turn OPD introduces a prefix trap: making histories more student-on-policy improves relevance to the student, but can query the teacher on histories where its target is unreliable. This creates a two-sided distribution shift between student occupancy and teacher reliability. ReOPD addresses this by treating multi-turn OPD as a reliability-aware prefix distribution design and implements it with a simple step-decaying sampling schedule that emphasizes early, lower-shift prefixes. Across mathematical reasoning with Python and search environments over multiple teacher and student model scales, ReOPD preserves or improves OPD-level accuracy, uses zero tool calls during student training, and is at least 4$\times$ faster per training step than OPD. ReOPD therefore turns expensive agent-environment interaction into a reusable offline resource, enabling scalable distillation across tools, tasks, and environments.

[259] Localized LoRA-MoE: Block-wise Low-Rank Experts With Adaptive Routing cs.LG | cs.AI | cs.CLPDF

Babak Barazandeh, Subhabrata Majumdar, Vinay Prithyani, George Michailidis

TL;DR: 本文提出了Localized LoRA-MoE，一个将局部空间分块与动态上下文条件路由相结合的统一框架，用于解决参数高效微调（PEFT）中的梯度冲突和静态拓扑适应性问题。该框架包含两种新范式：基于集中式宏观路由的Block-Wise LoRA-MoE和基于去中心化微观路由的Cell-Wise LoRA-MoE。

Details

Motivation: 标准PEFT方法（如LoRA）存在单一瓶颈结构，易受梯度冲突影响，导致权重退化为非特化的平均值；而现有的空间分区方法仍受限于静态拓扑，无法适应动态任务切换或环境传感器故障。

Result: 在涵盖高维SVD矩阵模拟、真实世界表格转换以及传感器退化下的空间视觉感知等一系列基准测试中，两种架构均解决了静态基线的优化死锁问题。去中心化的单元级门控实现了与全知全局协调器完全相当的统计性能，并提供了保护存活路径免受故障传播影响的鲁棒“梯度防火墙”。

Insight: 核心创新在于将MoE（专家混合）的动态路由思想与LoRA的局部低秩适应结构在空间维度上深度融合，特别是提出了去中心化的单元级自主门控机制，为动态模型适应提供了可扩展且参数高效的解决方案。

Abstract: Large Language Models (LLMs) and high-dimensional perception networks increasingly rely on parameter-efficient fine-tuning (PEFT) to adapt to diverse operational contexts. However, standard methods like LoRA are structurally limited by a monolithic bottleneck, making them highly susceptible to gradient warfare. Interleaved multi-task streams may trigger destructive optimization feedback, collapsing adapter weights into unspecialized averages. While recent spatial partitioning methods have introduced block-wise isolation, they remain trapped in static topologies, unable to adapt to dynamic task-switching or environmental sensor failure. In this work, we introduce Localized LoRA-MoE, a unified framework that fuses localized spatial blocking with dynamic, context-conditioned routing. We propose and evaluate two novel architectural paradigms: Block-Wise LoRA-MoE (Centralized Macro-Routing), which modulates the entire structural grid via a monolithic context signal, and Cell-Wise LoRA-MoE (Decentralized Micro-Routing), which empowers every coordinate cell in the matrix grid with autonomous, localized expert gating. Through a comprehensive suite of benchmarks, ranging from high-dimensional SVD matrix simulations and real-world tabular transformations to spatial vision perception under sensor degradation, we demonstrate that both architectures resolve optimization deadlocks inherent in static baselines. Our empirical results establish that decentralized cell-level gating achieves complete statistical parity with an omniscient global coordinator, providing a robust “gradient firewall” that protects surviving pathways from fault-propagated corruption. Our proposals consistently outperform static baselines, offering a scalable and parameter-efficient solution for dynamic model adaptation across granular coordinate fields and shifting operational regimes.

[260] Weak-to-Strong Generalization via Direct On-Policy Distillation cs.LG | cs.AI | cs.CLPDF

Shiyuan Feng, Huan-ang Gao, Haohan Chi, Hanlin Wu, Zhilong Zhang

TL;DR: 本文提出了一种名为直接策略蒸馏（Direct-OPD）的方法，用于实现从弱模型到强模型的泛化。该方法通过利用弱模型在强化学习（RL）训练前后的策略差异，生成密集的隐式奖励信号，直接指导强学生模型的策略优化，从而避免在强模型上重复昂贵的RL训练过程。

Details

Motivation: 为了解决在大型语言模型上重复进行可验证奖励的强化学习（RLVR）训练成本高昂的问题，研究如何将弱模型RL训练中学到的知识迁移到更强的目标模型上，以降低后训练瓶颈。

Result: 在AIME 2024基准测试中，该方法将Qwen3-1.7B模型的性能从48.3%提升至62.4%，仅需8块A100 GPU运行4小时，优于匹配步数的直接RL方法，并支持多个策略转移的顺序组合。

Insight: 创新点在于将弱模型RL训练前后的策略对数比作为密集隐式奖励，直接应用于强学生模型的在线策略状态，实现了RL监督信号的跨模型规模重用，而不仅仅是模仿最终模型；这为高效知识迁移提供了新思路。

Abstract: Reinforcement learning with verifiable rewards (RLVR) is a powerful recipe for improving language-model reasoning, but it is expensive to repeat on every new strong model because the target model must generate many rollouts during training. As models scale, post-training itself becomes a bottleneck. We study a weak-to-strong alternative: run RL on a smaller model where rollouts are cheaper, then reuse what that RL run learned to improve a stronger target model. Directly distilling the post-RL weak teacher is not enough, because the teacher’s final policy mixes useful RL gains with the limitations of the smaller model. We propose Direct On-Policy Distillation (Direct-OPD), which transfers the teacher’s RL-induced policy shift instead. Direct-OPD compares the post-RL teacher with its own pre-RL reference and treats their log-ratio as a dense implicit reward for the student. In plain terms, the checkpoint pair tells us which actions RL made the weak model more or less likely to take, and Direct-OPD applies that signal on the stronger student’s own on-policy states. This directly reuses the weak model’s RL supervision signal without training an explicit reward model or running sparse-reward RL on the target model. Empirically, Direct-OPD consistently leverages weaker teachers to improve stronger target models; notably, it boosts Qwen3-1.7B from 48.3% to 62.4% on AIME 2024 in just 4 hours on 8 A100 GPUs. It outperforms step-matched direct RL and enables the sequential composition of multiple policy shifts. Our results show that RL outcomes can be reused across model scales as implicit reward signals, not merely as final models to imitate.

[261] Federated Learning for Object Detection: Enabling Collaborative Drone Learning Without Centralizing Data cs.LG | cs.AI | cs.CV | cs.DCPDF

Daniel M. Jimenez-Gutierrez, Enrique Zuazua, Georgios Kellaris, Joaquin del Rio, Oleksii Sliusarenko

TL;DR: 本研究将联邦学习应用于无人机目标检测任务，使多架无人机能够协作训练共享模型而无需集中图像数据。通过在KIIT-MiTA数据集上使用Sherpa.ai联邦学习平台进行实验，结果表明联邦学习方法在保持接近集中式训练性能的同时，显著优于单无人机训练，尤其适合在资源受限的边缘设备上部署。

Details

Motivation: 解决分布式无人机部署中目标检测模型训练面临的隐私、监管、存储和带宽挑战，避免将敏感视觉数据集中到中央基础设施的需求。

Result: 在KIIT-MiTA数据集上，使用YOLO26 nano轻量模型，联邦学习方法在mAP@0.50和mAP@0.50:0.95指标上分别比单无人机训练相对提升52.89%和67.80%，性能接近集中式训练基准。

Insight: 将联邦学习框架与轻量化目标检测模型（如YOLO）结合，实现了隐私保护下的分布式协作学习；证明了在边缘设备资源受限场景下，联邦学习能有效平衡性能与隐私需求。

Abstract: Object detection is a fundamental capability for AI-driven perception in safety-critical drone and edge-vision systems, including disaster response, operational security environments, infrastructure monitoring and defense applications. Robust model performance in such environments depends on large, continuously updated datasets. However, training high-performing detectors typically requires centralizing aerial imagery, which raises privacy, regulatory, storage, and bandwidth challenges. This is especially problematic in distributed drone deployments, where visual data is generated onboard and is often impractical or undesirable to transfer to a centralized infrastructure. In this work, we apply Federated Learning (FL) for object detection, enabling drones to improve a shared model while keeping image data local and private. We implement a federated object detection pipeline using the Sherpa.ai FL platform on the KIIT-MiTA dataset, and compare it with Single-drone and Centralized baselines using mean Average Precision (mAP) at IoU thresholds of 0.50 and 0.50-0.95. In our experiments, the proposed FL approach remains close to Centralized training while dramatically improving over Single-drone training, with the best lightweight model (YOLO26 nano), suitable for deployment even on very limited edge infrastructure, achieving relative gains of 52.89% and 67.80% in mAP@0.50 and mAP@0.50:0.95, respectively. These results show that FL enables scalable, high-performing, and privacy-preserving object detection across distributed drone fleets without data centralization.

Shijie Cao, Qingyu Zhang, Boxi Yu, Yuzhong Zhang, Boxi Cao

TL;DR: 本文提出了一种名为OmniFocus的免训练、查询引导的令牌压缩方法，旨在解决全模态大语言模型在处理音视频输入时因生成大量令牌序列而导致的高推理成本问题。该方法通过对视频和音频进行独立的模态重要性估计，实现模态对称的压缩设计，在保留模态特异性显著证据的同时维持音视频对齐，从而缓解了单模态引导压缩可能带来的模态偏差问题。

Details

Motivation: 现有音视频令牌压缩方法通常依赖单模态指导，忽略了查询相关证据在音视频输入中的时间局部性，并隐含地假设两种模态共享时间对齐的信息密度分布，这可能导致模态偏差和效率低下。

Result: 在Qwen2.5-Omni模型家族上，于四个音视频基准测试的实验表明，OmniFocus在低令牌保留率下保持了强大的压缩性能，并在25%令牌保留率下于多个主要基准分数上优于现有基线。具体而言，在DailyOmni基准上使用Qwen2.5-Omni-7B模型，25%令牌保留率下，OmniFocus保持了59.40的准确率，同时相对于全令牌基线实现了高达1.38倍的预填充加速，展现了有利的准确率-效率权衡。

Insight: 创新点在于提出了一个免训练的、查询引导的模态对称压缩框架，通过独立估计视频和音频的重要性，实现了模态平衡的令牌压缩，有效缓解了模态偏差，并显著提升了推理效率，为全模态大模型的实用化提供了可行的优化路径。

Abstract: Omni modal large language models (OmniLLMs) have attracted wide attention for their ability to jointly process audio and video, but they generate large token sequences under audio-visual inputs, leading to substantial inference cost. Existing audio-visual token compression methods often rely on unimodal guidance, overlooking the temporal locality of query-relevant evidence in audio-visual inputs and implicitly assuming that the two modalities share a temporally aligned information density distribution. We propose \textbf{OmniFocus}, a training-free query-guided token compression method for OmniLLMs that performs independent importance estimation for video and audio, enabling a modality-symmetric compression design that preserves modality-specific salient evidence while maintaining audio-visual alignment, thereby mitigating the modality bias issue that can arise from unimodal-guided compression. Experiments on the Qwen2.5-Omni model family across four audio-visual benchmarks show that OmniFocus maintains strong compressed performance at low token retention ratios and outperforms existing baselines on several major benchmark scores at 25% token retention. On DailyOmni with Qwen2.5-Omni-7B at 25% token retention, OmniFocus maintains 59.40 accuracy while delivering up to 1.38$\times$ prefill speedup relative to the full-token baseline, highlighting a favorable practical accuracy-efficiency trade-off.

[263] When Geometry Aligns: Dihedral Hidden-State Transformations in UNet, ViT, and DiT Architectures cs.LG | cs.CVPDF

Mojtaba Faramarzi, Alex Lamb, Irina Rish

TL;DR: 本文研究了在扩散模型架构（如UNet、ViT和DiT）中，对中间隐藏状态应用二面体群（如反射）的几何变换的影响。通过统一的框架对比几何一致与不一致的变换变体，使用激活级诊断指标（如SCS、AMS和Drift）分析特征稳定性和几何漂移。主要实验在Stable Diffusion 2.1 U-Net上进行，评估了七种干预模式，并结合图像级指标（FID、KID、CLIP分数和LPIPS多样性）验证结果。研究表明，几何一致性是空间结构化视觉和扩散模型中稳定隐藏状态干预的关键原则。

Details

Motivation: 扩散模型架构包括卷积UNet和基于Transformer的设计（如DiT），但这些架构内部结构化几何扰动的影响尚不明确。本文旨在通过受控内部干预，探究几何变换对隐藏状态的影响，以理解模型稳定性和失败模式。

Result: 在Stable Diffusion 2.1 U-Net的主要研究中，评估了七种干预模式，发现一致变换提高稳定性，而不一致变换导致可预测的架构特定失败。图像级指标（如FID、KID）支持了内部诊断结果，ViT和受控DiT分析进一步验证了几何一致性的重要性。

Insight: 创新点在于提出一个统一框架，应用二面体群的反射元素作为隐藏状态干预，并使用激活级诊断指标（SCS、AMS、Drift）量化几何一致性效应。客观分析表明，几何一致性可视为扩散模型稳定性的设计原则，为架构干预提供了新视角。

Abstract: Diffusion architectures now encompass convolutional UNets as well as transformer-based designs such as Diffusion Transformers (DiTs), inspired by Vision Transformers (ViTs), yet the effects of structured geometric perturbations within these architectures remain poorly understood. We study this question through a unified framework that applies reflection-based elements of the dihedral group to intermediate hidden states as controlled internal interventions, contrasting geometrically consistent and inconsistent variants. Using activation-level diagnostics, including Self-Consistency Shift (SCS), Activation Mass Scatter (AMS), and Drift, we analyze feature stability and geometric drift. We find that consistent transformations improve stability, while inconsistent ones induce predictable, architecture-specific failures. In the main Stable Diffusion 2.1 U-Net study, we evaluate seven intervention modes over three seeds and complement the internal diagnostics with image-level FID, KID, CLIP score, and LPIPS diversity. Taken together with supporting ViT and controlled DiT analyses, these results establish geometric consistency as a key principle for stable hidden-state interventions in spatially structured vision and diffusion models.

[264] Phase-Preserving Trimodal Transformer for Tropical Forest Biomass Estimation Using Optical and PolInSAR Data cs.LG | cs.AI | cs.CVPDF

Luiz Felipe Parente Santiago, Rosiane Rodrigues de Freitas, Daniel Rodrigues dos Santos, Felipe Ferrari

TL;DR: 本文提出了一种名为Trimodal Coherent Co-attention Transformer (TCCT)的物理信息深度学习架构，用于融合光学（Landsat-5）和复杂值极化SAR干涉测量（PolInSAR）数据，以准确估算成熟热带森林的地上生物量（AGB）。该模型通过保留空间相位相干性和动态协同注意力机制，克服了SAR信号在高密度区域饱和及光学图像受云层影响的问题，并经过局部空间异速生长校准和两阶段评估，在亚马逊Paracou地区实现了优于基准模型的性能，满足了欧洲空间局BIOMASS任务的误差要求。

Details

Motivation: 解决成熟热带森林AGB估算中的关键挑战，即合成孔径雷达（SAR）信号在高密度区域饱和以及光学图像受持续云层覆盖影响，导致传统遥感方法精度受限。

Result: 在亚马逊Paracou地区的评估中，TCCT经过两阶段协议（包括5折交叉验证和局部微调），在冠层高度模型（CHM）上取得了绝对RMSE为3.78米和R²为0.33的结果，优于随机森林、CNN和Vision Transformer基线；转换为AGB后，在密度高于50 Mg/ha的森林区域，相对RMSE（rRMSE）达到4.51%，满足了欧洲空间局BIOMASS任务误差低于20%的要求。

Insight: 创新点包括：采用复杂值编码器保留空间相位相干性以缓解深冠层信号饱和，以及动态协同注意力机制作为自适应门控模块，减少云污染光学像素的权重并增强对微波相位数据的依赖；此外，通过Levenberg-Marquardt优化推导局部空间异速生长校准模型，针对特定木材密度进行定制化处理。

Abstract: The accurate estimation of Above-Ground Biomass (AGB) in mature tropical forests remains a critical challenge in remote sensing, primarily due to the saturation of Synthetic Aperture Radar (SAR) signals in high-density areas and persistent cloud cover affecting optical imagery. To overcome these physical limitations, we propose the Trimodal Coherent Co-attention Transformer (TCCT), a physics-informed deep learning architecture. The TCCT natively fuses optical surface reflectance (Landsat-5) with complex-valued Polarimetric SAR Interferometry (PolInSAR) data from both P and L bands. Unlike traditional fusion methods, our architecture employs complex-valued encoders to preserve spatial phase coherence, coupled with a dynamic co-attention mechanism that acts as an adaptive gating module, reducing the weight of cloud-corrupted optical pixels and shifting reliance to microwave phase data. We also derived a localized spatial allometric calibration model via Levenberg-Marquardt optimization, tailored to the specific wood density of the Paracou region in the Amazon basin. Evaluated using a two-stage protocol, the TCCT first underwent a rigorous 5-fold cross-validation to establish robust global weights (achieving a global RMSE of 4.19 m). Subsequently, following a localized spatial fine-tuning phase over 200 epochs, the model attained an absolute RMSE of 3.78 m and an $R^2$ of 0.33 for Canopy Height Models (CHM), outperforming standard Random Forest, CNN, and Vision Transformer baselines. Our ablation study confirms that preserving phase coherence mitigates deep-canopy signal saturation. When converted to AGB, the fine-tuned TCCT map yielded a Relative RMSE (rRMSE) of 4.51% in dense forest areas above 50 Mg/ha. By meeting the European Space Agency (ESA) BIOMASS mission requirement of less than 20% error, the TCCT provides a robust framework for continuous carbon stock mapping in tropical biomes.

Arunkumar Ramachandran

TL;DR: 本文提出了GlacierCastAI模型，将冰川边界预测重构为多模态时空预测问题，融合多时相Landsat卫星影像、ERA5再分析气候变量和Copernicus DEM地形特征，以预测跨越四种气候区的五个冰川的未来边界。模型架构结合了ResNet50空间编码器、ConvLSTM时序模型和交叉注意力气候融合模块。

Details

Motivation: 现有深度学习方法主要关注绘制当前冰川边界，而ERA5季节性气候变量包含超越卫星影像的、关于未来冰川退缩的预测信息，因此需要开发能够融合多模态数据以进行未来边界预测的方法。

Result: 在预测任务中，模型IoU值为0.320-0.337，虽无法与最先进的当前边界测绘模型直接比较，但相比持续性基线（IoU 0.160）和线性趋势基线（IoU 0.169）有89-99%的相对IoU提升。通过预注册消融实验，加入ERA5气候信号使仅用影像的IoU从0.326提升至0.337（+3.4%）。一个轻量级仅用气候的MLP基线（66.1万参数）达到了IoU 0.320，接近仅用影像的性能（98%），但参数少85倍。

Insight: 创新点在于将冰川边界预测重构为多模态时空预测问题，并设计了融合卫星影像、气候和地形数据的架构。客观分析表明，ERA5气候变量独立于卫星影像编码了显著的预测信号，其中春季太阳辐射（MAM）被SHAP归因分析识别为主导气候驱动因素，这与春季日照在决定融化季节轨迹中的已知作用一致，为理解气候对冰川退缩的直接影响提供了可解释的见解。

Abstract: ERA5 seasonal climate variables contain predictive information about future glacier retreat beyond what satellite imagery alone provides, yet existing deep learning methods focus on mapping current boundaries rather than forecasting future ones. This paper presents GlacierCastAI, which reframes glacier boundary prediction as a multi-modal spatiotemporal forecasting problem, fusing multi-temporal Landsat imagery with ERA5 reanalysis climate variables and Copernicus DEM terrain features to forecast glacier boundaries across five glaciers spanning four climate regimes. The architecture couples a ResNet50 spatial encoder with a ConvLSTM temporal model and a cross-attention climate fusion module. Because forecasting is inherently more uncertain than mapping current boundaries, the reported IoU values (0.320-0.337) are not directly comparable to state-of-the-art mapping models. Comparisons are against traditional baselines and experimental conditions. Through a pre-registered ablation study, adding ERA5 climate signals improves image-only IoU from 0.326 to 0.337 (+3.4%), suggesting that atmospheric forcing carries predictive information beyond imagery alone. All deep learning models substantially outperform persistence and linear trend baselines (IoU 0.160 and 0.169 respectively), with improvements of 89-99% relative IoU. A lightweight climate-only MLP baseline (661K parameters) achieves an IoU of 0.320 (98% of image-only performance) using 85x fewer parameters, suggesting that ERA5 variables encode substantial predictive signal independently of satellite imagery. SHAP attribution analysis suggests that spring solar radiation (MAM) is the dominant climate driver, consistent with the known role of spring insolation in setting melt season trajectories.

[266] Qantara: Bridge-Flow Training for Multi-Paradigm JEPA Control cs.LG | cs.CV | cs.ROPDF

Ruslan Rakhimov, George Bredis, Yuriy Maksyuta, Daniil Gavrilov

TL;DR: Qantara是一种联合嵌入预测架构（JEPA）世界模型，通过桥流训练实现多范式控制。它在状态轴上使用布朗桥插值器连接连续潜在状态，在动作轴上使用噪声到数据流匹配进行联合训练。同一模型支持三种推理范式：潜在规划、行为克隆动作采样和逆向动力学，无需重新训练。

Details

Motivation: 现有JEPA世界模型在训练时仅支持单一推理范式（轨迹优化或行为克隆），无法适应部署时的不同约束（如计算成本、观测可访问性）。Qantara旨在通过单一检查点支持多种推理范式，将选择推迟到推理阶段。

Result: 在LeWM控制套件中，Qantara达到91.2 SR（三个训练种子的平均值），在OGBench-Cube上创造了新的SOTA（比DINO-WM高7.7 SR，比LeWM高19.7 SR）。行为克隆和视频逆向路径在Push-T上达到82-83 SR，在Cube上达到71-73 SR。

Insight: 创新点在于将布朗桥插值器与流匹配结合，实现多范式控制；通过视频逆向组合（先预测无动作条件的潜在状态，再提取动作）支持逆向动力学推理；训练集中在噪声正方形的边缘以提高推理效率，均匀采样会显著降低性能（如Push-T规划从90.1 SR降至53.3 SR）。

Abstract: Joint-Embedding Predictive Architectures (JEPAs) underpin a growing family of latent world models for control from raw pixels, but every existing JEPA world model commits at training time to a single inference paradigm: either trajectory optimisation in a learned dynamics model, or direct behaviour cloning. A single checkpoint that serves both would defer this choice to inference, when deployment constraints (rollout cost, observation accessibility) determine which path wins. We present Qantara, an end-to-end JEPA whose joint training objective pairs a Brownian-bridge interpolant between consecutive clean latents on the state axis with noise-to-data flow matching on the action axis. The same checkpoint serves three inference paradigms without retraining: latent planning, behaviour-cloning action sampling, and inverse dynamics, which we query through a video-inverse composition that first predicts the next latent without action conditioning, then extracts the action. Training concentrates mass on the edges of the (action-time, state-time) noise square, where inference queries the predictor: replacing it with uniform interior sampling drops Push-T planning from 90.1 to 53.3 SR at matched compute. On the LeWM control suite, Qantara reaches a 91.2 SR three-train-seed average and sets new SOTA on OGBench-Cube (+7.7 SR over DINO-WM, +19.7 over LeWM). From the same weights, the behaviour-cloning and video-inverse paths reach 82-83 SR on Push-T and 71-73 SR on Cube. These results move JEPA world models from single-paradigm planners to multi-paradigm controllers.

[267] Beyond Modality Fusion: Deep Ensembles for Multimodal Classification cs.LG | cs.CVPDF

Ilya Burenko, Dmitry Vetrov

TL;DR: 本文提出了一种无需显式模态融合的多模态分类方法，即使用单模态网络的深度集成。研究表明，在参数数量相等的情况下，深度集成方法在应对模态不平衡问题时，其性能优于现有的晚期融合、中期融合及混合方法。

Details

Motivation: 针对多模态分类中模态不平衡问题，现有方法通常依赖模态融合和正则化技术来平衡学习过程，但本文旨在探索是否可以不进行显式模态融合，仅通过集成单模态网络来实现有效分类。

Result: 在多个真实和合成数据集上，深度集成方法在参数数量相等的条件下，一致超越了为处理模态不平衡而设计的最先进晚期融合方法，并且在极端模态不平衡和小规模集成情况下，仅使用强模态模型也能取得优势。

Insight: 创新点在于提出了一种无需显式融合的多模态分类深度集成框架，并通过经验验证的启发式方法选择每个模态的模型数量，避免了计算昂贵的穷举搜索；同时，通过合成多模态框架和拟合缩放定律，系统分析了模态不平衡下的优化挑战和集成方法的渐近性能。

Abstract: In multimodal classification, late-fusion approaches classify concatenated modality-specific features extracted by unimodal neural networks. When modality imbalance is pronounced, various regularization techniques have been proposed to balance the learning process and overcome the inferior performance of late-fusion networks. In contrast, this work demonstrates that multimodal data can be effectively classified without any explicit modality fusion, using deep ensembles of unimodal networks. We systematically compare deep ensembles to late-fusion networks at equal parameter count and show that ensembles consistently outperform state-of-the-art late-fusion methods designed to address modality imbalance. This advantage also holds over intermediate-fusion techniques we evaluated and over hybrid methods that combine unimodal and multimodal predictions. We propose and empirically validate a method for selecting the number of models per modality in an ensemble, avoiding computationally expensive exhaustive search. Under extreme modality imbalance and small ensemble sizes, the heuristic indicates that ensembles of unimodal models trained solely on the stronger modality are preferable; as the ensemble scales up, incorporating models from the weaker modality becomes beneficial. Both predictions align with our empirical findings. To systematically explore the challenges of optimizing multimodal models, we propose a synthetic multimodal framework that allows control over both the number of modalities and their predictive strength; our findings are consistent across synthetic and real-world datasets. Finally, by fitting scaling laws to bimodal datasets, we estimate the asymptotic performance of ensembles.

[268] Air Quality Downscaling with Station-Guided Pseudo-Supervision cs.LG | cs.AI | cs.CVPDF

Guorun Wang, Simone Foti, Andreas D. Demou, Leonidas Kotoulas, Theodoros Christoudias

TL;DR: 本文提出了一种用于欧洲PM2.5高分辨率降尺度的站点引导框架，该框架将粗分辨率的CAMS大气成分场与多种辅助信息结合，通过多尺度Transformer网络实现40倍超分辨（约1公里）和偏差校正，无需依赖时间序列建模。

Details

Motivation: 解决大气场降尺度到局部PM2.5变化时面临的空间支持不匹配问题：粗分辨率像素代表区域平均值，而地面观测是连续空间信号的离散、未对齐样本。

Result: 在欧洲范围内进行的广泛定性和站点级评估表明，该模型能够恢复细粒度的空间结构，并有效缓解了局部CAMS偏差。

Insight: 创新点在于引入了时间无关的传播策略，利用插值后的OpenAQ观测数据进行空间高斯混合，以解决用稀疏现场数据密集监督多尺度Transformer网络的挑战；框架不依赖时间序列建模，实现了联合超分辨和偏差校正。

Abstract: Super-resolving coarse atmospheric fields to local PM${2.5}$ variations is uniquely challenged by a mismatch in spatial support: while pixels represent regional averages, ground-truth observations are discrete, unaligned samples of a continuous spatial signal. To bridge this gap, we present a station-guided framework for high-resolution PM${2.5}$ downscaling over Europe. Taking coarse CAMS atmospheric composition fields alongside heterogeneous side information (i.e., human activity, land cover, elevation, satellite aerosol observations, and wind fields) our framework jointly super-resolves ($\times 40$, $\approx$ 1 km) and bias-corrects CAMS rasters, without relying on temporal sequence modelling. To address the challenge of densely supervising our multi-scale transformer network with sparse in-situ data, we introduce a time-agnostic propagation strategy that utilises spatial Gaussian blending of interpolated OpenAQ observations. Extensive qualitative and station-level evaluations across Europe demonstrate that our model recovers fine-grained spatial structures and effectively mitigates localised CAMS biases.

cs.CR [Back]

[269] Vision Token Manipulation Attacks on Cloud-Edge Inference of Large Vision-Language Models cs.CR | cs.AI | cs.CVPDF

Zikai Zhang, Rui Hu, Olivera Kotevska, Jiahao Xu

TL;DR: 本文研究了云边协同部署的大型视觉语言模型推理过程中的安全漏洞，提出了一种名为视觉令牌操纵攻击的黑盒中间人攻击方法。攻击者可以在预算约束下拦截并篡改传输中的部分视觉令牌，从而显著降低模型性能。实验表明，仅操纵10%的视觉令牌即可在多个基准测试上使模型准确率下降高达88.31%。

Details

Motivation: 云边协同LVLM推理通过将计算任务分割到边缘设备和云端服务器来提高效率，但传输中的中间视觉令牌暴露了新的攻击面。本文旨在探究在这种架构下，攻击者通过操纵视觉令牌可能造成的安全威胁。

Result: 在4个基准测试上对6个SOTA LVLM（参数量3B-72B）的实验表明，仅操纵10%的视觉令牌就能使模型准确率最大降低88.31%，揭示了该架构的关键脆弱性。

Insight: 创新点在于首次系统性地定义了云边LVLM推理中的视觉令牌操纵攻击场景，并提出了基于优化的令牌选择方法以最大化攻击效果。这揭示了模型部署中数据流安全的重要性，为未来设计鲁棒的云边协同系统提供了安全警示。

Abstract: Cloud-edge Large Vision-Language Model (LVLM) inference enables efficient deployment by splitting computation between edge devices and cloud servers. In this process, intermediate vision tokens are transmitted from the edge to the cloud over a communication link, thereby exposing a new attack surface. We study vision token manipulation attack (VTM-Attack) under a black-box man-in-the-middle setting, where an adversary intercepts and manipulates a subset of transmitted vision tokens under a budget constraint. We propose four naïve attack strategies and an optimization-based token selection method. Experiments on 6 state-of-the-art LVLMs (3B-72B) across 4 benchmarks show that manipulating only 10% of vision tokens can reduce accuracy by up to 88.31%. These results reveal a critical vulnerability in cloud-edge LVLM inference.

[270] PPE-Bench: A Benchmark for Evaluating MLLM Unlearning under Private-Public Entanglement cs.CR | cs.AI | cs.CVPDF

Xianren Zhang, Delvin Ce Zhang, Dongwon Lee, Suhang Wang

TL;DR: 该论文提出了PPE-Bench，一个用于评估多模态大语言模型在公私信息纠缠场景下遗忘能力的新基准。该基准使用包含待遗忘的私人个体和待保留的公共信息（如公众人物或地标）的复杂真实图像，以弥补现有基准的不足。

Details

Motivation: 现有MLLM遗忘基准存在两大局限：一是使用仅包含单一目标的简化图像，无法反映真实照片的视觉复杂性；二是假设遗忘集和保留集完全分离，忽略了私人信息常与良性公共信息视觉纠缠的现实。

Result: 实验发现，现有的遗忘方法虽然能减少私人信息泄露，但往往会对相邻的公共信息造成显著损害。

Insight: 主要创新点在于构建了首个考虑公私信息视觉纠缠的MLLM遗忘评估基准，并引入了两种简单有效的方法以在遗忘过程中更好地保留公共信息，这为现实场景下的隐私保护研究提供了更贴近实际的评估框架。

Abstract: Multimodal Large Language Models (MLLMs) have shown strong capabilities, but they may memorize private information from web data, raising privacy concerns. Machine unlearning offers a way to remove such private knowledge without retraining from scratch. However, existing MLLM unlearning benchmarks have two major limitations. First, they rely on simplified images that contain only the single target individual, failing to reflect the visual complexity of real-world photos. Second, they typically assume that the forget set and retain set are fully separated, ignoring the fact that private information is often visually entangled with benign public information. For example, a private individual may appear with a public figure or in front of a well-known landmark, where unlearning the private target should not damage the public context. To address these limitations, we propose PPE-Bench, a new benchmark for evaluating MLLM unlearning under private-public entanglement. Each image contains a target individual to be forgotten and public information to be preserved, including public figure and landmark. We further introduce two simple but effective methods to better preserve public information during unlearning. Through experiments, we find that existing unlearning methods can reduce private information leakage, but often substantially harm adjacent public information.

cs.HC [Back]

[271] Scalable Semantic Steering of Embedding Projections cs.HC | cs.AI | cs.CVPDF

Wei Liu, Eric Krokos, Kirsten Whitley, Rebecca Faust, Chris North

TL;DR: 本文提出了一种可扩展的语义引导方法，用于改进高维数据嵌入的低维投影，使其与分析师定义的语义关系对齐。该方法通过将语义计算从单个数据项转移到用户定义的组别，利用单次LLM调用为所有组生成结构化描述，并结合种子中心点形成混合语义原型，从而在保持全局对齐效果的同时，大幅减少了LLM调用次数和成本。

Details

Motivation: 现有基于LLM的语义引导方法通过逐项推理传播分析师意图，导致LLM调用次数和成本随数据规模线性增长，难以扩展到大规模嵌入集合。

Result: 在包含5K文档的LitCovid语料库上，该方法实现了与逐项LLM引导相当的全局对齐效果，同时将LLM调用次数减少了三个数量级以上；图像案例研究表明该基于原型的机制可扩展到多模态嵌入。

Insight: 创新点在于将语义意图的传播从逐项LLM推理转变为基于组级表示的混合原型计算，通过嵌入空间的软分配、弃权和对齐缩放更新实现无需重新训练的高效引导，为大规模嵌入集合的语义分析提供了更实用的解决方案。

Abstract: Low-dimensional projections support interactive visual analysis of high-dimensional data embeddings, but their structure often does not align with analyst-defined semantic relationships. Recent LLM-augmented semantic steering methods address this gap by externalizing analyst intent from user-defined groups of seed examples, but they propagate intent through per-item LLM reasoning, causing LLM calls and cost to grow linearly with collection size. We propose a scalable semantic steering method that shifts semantic computation from individual items to user-defined groups. A single LLM call generates structured profiles for all groups, which are embedded and combined with seed centroids to form hybrid semantic prototypes. The method then propagates intent without retraining, using embedding-space soft assignment, abstention, and alignment-scaled updates before reprojection. On a 5K-document LitCovid corpus, our method achieves global alignment comparable to per-item LLM steering while reducing LLM calls by over three orders of magnitude. An image case study shows that the same prototype-based mechanism extends to multimodal embeddings. These results suggest that group-level representations can make semantic steering more practical for larger embedding collections.

cs.GR [Back]

[272] StructuredEdit: Constraint-Aware Graphic Design Editing via Differentiable Parameter Propagation cs.GR | cs.CVPDF

Veeramanohar Avudaiappan, Ritwik Murali

TL;DR: 本文提出了StructuredEdit，一种将图形设计编辑重新定义为参数操作而非像素生成的流程。其核心技术贡献是可微参数传播（DPP），这是一种通过轻量级可微分栅格器反向传播像素级约束违规，从而将硬设计约束嵌入到视觉语言模型微调中的训练方法。该系统在约束满足度、元素匹配精度和字体识别准确率上显著超越GPT-4V基线，并在用户研究中大幅减少了编辑时间和修正迭代次数。

Details

Motivation: 当前基于像素操作的视觉语言模型在结构化设计编辑中约束满足率低（仅52%），限制了其在专业工作流程中的可靠性。本文旨在通过参数化方法解决图形设计编辑中精确操控版式、布局和视觉层次并满足严格设计约束的难题。

Result: 在结构化设计编辑任务上，StructuredEdit达到了89%的约束满足率（GPT-4V为52%），匹配元素的交并比为0.82，并在100种最常用设计字体上实现了76%的top-1字体识别准确率。用户研究（N=35）显示，相较于GPT-4V基线，编辑时间减少了33%，修正迭代次数减少了44%。

Insight: 核心创新点在于将设计编辑任务从像素生成范式转向参数操作范式，并提出了可微参数传播（DPP）训练方法，通过可微分栅格器将硬约束融入模型微调。这为在视觉语言模型中嵌入领域特定、非可微的规则或约束提供了一种新颖且有效的技术路径。

Abstract: Graphic design editing requires precise manipulation of typography, layout, and visual hierarchy under strict design constraints. Following the introduction of large language models, organizations have increasingly promoted vision-language models to enhance productivity. However, current models operate on pixels and achieve only 52% constraint satisfaction on structured design edits, thereby limiting their reliability for professional workflows. We present StructuredEdit, a pipeline that reframes design editing as parameter manipulation rather than pixel generation. Our core technical contribution is Differentiable Parameter Propagation (DPP), a training method that embeds hard design constraints into vision-language model fine-tuning by backpropagating pixel-level constraint violations through a lightweight differentiable rasterizer. A hybrid candidate-and-filter pipeline produces 125k validated edit triplets. The resulting system reaches 89% constraint satisfaction versus 52% for GPT-4V, 0.82 matched-element Intersection over Union, and 76% top-1 font accuracy over the 100 most-frequent design typefaces. In a user study (N=35), editing time drops 33% and correction iterations drop 44% relative to a GPT-4V baseline.

cs.SE [Back]

[273] Anchored Self-Play for Code Repair cs.SE | cs.CLPDF

Caroline Choi, Zeyneb Kaya, Shirley Wu, Tengyu Ma, Tatsunori Hashimoto

TL;DR: 本文提出了一种名为锚定自博弈（Anchored Self-Play, ASP）的方法，用于提升语言模型在代码修复任务上的能力。该方法通过单一模型进行生成器-修复器的自博弈强化学习，并引入一个小的参考集作为锚点，通过代码嵌入相似性奖励和混合参考错误来防止训练漂移，从而在涵盖多种错误来源的BugSourceBench基准测试上取得了最佳修复率。

Details

Motivation: 代码修复是语言模型的重要能力，但相关训练数据有限。为了扩展监督信号，本文旨在利用语言模型自身生成错误-修复任务，通过自博弈创建自动课程，但发现标准自博弈会漂移到困难但不现实的错误上。

Result: 在涵盖人类编写代码、LM生成代码和人类编辑的LM生成代码中错误的BugSourceBench基准上，ASP方法取得了最佳修复率。相比标准自博弈，ASP将平均修复率相对提升了24%（绝对提升7.0个百分点），在LM和人类产生的错误上均有增益。

Insight: 核心创新点在于提出了锚定自博弈（ASP），通过引入一个小型参考集作为锚点，结合代码嵌入相似性奖励和混合参考错误进行训练，有效防止了自博弈在自动课程学习中向不现实错误的漂移，提升了模型在多样化、现实错误来源上的泛化性能。

Abstract: Code repair is an important capability for language models (LMs): given a buggy program and unit tests, an LM must produce a fixed program that passes the tests. Because code repair data is limited, we aim to scale supervision by using an LM to generate bug–fix tasks. We propose generator–fixer self-play, in which a single model is trained with reinforcement learning to generate bugs and fix them. As the fixer improves, the generator adapts to produce more difficult bugs, yielding an automatic curriculum. To test whether this curriculum generalizes, we introduce BugSourceBench, a repair benchmark spanning realistic bug sources: bugs in human-written code, LM-generated code, and human-edited LM-generated code. On BugSourceBench, we find that self-play drifts toward difficult but unrealistic bugs, improving on synthetic bugs but degrading on human-authored ones. We propose Anchored Self-Play (ASP), which anchors self-play with a small reference set by adding a code-embedding similarity reward for generation and mixing reference bugs into fixer training. Across bug sources, ASP achieves the best fix rates, improving average fix rate over standard self-play by $+24%$ relative / $+7.0$ pp absolute, with gains on bugs from both LMs and humans.

[274] Teaching Code LLMs to Reason with Intermediate Formal Specifications cs.SE | cs.CLPDF

Minh Le-Anh, Cuong Chi Le, Tien N. Nguyen

TL;DR: 本文提出SpecCoder，一个用于训练代码大语言模型的验证引导框架，旨在教会模型生成可执行的中间形式化规约（即检查点断言），以描述程序内部关键点的预期状态。该研究引入了HumanExec基准，并在其上验证了SpecCoder在提升规约质量、程序正确性检查和程序修复任务上的有效性。

Details

Motivation: 现有基于LLM的方法主要推断整个程序的前置/后置条件，忽略了程序员在算法推理时所依赖的中间语义承诺，且生成的断言常存在语法无效、过于平凡或太弱而无法检测行为改变型错误的问题。

Result: 在HumanExec基准测试中，SpecCoder相比基础CodeLLM显著提升了检查点规约质量：在Qwen2.5-Coder模型上，内联规约的正确性提升高达55.8%，完整性提升高达358.1%，可执行断言的有效性提升高达26.6%。这些提升进一步转化为下游正确性推理和修复任务的改进。

Insight: 核心创新在于将形式化规约从被动的注释转变为可执行的证据，通过验证引导的训练框架，让模型学习从经过验证的参考程序、行为改变型变异体和多轮规约精炼轨迹中生成能区分正确与错误执行的中间检查点断言，为可靠验证提供了细粒度证据。

Abstract: Unlike natural-language specifications, executable formal specifications provide machine-checkable constraints for verifying, debugging, and repairing code. However, writing such specifications is labor-intensive, and existing LLM-based methods mainly infer whole-program pre/postconditions, missing the intermediate semantic commitments that programmers rely on when reasoning about an algorithm. Our study further shows that prompting current CodeLLMs often produces executable assertions that are syntactically invalid, trivial, or too weak to reject behavior-changing faults. In this paper, we study executable checkpoint specification generation, where assertions are inserted at meaningful internal program points to describe expected intermediate states. We introduce SpecCoder, a verification-guided CodeLLM training framework that learns from validated reference programs, behavior-changing mutants, and multi-turn specification-refinement traces. SpecCoder selects specifications that hold on correct executions while rejecting faulty executions, turning specifications from passive annotations into executable evidence. To evaluate this setting, we introduce HumanExec, a benchmark built from recent Codeforces competitive programming problems with test suites, reference solutions, and human buggy submissions, supporting three tasks: specification generation, program correctness checking, and program repair. Experiments on HumanExec show that SpecCoder substantially improves checkpoint-specification quality over base CodeLLMs. Across Qwen2.5-Coder models, SpecCoder improves inline-specification correctness by up to 55.8%, completeness by up to 358.1%, and executable assertion validity by up to 26.6%. These gains further translate to downstream correctness reasoning and repair, showing that executable checkpoints provide fine-grained evidence for reliable verification.

[275] Dashboard2Code: Evaluating Multimodal Models on Reconstructing Interactive Dashboards cs.SE | cs.AI | cs.CVPDF

Tianhao Niu, Ziyu Han, Qiguang Chen, Shiqi Zhou, Baocai Shan

TL;DR: 本文提出了Dashboard2Code任务，要求多模态大语言模型通过主动交互（如点击、筛选）探索交互式仪表盘，并生成能复现目标仪表盘的代码。为此，作者构建了DashboardMimic基准数据集，包含180个仪表盘-代码对，并设计了结合代码语义分析和动态交互测试的自动化评估框架。实验表明，即使是当前最强的模型在处理高复杂度仪表盘时仍面临困难，且开源与闭源模型在此任务上存在显著性能差距。

Details

Motivation: 现有自动数据可视化生成研究主要关注静态图表，忽视了现实世界中常用的交互式仪表盘。本文旨在填补这一空白，评估多模态模型在重建交互式仪表盘方面的能力。

Result: 在DashboardMimic基准（包含三个难度等级和八种常见交互模式）上的实验表明，所有测试的多模态模型（包括开源和闭源模型）在高复杂度仪表盘上都表现不佳，且开源模型与闭源模型之间存在显著的性能差距。

Insight: 论文的创新点在于首次定义了Dashboard2Code任务，并构建了首个针对Plotly+Dash交互式仪表盘的基准数据集DashboardMimic。同时，提出的自动化评估框架结合了代码语义分析和动态交互测试，能有效评估仪表盘的视觉和交互一致性，并与人工评估结果高度一致，为未来交互式可视化生成研究提供了新的评估范式。

Abstract: Automatic data visualization generation has advanced rapidly with multi-modal large language models, yet existing efforts largely focus on static charts and overlook the interactive dashboards commonly used for real-world data exploration. We introduce Dashboard2Code, a novel task that requires a model to proactively explore an interactive dashboard, acquire and integrate feedback from its own interactions (e.g., clicking and filtering), and generate code that reproduces the target dashboard. To support comprehensive evaluation, we present DashboardMimic, the first Plotly+Dash benchmark for Dashboard2Code, comprising 180 carefully designed and manually verified dashboard-code pairs spanning three difficulty levels and covering eight common real-world interaction patterns. We further propose an automated evaluation framework tailored to dashboards that combines code semantic analysis with dynamic interaction-based testing to assess visual and interaction consistency, showing strong agreement with human judgments. Experiments across a range of open- and closed-source multi-modal models reveal that even the strongest systems struggle on high-complexity dashboards and that a substantial performance gap remains between open-source and closed-source models on the Dashboard2Code task.

cs.IR [Back]

[276] Autonomous Information Seeking: A Roadmap for Agentic Recommender Systems cs.IR | cs.CLPDF

Xinyu Lin, Yashar Deldjoo, Sunhao Dai, Honghui Bao, Xiaopeng Ye

TL;DR: 这篇综述论文系统性地概述了基于大语言模型（LLM）的智能体在推荐系统中的整合趋势，提出了一个以自主性水平为核心、包含三种核心范式（智能体辅助推荐、智能体即推荐器、智能体即用户模拟器）的统一分类法。论文分析了各范式的智能体架构、关键组件增强方式、现有评估方法的局限性以及未来面临的开放性挑战。

Details

Motivation: 动机在于应对推荐系统从静态、基于排序的管道向能够推理、规划和行动的自主交互式系统的范式转变，旨在为这一新兴领域建立一个统一的理解基础和分析框架。

Result: 作为一篇综述论文，未报告具体的定量实验结果，但系统性地梳理和分析了现有方法、评估协议及其局限性，并指出了该领域未来的研究方向。

Insight: 创新点在于提出了一个以自主性为核心、涵盖主动性、上下文感知、交互灵活性和适应性等维度的统一分类框架，并深入探讨了评估智能体推荐系统时在推理质量、用户体验和系统行为捕捉方面存在的未解决问题，为后续研究提供了清晰的路线图。

Abstract: The rapid integration of large language model-based agents into recommender systems has driven a shift from static, ranking-based pipelines toward autonomous and interactive systems that can reason, plan, and act. This survey provides a comprehensive overview of this emerging landscape by introducing a unified taxonomy grounded in the level of autonomy and three core paradigms of agentic recommender systems: agent-assisted recommendation, agent-as-recommender, and agent-as-user-simulator. The autonomy framework organizes existing methods along increasing capabilities in proactivity, context awareness, interaction flexibility, and adaptivity. Building on this framework, the survey analyzes how each paradigm adopts different agentic architectures and how agents enhance key components such as profiles, memory, tool use, workflows, and optimization mechanisms. We further examine evaluation methodologies for agentic recommendation, covering automated metrics, LLM-based judging, and simulation-based assessment, and discuss their limitations in capturing reasoning quality, user experience, and system behavior. Beyond existing evaluation protocols, we further discuss unresolved issues in evaluating agentic recommender systems, including trajectory-level assessment, agent contribution analysis, and calibration of user simulation. Lastly, the survey outlines open challenges in lifelong user modeling, contextual abstraction, multimodal alignment, controllability, trustworthiness, privacy, scalability, and efficiency. Together, these analyses establish a unified foundation for understanding the current progress of agentic recommender systems and highlight promising opportunities for developing more autonomous, reliable, and human-aligned recommendation agents.

[277] Do All Visual Tokens Matter Equally? Object-Evidence Preserving Token Merging for Vision-Language Retrieval cs.IR | cs.AI | cs.CL | cs.CVPDF

Suhyeong Park, Junha Jung, Jungwoo Park, Jaewoo Kang

TL;DR: 本文提出了一种名为SaMer的对象感知令牌合并框架，用于解决多向量视觉语言检索中图像侧令牌存储和计算成本高的问题。该方法通过将图像侧投影后的令牌压缩为K个代表性中心点，在保持原始后期交互接口的同时，显著减少了令牌数量。

Details

Motivation: 现有令牌压缩方法可能会移除或破坏对象和区域级别的视觉证据，而这些证据是未来查询令牌可能需要的。因此，需要一种方法在压缩令牌的同时，保留可被查询选择的对象证据。

Result: 在Flickr30K和MSCOCO数据集上，当K=64时，SaMer移除了超过93%的图像侧令牌，将ColPali存储减少了16.09倍，同时提高了R@1检索精度，性能优于其他压缩基线方法。

Insight: 创新点在于提出了一种仅在训练时使用对象标注作为合并先验、无需推理时边界框或检测器的对象感知合并方法。核心见解是，高效的多向量检索不仅依赖于减少令牌数量，更关键的是保留未来查询令牌需要选择的证据。

Abstract: Multi-vector vision-language retrieval preserves fine-grained visual evidence through maximum-similarity late interaction, but dense image-side tokens make storage and scoring expensive. Existing token compression methods reduce this cost, yet they can remove or collapse object- and region-level evidence that future query tokens may need to select. We propose SaMer, an object-aware token merging framework that compresses image-side post-projector tokens into $K$ representative centroids while preserving the original late-interaction interface. SaMer uses object annotations only during training as a merge prior to discourage cross-instance mixing, requires no ground-truth bounding boxes or detectors at inference time, and adapts only the shared projection layer with frozen vision and language backbones. With $K=64$, SaMer removes more than 93% of image-side tokens and reduces ColPali storage by $16.09\times$, while improving R@1 on Flickr30K and MSCOCO. These gains arise because object-aware merging preserves query-selectable object evidence that pruning or feature-only pooling can remove or collapse. SaMer also outperforms compression baselines and shows stronger phrase-level grounding, suggesting that efficient multi-vector retrieval depends not only on reducing token count, but on preserving the evidence future query tokens need to select.

Table of Contents

cs.CL [Back]

[1] Gemma 4 Technical Report cs.CL | cs.AIPDF

[2] Echoes of Unrest: A Multimodal NLP Framework for Early Warning of Fake News and Violence-Driven Mob Activity cs.CL | cs.AIPDF

[3] Reinforcement Learning for Data-Efficient Code-Switched ASR cs.CL | cs.SDPDF

[4] Jointly Improving Dialect Identification and ASR in Indian Languages using Multimodal Feature Fusion cs.CL | eess.ASPDF

[5] Where do LLMs Fall Short in CBT-Guided Affective Reasoning? cs.CL | cs.AI | cs.HC | cs.IRPDF

[6] Seduced by the Narrative: Assessing Rule Adherence in Semi-Open Textual Sandboxes cs.CL | cs.AIPDF

[7] Distill Where the Student Goes: Teacher-Regularized RL for English-Evidence Cross-Lingual RAG cs.CLPDF

[8] Don’t Wait to Reply: Towards Responsive yet Thoughtful Dialogue through Proactive Thinking cs.CL | cs.AIPDF

[9] KARMA: Knowledge graph-based Automated Reasoning Materialization and Alignment cs.CL | cs.AI | cs.LGPDF

[10] From Judgments to Issues: Structured Extraction of Legal Reasoning with Citation-Hallucination Control cs.CL | cs.AI | cs.IRPDF

[11] Efficient Decentralized Multi-task Dataset Valuation via Model Merging cs.CL | cs.AI | cs.CV | cs.LGPDF

[12] Reading Between the Dots: Decoding Hidden Computation across Filler Tokens cs.CL | cs.AI | cs.LGPDF

[13] GRASP: Graph-Reasoning Aided Survey Planning for High-Fidelity Related Work Generation cs.CLPDF

[14] The Remarkable Effectiveness of Providing AI Agents with Natural Language Tools: A Replication Study Validating NLT Performance Across 14 Models cs.CL | cs.AIPDF

[15] NormWorlds-CF: Solver-Verified Counterfactual Normative Reasoning with Metamorphic-Relation GRPO cs.CL | cs.AIPDF

[16] BanglaMemeEvidence: A Multimodal Benchmark Dataset for Explanatory Evidence Detection in Bengali Memes cs.CL | cs.AI | cs.CVPDF

[17] Beyond Multilingual Averages: MTEB-PT, a Benchmark for Portuguese Sentence Encoders cs.CL | cs.AI | cs.LGPDF

[18] CausalGame: Benchmarking Causal Thinking of LLM Agents in Games cs.CL | cs.AI | cs.LG | stat.MLPDF

[19] Spinning Straw into Gold: Relabeling LLM Agent Trajectories in Hindsight for Successful Demonstrations cs.CLPDF

[20] AI Wizards at EXIST 2026: Hierarchical Soft-Label Learning for Multimodal Sexism Identification in Memes cs.CLPDF

[21] UI-MOPD: Multi-Platform On-Policy Distillation for Continual GUI Agent Learning cs.CL | cs.AI | cs.CV | cs.LG | cs.MMPDF

[22] dOPSD: On-Policy Self-Distillation for Diffusion Language Models cs.CL | cs.AIPDF

[23] Don’t Commit Alone: Joint Token Commitment in Diffusion Large Language Models cs.CLPDF

[24] FormalRx: Rectify and eXamine Semantic Failures in Autoformalization cs.CLPDF

[25] What You See Is What You Get: Observation-Aligned Supervision for Chart-to-Code Generation cs.CL | cs.LGPDF

[26] Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment cs.CL | cs.AI | cs.LGPDF

[27] LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure cs.CL | cs.LGPDF

[28] You Frame It: How Conceptual Representations Shape LLM Detection and Reasoning about Antisemitism cs.CLPDF

[29] Knowledge Knows, Verbalization Tells: Disentangling Latent Directions for Mathematical Solvability in LLMs cs.CL | cs.LGPDF

[30] Multi-Large Language Model Orchestrated Severity Assessment of Clinical Records (MOSAIC) cs.CLPDF

[31] Unified Audio Intelligence Without Regressing on Text Intelligence cs.CL | cs.AI | cs.LG | cs.SD | eess.ASPDF

[32] REDDIT: Correcting Model-Generated Timestamp Drift in ASR without Forgetting via Replay-Based Distribution Editing cs.CL | cs.AI | cs.SDPDF

cs.CV [Back]

[33] Learning 3D Affordances for Blade Insertion in Cluttered Stowing cs.CV | cs.ROPDF

[34] DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differences cs.CV | cs.AIPDF

[35] Reliability-Aware Monocular Depth Supervision for Sparse-View Neural Reconstruction cs.CV | cs.GRPDF

[36] Interpretable machine learning predicts Parkinson’s disease severity using motion-corrected QSM MRI and multiband multiecho fMRI features cs.CV | cs.LG | q-bio.NCPDF

[37] How many labels do you need? A decision framework for cross-habitat marine species recognition cs.CVPDF

[38] Entropy-Coded MS-VQ-VAE with Learned Priors for Ultra-Low Bitrate Video Compression cs.CVPDF

[39] Coordinate Singularities Break Conformal Coverage for Gaze and Head Pose cs.CV | cs.HCPDF

[40] MAGE: View-guided Point Cloud Completion with Efficient Modality Alignment and Adaptive Geometry Enhancement cs.CVPDF

[41] Dual-Adaptive SAM3: Hierarchical Routing over Low-Rank Expert Layers for Parameter-Efficient Medical Image Segmentation cs.CVPDF

[42] Uncertainty-Aware Last-Layer Adaptation of RETFound for Referable Diabetic Retinopathy Screening Under Dataset Shift cs.CV | cs.LGPDF

[43] Criterion-Conditional In-Context Learning: Evaluating Criterion-Shift Adaptation in Vision-Language Models cs.CV | cs.AIPDF

[44] Reliability-Aware CT-MRI Registration: A Quality Engineering Framework with Stability Analysis and Risk Classification cs.CVPDF

[45] RotateAttention: RoPE-Aware Rotation and Range Rectification for INT4 Quantized Attention in Video Generation cs.CVPDF

[46] Classroom Behavior Monitoring with YOLO An Empirical Study in Higher Education Settings cs.CV | cs.CYPDF

[47] Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoning cs.CV | cs.AI | cs.CLPDF

[48] H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation cs.CV | cs.LGPDF

[49] Evaluating Agentic Harness Systems for Autonomous Computational Pathology cs.CVPDF

[50] Token-level Response-visual Attention Guidance for Multimodal LLMs Knowledge Distillation cs.CV | cs.CLPDF

[51] Privacy-Preserving Industrial Ergonomics: mmWave-Based Automated REBA Scoring and Pose Estimation cs.CV | cs.LGPDF

[52] Latent Visual Cache for Video Reasoning cs.CV | cs.CLPDF

[53] DynaWM: A Base-VLA-Guided World Foundation Model for Moving-Object Manipulation cs.CV | cs.ROPDF

[54] Fusion: A Framework for Unified Sequential Token AdaptatIon in VisiOn TraNsformers cs.CV | cs.AI | cs.AR | cs.DC | cs.LGPDF

[55] EmoteGPT: 3D Human Facial Expressions from Natural Language Descriptions cs.CVPDF

[56] K9-Bench: Evaluating Multimodal LLMs on Canine-Centric Videos cs.CV | cs.AIPDF

[57] S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval cs.CV | cs.AIPDF

[58] VLRC: Vision-Language Reprojection Consistency as a scalable signal for better feed-forward 3D pretraining cs.CVPDF

[59] An Automated Multimodal Glaucoma Detection Framework Using ViT and a Stacking-Based Ensemble cs.CVPDF

[60] When Does Resolution Help a Frozen Backbone? Global Attention at Resolution Predicts Scalable Adaptation for Camouflaged and Marine Animal Segmentation cs.CVPDF

[61] Diagnosing Aerial-View Object Detectors with Foundational Image Generative Models cs.CV | cs.AI | cs.LGPDF

[62] Global Pose Control for Generative View Synthesis in Normalized Object Coordinate Space cs.CVPDF

[63] Signal from Space: Detecting Schools and Towers to Bridge the Digital Divide cs.CV | cs.AI | cs.CYPDF

[64] Biomechanics-aware Multi-view Markerless Motion Capture of Dexterous Hand Movements cs.CVPDF

[65] GRCD: Grounded Region Change Detection for Multi-Finding Chest X-Ray Pairs cs.CVPDF

[66] Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generation cs.CV | cs.GRPDF

[67] Prior Bias in Vision Language Models on UML Diagram Interpretation cs.CV | cs.SEPDF

[68] SPLIT: Training-Free AI-Generated and Partially Edited Video Detection via Spatial Patch-Level Incoherence and Temporal Roughness cs.CV | cs.AI | cs.LGPDF

[69] ProLaViT: Learning Progressive Latent Visual Thoughts in Structured Latent Space cs.CV | cs.CLPDF

[70] Learning Taxonomic Trees with Hierarchical Representation Regularization for Large Multimodal Models cs.CV | cs.AIPDF

[71] R3D: Quantitative 3D Spatial Reasoning for Egocentric Wearables cs.CV | cs.AIPDF

[72] STAC: Selective Spatiotemporal Aggregation and Compression for Video Reasoning Segmentation cs.CVPDF

[73] CL-Anomaly: Layer-Adaptive Mixture-of-Experts with Multimodal Large Language Model for Continual Learning in Anomaly Detection cs.CVPDF

[74] MatPhaseBench: A Semantics-Guided Benchmark for Materials Phase Diagrams Understanding cs.CV | cond-mat.mtrl-sci | cs.AIPDF

[75] VideoSearcher: Empowering Video Deep Research with Multi-Tool Agentic Reasoning via Reinforcement Learning cs.CV | cs.AIPDF

[76] MORE: A Multilingual Document Parsing Benchmark and Evaluation cs.CV | cs.CLPDF

[77] Incentivizing Vision Language Models to Search for Long Video Question Answering cs.CV | cs.LGPDF