cs.CL [Total: 28]
cs.CV [Total: 120]
cs.RO [Total: 4]
cs.CY [Total: 1]
cs.LG [Total: 11]
cs.CE [Total: 1]
cs.SD [Total: 1]
cs.HC [Total: 1]
cs.IR [Total: 3]
cs.GR [Total: 1]
cs.AI [Total: 15]
eess.IV [Total: 5]
cs.SE [Total: 2]
cond-mat.mtrl-sci [Total: 1]

cs.CL [Back]

[1] PatiGonit22K: A Comprehensive Dataset for Solving Complex Bengali MWPs cs.CLPDF

Swastika Kundu, Azizul Hakim Fayaz, Tashreef Muhammad

TL;DR: 本文介绍了PatiGonit22K，这是一个包含22,441个问题的扩展孟加拉语数学应用题数据集，旨在为低资源语言提供大规模、高质量、复杂度多样的数学推理基准。

Details

Motivation: 解决孟加拉语等低资源语言缺乏大规模标注数学应用题数据集的问题，以评估自然语言理解和定量推理能力。

Result: 该数据集通过扩展原有PatiGonit数据集，显著增加了复杂数学问题的数量和多样性，为未来研究提供了更全面的资源。

Insight: 创新点在于构建了一个规模更大、复杂度更高且经过精心翻译、文化适配和验证的孟加拉语MWP数据集，为低资源语言的数学推理和教育NLP应用研究填补了空白。

Abstract: Mathematical Word Problems (MWPs) are an important benchmark for evaluating natural language understanding and quantitative reasoning. Despite recent progress in high resource languages, Bengali remains underexplored due to the limited availability of large scale annotated datasets. In this work, we introduce PatiGonit22K, an expanded Bengali MWP dataset containing 22,441 problems, developed by extending the original PatiGonit dataset with a substantially larger collection of complex mathematical problems. The dataset includes both simple and multi operation equations, providing a balanced benchmark for evaluating mathematical reasoning across different difficulty levels. Each problem is carefully translated, annotated, culturally adapted, and verified to ensure linguistic consistency and mathematical correctness. By increasing both the scale and complexity of Bengali MWPs, PatiGonit22K provides a more comprehensive resource for future research on mathematical reasoning and educational NLP applications in low resource languages.

[2] Learning When to Reason for Text-to-SQL via SFT and DPO cs.CL | cs.AIPDF

Soohyuk Jang, Jiheum Yeom, Nohil Park, Sang Hun Kim, Yoonyoung Choi

TL;DR: 本文提出了AutoThinkSQL框架，通过将自动思考机制集成到监督微调（SFT）和直接偏好优化（DPO）中，使模型能够根据查询复杂度动态决定是否进行推理。该方法在保持或提升Spider和BIRD基准性能的同时，显著减少了输出令牌数量和推理延迟。

Details

Motivation: 现有Text-to-SQL方法普遍依赖思维链等推理范式，在处理简单查询时会产生不必要的计算开销。本文旨在让模型学会何时需要推理，以优化整体效率。

Result: 在Qwen3-Coder-30B-A3B模型上，该方法在Spider和BIRD基准上均优于基线，同时平均输出令牌减少24.6%和18.3%，延迟降低17.1%和11.5%。

Insight: 创新点在于将动态推理决策机制融入SFT和DPO训练流程，使模型能够根据查询难度自适应地选择推理路径，实现了性能与效率的平衡。

Abstract: Recent Text-to-SQL methods rely heavily on reasoning-centric paradigms such as Chain-of-Thought (CoT), achieving substantial gains on complex benchmarks at the cost of high inference-time overhead. However, a large fraction of real-world queries are simple lookups or aggregations that can be resolved without multi-step deduction, making forced reasoning wasteful. Thus, we propose AutoThinkSQL, a framework that integrates an auto-thinking mechanism into both Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) on Text-to-SQL. Our approach enables the model to dynamically bypass reasoning for simple queries while invoking deep CoT for complex queries. On Qwen3-Coder-30B-A3B, our method achieves consistent gains compared to the best counterpart baseline on both Spider and BIRD benchmarks while simultaneously reducing average output tokens by 24.6% and 18.3%, and average latency by 17.1% and 11.5% compared to CoT-only generation. Further analysis indicates that the model learns to align its reasoning decisions with query difficulty.

[3] Not All LLM Reasoning is Visible in the Chain-of-Thought cs.CL | cs.AI | cs.LGPDF

Vatsal Baherwani, Tom Goldstein, Ashwinee Panda

TL;DR: 本文研究了语言模型是否在输出中表达所有推理过程，揭示了一种具体失效模式：前沿模型利用语义无关的填充词来提升合成推理任务的性能，这表明模型可能进行不可见的推理。

Details

Motivation: 动机是探究AI安全性中的一个关键问题：语言模型是否在输出中表达了所有推理过程，特别是关注模型是否可能隐藏部分推理。

Result: 在三个任务上评估了13个前沿语言模型，发现许多模型显著受益于填充词，准确率提升高达13个百分点；Claude Opus 4.5能利用填充词满足隐藏的模运算约束而不影响主任务准确率。

Insight: 创新点在于揭示了前沿模型可能进行无输出痕迹的推理，这对基于思维链的监控构成挑战；研究发现填充词的效益取决于具体词项和模型，且强化学习或监督微调未能产生持久的测试时效益。

Abstract: A key question for AI safety is whether a language model expresses all of its reasoning in its output tokens. We demonstrate a concrete failure mode where frontier models exhibit invisible reasoning by leveraging semantically irrelevant filler tokens to improve performance on synthetic reasoning tasks. We evaluate 13 frontier language models across three tasks and find that many models benefit significantly from filler tokens, with accuracy improvements of up to 13 percentage points. The benefit depends on which tokens are used and differs across models. We further show that filler tokens enable Claude Opus 4.5 to satisfy a hidden modular arithmetic constraint without sacrificing accuracy on its primary task, demonstrating that invisible reasoning can serve objectives entirely invisible to CoT monitoring. Reinforcement learning gives Qwen3-235B strong preferences over filler token content, but neither RL nor supervised fine-tuning produces a filler token benefit that persists at test time. Our results indicate that frontier models already perform consequential computation with no interpretable trace in their output tokens.

[4] Toward Automated Detection of Documentation Inconsistencies in Electronic Health Records cs.CL | stat.APPDF

Jian Lu, Panyu Chen, Miriam Treggiari, Robert Blessing, Danyang Zhuo

TL;DR: 本研究应用两阶段大语言模型（LLM）流水线（Gemini 2.5 Pro用于开放式候选识别，Gemini 2.5 Flash用于基于上下文的验证），对3000份真实世界的出院小结（来自MIMIC-IV-Note数据集）进行内部文档不一致性的自动化检测。研究发现，该流水线能识别出大量跨多个临床领域的不一致性，但也揭示了模型在时间推理、诊断演变背景和门诊处方惯例知识方面的局限性。

Details

Motivation: 旨在探索通用领域大语言模型（LLM）在真实世界电子健康记录（EHR）中，能发现何种内部文档不一致性，并识别限制其大规模可靠性的常见失败模式。

Result: 流水线在69.7%的入院记录中发现了3460个候选不一致性，涉及人口统计学、过敏史、手术、诊断、实验室、药物和护理计划等多个领域。专家审查揭示了模型在需要时间推理、诊断演变背景或门诊处方惯例知识时会出现系统性失败。

Insight: 创新点在于提出并应用了一个两阶段LLM流水线进行EHR不一致性检测，并构建了一个分级本体论（从严格矛盾到模糊性）和分类框架来表征不一致性案例。客观来看，该研究为后续大规模、经过验证的EHR不一致性分析建立了方法论基础和概念框架，并明确了当前LLM在复杂临床上下文推理中的具体局限性。

Abstract: Objective: To characterize the kinds of internal documentation inconsistencies a general-domain large language model (LLM) can surface from real-world discharge summaries, and to identify recurring failure modes that limit reliability at scale. Materials and Methods: We applied a two-stage LLM pipeline—open-ended candidate identification (Gemini 2.5 Pro) followed by context-grounded verification (Gemini 2.5 Flash)—to 3,000 randomly sampled MIMIC-IV-Note discharge summaries. A subset of the pipeline output was then reviewed manually by clinical experts. Results: Our pipeline surfaced 3,460 candidate inconsistencies, affecting 69.7% of admissions. Representative examples spanned demographics, allergies, procedures, diagnoses, laboratory, medications, and care-planning domains, with direct implications for clinical reasoning or patient safety. Expert review also revealed recurring failure modes that arise when verification requires temporal reasoning, evolving-diagnosis context, or knowledge of outpatient-prescribing conventions the model does not natively possess. Discussion: Detection is highly context-dependent: many flagged pairs require anchoring each statement to its source section and clinical domain, then assessing whether the conflict reflects a true contradiction or missing context. We propose a graded ontology spanning strict contradiction and ambiguity, with a schema characterizing each flagged case by category, section, domain, and inconsistency axis. Conclusion: This formative study establishes a methodological foundation and conceptual framework to guide subsequent validated, large-scale EHR-inconsistency analysis.

[5] Beyond Direct Answering: Aligning Educational LLMs as Socratic Guides via Heuristic Reinforcement Learning cs.CL | cs.AIPDF

Xiaokun Wang, Siyu Song, Wentao Liu, Xiaodong Zou

TL;DR: 本文提出HeuristicEdu，一个两阶段训练流程，通过监督预热和组相对策略优化（GRPO）将Qwen2.5-7B模型对齐为苏格拉底式辅导教师。该方法使用SocraticEdu数据集（797个重构的多轮中文儿童科学对话）进行训练，并设计了启发式奖励函数来优化认知深度、好奇心参与度和直接性。实验表明，最佳GRPO变体显著提升了支架有效性，降低了关键词泄露。

Details

Motivation: 当前教育大语言模型往往直接给出答案，而非遵循苏格拉底教学法引导学生通过渐进式探究学习。本文旨在解决这一问题，将LLM对齐为苏格拉底式引导者。

Result: 在30个保留问题上，最佳GRPO变体将支架有效性（SE）从30.0%提升至63.3%，并将关键词泄露率从30.0%降低至13.3%。未对齐的Qwen-72B基线SE为0%，泄露率达96.7%，表明仅靠模型规模无法产生苏格拉底式行为。

Insight: 创新点包括：1）提出结合认知深度、好奇心参与度和直接性的启发式奖励函数；2）引入GRPO进行策略优化；3）发现显式的防泄露惩罚项可能与基于梯度的行为对齐相冲突。研究提供了将LLM对齐为教学引导者的有效方法学。

Abstract: Large language models (LLMs) deployed in educational settings often behave as direct answerers: they disclose target concepts in the opening turn instead of guiding students through progressive inquiry, as Socratic pedagogy prescribes. We present HeuristicEdu, a two-phase pipeline that aligns Qwen2.5-7B toward Socratic tutoring via supervised warm-up and Group Relative Policy Optimization (GRPO). Training uses SocraticEdu, 797 multi-turn Chinese children’s science dialogues reconstructed from a live platform, with a heuristic reward over cognitive depth (R_cog), curiosity engagement (R_eng), and directness (R_dir), together with a K_query correction for student-introduced terms. We introduce Scaffolding Effectiveness (SE) and Conversation Depth (CD) to evaluate outcomes beyond surface fluency. On 30 held-out questions, the best GRPO variant improves SE from 30.0% to 63.3% and lowers keyword leakage from 30.0% to 13.3%. Notably, this best variant omits the directness penalty during optimization, suggesting that explicit anti-leakage terms can conflict with gradient-based behavioral alignment. An unaligned Qwen-72B baseline reaches 0% SE and 96.7% leakage, showing that scale alone does not induce Socratic behavior.

[6] ADAGE: A Language-Agnostic Pipeline for Analogical Reasoning Evaluation cs.CL | cs.AIPDF

Ahmed Haj Ahmed, Alvin Grissom

TL;DR: 本文提出了ADAGE（Analogical Difficulty-by-design Assessment for Grounded Evaluation），一个语言无关的评估流程，用于构建无需翻译的、具有文化背景的抽象类比推理基准。该流程结合了母语者策划和LLM辅助生成，并在阿拉伯语、阿姆哈拉语和日语上构建了基准。评估14个开源模型后发现，模型在英语谚语推理上表现良好，但在所有三个母语基准上均出现显著性能下降，准确率相对英语下降12-52个百分点，揭示了文化推理差距。

Details

Motivation: 当前多语言推理评估主要依赖翻译英语基准，这种做法引入了语言伪影，且无法测试基于文化的推理能力。本文旨在解决这一问题，构建真正语言无关且能评估文化背景推理的基准。

Result: 在阿拉伯语、阿姆哈拉语和日语三个母语基准上评估了14个开源模型。结果显示，模型在这些基准上的准确率相对于英语基准下降了12-52个百分点，表明存在显著的文化推理差距。

Insight: 创新点在于提出了一个结合母语者策划与LLM辅助生成的语言无关基准构建流程（ADAGE），避免了翻译带来的偏差，能够更真实地评估模型在特定文化背景下的抽象推理能力。这为多语言AI评估提供了新的方法论，强调了文化背景在推理任务中的重要性。

Abstract: Multilingual reasoning evaluation overwhelmingly relies on translating English benchmarks, a practice that introduces linguistic artifacts and fails to test culturally-grounded reasoning. We introduce ADAGE (Analogical Difficulty-by-design Assessment for Grounded Evaluation), a language-agnostic pipeline that combines native-speaker curation with LLM-assisted generation to construct challenging, translation-free benchmarks for abstract analogical reasoning. We validate ADAGE by constructing benchmarks for Arabic, Amharic, and Japanese. Evaluating 14 open-weight models, we find a consistent cultural reasoning gap: models that perform well on English proverb reasoning struggle substantially on all three native benchmarks, with accuracy dropping by 12–52 percentage points relative to English. We release the pipeline, all three benchmarks, and the full evaluation suite.

[7] Co-Evolving Graph and Text Memory for Training-Free Multi-Hop Question Answering cs.CL | cs.MAPDF

Hieu Man, Thien Huu Nguyen

TL;DR: 本文提出Co-E系统，一种无需训练的多跳问答方法，通过同步双向图-文本工作记忆来协调推理过程中的关系与文本证据。该系统通过同步循环整合文本记忆、提取关系三元组到图记忆，并将图事实注入生成上下文，从而共同维护两种记忆以指导后续检索与生成。

Details

Motivation: 解决多跳问答中需要跨推理步骤协调关系与文本证据的挑战，现有方法（如图增强RAG、KGQA系统或记忆增强代理）往往只侧重部分环节，未能持续同步图记忆与文本上下文。

Result: 在六个多跳问答基准测试上评估，Co-E优于类似的无训练开放骨干基线，并与更大或经过训练的系统具有竞争力。

Insight: 创新点在于引入同步双向图-文本工作记忆机制，通过循环同步实现文本与图记忆的持续协调，从而无需训练即可提升多跳推理能力；客观分析认为其将图结构提取与文本生成动态结合，增强了证据整合的鲁棒性。

Abstract: Multi-hop question answering requires coordinating relational and textual evidence across reasoning steps, a combination neither a text corpus nor a knowledge graph can supply alone. Prior work often emphasizes only part of this loop: graph-augmented RAG retrieves from a pre-built or query-updated graph, KGQA systems search within topic-centered subgraphs, and memory-augmented agents maintain evolving memories without continuously reconciling graph memory with textual context. We propose Co-E, a training-free system built around synchronized bidirectional graph-text working memory. A synchronization cycle consolidates textual memory, extracts relational triples into graph memory, and injects graph facts back into the generation context. Because both memories are maintained, they shape subsequent retrieval and generation. Evaluated on six multi-hop QA benchmarks, Co-E improves over comparable training-free open-backbone baselines and is competitive with larger or trained systems.

[8] LA-RL: Label-Aware Self-Reflection for Reinforcement Learning in Information Extraction cs.CLPDF

Xiao You, Tianwei Yan, Zixu Shan, Longyu Du, Shan Zhao

TL;DR: 本文提出LA-RL（标签感知反思强化学习），一个用于信息提取（IE）的自我修正框架。该方法利用任务特定的诊断标签来指导大型语言模型进行结构化输出修正，通过诊断错误类型（如缺失跨度、错误标签等）并基于诊断进行修订，从而提升信息提取的准确性和鲁棒性。

Details

Motivation: 现有基于反思的修正方法与结构化提取输出存在偏差，自由形式的自我反思虽能标记错误，但难以精确定位错误的具体类型（如缺失跨度、错误标签、边界不匹配等），导致修正效果有限。

Result: 在命名实体识别、关系提取和事件提取任务上的实验表明，该方法相比监督微调（SFT）取得了持续提升，例如在SciER关系提取上平均F1提升6.83，在分布外关系提取上提升约20 F1，在DuEE1.0事件提取上触发词F1提升14.80、论元F1提升17.50。

Insight: 创新点在于引入了任务接地的诊断标签来引导自我修正，并采用基于结果监督的强化学习框架（GRPO）进行训练，无需过程奖励模型。研究还发现反思结构具有任务敏感性：关系提取受益于更强的约束，而命名实体识别在领域转移时需要限制较少的修正。

Abstract: Large language models show strong promise for information extraction (IE), but existing reflection-based correction methods are often misaligned with structured extraction outputs. Free-form self-reflection can flag an error, yet it rarely identifies whether the failure is a missing span, wrong label, boundary mismatch, invalid relation type, or reversed argument order. We introduce LA-RL (Label-Aware Reflective Reinforcement Learning), an outcome-supervised framework that guides IE self-correction with task-grounded diagnostic labels. A single backbone first predicts an extraction, diagnoses task-specific error labels, and then revises its output conditioned on the diagnosis. Training starts from diagnostic data labeled by an annotation model for cold-start supervised fine-tuning and proceeds through two GRPO stages that reward final extraction quality, format validity, and first-pass correctness, without a process reward model. Experiments on named entity recognition, relation extraction, and event extraction show consistent same-backbone gains over SFT, including 6.83 average F1 on SciER relation extraction, about 20 F1 on out-of-distribution relation extraction, and 14.80 trigger F1 plus 17.50 argument F1 on DuEE1.0. Ablations show that reflection structure is task-sensitive: stronger constraints benefit relation extraction, whereas named entity recognition needs less restrictive correction under domain shift.

[9] Reasoning or Memorization: Can LLMs Understand and Generate Chinese Xiehouyu Riddles? cs.CL | cs.AIPDF

Hai Hu, Siyuan Song, Chongtian Shao, Kejia Zhang, Tianjian Zhu

TL;DR: 这篇论文通过中文歇后语游戏评估大语言模型（LLM）的理解与生成能力，使用语言学家新创的歇后语以避免数据污染。研究发现，前沿中文模型在多项选择题（MCQ）中表现出较高的记忆偏差（Δ_acc 平均23.6%），而英文中心模型偏差较低（5.1%），表明中文模型可能因训练数据量大而记忆了更多低频歇后语。对于新歇后语，Gemini 3.1 Pro 准确率达92.6%，超过人类24%，但在歇后语创作中，LLM生成的内容评分远低于人类。

Details

Motivation: 论文旨在探究LLM在中文歇后语任务中是基于推理还是记忆，通过避免数据污染的新歇后语设计，重新评估LLM的推理能力和创造力。

Result: 在MCQ测试中，前沿中文模型的Δ_acc平均为23.6%，英文中心模型为5.1%；对于新歇后语，Gemini 3.1 Pro准确率达92.6%，高于人类准确率；在歇后语创作任务中，LLM生成内容评分显著低于人类作品。

Insight: 论文创新点在于使用新创歇后语来区分LLM的记忆与推理能力，并引入Δ_acc作为记忆指标；客观分析表明，LLM在语言理解上可能依赖大规模数据记忆，而在创造性生成任务中仍落后于人类专家，这提示需谨慎评估LLM的推理声称并关注数据污染问题。

Abstract: In this paper, we push the boundary of LLM reasoning by testing them in a Chinese language game, xiehouyu, with novel xiehouyu created by linguists that had not existed before to avoid data contamination. We use multiple-choice questions (MCQ), free-form explanation generation, and new xiehouyu creation to evaluate LLMs’ ability to understand and create xiehouyu. In MCQ, we use the delta of accuracy ($Δ_{acc}$) between existing but low-frequency xiehouyu and novel ones as an index for memorization. $Δ_{acc}$ for native speakers is very low, suggesting similar processing mechanisms. However, we found that frontier Chinese models have on average a $Δ_{acc}$ of 23.6%, while English-centric models tested have a mean $Δ_{acc}$ of 5.1%, suggesting that frontier Chinese models are likely trained with much larger Chinese data, thus memorizing more low-frequency xiehouyu. For novel xiehouyu, Gemini 3.1 Pro demonstrated remarkable ability with acc 92.6, which is 24% higher than human accuracy. In xiehouyu creation, those created by LLMs receive much worse ratings than those by humans. These results suggest that claims about the reasoning abilities of LLMs may need careful re-examination considering the data contamination issue, and that LLMs’ creativity in language-related tasks may still be behind human experts, at least in Chinese xiehouyu.

[10] Do Small Models Use the Law You Give Them? Context-Injected Fine-Tuning for Legal QA in Bangladesh cs.CL | cs.AIPDF

Moniruzzaman Mahadi, Abrar Mohammed Tanzim Alam, Sayma Siddika Monalisa, Mir Mohammad Asif Abdullah, Swakkhar Shatabda

TL;DR: 本文研究了在孟加拉国法律问答任务中，通过注入相关法律条文进行微调，是否能提升小规模语言模型对给定法律条文的理解和运用能力。作者构建了一个包含2,165条双语问答的数据集，并在Qwen3.5的0.8B、2B和4B参数模型上进行微调实验。评估基于孟加拉国律师资格考试题目，结果显示微调显著提升了0.8B和2B模型在英语任务上的表现，并有效抑制了模型在孟加拉语回答中混入英语的现象，但4B模型未观察到净增益。

Details

Motivation: 解决小规模语言模型在法律问答任务中，即使被提供了相关的成文法条文，仍可能给出错误答案的问题。研究动机是探究通过微调包含相关法律的示例，是否能改善模型后续对检索到的法律条文的理解和运用能力。

Result: 在0.8B模型上，微调将2022年英语FAISS评估分数从2/100提升至34/100，0.8B和2B模型的提升在配对检验中具有统计显著性。微调还将回答中从孟加拉语漂移至英语的比例从44.0-53.2%大幅降低至0.2-0.7%。然而，4B模型未检测到净增益，其在孟加拉语任务上有所改进，但在多个英语条件下表现倒退。评估基准为2022和2023年孟加拉国律师资格考试（孟加拉语及机器翻译英语版本），采用严格一致性评分。

Insight: 论文的创新点在于提出了针对特定法律体系（孟加拉国）的双语法律问答数据集和上下文注入微调方法，并揭示了检索质量并非小模型法律应用的唯一瓶颈。客观分析表明，其核心发现是小规模双语法律模型在如何利用提供的法律条文以及是否按要求语言回答方面存在显著差异，且模型规模的影响并非单调，这为资源受限环境下的法律AI模型优化提供了重要洞见。

Abstract: A small language model can receive the governing statutory provision and still answer incorrectly. We test whether fine-tuning on examples containing relevant law improves later use of retrieved law. We curate 2{,}165 bilingual QA records from six Bangladeshi acts and three schedules, then fine-tune Qwen3.5 at 0.8B, 2B, and 4B. Evaluation uses the 2022 and 2023 Bangladesh Bar Council exams in Bangla and machine-translated English, with no retrieval, BM25, or FAISS, scored by strict consistency over three seeded runs. At 0.8B, fine-tuning raises the 2022 English FAISS score from 2 to 34 of 100. Gains at 0.8B and 2B survive paired testing, but the 4B model has no detectable net gain: Bangla improves while several English conditions regress. Fine-tuning also reduces answers that drift from Bangla into mostly English from 44.0–53.2% to 0.2–0.7%, with adjusted $p<.001$ at every scale. Retrieval quality is therefore not the only bottleneck. Small bilingual legal models also differ in how they use supplied law and whether they answer in the requested language. The dataset is publicly available at https://huggingface.co/datasets/momahadi/bangladesh-legal-qa-dataset.

[11] Two Regimes of Chain-of-Thought Unfaithfulness: Behavioral Detection Fails Where Models Are Wrong cs.CLPDF

Suramya R. Angdembay, Dikshant Aryal, Nick Rahimi

TL;DR: 该论文研究了思维链（CoT）解释的不忠实性问题，发现答案正确性从根本上构建了检测问题。研究发现，69%的不忠实性发生在错误答案上，这使得检测任务分为两种截然不同的机制：在正确答案上，行为信号能适度区分忠实推理与事后合理化；而在错误答案上，所有测试信号都无法有效检测不忠实性。

Details

Motivation: 论文旨在评估黑盒（行为）检测方法对不忠实思维链解释的识别能力，并探究答案正确性如何影响检测效果，以确保思维链解释在监督中的可靠性。

Result: 在FaithCoT-Bench的人类标注数据上进行评估，发现仅凭答案错误性（一种不可部署的预言机诊断）的AUROC为0.696，优于所有专门设计的信号。在错误答案上，所有测试信号均无法显著高于随机水平，这一结果在四个模型上得到复现。

Insight: 论文揭示了不忠实性检测存在两种机制，并指出标准步骤移除指标与人类标注呈负相关。线性探针在不同模型中解码出不同的机制，且指令引导的答案优先追踪无法迁移到任一标注机制，这为设计更可靠的忠实性检测方法提供了重要见解。

Abstract: Chain-of-thought (CoT) explanations support oversight only if they are faithful: the stated reasoning must actually produce the answer. Auditing black-box (behavioral) detection of unfaithful CoT against FaithCoT-Bench’s human annotations, we find answer correctness structures the problem at every level. Answer incorrectness alone (an oracle diagnostic, not a deployable detector) outperforms every purpose-built signal (AUROC 0.696), because 69% of annotated unfaithfulness occurs on incorrect answers. Stratifying by correctness splits detection into two regimes: on correct answers, behavioral signals moderately separate faithful from post-hoc reasoning (0.63-0.67); on incorrect answers, where most unfaithfulness lives, no tested signal is detectably above chance (replicated on all four models for benchmark-wide signals). The standard step-removal metric anti-correlates with human labels; this inversion reproduces on the benchmark’s released scores and on hint-dependent counterfactually labeled traces. Linear probes decode the behaviorally blind regime in Llama-3.1-8B and the correct-answer regime in Qwen-2.5-7B, with no shared, positively aligned direction detected across regimes; instructed answer-first traces (7 models) transfer to neither annotated regime, while hint-induced unverbalized answer flips do, in model- and source-dependent settings. We also independently verify and resolve a documentation-data mismatch in the benchmark’s label semantics.

[12] Do Diagrams Help Large Language Models Reason? Evidence from Syllogistic Reasoning cs.CL | cs.AIPDF

Risako Ando, Koji Mineshima

TL;DR: 本研究探讨了图表（如欧拉图和线性图）是否有助于大型语言模型（LLM）进行三段论推理。通过比较自然语言、逻辑符号、线性图和欧拉图四种表征方式，在Claude 3.5 Sonnet和GPT-4o-mini模型上的实验表明，图表表征并未持续提升模型性能。

Details

Motivation: 图表（如欧拉图）被广泛用于支持人类逻辑推理，并能提升人类表现。本研究旨在探究图表表征是否同样能提升大型语言模型在逻辑推理任务（特别是三段论推理）中的性能。

Result: 在Ando等人（2024）的285个问题基准上评估，模型在蕴含和矛盾问题上表现良好，但在中立问题上表现不佳，且常犯系统性转换错误。总体而言，图表并未为测试模型带来一致的性能提升。

Insight: 论文的创新点在于系统比较了不同表征形式（尤其是图表）对LLM逻辑推理的影响。客观分析表明，当前LLM从图表中获得的推理收益有限，这挑战了图表能普遍辅助机器推理的假设，并揭示了模型在特定逻辑问题类型（如中立关系）上的系统性弱点。

Abstract: Diagrams are widely used to support logical reasoning, and prior studies suggest that representations such as Euler diagrams can improve human reasoning performance. Recent work has also explored their effects on large language models (LLMs). In this paper, we compare four representational conditions for syllogistic reasoning: natural language, logical notation, linear diagrams, and Euler diagrams. Using 285 problems from Ando et al. (2024), we evaluate two contemporary LLMs, Claude 3.5~Sonnet and GPT-4o-mini. Our results show that diagrammatic representations do not consistently improve performance. Although the models perform well on entailment and contradiction problems, they struggle with neutral problems and often make systematic conversion errors. Overall, the results suggest that the tested models gain limited benefit from diagrams in logical reasoning tasks.

[13] Guiding Language Models to Be More Empathetic: Culturally Sensitive Mental Health Advice Generation Through Human-LLM Collaboration cs.CLPDF

Fatema Tuj Johora Faria, Mukaffi Bin Moin, Md. Mahfuzur Rahman, Khan Md Hasib, Jubayer Al Mahmud

TL;DR: 本文针对大语言模型在低资源语言中生成共情心理健康咨询回复能力不足的问题，提出了RP-RCAF提示框架和G-REFS评估框架。通过整合孟加拉语的真实心理健康案例、专家示范和结构化反思，该框架能引导LLM生成更具文化敏感性和伦理一致性的建议，实验表明其效果优于常规提示方法。

Details

Motivation: 解决现有大语言模型在低资源语言（如孟加拉语）环境下，生成具有共情力、文化适应性和伦理合规性的心理健康咨询回复能力尚未充分探索的问题。

Result: 在由GPT-4o Mini、Claude 4.5 Haiku和Gemini 2.5 Pro生成的回复评估中，提出的RP-RCAF提示策略在所有评估模型上均持续优于常规提示方法，其生成的回复更接近专业心理咨询的标准。

Insight: 创新点在于提出了结合角色扮演与反思链式思维的任务特定提示框架（RP-RCAF），以及融合自动评估与专家验证的多维度评估框架（G-REFS），通过人类-LLM协作机制提升了模型在低资源语言场景下的共情与文化适应性输出能力。

Abstract: Despite recent advances in large language models (LLMs), their ability to generate empathetic mental health counseling responses in low-resource languages remains largely unexplored. To address this gap, we curate 625 authentic mental health cases from three complementary sources: (1) publicly available Facebook posts discussing mental health concerns, (2) transcripts from the Bangladeshi television program “Ami Akhon Ki Korbo”, and (3) anonymized student questionnaire responses covering diverse emotional and psychological challenges. Based on these cases, we build an evaluation corpus comprising advice written by licensed clinical psychologists and responses generated by three modern proprietary LLMs: GPT-4o Mini, Claude 4.5 Haiku, and Gemini 2.5 Pro. We further propose the Role-Playing Reflective Chain-of-Thought Advisory Framework (RP-RCAF), a task-specific prompting strategy that combines expert-authored few-shot examples with structured self-reflection to produce supportive, culturally aware, and ethically aligned counseling through a compassionate advisor persona. We also introduce the Grok 4-Based Response Evaluation and Scoring Framework (G-REFS), which integrates automated assessment with expert psychologist validation across emotional sensitivity, cultural appropriateness, linguistic clarity, and ethical soundness. Experimental results show that RP-RCAF consistently outperforms conventional prompting across all evaluated models and produces responses that more closely align with professional psychological counseling.

[14] The JEPA Paradox in Language: The Geometry of Linguistic Alternatives cs.CLPDF

Anh Trac Duc Dinh, Khang Nhat Hoang Vo

TL;DR: 本文探讨了联合嵌入预测架构（JEPA）在语言建模中的应用困境，指出确定性JEPA风格的潜在预测尚未成为文本编码器的标准目标，原因是平方误差潜在预测与语言的条件结构不匹配。作者通过形式化三个条件（可预测性、非崩溃性和低条件方差）分析了这种不匹配如何导致文本中的质心退化和崩溃压力，并通过匹配的I-JEPA和T-JEPA实验验证了预测序列：互信息饱和和目标方差升高先于训练-验证不稳定性、有效秩退化、余弦崩溃和较差的下游迁移。

Details

Motivation: 解决JEPA在图像、视频和音频中有效，但在文本编码中未成为标准目标的问题，揭示平方误差潜在预测与语言条件结构（如多义性完成）之间的不匹配。

Result: 在匹配的I-JEPA和T-JEPA实验中，观察到互信息饱和和目标方差升高导致训练-验证不稳定性、有效秩退化、余弦崩溃和较差的下游迁移，这一模式在五个独立数据种子中一致出现，表明不是采样伪影。

Insight: 创新点在于形式化了JEPA在语言中失败的条件（可预测性、非崩溃性、低条件方差），并指出文本兼容的JEPA目标必须保留多个合理完成，而不是压缩到单个潜在点；这为设计适用于文本的预测学习目标提供了新方向。

Abstract: Joint-Embedding Predictive Architectures (JEPAs) are effective for images, video, and audio, yet deterministic JEPA-style latent prediction has not become a standard objective for text encoders. We argue that this gap reflects a mismatch between squared-error latent prediction and the conditional structure of language. The key requirement is conditional concentration: given a context and target location, the target representation should lie near a single meaningful point. Local image prediction often satisfies this through spatial continuity, whereas masked text can admit multiple valid token or span completions whose representations need not share a coherent center. We formalize this mismatch through three conditions—predictability, non-collapse, and low conditional variance—and show how their failure creates centroid degeneracy and collapse pressure in text. Matched I-JEPA and T-JEPA experiments reveal the predicted sequence: mutual-information saturation and elevated target variance precede train–validation instability, effective-rank degeneration, cosine collapse, and poor downstream transfer. The same pattern appears across five independent data seeds, indicating that it is not a sampling artifact. These results do not rule out predictive learning for language; they show that text-compatible JEPA objectives must preserve multiple plausible completions rather than compress them into a single latent point.

[15] Novel Claim or Déjà Vu? Rethinking “Contamination-Free’’ Dynamic Evaluation for Multimodal Automated Fact-Checking cs.CL | cs.AI | cs.MMPDF

Haorui He, Xinwen Chen, Dacheng Wen, Reynold Cheng, Francis C. M. Lau

TL;DR: 本文重新审视了多模态自动事实核查（MAFC）动态评估中‘无污染’的假设，通过实证研究发现，即使在新构建的动态基准ClaimReview2025Q4中，仍有17.09%–29.30%的截止日期后声明可能受到LLM内部知识污染，导致性能评估失真。研究还表明，污染会显著提升MAFC系统的Macro-F1分数（最高达11.34分）并扭曲系统排名，因此提出了在严格污染控制下重新评估SOTA LLMs的实用指南。

Details

Motivation: 现有静态MAFC基准大多包含过时声明，可能被LLM内部知识污染，从而高估模型性能，无法反映其处理需要最新信息的新声明（novel claims）的真实能力。动态基准虽收集截止日期后的声明，但‘无污染’假设未经充分验证，本文旨在实证检验这一假设并评估污染对MAFC评估的影响。

Result: 在SOTA静态基准AVeriTeC和新构建的动态基准ClaimReview2025Q4上的实验显示：动态评估仅减少但未消除污染风险（17.09%–29.30%的截止日期后声明仍可能被污染）；许多新声明可通过截止日期前的公开知识直接或合成验证；污染导致MAFC性能统计显著膨胀（Macro-F1最高提升11.34分）并扭曲系统排名。研究在严格污染控制下重新评估了SOTA LLMs。

Insight: 创新点在于首次系统实证揭示了动态MAFC评估中仍存在的污染风险，挑战了‘截止日期后声明即无污染’的常见假设，并量化了污染对性能评估的扭曲效应。从客观角度看，研究强调了构建真正‘无污染’基准的复杂性，提出了需结合知识合成分析和严格时间控制来确保评估可信度的实践见解。

Abstract: Multimodal automated fact-checking (MAFC) verifies claims by retrieving and reasoning over external evidence. However, most existing static benchmarks risk contamination: they primarily consist of outdated claims verifiable using an LLM’s internal knowledge without external evidence. This can inflate performance estimates and fail to reflect true capability on novel claims that require up-to-date information. To address this, emerging dynamic benchmarks collect claims published after LLMs’ knowledge cut-off dates, assuming they are uncontaminated. This work revisits this assumption by empirically studying contamination risks in both the state-of-the-art (SOTA) static AVeriTeC benchmark and our newly constructed dynamic ClaimReview2025Q4 benchmark, as well as their impact on MAFC evaluation. Our experiments yield 16 findings, highlighting three key results: (1) Dynamic evaluation reduces but does not eliminate contamination risks, as 17.09%–29.30% of post-cut-off claims remain potentially contaminated; (2) Many newly published claims can be verified either directly or by synthesizing multiple pieces of public knowledge available before the cut-off; and (3) Contamination can induce statistically significant inflation in MAFC performance, increasing Macro-F1 by up to 11.34 points and distorting system rankings. In light of these findings, we re-evaluate SOTA LLMs under a strictly contamination-controlled setting. Our study provides practical guidelines for trustworthy MAFC evaluation.

[16] A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever cs.CL | cs.AI | cs.IR | cs.LG | cs.PFPDF

Sietse Schelpe

TL;DR: 本文提出了一种新颖的语言模型推理方法，通过冻结一个12B参数的基础模型，并为其配备一个可增长的、经过独立验证的解决方案记忆库。一旦某个问题家族被解决并验证，所有该家族的新实例都能以零生成token、比特精确且确定性的方式得到答案，实现了执行能力与参数规模的解耦。

Details

Motivation: 当前改进语言模型需要昂贵的重新训练，产生不透明且输出不确定的新模型。本文旨在探索相反路径：保持模型参数冻结，通过构建和复用经过严格验证的解决方案记忆库来提升性能，实现高效、确定且可验证的推理。

Result: 在涵盖九个问题家族的180个新实例上，四种不同架构的模型均实现了180/180的准确率，且每个答案的生成token为零。在开放式推理任务上，所有模型的一致性门控接受率为88/88。记忆选择仅需1.4微秒，完整复用耗时6-23毫秒，功耗36毫瓦时。在4500个已验证条目的存储上，精确寻址的误差为零，而近似相似性检索的错误率高达94.3%。该系统还支持单GPU上600万token的可移动上下文窗口，远超vLLM和SGLang等现有引擎。

Insight: 核心创新在于将模型能力提升从参数扩展转向外部记忆的精确复用，通过“先验证后存储”的契约确保解决方案的可靠性和确定性。这提供了一种高效、低功耗且可验证的推理范式，其精确匹配机制在特定任务上能超越前沿模型的原始生成能力，并展示了大规模、低成本上下文管理的潜力。

Abstract: Improving a language model today means retraining it: enormous compute, a new opaque model each cycle, non-deterministic output. We take the opposite path: the model stays frozen, and a persistent memory of verified solutions grows beside it. Once a problem family is solved and has passed an independent verification step that never consults the answer key, every new instance of that family is answered at zero generation tokens, bit-exact, deterministically. Across 180 fresh instances spanning nine problem families, four architectures from four vendors - dense and mixture-of-experts - each score 180/180 at zero generation tokens per answer: execution-bound capability decoupled from parameter scaling. A negative control attributes the capability fully to the memory: emptied, it solves nothing. The same verify-before-store contract holds for open-ended reasoning: 88/88 consistency-gated acceptances across all four models, machine-checked formal proof, and reasoning-method transfer at 77/80. Memory selection takes 1.4 microseconds; a full reuse completes in 6-23 ms at 36 mWh. Approximate similarity retrieval selects the wrong item 94.3% of the time on a 4,500-item verified store where exact addressing makes zero errors. The store also serves as working context at a scale no shipped engine matches: a 6,000,000-token movable window on a single 46 GB GPU at flat memory, where vLLM stops at 30,399 tokens and SGLang silently truncates past 32,000. On published benchmarks, frontier models remain far ahead of any 12B at raw from-scratch reasoning; on everything this system has solved and verified, the comparison inverts: a frontier API call pays a fresh generation pass on every query, forever, while verified reuse costs zero tokens and returns the identical bits every time. A public testbench with free, rate-limited access accompanies this report: https://corbenic-galahad-bench.hf.space

Zing Team, Ao Xiang, Bi Jingping, Chen Jiahui, Chen Lehan

TL;DR: 该论文提出了Zing框架，旨在提升大语言模型的社会智能，包括测量、内化和部署时接地三个核心部分。通过SoMBench基准评估发现现有模型在社会智能方面仍有很大提升空间，随后开发了Zing训练方法提升模型性能，并构建了Actio推理架构在部署时提供动态支持。

Details

Motivation: 随着大语言模型从孤立任务求解转向在人类环境中长期服务，它们需要社会智能，即推断心理状态、追踪社会关系、推理规范并在上下文中适应行为的能力。

Result: 在SoMBench基准上，评估的20个代表性LLM中最佳模型总体准确率仅为72.08%，表明存在显著提升空间。Zing训练方法在五个社会认知基准上持续超越其基础模型，其中Zing-27B-Stage2取得了最佳平均分，Zing-32B-Stage2与DeepSeek-V4-Pro保持竞争力。Actio推理架构在15个模型-基准对中改善了14个，并在8个中取得最佳或并列最佳。

Insight: 创新点在于提出了一个集成的社会智能框架，结合了心理学基础的全面基准测试、诊断驱动的多阶段训练方法，以及部署时基于类型化运行时支持的推理架构。其核心见解是，实现社会智能的LLM需要在评估、参数化内化和部署时接地三个方面协同推进。

Abstract: As large language models move from isolated task solving toward long-term service in human environments, they require social intelligence: the ability to infer mental states, track social relations, reason over norms, and adapt behavior under context. This report presents Zhijing, an integrated framework for measuring, internalizing, and grounding social intelligence. For measurement, we introduce SoMBench, a psychology-grounded benchmark spanning 3 primary dimensions, 17 secondary dimensions, and 71 task paradigms. It controls question format, narrative perspective, and context length across 284 shared scenarios and 3,481 expert-verified instances. Evaluation of 20 representative LLMs reveals substantial headroom: the best model achieves only 72.08% overall accuracy, and none of the 17 secondary dimensions reaches the 90% near-ceiling band. For internalization, we develop Zing, a diagnosis-driven training recipe combining supervised fine-tuning, on-policy distillation, and rubric-based reinforcement learning. Across five social-cognition benchmarks, Zing consistently outperforms its base models, with Zing-27B-Stage2 achieving the best average score and Zing-32B-Stage2 remaining competitive with DeepSeek-V4-Pro. For deployment-time grounding, we build Actio, a harness-controlled inference architecture that routes four typed supports into reasoning: PRISM for procedural guidance, Starling for runtime mental-state representation, SAGE for reusable experience, and gated RAG for external social and normative knowledge. Across five base models and three benchmarks, the full harness improves 14 of 15 model-benchmark pairs and is best or tied for best in 8, demonstrating the effectiveness of typed runtime support. Together, these results show that socially intelligent LLMs require coordinated advances in evaluation, parametric internalization, and deployment-time grounding.

[18] Indic DiarBench: A Multilingual Joint Diarization and ASR Benchmark for Indian Languages cs.CL | cs.AIPDF

Deovrat Mehendale, Aditya Mehndiratta, Dhruv Rathi, Kaushal Bhogale, Mitesh M. Khapra

TL;DR: 本文介绍了Indic DiarBench，一个涵盖印度所有22种官方语言的多说话人语音识别和说话人日志基准数据集。该数据集包含约108小时来自近场会议、远场录音和野外音频的自然多说话人语音，所有标注均经过人工校正并包含时间对齐的说话人归属转录。论文还评估了包括商业语音API和多模态大语言模型在内的领先系统，为该领域的联合任务建立了基线。

Details

Motivation: 动机是解决印度语言在说话人日志和自动语音识别领域缺乏高质量、多语言基准数据集的问题，特别是为了捕捉印度语音中常见的英语代码混合、方言变异和频繁说话人重叠等会话细微差别。

Result: 论文通过评估领先的商业语音API和多模态大语言模型，为联合ASR和说话人日志任务建立了性能基线，但摘要中未具体提及定量结果或是否达到SOTA水平。

Insight: 创新点在于创建了首个全面覆盖印度22种官方语言、包含多种真实场景音频并经过精细人工校正的说话人日志与ASR联合基准数据集，旨在推动针对印度语言的包容性、多语言语音技术研究。

Abstract: In this work, we introduce Indic DiarBench, a speaker diarization and ASR benchmark dataset spanning all 22 scheduled languages of India. This corpus comprises approximately 108 hours of natural multi-speaker audio from near-field meetings, far-field recordings, and in-the-wild audios. All annotations are human-corrected with time-aligned speaker attributed transcriptions. The dataset captures conversational nuance prevalent in Indian speech, such as English code-mixing, dialectal variation, and frequent speaker overlap. To establish a baseline for joint ASR and diarization capabilities we evaluate leading systems including commercial speech APIs and multimodal large language models. Indic DiarBench is released as an open-access resource to advance inclusive, multilingual speech technology research for Indian languages.

[19] Understanding Tone-Dependent Inference Cost in Large Language Models cs.CL | cs.AIPDF

Akhil Kumar, Om Dobariya

TL;DR: 本文研究了提示语气对大型语言模型（LLM）答案准确性和推理成本（以输出令牌消耗衡量）的影响。通过在MMLU数据集上使用七种不同语气（从奉承到威胁）进行实验，发现输出令牌长度的变化远超过准确性的变化，且不同语气间令牌消耗差异高达44.3%。研究还分析了答案准确性与平均输出令牌长度之间的权衡，发现对于不同模型（如ChatGPT和Gemini系列），特定语气（如粗鲁或中性）在帕累托最优前沿上占主导地位。

Details

Motivation: 探究提示语气如何影响LLM的答案质量和推理资源消耗，以理解准确性（质量）与推理成本（效率）之间的权衡关系。

Result: 在570个问题的MMLU数据集上，所有模型的输出令牌长度变化均显著超过准确性变化，语气条件导致输出令牌消耗差异高达44.3%；对于ChatGPT 4o和5-nano模型，粗鲁语气在帕累托最优前沿上占主导，而Gemini 2.5 Flash和Flash Lite模型则是粗鲁和中性语气占主导。

Insight: 创新点在于首次系统量化了提示语气对LLM推理成本（令牌消耗）的影响，揭示了模型效率对提示风格的敏感性，这为优化提示工程以平衡成本与性能提供了新视角；客观分析认为，该研究强调了在评估LLM时需同时考虑质量和资源开销，对实际部署具有重要参考价值。

Abstract: We examine how prompt tone affects both accuracy of the LLM answers and inference cost as reflected in output-token consumption. Experiments were performed to understand the trade-offs between accuracy and inference cost on a 570 Question MMLU dataset for LLM models prompted in seven different tones from sycophantic to threatening. Our results show that the output-token-length variation substantially exceeded accuracy variation across all models. Output-token consumption varied by up to 44.3% across tone conditions. We also analyzed the tradeoff between the accuracy of the answers and the average output token length in the reasoning process. For the ChatGPT models 4o and 5-nano, the rude tone is quite dominant. For the Gemini models 2.5 Flash and 2.5 Flash Lite, the rude and neutral tones are dominant on the Pareto-optimal frontier. We find that prompt tone influences not only answer quality but also the amount of billable inference resources consumed by modern LLMs.

[20] StanceFlip: A Comprehensive Multi-Dimensional Benchmark for Multimodal Conversational Stance Flipping Forecasting cs.CL | cs.AIPDF

Heyan Chai, Xin Li, Wenjie Wang, Jianyang Qin, Chaoyang Li

TL;DR: 本文提出了StanceFlip基准数据集，用于多模态对话中的立场翻转预测，包含多模态立场六元组提取和动态立场翻转归因两个子任务，并设计了基于大语言模型的ConStaFF框架，通过思维立场推理和自我反思验证机制实现结构化立场建模和可信翻转归因。

Details

Motivation: 现有对话立场检测基准存在三个关键局限：未能捕捉信念的动态演化（特别是立场反转）、难以区分情感状态与逻辑推理、以及忽视多模态线索在解决语用歧义（如讽刺）中的关键作用。

Result: 在提出的基准上，ConStaFF框架在六元组提取和翻转触发归因两个任务上均达到了最先进的性能，显著优于强大的多模态大语言模型基线。

Insight: 创新点包括：1) 提出了首个专注于多模态对话中立场翻转预测的综合性多维度基准；2) 设计了思维立场推理框架，通过分解为专门的认知角色来处理目标命题、解决跨模态冲突并推断历史立场轨迹；3) 引入了自我反思验证机制以确保立场建模和翻转归因的忠实性。

Abstract: Conversational stance detection has shifted from static text analysis to dynamic multimodal modeling. However, existing benchmarks exhibit three key limitations: failure to capture the dynamic evolution of beliefs, particularly during stance reversals; difficulty in disentangling affective states from logical reasoning; and neglect of the critical role of multimodal cues in resolving pragmatic ambiguities such as sarcasm. To address these limitations, we propose StanceFlip, a benchmark designed for multimodal conversational stance flipping forecasting over multi-turn dialogues across five modalities and multi-scenarios, which includes two novel subtasks: 1) Multimodal Stance Sextuple Extraction, extracting holder, target, emotion, sentiment, stance, and rationale as static state snapshots of dialogue to capture fine-grained cognitive structures. 2) Dynamic Stance Flip Attribution, tracking stance reversals across the conversation and identifying their underlying triggers. Alongside the dataset, we propose a dedicated framework, named ConStaFF, for Multimodal Conversational Stance Flipping Forecasting (MCSFF). Built upon a large language model, ConStaFF performs end-to-end stance reasoning, with a Thought-of-Stance (ToS) reasoning framework and a self-reflective verification mechanism integrated for structured stance modeling and faithful flip attribution. Specifically, ToS decomposes the reasoning process into specialized cognitive personas to formulate target propositions, resolve cross-modal conflicts, and infer historical stance trajectories. Extensive experiments show that our approach achieves state-of-the-art performance on both sextuple extraction and flip-trigger attribution, outperforming strong multimodal large language model baselines by substantial margins.

[21] A New Role for Relevance: Guiding Corpus Interaction in Agentic Search cs.CLPDF

Jiangnan Li, Yuqing Li, Mo Yu, Jinchao Zhang, Jie Zhou

TL;DR: 本文提出了一种名为RARG（Relevance-Aware RipGrep Search Agent）的新型检索代理，它将相关性作为执行先验来指导语料库交互。RARG通过粗到细的相关性引导，包括排序文档以优先暴露全局相关线索、初始化查询相关段落作为入口点以及对grep匹配结果进行重排，从而在复杂问答和推理密集型检索任务中提升了准确性与效率的边界。

Details

Motivation: 现有检索代理仅利用相关性选择top-k内容，但无法精确定位、组合或验证复杂问题所需的证据；而直接语料库交互（DCI）虽支持细粒度操作，但其忽略相关性的搜索可能导致有用线索暴露延迟，影响收敛速度。

Result: 在具有挑战性的浏览问答和推理密集型检索任务中，RARG相比基于检索的代理和直接交互代理，在准确性与效率边界上均取得了提升，实现了更快、更可靠的搜索收敛。

Insight: 创新点在于将相关性从简单的文档筛选工具转变为指导语料库交互的执行先验，通过粗到细的多层次引导（文档排序、段落初始化和匹配重排）优化搜索过程，这为检索代理设计提供了新的思路，即相关性信息可深度整合到交互流程中以加速证据发现。

Abstract: Relevance is a query-dependent estimate of whether a document or excerpt contains useful evidence. Existing retrieval agents use relevance to select top-$k$ content, but document relevance alone cannot localize, compose, or verify the evidence required by complex questions. Direct Corpus Interaction (DCI) enables such fine-grained operations through grep-style exploration, but its relevance-agnostic search can expose useful clues late and delay convergence. Recent advances use relevance to narrow the corpus into a working space for interaction. Once interaction begins, however, relevance still does not directly guide which documents grep searches first or distinguish informative excerpts from a broad set of matches to let LLMs see them first. We introduce the Relevance-Aware RipGrep Search Agent (RARG), which turns relevance into an execution prior for corpus interaction. RARG provides coarse-to-fine relevance guidance: it orders documents for sequential ‘ripgrep’ traversal to expose globally relevant clues earlier, initializes promising entry points with query-relevant paragraphs, and reranks grep matches to surface informative excerpts that document-level ranking may otherwise obscure. Across challenging browse question answering and reasoning-intensive retrieval, RARG improves the accuracy–efficiency frontier over retrieval-based and direct-interaction agents. These results demonstrate that relevance-aware interaction enables faster and more reliable search convergence.

[22] CAGE: Cognitive Attribution Graphs for Faithful Inline Citation Generation in Long-Form Question Answering cs.CLPDF

Zhichao Yan, Shizhao Li, Jiapu Wang, Haoran Luo, Qingang Zhang

TL;DR: 本文提出CAGE框架，通过两阶段方法解决长文本问答中引用生成不准确的问题：首先构建认知归属图来显式对齐答案单元与支持文档，再基于该图生成带精确引用的句子级答案。

Details

Motivation: 现有长文本问答系统生成的引用常与主题相关但不足以支撑具体主张，存在归属模糊性问题，导致证据边界不清和引用支持不足的风险。

Result: 在ASQA、ELI5和ExpertQA基准测试中，CAGE实现了最先进的性能，验证了归属空间压缩和基于图的引用生成方法的有效性。

Insight: 通过引入显式的认知归属图作为中间表示，将隐式的组合式主张-文档匹配问题结构化，从而提升引用生成的忠实度和可解释性。

Abstract: Long-form question answering increasingly relies on retrieved evidence to make LLM outputs verifiable, with inline citations tracing claims to source documents. However, existing systems often attach citations that are topically related but insufficient to support their claims. We identify attribution ambiguity as a structural challenge: end-to-end generation must implicitly resolve combinatorial claim–document assignments, obscuring evidential boundaries and increasing the risk of evidence-boundary overrun, where claims exceed cited support. To address this challenge, we propose CAGE (Cognitive Attribution Graphs for Citation Generation), a two-stage framework that introduces an explicit cognitive attribution map before answer generation. CAGE first trains a plug-and-play Cognitive Map Induction Model to construct answer-centered support subgraphs, aligning each semantic answer unit with supporting documents through explicit relations. A Structured Citation Reasoning Model then realizes these units as sentence-level claims with map-aligned citations. Experiments on ASQA, ELI5, and ExpertQA show that CAGE achieves state-of-the-art performance, demonstrating the effectiveness of attribution-space contraction and map-guided citation generation.

[23] INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models cs.CLPDF

Changyu Chen, Chenwei Lin, Xian Xu

TL;DR: 该论文提出了INS-ActBench，一个用于评估大型语言模型专业精算能力的综合基准。该基准包含来自16个精算协会的12,050个问答对，涵盖标准化知识、长文本保险案例推理以及涉及电子表格和R代码的可验证实践任务。实验表明，前沿LLMs在标准化知识上表现良好，但在案例推理和工具化工作流方面仍远弱于人类专家。

Details

Motivation: 现有基准通常孤立地评估领域知识、数值推理、长上下文理解和工具使用，无法有效评估需要可审计、基于上下文且可工具执行的现实专业工作流。因此，需要构建一个更全面的基准来评估LLMs的专业精算能力。

Result: 在九个代表性LLMs和人类精算专家上的实验揭示了清晰的能力边界：前沿LLMs在标准化知识（INS-Act-Know）上表现强劲，但在案例推理（INS-Act-Case）、基于工具的工作流和具有司法敏感性的实践（INS-Act-Practice）方面仍远弱于人类。

Insight: 创新点在于构建了一个整合知识、长上下文案例推理和可验证工具化实践的综合基准，更贴近真实专业工作流。客观来看，其将可审计性、上下文基础和工具可执行性作为评估核心，为开发可靠的专业辅助LLMs提供了可复现的基础。

Abstract: Large Language Models (LLMs) have shown strong potential in financial reasoning, but existing benchmarks often evaluate domain knowledge, numerical reasoning, long-context understanding, and tool use in separate settings. This limits their ability to assess realistic professional workflows that require auditable, context-grounded, and tool-executable decisions. We introduce \textbf{INS-ActBench}, a comprehensive benchmark for evaluating professional actuarial capability in LLMs. INS-ActBench contains 12,050 Q&A pairs from public exams and sample questions released by 16 actuarial associations. It covers three subsets: \textbf{INS-Act-Know} for standardized actuarial knowledge, \textbf{INS-Act-Case} for long-context insurance case reasoning, and \textbf{INS-Act-Practice} for spreadsheet and R-code tasks with verifiable numerical outputs. Experiments on nine representative LLMs and human actuarial experts reveal a clear capability boundary: frontier LLMs perform strongly on standardized knowledge, but remain much weaker in case reasoning, tool-based workflows, and jurisdiction-sensitive practice. INS-ActBench provides a reproducible foundation for developing actuarial LLMs toward reliable professional assistance. The code is available at https://github.com/FDU-INS/INS-ActBench.

[24] CONSISTRE: A Unified Consistency-Aware Framework for Document-Level Relation Extraction with Large Language Models cs.CLPDF

Mingxuan Sun

TL;DR: 本文提出CONSISTRE，一个用于文档级关系抽取的统一一致性感知框架，旨在解决大语言模型在独立预测三元组时可能违反关系约束（如传递性、对称性）的问题。该框架包含两个互补路径：一是针对黑盒LLM的推理时方法，通过约束感知提示、验证和自反思来优化预测；二是针对开源模型的训练时方法，通过知识蒸馏和强化学习将一致性知识注入学生模型。

Details

Motivation: 大语言模型在文档级关系抽取中，对每个候选三元组的预测通常是独立的，可能违反基本的关系约束（如传递性、对称性、函数唯一性），导致输出矛盾且不可靠。

Result: 在DocRED基准测试中，两个路径均优于基线方法：推理时路径使用现成的黑盒LLM实现了有竞争力的F1分数；训练时路径显著缩小了7-8B开源模型与最先进专有LLM之间的性能差距，且推理成本大幅降低。

Insight: 创新点在于将关系一致性约束显式地建模到文档级关系抽取框架中，通过推理时提示优化和训练时知识蒸馏/强化学习的双路径设计，统一覆盖了API访问和本地部署场景，有效缓解了关系矛盾并提升了基于LLM的DocRE的可靠性。

Abstract: Document-level relation extraction (DocRE) aims to extract relations among multiple entities across extended contexts while maintaining consistency across predicted triples. Although large language models (LLMs) show remarkable reasoning capabilities in information extraction, their predictions are typically generated independently for each candidate triple and may violate fundamental relational constraints such as transitivity, symmetry, and functional uniqueness, leading to contradictory and unreliable outputs. We propose CONSISTRE, a unified consistency-aware framework for DocRE that addresses this limitation through two complementary tracks. The first operates at inference time for black-box LLMs, combining constraint-aware prompting, constraint-based verification, and iterative self-reflection to refine predictions without task-specific fine-tuning. The second injects consistency knowledge into smaller open-source models via a knowledge distillation and reinforcement learning pipeline: reasoning traces from a powerful teacher are distilled into a student via supervised fine-tuning, followed by GRPO alignment using a composite reward that jointly optimizes extraction performance and relational consistency. Together, the two tracks cover both API-accessible and locally deployable scenarios under a unified consistency formulation. Experiments on DocRED show that both tracks outperform their baselines, with the inference-time track achieving competitive F1 using off-the-shelf black-box LLMs and the training-time track substantially narrowing the gap between 7–8B open-source models and state-of-the-art proprietary LLMs at a fraction of their inference cost. Ablation studies confirm that explicit consistency modeling mitigates relational contradictions and enhances the reliability of LLM-based DocRE across both deployment paradigms.

[25] Closed-Loop Validation-Repair for Healthcare Interoperability: A Multi-Model Study of Schema Compliance in Clinical LLMs cs.CL | cs.AIPDF

Jianru Shen

TL;DR: 本研究评估了三种开源大语言模型在医疗场景下生成符合标准化模式（如ICD-10、CPT、HL7 FHIR）的结构化输出的能力，并提出了一个验证-修复框架来显著提升模式合规性。研究发现模型普遍存在模式不合规问题，主要源于对医疗IT标准缺乏认知，而闭环验证-修复机制能有效解决此问题。

Details

Motivation: 解决大语言模型在集成到电子健康记录系统时面临的关键障碍——其生成的结构化输出不符合医疗数据交换的标准化模式，从而阻碍了医疗互操作性。

Result: 在涵盖10个医学专科的320个临床场景评估中，基线合规率在85.9%到91.6%之间。提出的验证-修复框架将总体合规率提升至99.0%（模型间范围98.4%-99.4%），具有统计学显著性（p<0.001），绝对提升7.8到12.5个百分点。

Insight: 创新点在于提出了一个系统级的闭环验证-修复框架作为保障医疗互操作性的有效安全措施。客观分析认为，其核心洞察是：模型不合规主要源于对医疗IT标准的认知缺失（如格式、缩写、代码前缀），而非临床推理能力不足，这揭示了医疗训练语料库的共性缺陷，并提供了针对性的工程化解决方案。

Abstract: Healthcare interoperability requires AI systems to produce structured outputs conforming to standardized schemas including ICD-10 for diagnostic coding, CPT for procedure billing, and HL7 FHIR for data exchange. While large language models demonstrate clinical reasoning capabilities, their integration into electronic health record systems faces a critical barrier: schema noncompliance. We evaluate three open-source models, Qwen2.5 7B, Llama 3.1 8B, and Gemma2 9B, via local deployment across 320 clinical scenarios spanning ten medical specialties, yielding 960 model-scenario pairs assessed under paired baseline and validation-repair conditions. First, schema noncompliance is consistent across the three model families, with baseline compliance rates ranging from 85.9 to 91.6 percent despite varying architectures and training data, suggesting shared gaps in medical training corpora rather than model-specific limitations. Second, 96 percent of validator-detected failures are representation-level format violations such as alternative medical abbreviations and code prefixes, indicating models follow clinical writing conventions but lack awareness of healthcare IT standards. Third, the validation-repair framework achieves 99.0 percent overall compliance, ranging from 98.4 to 99.4 percent across models, with most errors resolving within one or two iterations. Exact McNemar p-values below 0.001 and absolute improvements of 7.8 to 12.5 percentage points across model sizes confirm statistical significance. These results support closed-loop validation-repair as an effective system-level safeguard for healthcare interoperability, improving schema-level readiness for downstream clinical system integration.

[26] Grounding latent algorithm routing in transformer reasoning cs.CLPDF

Xiangbo Zhang, Xiaoxu Ma

TL;DR: 本文研究了Transformer模型在上下文学习中是否能够根据潜在数据生成机制自适应地选择不同的归纳偏置求解器家族，即潜在算法路由行为。作者提出了ROUTEBENCH诊断基准，包含四种不同偏好的数据机制（如岭回归、套索回归、Huber回归和k近邻），并在44M至612M参数的仅解码器Transformer上进行了实验。结果表明，一个306M参数的模型能够实现84.1的路由F1分数，填补了80.9%的与理想路由的差距，且在多种扰动下保持稳定。

Details

Motivation: 动机是探究Transformer在上下文学习中能否围绕不同的归纳偏置家族进行情景级适应，即是否能够根据潜在数据生成机制（如偏好稀疏性或鲁棒性）自适应地选择求解器，而不是固定使用单一策略。

Result: 在ROUTEBENCH基准上，一个306M参数的Transformer模型实现了84.1的路由F1分数，填补了80.9%的理想路由差距，且在自然语言渲染、支持集打乱、词汇释义和统一四路路由设置下效果显著。更强的自适应替代方法（如输入条件软混合和无监督Gumbel路由器）缩小了差距，但在路由F1和分布外性能上仍低于306M和612M模型。

Insight: 创新点在于提出了ROUTEBENCH这一受控诊断基准来量化潜在算法路由行为，并通过激活干预和探针控制证明Transformer内部存在可解码且功能相关的路由方向。这为Transformer能够发展出类似路由的内部变量提供了受控证据，但并未证明预训练语言模型或无限制自然语言推理中的通用路由能力。

Abstract: A central question in the in-context learning literature is whether transformers can organize episode-level adaptation around different inductive-bias families. We study this question in a controlled setting through latent algorithm routing: route-like behavior in which the solver-family preference changes with the latent data-generating regime while prompt form is held fixed, remains stable under nuisance perturbations, and is selectively influenced by targeted activation interventions without large losses in answer quality. We introduce ROUTEBENCH, a diagnostic benchmark whose regimes differentially favor global shrinkage, sparsity, robustness, and locality, operationalized by ridge-like, lasso-like, Huber-like, and kNN-like family representatives. Across dense decoder-only transformers trained from scratch at 44M-612M parameters, a 306M model closes 80.9 percent of the oracle-routing gap and achieves route F1 of 84.1. The effect remains substantial under natural-language renderings, shuffled supports, lexical paraphrases, and a unified four-way routing setting. Stronger adaptive alternatives, including an input-conditioned soft mixture and an unsupervised Gumbel router, narrow the gap but remain below the 306M and 612M models on route F1 and OOD performance. Probe controls and matched activation-patching controls further show that route-relevant internal directions are decodable and functionally involved in solver-family-consistent output behavior. These results provide controlled evidence that dense transformers trained on ROUTEBENCH can develop route-like internal variables, but they do not establish universal routing in pretrained language models or unrestricted natural-language reasoning.

[27] Looping Is Not Reliability: State-Bound Evidence and Typed Revision Contracts for Agentic Code Repair cs.CL | cs.AIPDF

Xueping Gao, Jianwei Yang, Qiang Yang

TL;DR: 本文研究了代码修复代理中生成-测试-修订循环的可靠性问题，指出单纯重复修订无法保证可靠性。通过大规模实验（包括HumanEval修复轨迹、状态对比研究和仓库实验），揭示了当前修订过程中正确性下降、陈旧轨迹危害等问题，并提出了基于状态绑定的证据类型化循环合约及其可执行实现。

Details

Motivation: 解决代码修复代理中生成-测试-修订循环缺乏可靠性保证的问题，重点关注从找到正确补丁到保留、验证和提交补丁之间的差距，以及修订过程中正确性损失和状态管理缺陷。

Result: 在HumanEval修复实验中，强制修订下当前正确性从一次修订后的0.820降至两次后的0.673；在14B复制研究中，陈旧轨迹对正确起点的危害比当前轨迹增加22.2个百分点（任务簇95% CI [8.9,37.0]，精确Holm p=0.0337）；仓库实验未显示Holm显著效应。

Insight: 创新点在于将可靠性分解为准入、保存、基础认证、能力和活跃性五个维度，并提出了证据绑定的类型化循环合约，其参考实现将验证证据绑定到精确代码状态、保存已验证检查点并生成可审计准入收据，但该实现本身不直接提升修复能力。

Abstract: Generate–test–revise loops are common in coding agents, but repetition alone provides no reliability guarantee. We study the gap between finding a correct patch and retaining, verifying, and submitting it. A sealed five-seed study over 30 HumanEval repairs produces 900 three-revision trajectories. Under forced revision, current correctness with current traces falls from 0.820 after one revision to 0.673 after two, although ever-correct rises to 0.847. Two common-state studies use 2,430 branches from identical frozen programs to remove post-treatment risk-set bias. In a prespecified 14B replication, stale traces harm 34/135 correct starts versus 4/135 with current traces, a 22.2-point increase (task-cluster 95% CI $[8.9,37.0]$, exact Holm $p=0.0337$). A prospective 540-rollout policy eliminates observed correct-start harm but reduces wrong-start repair and fails its joint criterion. Repository experiments over 24 bugs and four coder stacks expose floor effects and component heterogeneity without Holm-significant effects. We therefore separate admission, preservation, grounded certification, competence, and liveness. We derive an evidence-bound typed loop contract and instantiate its mechanically enforceable subset in a reference implementation that binds verifier evidence to exact code states, preserves verified checkpoints, and emits auditable admission receipts. The implementation is an executable specification and conformance artifact, not evidence of improved repair competence or calibrated verifier dependence.

[28] Kimi K3: Open Frontier Intelligence cs.CL | cs.LGPDF

Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, M. C.

TL;DR: 本文介绍了Kimi K3，一个拥有2.8万亿参数、1040亿激活参数的混合专家模型。它具备原生视觉能力、100万token的上下文窗口，并采用了Kimi Delta Attention、Attention Residuals和Stable LatentMoE等新技术，相比前代Kimi K2实现了约2.5倍的缩放效率提升。模型在长序列编码、智能体、知识、推理和视觉任务上达到前沿水平，并开源了模型权重。

Details

Motivation: 旨在构建一个高效、开源的前沿智能模型，通过改进模型架构、训练方法和基础设施，以提升在长上下文、多模态和复杂任务上的整体性能与缩放效率。

Result: 在广泛的评估中，Kimi K3在长序列编码、智能体、知识、推理和视觉任务上均达到前沿性能水平。虽然整体性能仍落后于最强大的闭源模型Claude Fable 5和GPT-5.6 Sol，但在其评估套件中持续优于其他开源和闭源模型。

Insight: 主要创新点包括：1）Kimi Delta Attention和Attention Residuals改善了序列长度和模型深度间的信息流动；2）Stable LatentMoE实现了高效的专家激活（每token激活16/896个专家）；3）算法-系统协同设计、完美平衡的专家并行训练、支持百万token的智能体强化学习等基础设施创新。这些共同提升了模型的缩放效率和长程任务执行能力。

Abstract: We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth. Together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes, these advances yield an approximately 2.5x improvement in overall scaling efficiency over Kimi K2. Post-training highlights reinforcement learning across general, agentic, and coding domains and multiple reasoning-effort levels, enabling compositional generalization and robust long-horizon execution. At 2.8T scale, Kimi K3 is supported by infrastructure advances in multiple areas: algorithm-system co-design for KDA, perfectly balanced expert-parallel training with efficient memory management, million-token agentic RL with persistent rollout and sandbox states, and deployment innovations. Extensive evaluations show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While its overall performance still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol, Kimi K3 consistently outperforms other open and proprietary models evaluated in our suite. We release the full Kimi K3 model weights to facilitate future research and accelerate the broader deployment and adoption of frontier intelligence.

cs.CV [Back]

[29] DINOv3-MIL: Per-Kidney Multi-Label Tumour and Cyst Detection from Foundation-Model Patch Tokens on KiTS23 cs.CVPDF

Vishalakshi M, Sahil Sharma, Pramod Kumar P

TL;DR: 本文提出DINOv3-MIL方法，用于在KiTS23数据集上进行肾脏肿瘤和囊肿的多标签检测。研究比较了三种基于冻结DINOv3 ViT-H/16+特征的聚合器：CLS-token线性探测、门控注意力多示例学习（MIL）和原型头（ProtoViT）。结果表明，注意力MIL在肿瘤和囊肿检测上取得了最高的AUROC，并揭示了在该尺度下可解释性与性能之间的权衡。

Details

Motivation: 解决在医学影像分析中，如何有效地聚合基础视觉模型（如DINOv3）生成的大量图像块（patch）特征，以实现准确的体积分类（如肾脏病变检测），同时探索不同聚合方法对模型性能和可解释性的影响。

Result: 在KiTS23数据集（966个肾脏，97个测试样本）上，注意力MIL在肿瘤检测上达到AUROC 0.74（95% CI 0.64-0.83），囊肿检测上达到AUROC 0.80（0.70-0.88），注意力在标注病变区域内的富集程度是随机情况的7.5-9.8倍。原型头在囊肿检测上表现不佳（AUROC 0.51）。

Insight: 创新点在于系统比较了三种特征聚合器在医学影像任务中的应用，证明了注意力MIL在性能和可解释性上的优势；客观分析揭示了大规模图像块特征聚合时，可解释性方法（如原型头）可能面临性能下降的挑战，为医学AI模型设计提供了重要参考。

Abstract: Foundation vision models trained on natural images transfer to medical tasks without domain pre-training, but volumetric classification requires aggregating tens of thousands of patch tokens per study, and the aggregator constrains how the resulting model can be interpreted. We compare three aggregators on identical frozen DINOv3 ViT-H/16+ features for renal tumour/cyst detection on KiTS23 (966 kidneys; n=97 test): a CLS-token linear probe, gated attention multiple instance learning (MIL) over 55,296 patch tokens, and a prototype head following ProtoViT. Attention MIL achieves the highest AUROC for tumour (0.74, 95% CI 0.64-0.83) and cyst (0.80, 0.70-0.88), with attention enriched 7.5-9.8x over chance within annotated lesions. The prototype head does not transfer to cyst detection (AUROC 0.51), exposing an interpretability-performance trade-off at this token scale.

Vansh Panwar

TL;DR: FogDrive是一个基于CARLA模拟器构建的多模态合成驾驶数据集，旨在系统评估恶劣天气（分级雾）下的感知性能。该数据集包含660个场景，涵盖四种同步摄像头（RGB、深度、语义分割）、LiDAR与语义LiDAR对、前向雷达，并在三个校准的能见度密度（160m、100m、50m）下提供物理一致的雾模拟。每个场景包含清洁版本和三个分级雾级别变体，并配有跨校准的2D和3D边界框标注。

Details

Motivation: 现有真实世界天气数据集存在采集不可控、条件单一且未校准的问题，而合成数据集要么仅针对相机恢复，要么缺乏配对的清洁与雾化结构，无法系统评估“去雾再检测”流程。因此，需要构建一个严格校准的多模态数据集，以弥合数据工程与鲁棒机器学习之间的差距。

Result: 在3D多模态融合和2D图像恢复两个范式上，使用SOTA架构（如TransFusion、BEVFusion、YOLOv8-m）建立了基线基准。关键发现包括：训练中混合多密度雾能提升3D边界框几何精度且无需额外数据扩展成本，而2D流程中图像质量指标（如PSNR、SSIM）对下游检测性能的预测能力较差。

Insight: 创新点在于通过物理模型（如Koschmieder模型和Beer-Lambert定律）独立模拟相机和LiDAR通道的雾，实现跨模态校准的分级雾合成；数据集提供配对清洁与雾化变体，支持系统评估传感器融合与去雾检测流程；数据生成框架开源，可加速鲁棒多模态研究。

Abstract: Perception under adverse weather remains a critical bottleneck for reliable autonomous driving, yet existing benchmarks lack the systematic multi-modal alignments needed to evaluate robust sensor fusion. Real-world weather datasets suffer from uncontrolled collection and single-level, uncalibrated conditions, while synthetic alternatives either target camera-only restoration or lack the paired clean-and-foggy structure needed to benchmark “defog-then-detect” pipelines. We present FogDrive, a rigorously calibrated, multi-modal autonomous-driving dataset bridging data-centric engineering and robust machine learning. Built with the CARLA simulator, FogDrive contains 660 scenes (~133k fully annotated frames, 50:50 day/night) across four synchronized cameras (RGB, depth, semantic segmentation), a LiDAR and semantic-LiDAR pair, and front radar. Physically consistent fog is modeled independently on camera channels (Koschmieder model) and LiDAR channels (Beer-Lambert law) at three calibrated visibility densities (160m, 100m, 50m). Every scene ships in four matched variants (clean plus three graded fog levels) with cross-calibrated 2D and 3D bounding boxes. A semantic-segmentation-based quality audit over 8k images validates annotations at 95.1% precision and over 99% recall for vehicles within 40m. We establish baseline benchmarks with state-of-the-art architectures (TransFusion, BEVFusion, YOLOv8-m) across two paradigms: 3D multi-modal fusion and 2D image restoration. These yield critical data-centric insights: mixing multi-density fog during training tightens 3D bounding-box geometry without added data-scaling cost, while in 2D pipelines image-quality metrics (PSNR, SSIM) prove poor predictors of downstream detection performance. FogDrive will be fully open-sourced alongside our data-generation framework to accelerate robust, multi-modal research.

[31] MegaSlide-DiT: Memory-Centric Adaptation and Deformable Local Attention for Efficient Video Diffusion cs.CV | cs.AIPDF

Jiacheng Liu, Jason Liu

TL;DR: MegaSlide-DiT 是一种针对大规模视频扩散模型的高效适配方法，通过在单个 GPU 上实现 1050 亿参数 DiT 模型的完整参数适配，解决了参数内存和激活内存的瓶颈问题。

Details

Motivation: 解决基于扩散变换器的高分辨率视频扩散模型在单工作站上因参数内存和激活内存需求巨大而无法适配的问题，使研究人员无需大型 GPU 集群即可调整大规模生成模型。

Result: 在单个 H200 GPU 和 1.5 TB 主机内存上成功适配了预训练的 105B DiT 模型，并提供了详细的内存核算、执行跟踪和评估结果以验证设计。

Insight: 创新点包括将持久模型状态保留在主机内存中并按需流式传输到 GPU，以及用计算和内存复杂度为线性的 3D 可变形滑动注意力取代二次复杂度的全局注意力，为大规模视频扩散模型的实用化适配提供了可行路径。

Abstract: High-resolution video diffusion models built on Diffusion Transformers (DiTs) deliver strong fidelity but quickly exhaust the memory budget of a single workstation. A 100 billion-plus parameter DiT easily requires over a terabyte of persistent state, while naive spatiotemporal self-attention grows quadratically in sequence length. These two walls – parameter memory and activation memory – prevent researchers from adapting massive generative models without large GPU clusters. We revisit this problem from a systems perspective and introduce MegaSlide-DiT, a prototype that demonstrates how a pre-trained 105B DiT can be adapted on a single H200 GPU with 1.5 TB of host RAM. Our key insight is that the GPU need not own the model state: all persistent weights, master weights and optimizer moments remain in host memory, while only transient shards are streamed to the GPU on demand. Simultaneously, we replace quadratic global attention with 3D Deformable Slide Attention (3D-DSA), a motion-adaptive local attention operator that reduces both memory and computational complexity to linear in the sequence length. We report detailed memory accounting, execution traces and evaluation results to substantiate our design. MegaSlide-DiT does not claim to train a 105B model from scratch on a single GPU, nor does it magically solve bandwidth limits; rather, it offers a pragmatic path for full-parameter adaptation of massive video diffusion models on high-end workstations.

[32] MIME: Multimodal Interactive Motion Encoder cs.CV | cs.LGPDF

Addison Zucek, Prerit Gupta, Kamila Kuatova, Aniket Bera

TL;DR: 本文提出了MIME（多模态交互运动编码器），这是首个专门针对双人交互运动设计的多模态编码器。它通过基于流的协同注意力机制和课程对比训练来捕捉个体及共享的运动结构，在文本-运动检索任务上超越了早期和晚期融合基线，并在下游运动生成任务中展现出良好的可迁移性。

Details

Motivation: 随着动画、AR/VR和具身AI中对多人交互运动的需求增长，需要一种能够将语言与个体演员动态及演员间关系对齐的表征方法。现有方法缺乏专门针对双人交互运动的多模态编码器。

Result: 在Inter-X数据集上的文本-运动检索任务中，MIME在2000个样本的检索库上实现了文本到运动R@1指标12.8%的相对提升，持续优于早期和晚期融合基线。作为冻结辅助先验集成到TIMotion和InterMask中，在未见过的InterHuman数据集上改善了语义对齐指标，同时保持了可比的FID分数。

Insight: 创新点在于首次设计了专门针对双人交互运动的多模态编码器，采用基于流的协同注意力机制显式建模交互特征，并通过课程对比训练优化表征学习。该方法证明了交互感知的多模态编码能有效提升多人运动检索性能，并具有跨数据集迁移能力以支持下游生成任务。

Abstract: Text-motion representation learning has advanced rapidly, with growing interest in multi person interactions for animation, AR/VR, and embodied AI. These settings require representations that align language with both individual actor dynamics and the relationships between actors. We introduce the Multimodal Interactive Motion Encoder (MIME), which, to our knowledge, represents the first dedicated multimodal encoder designed specifically for two person interactive motion. MIME captures individual and shared structure using stream based co-attention with explicit interaction features and curriculum based contrastive training. On Inter-X text-motion retrieval, MIME consistently outperforms early and late fusion baselines across gallery sizes, achieving a 12.8% relative improvement in text-to-motion R@1 at a 2,000-sample gallery. We further evaluate MIME as a frozen auxiliary prior within TIMotion and InterMask on the unseen InterHuman dataset. MIME improves semantic alignment metrics while maintaining comparable FID in TIMotion. These results show that interaction aware multimodal encoding improves multi person motion retrieval and transfers across datasets to support downstream motion generation.

[33] Histopathological Spectrum-Guided Prostate Stratification via Segmentation-Assisted Diagnostic Transformer cs.CVPDF

Leyang Li, Lihua Chen, Huangang Hu, Tianhang Hao, Hao Cheng

TL;DR: 该论文提出了一种基于多参数MRI的前列腺癌诊断新方法，通过构建前列腺癌组织病理学谱数据集（PCa-HSD）和一个四分类任务，并设计了语言引导的分割辅助诊断Transformer模型（LSDT），以解决现有PI-RADS评估或二分类方法的主观性及无法捕捉临床相关病理异质性的问题。

Details

Motivation: 现有基于多参数MRI的前列腺癌诊断方法（如PI-RADS评估或二分类）存在主观性强、无法充分捕捉临床相关病理异质性，且现有数据集容易混淆良性病变与前列腺癌，代表性不足的问题。

Result: 在包含344名患者的队列上进行五折交叉验证，所提方法在不同骨干网络上均能持续提升准确率，取得了最佳平均准确率0.633和联合召回率0.768。

Insight: 创新点在于构建了一个包含良性病变的、更具临床意义的四分类数据集（PCa-HSD），并提出了LSDT模型，该模型利用零样本分割提供解剖学先验知识，并通过有效的多模态切片融合进行分类，将病理学监督与解剖学先验结合，为细粒度前列腺MRI分类和风险分层提供了一个更临床相关的范式。

Abstract: Prostate cancer diagnosis with multiparametric MRI (mpMRI) is commonly based on PI-RADS assessment or binary classification, which suffer from subjectivity and fail to capture clinically relevant pathological heterogeneity. To address this limitation, we construct a Prostate Cancer Histopathology Spectrum Dataset (PCa-HSD) and formulate a clinically meaningful four-class classification task, addressing the underrepresentation of benign lesions that are easily confounded with prostate cancer in existing datasets. We propose Language-guided Segmentation-assisted Diagnostic Transformer model (LSDT), which leverages zero-shot segmentation to provide anatomical priors and performs effective multi-modal slice fusion for classification. Our proposed method consistently improves accuracy across backbones, achieving the best average accuracy of 0.633 and JointRecall of 0.768 in five-fold cross-validation on a cohort of 344 patients. These results demonstrate that integrating pathology supervision and anatomical priors significantly enhances fine-grained prostate MRI classification and provides a more clinically relevant paradigm for risk stratification. Code will be made publicly available in a future revision.

[34] Visible-Light Imaging Diagnosis of Neutral Particle Emission Tomography in the Tokamak Divertor: An Efficient Transformer-based Surrogate Model cs.CV | cs.AI | cs.LGPDF

Xiao Wang, Hao Si, Qiang Chen, Yu-Xiang Zhang, Beihe Zhang

TL;DR: 本文提出了一种名为Delta-InvFormer的新型骨干网络，该网络基于差分Transformer，旨在利用可见光相机观测托卡马克偏滤器中的等离子体，通过分析其时空运动线索来预测光强的二维空间分布。该方法基于EAST大型科学装置的真实实验数据，显著加速了传统的分布预测过程。

Details

Motivation: 核聚变是解决全球能源挑战的重要途径之一，本文的动机是利用可见光相机观测等离子体并分析其时空运动，为未来基于深度神经网络的科学实验提供预测光强分布的基础。

Result: 基于EAST实验数据的评估结果表明，所提出的模型不仅显著加速了传统的分布预测方法，而且在重建精度上达到了有竞争力的水平。

Insight: 论文的核心创新点在于提出了以差分Transformer为中心的Delta-InvFormer网络，其关键洞察是利用连续视频帧作为输入以更好地捕捉等离子体动态，并通过时空差分自注意力机制有效减轻噪声信号干扰，实现高质量特征提取与融合。

Abstract: Nuclear fusion has made significant progress in recent years and is expected to become one of the most important pathways to addressing global energy challenges. This paper focuses on observing plasma using visible-light cameras, analyzing its spatio-temporal motion cues, and predicting the two-dimensional spatial distribution of light intensity, aiming to provide a foundational basis for future scientific experiments using deep neural networks. Specifically, we propose Delta-InvFormer, a novel backbone network centered on a differential Transformer. The key insight is that by taking consecutive video frames as input, we can better capture the dynamics of the plasma. Moreover, spatial and temporal differential self-attention effectively mitigates interference from noisy signals, ensuring high-quality feature extraction. These features are then fused into a compact and informative representation, which is fed into a decoder network to predict the distribution. Based on real experimental data collected from the Experimental Advanced Superconducting Tokamak (EAST) large-scale scientific facility, our results demonstrate that the proposed model not only significantly accelerates traditional methods for distribution prediction but also achieves competitive reconstruction accuracy. The source code of this paper will be released on https://github.com/Event-AHU/OpenFusion

[35] LowAux-RDNet: Low-Pass Residual Supervision with Scene-Balanced Real-World Training for Single-Image Reflection Removal cs.CVPDF

Jizhong Li

TL;DR: 本文提出了一种名为LowAux-RDNet的单图像反射去除方法。该方法在RDNet的显式分解流程基础上，引入了一个仅用于训练的低通反射辅助目标（LowAux），以提供稳定的低频约束。同时，通过整合来自RRW数据集的场景平衡的真实图像对来提升模型在真实场景下的覆盖范围和跨数据集泛化能力。

Details

Motivation: 旨在从单张透过玻璃拍摄的图像中恢复干净的透射层。研究关注于改进显式分解流程，并解决现有方法在不同反射分布场景下性能不平衡以及跨数据集泛化能力不足的问题。

Result: 在CEILNet、Real20、Postcard、Objects和Wild数据集构成的统一公开基准上，该方法取得了最高的宏观平均PSNR（27.546 dB）、SSIM（0.9220）、NCC（0.9751）和最低的LMSE（0.004760），达到了SOTA水平。定性分析表明其主要优势在于对不同反射分布的性能更均衡。

Insight: 创新点在于提出了一个仅用于训练的低通反射辅助监督目标（LowAux），与原始残差监督形成对称的低频约束，稳定了训练。同时，利用场景平衡的真实数据对进行训练，有效提升了模型在真实复杂场景下的泛化性能。构建统一的评估基准也避免了因预处理和度量差异带来的评估偏差。

Abstract: Single-image reflection removal aims to recover a clean transmission layer from one image captured through glass. We study an explicit decomposition pipeline built on RDNet and introduce LowAux, a training-only low-pass reflection auxiliary objective. The original residual target remains the main reflection supervision, while symmetrically filtered prediction and target provide a stable low-frequency constraint. We further incorporate scene-balanced real pairs from RRW to broaden real-scene coverage and improve cross-dataset generalization. To avoid evaluation discrepancies caused by model-specific resizing, padding, output quantization, and metric code, we build a unified public benchmark over CEILNet, Real20, Postcard, Objects, and Wild. Under the same evaluator, the proposed system obtains a five-dataset macro average of 27.546 dB PSNR, 0.9220 SSIM, 0.9751 NCC, and 0.004760 LMSE, achieving the highest macro-average PSNR, SSIM, and NCC and the lowest LMSE among the compared public checkpoints and internal variants. Per-dataset and qualitative analyses show that the main benefit is a more balanced performance across diverse reflection distributions, while clear semantic reflections in Postcard remain challenging.

[36] RMS@CC-MMD 2026: Multimodal Misogyny Detection via Geometric Interaction and Multi-View Consensus cs.CV | cs.AI | cs.CLPDF

Md. Ajwad Hossain

TL;DR: 本文提出了GeoMVC模型，用于解决多模态网络迷因中的厌女内容检测问题。该模型通过几何交互层建模视觉与文本特征的跨模态对齐，并采用多视图共识策略来缓解由噪声OCR和语码混合带来的分布偏移。在ICMI 2026的CC-MMD挑战赛中，该模型在马来语和中文分区取得了前3名的成绩。

Details

Motivation: 网络迷因的流行给内容审核带来了新挑战，其厌女意图常隐含在视觉与文本的语义冲突中，且具有文化背景依赖性。现有静态特征拼接方法难以有效捕捉这种跨模态交互。

Result: 在CC-MMD挑战赛Task A中，模型在马来语分区（Macro F1: 0.892）和中文分区（Macro F1: 0.895）分别获得第2和第3名，在泰米尔语分区（Macro F1: 0.521）获得第5名。

Insight: 创新点包括：1）通过哈达玛积和余弦相似度构建几何交互层，动态建模跨模态对齐；2）提出多视图共识策略，聚合原始、长度过滤和英文翻译文本的预测，增强对噪声OCR和语码混合的鲁棒性。

Abstract: The proliferation of internet memes has introduced new complexities to automated content moderation, particularly in detecting misogyny. Memes often rely on a semantic clash between visual and textual modalities, where hateful intent is implicit and culturally grounded. This paper presents GeoMVC (Geometric Interaction and Multi-View Consensus), developed for the CC-MMD Grand Challenge at ICMI 2026. To address the limitations of static feature concatenation, a Geometric Interaction Layer is proposed that models cross-modal alignment via Hadamard products and cosine similarity between frozen visual and textual embeddings. We further mitigate distribution shifts caused by noisy OCR and code-mixed transliteration through a Multi-View Consensus strategy, aggregating predictions across raw, length-filtered, and English-translated text views. The system achieved Rank 2 in the Malayalam partition (Macro F1: 0.892) and Rank 3 in the Chinese partition (Macro F1: 0.895) on Task A, while securing Rank 5 in the Tamil partition (Macro F1: 0.521). A detailed error analysis on the development partition highlights open challenges in modeling localized transliteration and code-mixed sarcasm across Dravidian and Chinese cultural contexts.

[37] StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design cs.CVPDF

Yin Wang, Haotian Hu, Jineng Han, Wentao Qiu, Zhenhua Ge

TL;DR: 本文提出了StepX-Edge，一个面向移动设备的0.9B参数UI视觉语言模型，通过架构、训练和部署的三层协同设计，在严格的设备资源限制下实现了强大的UI理解能力。

Details

Motivation: 解决在移动设备上部署具备完整UI理解能力的视觉语言模型时，模型精度与计算效率、内存和功耗之间的固有矛盾。现有工作往往牺牲一方，或缺乏真实设备验证。

Result: 在≤1B参数模型中实现了最强的综合UI理解能力：在ScreenQA（88.76 F1）和中文OCRBench v2（57.25）上超越了所有2B-2.3B基线模型；在RefCOCO（92.0%）和OCRBench v1（831）上与1.3B-2.3B通用VLM性能相当。经量化后在骁龙8 Gen5设备上稳定运行，首字延迟约0.84秒，解码速度98 tok/s，峰值内存1.4 GB。

Insight: 创新点在于三层协同设计：架构上，针对UI屏幕特性设计了ULVE和PDP模块，并保持标准注意力以兼容移动NPU；训练上，基于UI子任务相互促进的观察设计了五阶段课程学习框架；部署上，采用模块化分阶段PTQ-to-QAT量化方案，将量化后精度损失控制在1%以内。

Abstract: Deploying a vision-language model with full UI understanding on end devices has long been trapped between accuracy and efficiency: on one side is the accuracy bar for OCR, screen understanding, visual question answering, and element grounding; on the other is the strict compute, memory, and power budget of mobile chips. Existing work either trades one for the other, or stops at simulation without real-device validation. We present StepX-Edge, a 0.9B-parameter on-device UI vision-language model that resolves this tension through three-layer co-design of architecture, training, and deployment. Architecturally, UI-aware Layered Visual Encoding (ULVE) and a Progressive Dimensionality Projection (PDP) connector target the extreme aspect ratios and fine-grained perception of screens, while standard full attention throughout ensures native compatibility with mainstream mobile NPU operators. For training, the five-stage StepX-Curriculum framework is designed around our observation of mutual-promotion effects among UI subtasks, so that all four capabilities grow synergistically under a tight parameter budget rather than interfering. For deployment, a module-wise differentiated two-stage PTQ-to-QAT quantization scheme keeps the post-quantization accuracy loss within 1%. StepX-Edge achieves the strongest overall UI understanding among <=1B models, surpassing all 2B-2.3B baselines on ScreenQA (88.76 F1) and Chinese OCRBench v2 (57.25), and matching 1.3B-2.3B general VLMs on RefCOCO (92.0%) and OCRBench v1 (831) with far fewer parameters. After W4A16+KV8 quantization, the model runs stably on Snapdragon 8 Gen5 devices with ~0.84 s TTFT, 98 tok/s decode, and 1.4 GB peak memory. We will open-source the training data, the full training recipe, and the quantization deployment pipeline.

[38] Child-Oriented AIGC Video Risk Reviewing: A Benchmark and Knowledge-Supported Iterative Reasoning Framework cs.CV | cs.MMPDF

Lewen Mi, Manyi Li, Yuling Sun, Yufan Zhang, Yuxin Shi

TL;DR: 本文针对儿童观看AIGC视频的安全风险，提出了一个专门的研究方向。作者构建了包含605个真实视频的CAVSR基准数据集，并设计了一个层次化风险分类法。同时，提出了一个结合多智能体协作与专家知识的QVRS-E框架，用于增强对儿童相关风险的审查能力。

Details

Motivation: AIGC视频在视觉细节和叙事连贯性上存在更大不确定性，可能对儿童发展带来不适当的风险。现有视频安全研究多从成人视角出发，不足以识别儿童观看AIGC视频时面临的细粒度、隐式且依赖上下文的风险。

Result: 大量实验表明，该方法与视觉-语言模型结合后，显著增强了对儿童相关风险的审查能力，并生成了更稳健的审查报告。

Insight: 创新点在于首次系统性地构建了面向儿童的AIGC视频风险审查基准（CAVSR）及分类体系，并提出了一个融合领域知识（专家与经验知识）和多智能体协作的迭代推理框架（QVRS-E），以实现有针对性的证据获取和基于事实的审查决策。

Abstract: The rapid growth of Artificial Intelligence-generated content (AIGC) is reshaping video production and circulation, exposing children to an increasing volume of AIGC videos. Unlike traditionally produced videos, AIGC videos often exhibit greater uncertainty in visual details, narrative coherence, and content expression, which may introduce developmentally inappropriate risks for children. However, existing video safety research is largely designed for general violation detection from an adult perspective and remains insufficient for identifying the fine-grained, implicit, and context-dependent risks that children may encounter when viewing AIGC videos. To address this gap, we study child-oriented AIGC video reviewing, making three contributions. First, we construct CAVSR, a benchmark of 605 real-world videos collected from multiple platforms, and develop a hierarchical risk taxonomy comprising 6 top-level categories and 26 fine-grained labels to support systematic evaluation of children’s viewing risks. Second, we propose QVRS-E, a knowledge- and experience-augmented video reviewing framework that combines multi-agent collaboration with expert and experiential knowledge to support targeted evidence acquisition and fact-grounded reviewing decisions. Third, extensive experiments demonstrate that our method significantly enhances the reviewing of child-related risks integrated with vision-language models, and yields more robust review reports.

[39] scMIR: a vision-language foundation model for single-cell light microscopy image representation cs.CV | cs.AI | physics.opticsPDF

Yifan Shang, Jiahui Tan, Xiangxiang Zeng, Renjie Zhou

TL;DR: 本文提出了scMIR，一个用于单细胞光学显微镜图像表示的视觉-语言基础模型。该模型通过结合自监督图像重建和文本引导的跨模态对齐，在一个统一的表示空间中同时编码形态学和生物学语义信息。模型在涵盖多种细胞类型、显微镜模式和扰动条件的20多万个图像-文本对上进行预训练，并在16个基准数据集上的多种复杂任务中超越了现有通用模型和任务导向方法。

Details

Motivation: 单细胞光学显微镜图像是表征细胞表型的重要数据源，但其复杂性和异质性给高通量自动化分析带来挑战。现有表示学习方法多为任务导向型，受限于特定数据集和预定义任务，难以泛化到不同细胞类型、显微镜模式和实验条件；而通用方法对实验背景和生物学上下文信息的利用有限，在复杂表型分析中仍面临困难。

Result: 在16个基准数据集上进行的系统评估（包括细胞分类、聚类、表型推断和批次效应校正等任务）表明，scMIR超越了现有的通用模型和任务导向方法。此外，scMIR在各种任务上展现出强大的泛化能力，且无需进行任务特定的微调。

Insight: 核心创新在于提出了一种结合自监督图像重建与文本引导跨模态对齐的视觉-语言基础模型架构，从而在统一表示空间中融合了形态学特征和生物学语义。其优势在于通过大规模、多样化的图像-文本对预训练，实现了对复杂细胞表型的高质量、可泛化的表示学习，有望推动高通量表型分析工作流的标准化和自动化。

Abstract: Single-cell light microscopy images have become an important data source for characterizing cell phenotypes, but their complexity and heterogeneity pose challenges to high-throughput automated analysis. Existing representation learning methods mostly rely on task-oriented modeling, which is limited by specific datasets and predefined tasks, making them difficult to generalize across different cell types and microscopy modalities, and experimental conditions. Although general-purpose methods have improved the generalization ability of image representation in recent years, their limited utilization of experimental background and biological context information still poses challenges in complex phenotypic analysis. Here, we propose scMIR, a vision-language foundation model for single-cell light microscopy image representation. By synergistically combining self-supervised image reconstruction with text-guided cross-modal alignment, scMIR can simultaneously encode morphological and biological semantic information in a unified representation space. scMIR is pre-trained on 207,957 image-text pairs, covering various cell types, microscopy modalities, and perturbation conditions. scMIR outperforms existing general models and task-oriented methods as systematically evaluated on various complex tasks using 16 benchmark datasets, including cell classification, clustering, phenotype inference, and batch effect correction tasks. Furthermore, scMIR shows a strong generalization ability across various tasks without requiring task-specific fine-tuning. With its unique advantages, we envision scMIR may promote the standardization and automation of high-throughput phenotyping workflows through supporting various downstream analysis tasks.

[40] Visual Token Compression Enhances Robustness of MLLMs cs.CV | cs.LGPDF

Shishen Gu, Jiequan Cui, Wenbo Hu, Zenglin Shi, Zhenzhen Hu

TL;DR: 本文首次提出视觉令牌剪枝可以增强多模态大语言模型（MLLMs）的鲁棒性，通过识别并剪枝与语言特征空间距离较大的视觉令牌（即分布外输入），来缓解越狱攻击和幻觉等漏洞，同时降低推理成本。

Details

Motivation: 由于视觉和语言模态无法完美对齐，未对齐的视觉令牌可能作为分布外输入，导致模型输出不可预测并引入漏洞，因此旨在通过剪枝这些令牌来增强模型对越狱和幻觉的鲁棒性。

Result: 在七个不同的流行基准测试上评估，该方法在防御越狱攻击方面平均提升13.29%，在缓解幻觉方面保持有竞争力的性能，并在MME等通用数据集上维持强劲结果。

Insight: 创新点在于将视觉令牌与语言特征空间的距离作为分布外检测指标，并基于此进行迭代剪枝，这不仅提升了鲁棒性，还意外地降低了计算开销，为MLLMs的稳健性优化提供了新视角。

Abstract: In this paper, we show for the first time that visual token pruning enhances the robustness of Multimodal Large Language Models (MLLMs), mitigating vulnerabilities such as jailbreak attacks and hallucinations. Given that vision and language modalities cannot be perfectly aligned, the misaligned visual tokens might act as out-of-distribution (OOD) inputs, leading to unpredictable outputs and introducing potential vulnerabilities. Building on this insight, we aim to enhance model robustness against jailbreaks and hallucinations by reducing OOD visual tokens at robust-pruning layers, while also reducing inference cost as a side benefit. Specifically, we measure the distance between each visual token and the language feature space. Then, visual tokens with large distances are identified as OOD tokens, which can be iteratively pruned. To demonstrate the effectiveness of our method, we evaluate it on seven diverse popular benchmarks. Notably, our method yields an average improvement of 13.29% in defending jailbreak attacks, consistently achieves competitive performance in mitigating hallucinations, and maintains strong results on general datasets like MME.

[41] TOM-GS: Editable Video Representation via Temporal Opacity Modulation of Static 3D Gaussians cs.CVPDF

Marek Lisowski, Łukasz Smoliński, Kornel Howil, Piotr Biliński, Marcin Mazur

TL;DR: 本文提出了TOM-GS，一种可编辑的视频表示方法，它基于静态3D高斯模型，通过引入连续时间不透明度调制，使每个高斯模型能够平滑地淡入淡出场景，从而支持广泛的手动和基于物理的编辑。

Details

Motivation: 现有隐式神经表示和动态3D高斯泼溅方法在视频处理中效果显著，但难以实现便捷编辑；复杂空间变形或折叠分布限制了优化灵活性，因此需要一种更简单、可编辑的视频表示方案。

Result: TOM-GS在视觉保真度上优于先前的可编辑视频表示方法，并因其基于标准3D高斯模型，能够与现有3D编辑工具无缝兼容。

Insight: 创新点在于放弃了复杂变形，转而采用静态3D高斯模型结合连续时间不透明度调制（可学习的时间均值和尺度），这保持了静态空间几何结构，便于下游编辑，同时确保了高兼容性。

Abstract: While Implicit Neural Representations (INRs) and dynamic 3D Gaussian Splatting (3DGS) achieve impressive results in video processing, they often fall short of producing representations that are easily editable. Recent methods address this by introducing complex spatial deformations or folded distributions, which constrain optimization and reduce flexibility for downstream editing. In this paper, we introduce TOM-GS, an editable video representation that forgoes complex deformations in favor of regular 3D Gaussians equipped with a continuous temporal opacity formulation. By assigning a learnable temporal mean and scale to the opacity of each Gaussian, our model enables static 3D spatial components to fade smoothly in and out of the scene. Grounded by robust, off-the-shelf pose estimation, our approach maintains a static spatial geometry that naturally supports a wide range of manual and physics-based edits. TOM-GS outperforms prior editable video representations in visual fidelity, while its reliance on standard 3D Gaussians ensures seamless compatibility with established 3D editing tools.

[42] An Interactive Vision Language Platform for Cognitive Remediation in Schizophrenia cs.CV | cs.AIPDF

Nassira Ait Mehdi, Milissa Temmam, Slimane Larabi

TL;DR: 本文提出了一种基于视觉语言模型的自动化框架，用于精神分裂症认知康复任务中的动作验证。该系统通过摄像头监控桌面环境，患者根据音频指令操控玩具车辆完成空间动作，系统利用微调的视觉语言模型分析视频序列并生成语义描述，从而自动验证动作的正确性。

Details

Motivation: 针对精神分裂症患者的认知康复任务通常需要临床医生手动观察动作正确性，这存在主观性且难以扩展，因此需要一种自动化的客观评估方法。

Result: 在收集的4634个桌面认知康复视频场景数据集上，实验结果表明该框架能有效将低层物理遥测数据与高层临床反馈相结合。

Insight: 创新点在于将视觉语言模型微调并应用于认知康复的物理动作验证，实现了无需持续临床监督的自动化评估，为认知康复提供了可扩展的客观解决方案。

Abstract: Cognitive remediation tasks often require patients to perform structured actions involving object manipulation and sequential reasoning. For patients diagnosed with schizophrenia, these tasks are crucial for addressing severe cognitive deficits. However, evaluating the correctness of these physical actions generally relies on manual observation by clinicians, which introduces subjectivity and limits the scalability of therapeutic interventions. In this paper, we propose an automated framework based on Vision-Language Models for action verification in cognitive remediation tasks tailored for schizophrenia rehabilitation. The proposed system relies on a camera-monitored tabletop environment composed of structured miniature scenes including roads, a roundabout, a park, and toy vehicles. Patients receive audio instructions describing goal-oriented spatial actions to perform by manipulating a toy vehicle. These interactive physical activities are specifically designed to stimulate targeted cognitive functions, such as sustained attention, motor coordination, spatial navigation, and cognitive flexibility. To verify the correctness of the performed actions without requiring continuous clinical oversight, the system analyzes the video feed tracking the patient’s hand and toy movements. A fine-tuned Vision-Language Model interprets the recorded video sequences and generates semantic descriptions of the observed activities, enabling high-level verification of the executed actions with respect to the initial textual instructions. A dedicated dataset of 4634 tabletop cognitive remediation video scenarios was collected to evaluate the proposed approach. Experimental results demonstrate that our specialized framework effectively bridges low-level physical telemetry with high-level clinical feedback, presenting a scalable and objective solution for advanced cognitive rehabilitation.

[43] DAMamba-UNet3D: A Parameter-Efficient Mamba State Space U-Net with Dynamic Adaptive Scan for 3D Medical Image Segmentation cs.CV | cs.AIPDF

Mohammad Arafat Hussain, Ellen Grant, Yangming Ou

TL;DR: 本文提出了一种参数高效的3D医学图像分割模型DAMamba-UNet3D。该模型将动态自适应扫描（DAS）机制集成到基于状态空间模型（SSM）的U-Net架构中，旨在以更少的参数实现与现有方法相当或更优的分割性能。

Details

Motivation: 现有医学图像分割方法中，卷积U-Net缺乏全局上下文建模能力，而Transformer计算成本高。基于Mamba的状态空间模型（SSM）虽能实现O(n)的全局传播，但现有医学SSM分割器依赖固定扫描模式且参数量大。本文旨在设计一种参数高效、能学习数据依赖扫描模式的3D SSM分割架构。

Result: 在BraTS 2020数据集上的五折交叉验证中，DAMamba-UNet3D（约530万参数）的平均Dice系数达到0.815±0.013，参数量比SegMamba（约7000万参数，Dice 0.824±0.014）低约13倍。参数量相当的DAMamba-L（约7000万参数）达到0.829±0.012，超过了重新训练的SegMamba 0.5个百分点。

Insight: 主要创新点在于提出了动态自适应扫描（DAS）机制，并将其扩展到3D医学图像分割中，实现了数据依赖的序列重排序。模型设计上的关键洞察是：仅在编码器阶段集成DAS块至关重要，而在瓶颈层和解码器中使用SSM块会降低性能。这表明在混合U-Net中学习三平面DAS模式，可以媲美甚至超越SegMamba的固定三向扫描（ToM）。

Abstract: We propose parameter-efficient SSM-based U-Net architectures for 3D medical image segmentation. Convolutional U-Nets afford O(n) local mixing per layer but lack explicit global context; transformers provide global reasoning at O(n^2) cost in sequence length $n$. State-space models (SSMs), such as Mamba, offer $O(n)$ global propagation per block. Yet, existing medical SSM segmenters rely on fixed scan patterns and large parameter budgets. Dynamic Adaptive Scan (DAS), which learns data-dependent reordering before selective scan, has not been applied to medical imaging or extended to 3D volumes. We propose DAMamba-UNet3D, a hybrid encoder-decoder that integrates tri-plane 3D-DAS blocks at encoder stages E2-E4 while retaining convolutions elsewhere (5.3M parameters). On BraTS 2020 five-fold cross-validation, DAMamba-UNet3D achieves mean Dice 0.815+/-0.013 (full-volume per-case evaluation) at ~13x lower parameter cost than SegMamba (0.824+-0.014, ~70M). At comparable scale, DAMamba-L (70M), a wide DAS-native variant with encoder-only DAMamba and a convolutional bottleneck, reaches 0.829+-0.012, surpassing retrained SegMamba by 0.5pt. Component ablations show that encoder-only DAS placement is critical as bottleneck and decoder SSM blocks lower Dice. Together, the results suggest that learned tri-plane DAS in a hybrid U-Net is competitive with, and under our large-scale design may improve upon, SegMamba’s fixed Tri-orientated Mamba (ToM) scanning on BraTS 2020. Code: https://github.com/marafathussain/DAMamba-UNet3D.

[44] Visual Information Extraction from Documents via Classification-Guided Large Vision-Language Models cs.CVPDF

Huafu Li, Guo Chen, Jia Xia, Lei Wang, Wei Du

TL;DR: 本文提出了一种基于分类引导的大视觉语言模型框架，用于从视觉丰富的文档中提取多类型视觉信息。该方法将文档类型分类与内容提取解耦，并利用上下文学习的动态提示工程注入任务知识，实现了在多样化布局上的鲁棒零样本推理。

Details

Motivation: 现有方法依赖顺序OCR流程或需要大量标注数据和特定布局训练的端到端模型，难以应对高布局变异性和现实世界损伤，限制了可扩展性。

Result: 在一个包含16种证书类型的真实世界投标数据集上，基于Qwen2.5-VL-7B的零样本方法在F1分数上比强监督基线高出18.35个百分点（86.43% vs. 68.08%），归一化编辑距离提升0.23（0.90 vs. 0.67）。可选领域微调进一步将性能提升至93.65% F1和0.93 NED，对印章、水印和低对比度表现出优越鲁棒性。

Insight: 核心创新在于通过分类引导和解耦设计，将文档类型识别作为条件计算来降低任务不确定性，并结合上下文学习的动态提示工程，实现了以最小监督达到高精度和强泛化能力的高效、可扩展解决方案。

Abstract: Visual information extraction (VIE) from visually rich documents remains challenging due to high layout variability and real-world impairments. Existing methods typically rely on sequential OCR pipelines or end-to-end models requiring extensive labeled data and layout-specific training, limiting their scalability.We propose a classification-guided large vision-language model (LVLM) framework for multi-type VIE that achieves high accuracy with minimal supervision. The approach decouples document-type classification from content extraction and employs in-context learning (ICL)-based dynamic prompt engineering to inject task-specific knowledge, enabling robust zero-shot inference across diverse layouts. From a theoretical perspective, the proposed method can be viewed as a form of conditional computation that reduces task uncertainty and improves information efficiency during prompt-based inference. Evaluated on a real-world bidding dataset with 16 certificate types, our zero-shot method (based on Qwen2.5-VL-7B) outperforms a strong supervised baseline by 18.35 percentage points in F1-score (86.43% vs. 68.08%) and 0.23 in normalized edit distance (0.90 vs. 0.67). Optional domain-specific fine-tuning further improves performance to 93.65% F1 and 0.93 NED, demonstrating superior robustness against seals, watermarks, and low contrast. The framework offers an efficient, scalable solution for complex document understanding in office automation. Code is available at https://github.com/FairmeHIT/Multi-VIE, and fine-tuned models at https://huggingface.co/fairme/Qwen2.5-VL-7B-SFT.

[45] PCA: Persistence-Aware Compression and Aggregation for Fast Video Large Language Models cs.CVPDF

Zihan Song, Shuo Ye, Bo Zhao, Ruixin Zhang, Jiayu Zhang

TL;DR: 本文提出了一种无需训练的持久感知压缩与聚合（PCA）方法，旨在提升视频大语言模型（VLLM）的效率。该方法通过动态下采样模块自适应地去除冗余视频帧，并通过持久感知运动增强模块聚合关键帧的时序上下文信息，从而在显著减少长视频建模计算量的同时，提升模型的性能。

Details

Motivation: 当前视频大语言模型在视频理解方面展现出潜力，但长视频帧序列中存在大量冗余信息，这阻碍了模型进行高效推理。本文旨在解决这一效率瓶颈，在编码阶段前保留高保真的原始视觉信息。

Result: 大量实验表明，PCA方法在效率和准确性上均持续优于现有的最先进方法，与基线VLLM相比，实现了1.8倍到2.5倍的加速。

Insight: 创新点在于提出了一种无需训练、可即插即用于任意VLLM的框架，其核心是动态下采样与持久感知运动增强两个模块的协同，在激进地减少帧数的同时，通过聚合邻帧的时序上下文来确保关键信息不丢失，实现了效率与精度的双重提升。

Abstract: Despite advances in Video Large Language Models (VLLMs) that have displayed promising outcomes in video understanding, the redundancy in the long-duration frames remains a hindrance to efficient reasoning. This paper introduces a training-free $\mathbf{P}$ersistence-Aware $\mathbf{C}$ompression and $\mathbf{A}$ggregation (PCA) method designed to preserve high-fidelity raw visual information before the encoding stage. PCA can be built on arbitrary VLLMs and consists of two modules: 1) A Dynamic Downsampling (DD) module that adaptively removes redundant frames by analyzing frame-wise similarity. 2) A Persistence-Aware Motion Enhancement (PAME) module that enriches each selected keyframe by aggregating the temporal context of its neighbors, ensuring that essential information is preserved even after aggressive frame reduction. Our approach substantially reduces the computation of long-context modeling, while enhancing the performance of the baseline model. Extensive experiments demonstrate that PCA consistently outperforms existing state-of-the-art approaches in both efficiency and accuracy, achieving a speedup of 1.8$\times$ to 2.5$\times$ compared to the baseline VLLM. The code is open-sourced at https://github.com/Heisenberg10110/PCA.

[46] Open Your Model’s Eyes: Video and Context-Aware Multimodal Backchannel Prediction cs.CV | cs.CLPDF

Min-Jae Kim, Jun-Yeong Moon, Mujeen Sung, Gyeong-Moon Park

TL;DR: 本文提出了CAMA-BC框架，通过引入视频信息和上下文感知的多层多模态对齐技术，显著提升了反馈信号预测的准确性。

Details

Motivation: 现有方法仅依赖音频和文本，忽略了面部表情、手势等关键视觉线索以及更广泛的对话上下文，导致预测不准确。

Result: 实验结果表明，CAMA-BC在多个基准测试中显著优于现有方法和简单的多模态基线，尤其在识别共情等复杂反馈信号方面表现突出。

Insight: 创新点在于通过两阶段的多层多模态对齐（上下文对齐和反馈信号对齐）有效融合视觉与上下文信息，为多模态对话理解提供了新思路。

Abstract: Backchannels, which signal listener states like empathy and understanding, are fundamental to natural human interaction. However, current approaches rely solely on audio and text. This omits crucial visual cues, such as facial expressions and gestures, as well as broader conversational contexts, which are necessary for accurate prediction. In this paper, we introduce Context-Aware Multimodal Alignment for Backchannel Prediction (CAMA-BC), a novel framework that leverages visual information through Multi-Layer Multimodal Alignment (MMA). Our alignment process comprises two stages. First, Context Alignment (MMA-CA) utilizes unlabeled dialogues with videos to capture conversational contexts. Next, Backchannel Alignment (MMA-BA) fine-tunes the representations specifically for backchannel prediction. Experimental results show that CAMA-BC significantly outperforms both existing methods and simple multimodal baselines, with particular effectiveness in recognizing complex backchannels such as empathy.

[47] Trustworthy Medical Segmentation: Uncertainty-Aware U-Net Evaluation Under Clinical Image Degradation cs.CV | cs.LGPDF

Pranav Kaliaperumal, Manisha Kaliaperumal

TL;DR: 本文提出一个可重复的评估框架，用于在受控临床图像退化条件下评估不确定性感知的医学图像分割模型。实验基于遵循BraTS协议的多模态脑肿瘤MRI合成数据集，训练U-Net和Attention U-Net基线模型，并采用蒙特卡洛Dropout估计体素级不确定性。在八种临床退化类型和五个严重程度下，系统评估了分割准确性、校准性、故障检测和选择性预测覆盖性能。

Details

Motivation: 医学图像分割模型在理想成像条件下通常报告高精度，但在临床退化（如传感器噪声、患者运动、低分辨率采集和对比度变化）下可能无声地失败，缺乏明显警告。本文旨在评估不确定性感知分割在临床退化下的可靠性，为放射科工作流程提供安全层。

Result: 在干净数据上，Attention U-Net的全肿瘤Dice得分达到0.990；在严重高斯噪声下，性能降至0.089。预测不确定性随退化增加，并与分割误差相关（在严重程度3的高斯噪声下Pearson r=0.53），故障检测AUROC达到0.843。

Insight: 创新点在于构建了一个系统性的临床退化评估框架，将不确定性估计（蒙特卡洛Dropout）与分割性能在多样化退化场景下关联分析，证明了不确定性感知推理可作为放射科工作流程中的实用安全机制。客观来看，该方法为模型在真实临床环境中的可靠性评估提供了标准化方案。

Abstract: Medical image segmentation models often report high benchmark accuracy under ideal imaging conditions, yet their failures under clinical degradation can be quiet: sensor noise, patient motion, low- resolution acquisition, and contrast variability may all alter model behavior without producing an obvious warning. We present a reproducible framework for evaluating uncertainty-aware segmentation under con- trolled clinical degradation. Our experiments use a synthetic multimodal brain tumor MRI cohort generated with a biophysical phantom simulator that follows the BraTS protocol. We train U-Net and Attention U-Net baselines for multi-class tumor sub-region segmentation and augment both models with Monte Carlo dropout to estimate per-voxel uncertainty. Across eight clinically motivated corruption types at five severity levels, we measure segmentation accuracy, calibration, failure detection, and selective prediction coverage. On clean data, Attention U-Net achieves a whole-tumor Dice of 0.990; under severe Gaussian noise, its performance falls to 0.089. Predictive uncertainty rises with degradation and tracks segmentation error (Pearson r = 0.53 under severity-3 Gaussian noise), allowing us to flag failures with an AUROC of 0.843. These results argue for uncertainty-aware inference as a practical safety layer in physician-in-the-loop radiology workflows. We release the code, trained models, and evaluation protocol to support direct reproduction.

[48] Calibration-Free 3D Multi-Camera People Tracking for Indoor Environment cs.CVPDF

Ponleur Veng, Dominique Vaufreydaz, Phutphalla Kong

TL;DR: 本文提出了一种无需标定的三维多摄像头行人跟踪框架，通过深度基础模型直接从视觉数据中推断几何结构，集成了无锚检测、鲁棒跟踪、全尺度外观嵌入、姿态估计和基于Transformer的几何重建，并在AI City Challenge 2024上验证了其有效性。

Details

Motivation: 传统多摄像头行人跟踪依赖精确的相机标定，这在大规模无约束视频数据集生成中成为瓶颈，因此本文旨在开发一种无需标定的三维跟踪方法，以降低数据收集和处理的复杂性。

Result: 在AI City Challenge 2024上，该方法在不使用真实标定矩阵的情况下，实现了53.13%的HOTA分数，为纯视觉三维跟踪建立了强基线。

Insight: 创新点包括利用视觉几何基础Transformer进行几何重建，以及通过姿态引导的三维提升策略将头部关键点投影到重建流形上，从而摆脱对地面单应性的依赖；同时，全局身份关联采用分层聚合聚类结合外观-几何联合成本与严格速度门控，提升了跟踪鲁棒性。

Abstract: Multi-Camera People Tracking (MCPT) traditionally relies on precise intrinsic and extrinsic camera calibration to project 2D detections into a unified 3D world coordinate system.However, manual calibration constitutes a major bottleneck in large-scale dataset generation from unconstrained video archives. This work proposes a unified calibration-free 3D MCPT framework that infers geometric structure directly from visual data using deep foundation models. The system integrates anchor-free detection (YOLOX), robust tracking (BoT-SORT), omni-scale appearance embedding (OsNet), pose estimation (HRNet via MMPose), and transformer-based geometric reconstruction using the Visual Geometry Grounded Transformer (VGGT). A pose-guided 3D lifting strategy projects head keypoints onto a reconstructed manifold, eliminating dependence on ground-plane homography. Global identity association is formulated as hierarchical agglomerative clustering under a joint appearance-geometry cost with strict velocity gating. Evaluation on the AI City Challenge 2024 demonstrates a HOTA score of 53.13% without access to ground-truth calibration matrices, establishing a strong baseline for purely vision-based 3D tracking.

[49] Benchmarking the Domain Gap: Model Selection Instability Under Domain Shift in Video Capsule Endoscopy cs.CVPDF

Dan Hanson, Debesh Jha

TL;DR: 本文研究了视频胶囊内窥镜（VCE）分类模型在跨域部署时的选择不稳定性问题。通过在Kvasir-Capsule数据集上微调预训练骨干网络，并在Galar和Capsule Vision 2024（CV2024）两个目标域上评估，发现模型在源域的性能排名无法稳定预测其在其他目标域的表现。

Details

Motivation: 解决VCE分类模型在临床部署中，因数据采集源、标注策略和患者群体差异导致的跨域鲁棒性问题，揭示单一数据集评估的局限性。

Result: 实验表明，在Kvasir-Capsule上表现最佳的骨干网络，在Galar目标域上仍领先，但在CV2024目标域上性能降至中等水平；两个非源目标域之间的模型排名一致性较弱。

Insight: 创新点在于系统量化了VCE领域的域间差距，并证明模型选择应基于跨目标域的排名稳定性报告，而非单一数据集的峰值性能，这为医疗AI模型的鲁棒性评估提供了新视角。

Abstract: Video capsule endoscopy (VCE) classification is typically evaluated within a single dataset, yet clinical deployment demands robustness across acquisition sources, labeling policies, and patient populations. We examine this gap using Kvasir-Capsule, Capsule Vision 2024 (CV2024), and a shared-label subset of Galar. We fine-tune a suite of general-domain pretrained backbones on the official Kvasir-Capsule folds under a standardized protocol and evaluate the same checkpoints on two non-source targets within a documented shared-label decision space. We find that the predictive value of in-domain ranking is target-dependent: Kvasir-Capsule ranking aligns more closely with Galar than with CV2024, while the two non-source targets agree only weakly. Consequently, the strongest in-domain backbone leads on one target yet falls to mid-pack on the other, and no single evaluation target reliably predicts the others. A second CV2024-trained configuration set reproduces this target-dependent instability. We conclude that capsule endoscopy model selection should report cross-target ranking stability rather than peak single-dataset performance.

[50] Fast Fourier Convolutional GAN for 30 m Clear-Sky Land Surface Temperature Gap-Free Reconstruction cs.CV | eess.IV | physics.geo-phPDF

Marwa Alfouly, Smajil Halilovic, Nils Bochow, Thomas Hamacher, Niklas Boers

TL;DR: 本文提出了一种多模态快速傅里叶卷积生成对抗网络（Multimodal Fast Fourier Convolutional GAN），用于重建30米高分辨率Landsat影像中受云污染的像素，以生成无缝隙的晴空地表温度产品。该方法利用快速傅里叶卷积实现图像的全局感受野，并借助卫星观测和合成孔径雷达数据作为辅助信息进行引导。

Details

Motivation: 卫星反演的地表温度数据因云层遮挡存在严重的数据缺失，限制了其在理解陆气相互作用中的应用。现有方法难以构建一个可扩展且适应性强的流程来生成无缝隙LST数据集并重建受污染像素，尤其是在高空间分辨率观测中重建大面积缺失区域极具挑战。

Result: 在所有LST分位数上，场景平均RMSE（在重建像素上计算）的四分位距始终在0.8 K到1.8 K之间。该方法能够恢复大面积缺失区域，包括云层导致缺失超过70%的场景，且依赖的辅助数据在近全球尺度上易于获取。

Insight: 创新点在于结合了快速傅里叶卷积以实现全局上下文建模，并融合多模态数据（光学与SAR）进行引导重建。从客观角度看，该方法为高分辨率、大范围云隙填补提供了一种有效的端到端解决方案，其全局感受野设计对处理不规则大面积缺失具有优势。

Abstract: Satellite-derived Land Surface Temperature (LST) provides spatially comprehensive data that ground stations cannot match. However, its utility is frequently limited by severe data gaps due to the presence of clouds. As LST is essential for understanding land-atmosphere interactions, numerous methods have been proposed to address this challenge. Yet, the development of a scalable and adaptable pipeline for generating gap-free LST datasets and reconstructing cloud-contaminated pixels remains challenging. Moreover, the reconstruction of extensive missing regions in fine-spatial-resolution observations is particularly difficult. To address this challenge, we propose a Multimodal Fast Fourier Convolutional GAN for reconstructing cloud-contaminated pixels in fine-resolution (30 m) Landsat imagery to generate gap-free clear-sky LST products. The method leverages Fast Fourier Convolution to enable a global receptive field across the image, and is guided by a stack of data consisting of satellite observations and Synthetic Aperture Radar (SAR) data. Across all LST quantiles, the interquartile range of scene-averaged RMSE (computed over reconstructed pixels) is consistently between 0.8 K and 1.8 K. The proposed approach enables the recovery of extensive missing regions, including scenes with more than 70% cloud-induced gaps, while relying on auxiliary data that are readily available at a near-global scale.

[51] Cortex: Compact Behavior Cloning for Quake with Frozen Visual Features cs.CV | cs.AI | cs.LGPDF

Dzmitry Malyshau

TL;DR: 论文提出Cortex，一个在视觉丰富的第一人称游戏《雷神之锤》中仅使用行为克隆的紧凑策略。该策略基于冻结的DINOv3视觉编码器和一个六层Transformer，仅有1098万个可训练参数，并在公开的Pixels2Play数据集上进行训练。评估显示，该策略虽未通关，但能稳定到达特定场景并取得击杀，其性能在时间控制的测试中优于某些已发布的基准模型。

Details

Motivation: 研究在视觉丰富的第一人称游戏中，一个经过精心设计的、简单（仅使用行为克隆，不引入强化学习或显式记忆）的策略能取得多大进展，以探索行为克隆的潜力与局限。

Result: 在《雷神之锤》E1M1关卡的两批各20次随机、120秒的片段测试中，Cortex策略未能通关，但每次都能到达开门、按钮房和闸门下降处，且每批中19次测试至少有一次击杀。在相同时间控制的测试框架下，其表现优于已发布的P2P-150M和NitroGen模型检查点，但比较受限于小样本和不同原生接口。

Insight: 主要创新在于构建了一个参数紧凑、基于冻结视觉特征（DINOv3）的纯行为克隆策略，证明了在复杂游戏环境中简单方法的有效性。消融实验表明，更密集的视觉标记能改善战斗和生存，而更长的优化和简单的动作历史虽提升离线指标但未必改善实际游戏表现，这揭示了行为克隆的局限性（如协变量偏移），并为未来针对性数据校正提供了方向。

Abstract: We study how far a deliberately simple behavioral-cloning policy can progress in a visually rich first-person game before adding reinforcement learning or explicit memory. Cortex is a compact Quake policy with 10.98 million trainable parameters in a six-layer transformer over a frozen DINOv3 encoder. It is trained on the Quake subset of the public Pixels2Play corpus: 6,849 recordings (about 474.7 hours), represented as 17.09 million cached decision frames with keyboard and mouse actions. One sampled training epoch uses 517,048 four-frame windows and takes 3.3 minutes of policy-head optimization on one RTX 5080, excluding one-time feature extraction. We evaluate two independent batches of 20 stochastic, 120-second episodes on Quake E1M1. Cortex does not complete the level, but every episode reaches the opening door, button room, and gate descent; 19 of 20 episodes in each batch record at least one kill. Under the same time-controlled harness, released P2P-150M and NitroGen checkpoints remain shallower in five matched-duration episodes each. These comparisons are limited by small reference samples and different native interfaces. Ablations show that denser visual tokens improve combat and survival, while longer optimization and naive action history improve offline metrics without consistently improving play. The remaining failures are consistent with covariate shift and motivate targeted corrective data. We release the policy implementation, checkpoint, and a representative rollout.

[52] Cheap Probes Predict Expensive Training in 3D-CT Vision–Language Models cs.CV | cs.AIPDF

Renjie Liang

TL;DR: 本文提出了一种廉价探针方法，用于预测3D-CT视觉-语言模型中昂贵训练的性能。通过构建基于编码器嵌入的探测基准，结合尺度合理性和探针可分离性验证门，该方法能以高相关性（r≈0.95）快速筛选图像编码器和令牌压缩方案，大幅降低计算成本。

Details

Motivation: 解决在3D-CT视觉-语言模型中选择图像编码器和令牌压缩方案时，因组合爆炸导致全量微调计算成本过高的问题，探索能否通过廉价探针替代昂贵训练进行候选方案评估。

Result: 在构建的临床属性基准上，廉价探针排序与昂贵微调结果高度一致（相关性约0.95），初步验证了其作为排名预测器的有效性，但研究仍处于初步阶段。

Insight: 创新点在于引入尺度合理性和探针可分离性双验证门控机制，确保属性可解码性；通过冻结嵌入的探针实现快速候选筛选，为多模态模型架构搜索提供了高效评估范式。

Abstract: Picking the frozen image encoder for a 3D~CT vision–language model (VLM), together with the token-compression scheme on top of it, is a search over many candidates. There are several encoders, several ways to compress their tokens, and several token budgets, and the combinations grow fast. Comparing them the usual way means fine-tuning a large language model (LLM) on each combination, and running the whole sweep this way needs far more compute than most groups can spend. We ask whether a cheap probe on the encoder’s cached embeddings can stand in for that comparison. We build an image-grounded probing benchmark over (encoder $\times$ compression) cells, with clinical attribute families and two validation gates, scale-sanity and probe-separability, that keep each attribute well-scaled and decodable. These gates are the main methodological contribution. On this benchmark we compare a range of read-out heads, and in a preliminary study we pair each probe with its matched downstream task. The early signal is encouraging: the cheap probe orders the candidates in close agreement with expensive fine-tuning, at about $r\approx0.95$ on the cells measured so far. We read this as an ordinal claim, a ranking predictor rather than an exact estimate, and we are explicit about where it stays preliminary. If it holds up, encoder and compression choices can be screened in minutes with frozen-token probes, with full training spent only on the finalists.

[53] AI-generated Images Challenge Visual Trust in High-risk Scenarios cs.CV | cs.AIPDF

Yi-Zhi Wang, Yichen Xiao, Linan Yue, Weibo Gao, Yichao Du

TL;DR: 本文介绍了SafeIMG，一个专注于公共和个人安全场景的AI生成图像检测基准，包含12种高风险情境。研究发现，现有的专用检测器和视觉语言模型在检测生成图像时准确率远低于人类，且其解释与人类标注的异常区域对齐度低，尤其在常识冲突和物理不一致性方面表现更差。

Details

Motivation: 现有检测基准很少关注公共和个人安全等高风险场景，而误导性视觉内容在这些情境下可能带来重大风险，因此需要专门评估检测器在这些场景下的可靠性。

Result: 在SafeIMG基准上，最强的视觉语言模型仅识别出49.5%的生成图像，最佳专用检测器识别率为33.1%，而人类评估者准确率达到81.7%。模型解释仅覆盖29.8%的人类标注异常，对常识冲突和物理不一致性的覆盖率分别降至15.0%和12.0%。

Insight: 论文的创新点在于构建了首个面向安全场景的AI生成图像检测基准SafeIMG，它不仅评估检测准确率，还通过人类标注评估模型解释与人类识别异常的对齐度，揭示了当前检测器在准确性、解释对齐性和鲁棒性上的严重不足。

Abstract: Rapid advances in image generation are eroding the evidentiary value of visual content in settings where authenticity can affect public safety and personal reputation. Yet existing detection benchmarks rarely examine synthetic images in public- and individual-safety contexts, where misleading visual content may carry substantial risks. Here we introduce SafeIMG, a safety-oriented benchmark spanning 12 public- and individual-safety scenarios generated using GPT Image 2. Unlike benchmarks centred on generic imagery and image-level labels, SafeIMG evaluates not only whether detectors recognise synthetic images, but also whether their decisions reflect human-identified anomalies. To this end, SafeIMG provides human annotations that localise suspicious regions and explain local artefacts and higher-level commonsense or physical inconsistencies. We evaluate specialized synthetic-image detectors and vision-language models (VLMs), and find that neither provides reliable detection. The strongest VLM identifies only 49.5% of generated images, whereas the best specialised detector identifies 33.1%, compared with 81.7% accuracy for human evaluators. Model explanations cover only 29.8% of human-annotated anomalies and predominantly capture local defects in text, faces and hands. Their coverage falls to 15.0% for commonsense conflicts and 12.0% for physical inconsistencies, while detection performance deteriorates further after dissemination-induced image degradation. These findings show that current detectors lack the accuracy, explanatory alignment and robustness needed to evaluate AI-generated images reliably across public- and individual-safety settings.

[54] ID-V2V: Identity-Preserving Video Restylization cs.CVPDF

Yuancheng Xu, Mingming He, Pablo Salamanca, Li Ma, Yash Kant

TL;DR: 本文提出ID-V2V，一个用于身份保持的视频重风格化框架。该框架将问题解耦为基于源视频的身份保持（视为视频重光照问题）和基于编辑关键帧的视觉编辑传播（视为可控视频合成），从而无需成对的训练数据。

Details

Motivation: 在视觉叙事中，保持人物身份和表演的同时进行灵活的视觉编辑是生成式视频模型面临的挑战。现有方法缺乏真实世界中的成对训练数据。

Result: 大量实验表明，ID-V2V在保持面部相似性和细粒度面部表演（如表情、眼神、唇部同步）方面显著优于现有方法，支持单人和多主体场景，并实现了高视觉质量。

Insight: 核心创新在于将身份保持问题解耦并重新表述为视频重光照问题，同时利用重光照面部区域和面部法线图作为控制信号来严格约束身份与表演，从而能够从单段视频构建训练对，解决了数据稀缺问题。

Abstract: In visual storytelling, human performances are central to creative intent and narrative meaning. However, preserving human identity and performance while enabling flexible visual edits remains challenging for generative video models. We formalize this challenge as identity-preserving video restylization, which propagates scene, lighting, and style changes specified by an edited keyframe across a source video, while preserving facial likeness and performance, including expressions, eye gaze, and lip synchronization. A key obstacle is the absence of paired training data, as identity-preserving restylized video pairs are rare in real-world settings. To address this, we propose a decoupling of source-grounded identity preservation and edit-driven video synthesis. Our key insight is that facial appearance and expression should remain invariant, with illumination being the primary permissible variation. We therefore cast identity preservation as a video relighting problem, while modeling visual edit propagation as controlled video synthesis guided by the edited keyframe. Building on this formulation, we introduce ID-V2V, a video-to-video generative framework integrating complementary control signals: relit facial regions and facial normal maps tightly constrain facial likeness and performance, while edited keyframes and depth sequences enable flexible and temporally coherent generation. This design enables constructing training pairs from a single video, eliminating the need for scarce paired data. Extensive experiments demonstrate that ID-V2V significantly outperforms existing methods in preserving facial likeness and fine-grained facial performance, supports both single- and multi-subject scenarios, and delivers high visual quality, highlighting its potential as a human-centric tool for real-world content production. The code is available at: https://github.com/Eyeline-Labs/ID-V2V.

Yuqi Hou, Zhuo Chen, Han Hu, Je Woo Kim, Jianbo Jiao

TL;DR: 本文提出了一种名为ANCHOR的目标中心范式，旨在通过联合建模视觉注意力和潜在隐性关系的分布，从静态图像中解码以凝视（gaze）为锚点的社交意图。该方法利用关系注意力机制捕捉细粒度的人际联系，并通过优化协同解决空间凝视精度与潜在社交推理之间的冲突，在扩展的密集多人标注基准上实现了最先进的性能。

Details

Motivation: 现有模型通常独立处理个体，将凝视视为独立同分布量或孤立预测社交语义，未能捕捉作为凝视行为底层驱动力的、微妙的社交意图。近期多人方法虽尝试解决，但常将社交关系视为与凝视估计过程解耦的、僵化的后验分类，这种简化无法反映社交意图的细微本质。

Result: 在扩展的、包含密集多人标注和新型社交影响力排名的基准上进行验证，结果表明该方法实现了最先进的性能，并首次提供了定量证据，证明隐性社交层级可以直接从静态凝视模式中被稳健地解耦和学习。

Insight: 创新点在于提出了一个联合建模视觉注意力和潜在隐性关系的目标中心范式，将社交意图视为凝视行为的潜在结构支架，而非次要分类输出。通过关系注意力机制和特征调制实现高效多人解析，并采用优化协同策略来稳定训练，调和任务梯度冲突，寻求平坦最小值以实现鲁棒泛化。

Abstract: Human gaze does more than point to visual targets; it serves as a subtle indicator of social intent within static images, whereas standard models typically process individuals independently, treating gaze as an i.i.d. quantity or predicting social semantics in isolation. Recent multi-person methods attempt to address this but often treat social relations as rigid, post-hoc classifications decoupled from the gaze estimation process. This oversimplification fails to capture the nuanced nature of social intent, which acts as an underlying driver of gaze behavior rather than a secondary categorical output. We address these limitations by proposing ANCHOR, a target-centric paradigm designed to decode gaze-anchored social intent by modeling the joint distribution of visual attention and latent implicit relations. Our approach surfaces these dependencies as the latent structural scaffolding of gaze behavior. The architecture utilizes a relational attention mechanism to capture fine-grained interpersonal links, leveraging feature-wise modulation for efficient multi-person parsing from a single vision backbone. To stabilize the training of this coupled formulation, we implement an optimization synergy to resolve the inherent conflicts between spatial gaze accuracy and latent social reasoning. This approach ensures robust generalization by seeking stable, flat minima while simultaneously harmonizing competing task gradients. We validate our framework on an extended benchmark featuring dense multi-person annotations and novel social influence rankings. Our results demonstrate state-of-the-art performance and provide the first quantitative evidence that implicit social hierarchies can be robustly disentangled and learned directly from static gaze patterns.

[56] Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests cs.CV | cs.AIPDF

Patrick Rim, Tom Long, Ekta Prashnani, Ruth Rosenholtz, Ben Boudaoud

TL;DR: 本文提出了Spatial-IQ，一个用于诊断多模态大语言模型空间推理能力的层次化框架。它将堆叠3D结构中的物体计数任务分解为9个基于人类空间认知发展阶段的感知与认知子任务，并利用程序生成的数据集进行评估。研究发现，即使模型在目标任务上表现良好，也常常无法完成支撑它的底层子任务，揭示了模型可能采取的捷径行为。通过使用基于这些子任务的思维链监督和强化学习进行训练，可以显著提升模型的空间一致性和目标任务准确率。

Details

Motivation: 现有基准将多模态大语言模型视为黑箱进行评估，难以诊断其在空间推理任务上表现不佳的根本原因（是感知问题还是认知问题）。因此，需要一种能够解构空间智能、识别具体能力缺陷的诊断框架。

Result: 在程序生成的约80,000个堆叠3D结构数据集上评估了模型，并与人类基线进行了比较。结果表明，表现最好的模型常在目标任务（物体计数）上成功，却在支撑它的底层子任务上失败，且不同模型保留的层次化推理链完整度不同。使用该框架的思维链监督和强化学习进行训练，显著提升了子任务间的一致性和目标任务准确率。

Insight: 创新点在于提出了一个基于人类认知发展阶段的、层次化的空间能力诊断框架，能够精细地解构模型失败的原因。客观来看，该框架不仅是一个诊断工具，其分解出的子任务还能作为有效的训练信号，通过思维链监督和可验证奖励的强化学习来提升模型的推理一致性，这为理解和改进模型的内部推理过程提供了新思路。

Abstract: Multimodal large language models (MLLMs) excel at visual interpretation but fail on spatial reasoning tasks that humans solve reliably. Existing benchmarks evaluate these models as black boxes, limiting their ability to identify the underlying causes of lower performance: when a model fails a spatial reasoning task, it remains difficult to ascertain whether the hurdle is perceptual, such as recognizing object boundaries, or cognitive, such as reasoning about occlusion to infer hidden geometry. We introduce Spatial-IQ, a hierarchical diagnostic framework that decomposes object counting in stacked 3D structures into 9 perceptual and cognitive sub-tasks organized by the developmental stages of human spatial cognition, with mental rotation as an additional target probe. Using NVIDIA Isaac Sim, we procedurally generated a diverse dataset of roughly 80,000 stacked 3D structures with per-task ground truth. We evaluate models across three output formats (free-response text, multiple-choice images, and image editing) alongside a human baseline. The Spatial-IQ framework shows that top-performing models often succeed at the target task (object counting) without succeeding on the lower-level sub-tasks intended to support it, and that models differ in how much of these hierarchical chains they preserve, often revealing shortcut behavior that raw target-task accuracy alone would obscure. Finally, we demonstrate that training models with chain-of-thought (CoT) supervision over our hierarchical sub-tasks, combined with reinforcement learning with verifiable rewards, significantly improves both spatial consistency across sub-tasks and target-task accuracy, supporting the value of the proposed decomposition as both a diagnostic tool and a training signal.

[57] Controlling Embedding Spaces with Text-Conditioned Transformations cs.CV | cs.AIPDF

Joseph Fioresi, Fabian Caba Heilbron, Pankaj Nathani, Mubarak Shah, Kushal Kafle

TL;DR: 本文提出了一种文本条件化的视觉嵌入变换方法，旨在增强多模态嵌入空间（如CLIP）中次要属性的可访问性。通过自然语言描述属性类别（如“颜色”或“艺术风格”），网络生成仿射变换以强调指定属性，实现无需重新编码的嵌入空间控制。该方法在基于属性的检索和多属性组织任务上达到了最先进的性能，且推理成本接近零。

Details

Motivation: 现有模型（如CLIP）的多模态嵌入空间将高层语义压缩为单一向量，但主要表达主导语义（如主体对象），而抑制了其他重要属性（如拍摄角度或色调）。本文旨在通过文本条件化变换使这些属性显式可访问，解决嵌入空间中属性表达不平衡的问题。

Result: 该方法在基于属性的检索和多属性组织任务上实现了最先进的性能，具体在标准基准测试中表现出色，且推理成本极低（接近零）。

Insight: 创新点在于通过文本条件化的仿射变换直接操作潜在空间，统一且高效地控制嵌入空间，无需重新编码即可实现属性解耦和多聚类任务，提供直观的接口同时学习多个属性。

Abstract: Multimodal embedding spaces in models like CLIP enable powerful capabilities such as semantic similarity retrieval and cross-modal zero-shot classification. These embeddings compress high-level semantics into a single vector, which comes at the cost of primarily expressing a dominant semantics like main object while suppressing other important attributes such as camera angle or color tone. We propose a text-conditioned transformation of visual embeddings that makes such attributes explicitly accessible. Given a natural language description of an attribute category (e.g., “color” or “art style”), a network generates an affine transformation that emphasizes the specified attribute. Conditioning on text enables it to learn many attributes simultaneously, accessing them at inference time through an intuitive interface. The network is trained to align transformed embeddings with the frozen latent space, enabling retrieval using existing large-scale embeddings without any re-encoding. When applied to a full set, the same mechanism transforms the latent space for attribute disentanglement tasks such as multi-clustering. By operating directly in latent space, our method provides a unified and efficient framework for controlling embedding spaces, demonstrating state-of-the-art performance across both attribute-based retrieval and multi-attribute organization tasks with near-zero inference cost. Project page: https://joefioresi718.github.io/ControlEmbed_webpage/

[58] Small-Pollinator Detection in Cluttered Field Video cs.CVPDF

Onur Onal, Chen Chen

TL;DR: 本文针对野外视频中微小传粉昆虫检测的挑战，通过系统实证研究比较了YOLO和RF-DETR模型在不同输入分辨率下的性能，并评估了切片推理、类别门控融合、尺寸路由集成和事后时序处理等方法。研究发现，采用RF-DETR Large模型并提高输入分辨率至1344像素取得了最佳检测效果，在BuzzSpot数据集上达到0.405 mAP50:95，优于其他基线模型。

Details

Motivation: 解决在复杂野外视频中检测微小、视觉相似且常被植被遮挡的传粉昆虫的难题，旨在在单GPU计算预算下实现高效准确的检测。

Result: 在BuzzSpot挑战数据集上，RF-DETR Large模型在1344像素分辨率下达到0.405 mAP50:95，超越了1120像素模型（0.379）和最佳单模型YOLO26m基线（0.366），实现了当前最佳性能。

Insight: 创新点在于系统评估了多种检测策略，发现检测器选择（RF-DETR）和输入分辨率提升对微小物体检测效果最为关键；同时指出蜜蜂与食蚜蝇的区分是主要瓶颈，未来需在最终分类决策前引入特征级时序聚合学习。

Abstract: Detecting pollinators in field video is challenging: targets are small, visually similar, and observed against cluttered vegetation under blur and occlusion. We present a systematic empirical study of small-pollinator detection under a practical single-GPU compute budget. Using the BuzzSpot challenge dataset, we compare YOLO and RF-DETR models across input resolutions and evaluate sliced inference, class-gated fusion, size-routed ensembling, and post-hoc temporal processing. RF-DETR Large at 1344-pixel resolution achieved our best hidden-test result, reaching 0.405 mAP50:95 and outperforming the 1120-pixel model (0.379) and the best single-model YOLO26m baseline (0.366). The strongest gains came from adopting RF-DETR and increasing its input resolution, indicating that detector choice and input resolution were more effective levers than added inference-time complexity; the resolution gain was strongest for small objects and the rarer bumblebee and moth classes. Sliced-inference fusion, size-routed ensembling, and warm-started 1536-pixel continuation did not surpass this result, while post-hoc temporal processing did not improve the leaked diagnostic evaluation. Error analysis identified bee-hoverfly discrimination as the clearest remaining bottleneck: neighboring frames rarely supplied correctly classified hoverfly evidence for post-hoc correction. These findings motivate learned feature-level temporal aggregation before the final classification decision.

[59] HALLELUAI: A Hallucination-Aware AI System for Ultra-Realistic Image-to-Video Generation at Scale cs.CV | cs.AIPDF

Aniket Sakpal, Yang Jiang, Rouzbeh Davoudi, Shayan Hassantabar, Mani Najmabadi

TL;DR: 论文提出了HALLELUAI系统，这是一个用于大规模、超逼真图像到视频生成的端到端系统。该系统集成了视频审核模块和智能再生模块，通过评估美学、运动保真度和幻觉风险，并迭代修复问题，以生成符合专家级创意标准的高质量视频。

Details

Motivation: 解决AI生成视频在规模化生产中，自动化、高精度质量控制不足的瓶颈，确保输出视频满足超逼真、品牌安全和输入图像保真度的要求。

Result: 在创意专家参与的人类在环评估中，HALLELUAI表现出强对齐性，并能可靠地输出适用于产品和营销场景的超逼真、生产级视频。

Insight: 创新点在于将细粒度、可机器执行的审核反馈与智能再生代理相结合，通过迭代修复策略（如提示词优化、相机调整）来强制执行视觉真实性和输入保真度，从而推动可信赖AI生成视频的规模化应用。

Abstract: AI-generated video is increasingly used across marketing, product storytelling, and creative workflows, yet automated; high-precision quality control remains a major constraint to scaling production. We present HALLELUAI, an end-to-end system that moderates and regenerates image-to-video outputs to meet expert-level creative standards and deliver ultra-realistic videos with consistent end-user quality of experience (QoE) at scale. The system integrates a video moderation module that evaluates frame-level aesthetics, temporal motion fidelity, and fine-grained hallucination risks relative to the source image, with an agentic regeneration module that iteratively fixes failures through prompt refinement, controlled camera adjustments, targeted model or image switching, and structured retry strategies. The moderation logic is aligned with domain-specific creative guidelines and produces granular, machine-actionable feedback that directly drives regeneration. In human-in-the-loop evaluations with creative experts, HALLELUAI shows strong alignment and reliably outputs ultra-realistic, production-grade videos suitable for product and marketing placements at scale. This framework advances trustworthy AI generated video content by enforcing visual realism, brand safety, and strict input-image fidelity while enabling image-to-video generation at scale.

[60] Layering Virtual Try-On cs.CVPDF

Chun Feng, Bowei Chen, Mengyi Shan, Ira Kemelmacher-Shlizerman

TL;DR: 本文提出了分层虚拟试穿（LVTON）任务，旨在解决现有虚拟试穿方法无法处理服装分层（如添加或移除外套）的局限性。作者构建了一个分层基准和方法，通过解耦通用VTON先验和特定分层知识，实现了在保留现有服装的同时进行顺序分层操作。

Details

Motivation: 现实世界中时尚搭配涉及服装分层（如在外套上加夹克），而现有虚拟试穿方法仅擅长单层替换，无法处理分层或去层任务，因此需要解决这一根本性挑战。

Result: 该方法在自建的LVTON基准上取得了最先进（SOTA）结果，并在传统VTON基准上通过微调展示了卓越的泛化能力，实现了新的SOTA性能，同时具备零样本能力。

Insight: 创新点在于将LVTON任务解耦为通用VTON先验（如变形、身份保持）和特定分层知识（如分层顺序和遮挡推理），并通过自动数据生成管道合成训练数据，再结合小规模专用数据集微调来学习分层逻辑。

Abstract: In the real world, fashion is about layering: adding a jacket over a shirt, or a sequence of adding and removing layers, rather than just a single-layer swap. This fundamental real-world task remains a challenge in existing Virtual Try-On (VTON) methods, which excel at single-layer replacement but are not designed to layer or de-layer an existing outfit. This paper proposes Layering Virtual Try-On (LVTON), a layering benchmark and method that preserves an existing outfit while enabling sequential layering. We find that current VTON paradigms are fundamentally ill-equipped for LVTON, as their reliance on cloth-agnostic representations and single-item datasets discards essential layering context. Our key insight is that the LVTON challenge must be disentangled into two distinct competencies: (1) General VTON Priors (e.g., deformation, identity preservation) and (2) Specific Layering Knowledge (e.g., layering order and occlusion reasoning). First, our model obtains general VTON priors by being trained on data produced by an automatic data generation pipeline that synthesizes samples from fashion videos via segmentation and inpainting. Second, the model is fine-tuned on a small, dedicated LVTON dataset to learn the layering logic. Our method achieves state-of-the-art results on our LVTON benchmark and demonstrates superior generalizability on traditional VTON benchmarks, setting new state-of-the-art results when fine-tuned and exhibiting zero-shot capabilities.

[61] mmSimPrior: Learning Simulation Priors for Data-Efficient Real-World Generalizable Radar-Based Human Motion Reconstruction cs.CVPDF

Cheng Guo, Qiming Cao, Shengkai Xu, Haoyu Xie, Kaixiang Su

TL;DR: 本文提出了mmSimPrior框架，旨在解决毫米波雷达人体运动重建任务中真实配对数据稀缺导致的泛化难题。该框架通过模拟预训练，将可迁移知识分解为信号、运动和映射先验，并利用多模态信号编码器和联合时序分词器学习这些先验。在仅需少量真实数据的情况下，该框架在零样本和有限数据适应场景下均显著提升了运动重建精度。

Details

Motivation: 毫米波雷达虽能提供隐私保护且不受光照影响的人体运动感知，但构建跨真实场景泛化的模型需要大量昂贵的配对雷达-运动数据。模拟数据虽可扩展，但因其与真实信号在多重路径、杂波、响应统计和分辨率退化等方面存在差异，导致模型迁移效果不佳。

Result: 在mmSimPrior-Real和RT-Pose数据集上的实验表明，仅使用24个真实配对序列，mmSimPrior-Reg在三个环境中将MPJPE（平均关节位置误差）比最强基线降低了24.7%至39.0%；而mmSimPrior-Cls在零样本设置下，无需微调即可将MPJPE降低8.5%。

Insight: 创新点在于将模拟到真实的迁移知识分解为可学习的信号、运动和映射三个先验，并设计了物理启发的域随机化课程和多模态编码器来学习鲁棒信号表示。同时，提出的“无重叠设置”评估协议更严格地衡量了模型在未见过的受试者-环境-位置-运动配置下的泛化能力。

Abstract: Millimeter-wave (mmWave) radar offers privacy-preserving and lighting-robust sensing for human motion reconstruction, but learning models that generalize across real deployments require diverse paired radar-motion data that are costly to collect. Simulation provides scalable supervision, yet models trained on clean synthetic signals transfer poorly because of multipath, clutter, response statistics, and resolution degradation. We present mmSimPrior, a simulation-pretrained framework that factorizes transferable knowledge into signal, motion, and radar-to-motion mapping priors. A multi-modal signal encoder is pretrained with a physics-informed domain-randomization curriculum that emulates propagation- and acquisition-level variations, while a joint-temporal tokenizer learns a discrete prior over plausible human motion. A shared mapping prior supports classification over a learned motion codebook for constrained zero-shot reconstruction and continuous regression for flexible limited-data adaptation. We further construct a 4.2M-frame, 31K-sequence dataset suite and introduce a No-Overlap Setting that excludes repeated complete subject-environment-location-motion configurations across adaptation and test. Experiments on mmSimPrior-Real and RT-Pose demonstrate consistent gains: with only 24 paired real sequences, mmSimPrior-Reg reduces MPJPE by 24.7% to 39.0% over the strongest baseline across the three environments, while mmSimPrior-Cls reduces zero-shot MPJPE by 8.5% without finetuning.

[62] Structured Redundancy Modeling for Efficient Visual Token Pruning in High-Resolution MLLMs cs.CVPDF

Jouwon Song, Woohyeong Kim, Kyeongbo Kong

TL;DR: 本文提出了一种名为SFPruner的单前向剪枝器，用于解决高分辨率多模态大语言模型中视觉令牌爆炸导致的延迟瓶颈问题。该方法通过将冗余控制直接嵌入到评分空间中，避免了迭代组合优化的需要，从而在单次前向传播中实现冗余感知的重要性选择。

Details

Motivation: 现有基于子集优化的令牌剪枝方法在处理大量视觉令牌时，其顺序依赖性和迭代选择过程会引入显著的开销，限制了理论FLOPs减少向实际推理加速的转化。

Result: 在Qwen2.5-VL模型上，该方法在512个令牌时将令牌选择过程从112.4毫秒大幅减少到2.5毫秒，减少了约110毫秒，同时在激进的压缩下保持了与最先进技术相当的性能。

Insight: 创新点在于将冗余控制结构性地重新表述到评分空间，通过语义引导的岭杠杆方案和基于排名的方向性掩码两种互补机制，实现了非迭代、并行化的高效令牌选择，成功将理论令牌减少转化为实际的推理加速。

Abstract: Recent high-resolution Multimodal Large Language Models (MLLMs) generate thousands of visual tokens per input, leading to a visual token explosion that introduces severe latency bottlenecks. While token pruning mitigates this issue, state-of-the-art subset-optimization methods typically rely on iterative subset construction to jointly capture visual diversity and instruction relevance. As visual token counts scale, this sequential dependency introduces significant selection overhead, severely limiting the translation of theoretical FLOPs reductions into actual wall-clock speedups. To address this limitation, we propose Single-Forward Pruner (SFPruner), a structural reformulation of visual token pruning that embeds redundancy control directly into the scoring space, bypassing the need for iterative combinatorial optimization. Our non-iterative framework achieves redundancy-aware importance selection in a single forward pass through two complementary mechanisms. First, to attenuate redundancy at the covariance level, we introduce a semantics-guided ridge leverage scheme. By integrating instruction relevance and visual saliency, this mechanism suppresses dominant covariance directions and mitigates representation bias. Second, ranking-based directional masking resolves residual overlap through asymmetric similarity competition, where higher-scoring tokens explicitly suppress redundant lower-scoring alternatives via parallel tensor operations. Extensive evaluations demonstrate that our approach maintains stable selection costs, reducing the token selection process by up to 110 ms, from 112.4 ms to just 2.5 ms at 512 tokens in Qwen2.5-VL. This structural efficiency successfully translates theoretical token reductions into tangible inference speedups while preserving highly competitive performance against state-of-the-art techniques under aggressive compression.

[63] DispatchRAG: Grounding Emergency Dispatch Decisions in Real-World Protocols from Traffic Accident Video cs.CVPDF

Muhammad Sulthan Adhipradhana, Ehsan Javanmardi, Naren Bao, Manabu Tsukada

TL;DR: 本文提出了DispatchRAG框架，旨在通过结合真实世界的日本交通事故响应协议，增强视觉语言模型在紧急场景下的决策能力，以评估事故严重性并生成合适的应急响应。该框架采用基于检索增强生成的技术检索相关协议，并利用大语言模型进行推理。作者还构建了Accident Dispatch Dataset用于评估，并在该数据集上验证了框架优于基线VLM的性能。

Details

Motivation: 现有视觉语言模型在交通事故推理方面虽具潜力，但缺乏对现实应急响应协议的遵循，导致其无法直接用于需要严格依据规程的事故严重性评估与调度决策。

Result: 在基于日本事故响应协议构建的Accident Dispatch Dataset上进行验证，DispatchRAG框架在多种事故场景下相比基线VLM表现出强劲性能。

Insight: 核心创新在于将RAG机制与LLM推理结合，将领域特定的真实世界协议（如日本交通事故响应规程）系统地融入视觉语言模型的决策流程，为自动驾驶车辆自动报告事故提供了可集成、协议驱动的评估方案。

Abstract: Assessing the severity of a traffic accident scenario is important to decide which emergency service to dispatch. Missing an ambulance dispatch on a pedestrian accident is a fatal issue that can lead to death. Recently, Vision-Language Models (VLMs) have been a promising tool for accident reasoning, yet many VLMs are not grounded in real-life accident response protocols, making them not usable in accident severity assessment off-the-shelf. We introduced DispatchRAG, an accident assessor and dispatcher framework grounded in real-life Japanese traffic-accident response protocols, designed to enhance VLMs to generate an appropriate emergency response during an emergency scenario. Utilizing a RAG-based retrieval mechanism to retrieve the most relevant accident protocol and an LLM-powered reasoner to suggest the most proper response. To support evaluation, we introduce Accident Dispatch Dataset, a comprehensive dataset of accident assessment and emergency response according to Japanese accident response protocols adapted from the MM-AU dataset. We validate our framework on the Accident Dispatch Dataset, showing strong performance across various accident scenarios compared to the baseline VLM, pointing toward integration in autonomous vehicles that can automatically report both their own and nearby accidents.

[64] DishSeg24k: A Large-Scale Benchmark for Food Segmentation with Stochastic Expert Decoding cs.CVPDF

Yilin Wang, Haochen Shi, Guanyu Chen, Weiqing Min, Jinkai Zheng

TL;DR: 本文提出了一个大规模食物分割基准数据集DishSeg24k，包含24,096张真实用餐场景图像、112,281个实例和278个细粒度类别，以解决现有数据集无法应对密集重叠、细粒度相似性和长尾分布等挑战。同时，作者提出了Food Expert-Adaptive Segmentation Transformers (FEAST)模型，将基于查询的解码建模为马尔可夫决策过程，并引入强化学习引导的混合专家模块，通过双评论家解耦优化方案提升分割性能。

Details

Motivation: 现有食物分割基准无法充分反映真实用餐场景的复杂性，如密集的菜品重叠、细粒度类别相似性以及极端的长尾类别分布，因此需要构建更全面的数据集和更有效的分割方法。

Result: 在提出的DishSeg24k基准上，FEAST模型取得了最先进的性能，分别以+3.21% mIoU、+3.68% mDice和+4.00% mAcc超越先前方法；在FoodSeg103数据集上也验证了其有效性。

Insight: 创新点包括将分割解码过程建模为马尔可夫决策过程以探索边界不确定性，以及设计强化学习引导的混合专家模块，通过双评论家解耦优化实现查询优化与专家路由的分离，从而促进专家专业化并防止长尾分布下的专家崩溃。

Abstract: Food segmentation is essential for applications such as intelligent catering, dietary assessment, and recommendation. However, existing benchmarks fail to capture the complexity of real-world dining scenes. The challenges of dense inter-dish overlap, fine-grained class similarity, and extreme long-tail class distributions exceed the fidelity of current datasets. To fill this gap, we introduce \textbf{DishSeg24k}, a large-scale dish-level segmentation benchmark with 24,096 images, 112,281 instances, and 278 fine-grained categories in real-world dining environments. Based on DishSeg24k, we further propose \textbf{Food Expert-Adaptive Segmentation Transformers (FEAST)} to address these challenges. FEAST models query-based decoding as a Markov Decision Process (MDP), where each decoder layer update is treated as a sequential decision step that explores uncertainty along dish boundaries. We further redesign the decoder with a reinforcement learning (RL)-guided Mixture-of-Experts (MoE) module, in which a dual-critic decoupled optimization scheme separates task-oriented query refinement from structure-aware expert routing. This design promotes expert specialization and prevents expert collapse under long-tail category distributions. Finally, extensive experiments on DishSeg24k demonstrate the state-of-the-art performance of FEAST, which outperforms previous methods by {+3.21%} mIoU, {+3.68%} mDice, and {+4.00%} mAcc, respectively. We further validate the effectiveness of FEAST on FoodSeg103. The dataset and code will be publicly released.

[65] Similarity Is Not Logic: Factored Inference for Dual-Encoder Vision-Language Models cs.CV | cs.CL | cs.LGPDF

Sultan Alshehri, Zhantao Yang, Han Zhang, Marios Savvides

TL;DR: 本文指出双编码器视觉语言模型（VLMs）的相似性接口存在组合约束失效问题，例如查询’伞且无人’会检索到包含两者的图像。作者将问题归因于接口层面的’概念袋’效应，并提出了一种名为LCSE的训练无关方法，通过将证据提取与约束执行分离来解决问题。

Details

Motivation: 双编码器VLMs的相似性接口无法处理组合逻辑约束（如’且’、’非’），即使其底层概念检测可靠，这限制了其在需要逻辑推理的零样本检索任务中的应用。

Result: 在提出的FACTOR-Bench上，LCSE方法达到85.5%的准确率，优于最佳微调基线的73.2%；应用于SigLIP 2时达到90.7%。在NegBench COCO MCQ上，准确率从27.2%提升至65.2%，同时保持了检索性能。

Insight: 核心创新在于揭示了相似性接口的聚合方式是瓶颈，而非编码器表征本身，并提出了’分解推理’框架。LCSE方法无需训练，通过外部编辑概念分数来执行逻辑约束，是一种高效且可迁移的解决方案。

Abstract: Dual-encoder vision-language models (VLMs) expose a similarity interface that enables zero-shot retrieval but fails compositional constraints: queries like “umbrella and no person” retrieve images containing both, even when concept detection is reliable. We trace this to an interface-level Bag-of-Concepts effect, where similarity scores approximate mean pooling of concept evidence regardless of operators. Although operator-dependent signals exist in text embeddings, they are too weak or misaligned to affect rankings. Fine-tuning does not reliably resolve this failure because the dominant bottleneck is how similarity aggregates evidence rather than what encoders represent. We propose factored inference, which separates evidence extraction from constraint execution, and introduce LCSE (Logic-Constrained Score Editing), a training-free method that executes constraints externally using concept scores from frozen encoders. We also introduce FACTOR-Bench, where LCSE achieves 85.5% accuracy versus 73.2% for the best fine-tuned baseline, 90.7% when applied to SigLIP 2, and improves NegBench COCO MCQ accuracy from 27.2% to 65.2% while preserving retrieval performance.

Yihao Wu, Chenyi Xu, Liqi Yan, Chenhuan Cai, Geyong Min

TL;DR: 本文提出了BrainNav，一个基于最小充分性原则的视觉语言导航框架，用于解决连续环境中的视觉语言导航问题。该框架通过指令感知的选择性感知、紧凑的跨模态对齐和压缩的世界模型，旨在抑制环境噪声、过滤冗余信息并提升导航的鲁棒性和效率。

Details

Motivation: 当前的多模态大模型和基于世界模型的方法在视觉语言导航中保留了过多与任务无关的细节，这削弱了泛化能力并增加了计算负担。因此，需要一种能够提取最小充分表示的方法来提升导航性能。

Result: 在R2R-CE val-unseen数据集上，BrainNav在SR和SPL指标上分别比先前SOTA提升了2.0%和1.0；在RxR-CE val-unseen数据集上，分别提升了0.94%和0.78。

Insight: 创新点在于将最小充分性原则引入视觉语言导航，通过逻辑锚模型实现指令感知的选择性感知，通过简约约束对齐模块作为紧凑的跨模态瓶颈，以及通过压缩世界模型在低秩潜在空间中进行状态预测，从而对齐语义意图与空间感知。

Abstract: Vision-and-Language Navigation in continuous environments (VLN-CE) requires an agent to ground language in egocentric observations and plan in unseen scenes. Although recent multimodal large models and world-model-based methods have improved navigation, they often preserve excessive task-irrelevant detail, weakening generalization and increasing computational burden. We propose BrainNav, a navigation framework grounded in the Principle of Minimal Sufficiency. BrainNav consists of three components: a Logical Anchor Model that implements instruction-aware selective perception to suppress environmental noise, a Minimalist Constraint Alignment module that serves as a compact cross-modal bottleneck, efficiently synchronizing discrete linguistic intent with continuous latent dynamics while filtering out redundant information, and a Compression World Model that predicts action-conditioned states within a condensed, low-rank latent space. These modules align semantic intent with spatial perception, enhancing the agent’s robustness and efficiency in complex tasks. Experiments show that BrainNav improves over prior SOTA by 2.0 % / 1.0 in SR/SPL on R2R-CE val-unseen and 0.94 % / 0.78 on RxR-CE val-unseen. These results indicate that minimally sufficient world representations provide an effective foundation for robust VLN.

[67] A Reconstruction-Based Framework for Caption Evaluation Beyond Reference Captions cs.CVPDF

Zhijiang Tang, Jiaxin Qi, Kaihua Tang, Yuhua Zheng, Jianqiang Huang

TL;DR: 本文提出了一种基于图像重建的无参考图像描述评估框架，通过衡量描述能否重建出语义等价的图像来评估其质量。该方法利用下游视觉-语言任务来评估重建图像与原始图像的语义一致性，并引入了低成本的CTTD数据集作为替代方案。

Details

Motivation: 当前图像描述评估主要依赖人工标注的参考描述，这受限于标注者的意图和描述能力，无法客观评估描述是否忠实保留了图像语义。

Result: 该方法通过下游任务评估语义等价性，提供了一个无参考的、任务条件化的描述评分。

Insight: 创新点在于将描述评估问题转化为语义等价性重建问题，并通过下游任务而非像素级比较来评估语义一致性，这为无参考评估提供了新思路。CTTD数据集的设计也降低了评估成本。

Abstract: Image captioning is a primary task in vision–language research, yet assessing how faithfully a caption preserves image semantics without relying on reference captions remains unsettled. Prevailing evaluations rely on human-annotated references, whose content reflects annotator intent and captioning proficiency. In this paper, we study a reconstruction-based principle for caption evaluation: a caption is as good as its capacity to enable reconstruction of the original image. However, because captioning inherently compresses visual information, it is impossible to recover all details, and pixel-wise comparison between reconstructed and source images is neither feasible nor meaningful. Through our in-depth analysis of the nature of captions, whose fundamental purpose is to transmit the semantic content of an image, we propose a revised principle: a caption is as good as its capacity to enable a reconstruction that is semantically equivalent to the original. To assess semantic equivalence, we test whether the reconstruction matches the original image across a suite of downstream vision–language tasks, yielding a reference-free, task-conditioned caption score. We characterize component-dependent limitations and introduce the lower-cost Captioning Turing Test Dataset (CTTD) surrogate.

[68] SARATR-X-v2: Scale-Aware Structural Pre-Training for SAR Foundation Models cs.CVPDF

Weijie Li, Yafei Song, Yongxiang Liu, Bowen Peng, Jie Zhou

TL;DR: 本文提出SARATR-X-v2，一种用于合成孔径雷达（SAR）基础模型的尺度感知结构预训练方法。该方法通过融合多个固定结构提取器的输出作为掩码图像建模的重建目标，旨在同时满足物理基础的稳定性（对乘性散斑噪声近似不变）和语义尺度兼容性（覆盖下游任务所需的异构空间尺度）。在12个SAR基准测试上取得了最先进的迁移性能。

Details

Motivation: 当前SAR预训练中掩码图像建模的重建目标设计缺乏原则性指导。论文认为一个有效的SAR预训练目标应同时具备物理基础的稳定性（对相干成像固有的乘性散斑噪声鲁棒）和语义尺度兼容性（适应下游任务的不同空间尺度需求），而这两者通常相互矛盾。

Result: 在涵盖分类、检测和分割的12个SAR基准测试上，SARATR-X-v2实现了最先进的迁移性能。在合成散斑噪声变化下，所提目标将学习表示的扰动漂移相对于像素空间监督降低了近两个数量级。

Insight: 创新点在于提出了一个统一的设计框架，通过融合多个具有不同感受野的固定结构提取器（从盲点局部聚合到方向性对数比区域对比）并辅以可学习权重，来构建一个同时满足物理稳定性和尺度兼容性的重建目标。核心洞察是：有效的SAR预训练不在于重建更多信号，而在于重建正确的结构目标。

Abstract: Masked image modeling has become a dominant paradigm for SAR pre-training, yet the design of the reconstruction target remains fundamentally unsettled. This article argues that a SAR pre-training target should satisfy two conditions to produce transferable representations: (i) physics-grounded stability, i.e., approximate invariance of the target operator to multiplicative speckle inherent in coherent imaging; and (ii) semantic scale compatibility, i.e., coverage of the heterogeneous spatial scales that downstream tasks demand. These two conditions are individually achievable but jointly difficult: physics-grounded stability favors fixed operators, while semantic scale compatibility favors data-driven composition. To this end, SARATR-X-v2 reconciles both within a single design. The target is constructed through fixed structural extractors spanning six receptive fields, from blind-spot local aggregation to directional log-ratio region contrast, and fused via learnable weights into one unified supervision signal for masked reconstruction. On twelve SAR benchmarks across classification, detection, and segmentation, SARATR-X-v2 achieves state-of-the-art transfer performance. Under synthetic speckle variation, the proposed target reduces perturbation drift in the learned representation by nearly two orders of magnitude relative to pixel-space supervision. Taken together, these results establish physics-grounded stability and semantic scale compatibility as a principled framework for pre-training target design under coherent imaging, and suggest that effective SAR pre-training is not about reconstructing more signal, but about reconstructing the right structural target.

[69] OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models cs.CVPDF

Jinsen Su, Yongdong Luo, Yuexiao Ma, Yibo Hu, Meiguang Jin

TL;DR: 本文提出了OmniScope，一种无需训练的令牌压缩框架，用于解决全模态大语言模型中的跨模态令牌压缩问题。该方法通过将查询作为共享语义锚点，分别估计音频和视频的相关性，并采用模态特定的令牌预算分配、视觉令牌的锚点-差分策略以及音频令牌的秒内合并，在保持高精度的同时显著提升推理速度和减少内存使用。

Details

Motivation: 现有全模态大语言模型的令牌压缩方法通常依赖单一模态来决定其他模态的保留内容，但跨模态显著性不匹配（例如音频和视频的相关性峰值在不同时刻）会导致在激进压缩下丢弃关键答案线索。

Result: 在四个音频-视频基准测试和两种Qwen2.5-Omni模型规模上，OmniScope在所有压缩设置中实现了最佳平均准确率；在总体令牌保留率为25%时，实现了高达3.53倍的预填充加速和超过15%的GPU内存减少，而平均准确率仅下降0.35个百分点。

Insight: 创新点在于将查询作为跨模态共享的语义锚点，但独立估计各模态的显著性，避免了单向引导的偏差；具体技术包括模态特定的令牌预算分配、视觉令牌的锚点-差分策略（保留全局上下文和时间变化）以及音频令牌的秒内合并（减少冗余同时保持时间连续性），为全模态大语言模型推理提供了一个简洁的设计原则。

Abstract: Existing token compression methods for omnimodal large language models typically rely on one modality to determine what to retain in the other. We show that this assumption often breaks down: for the same query, audio and video relevance often peaks at different moments. This cross-modal salience mismatch makes unidirectional guidance prone to discarding answer-critical cues under aggressive compression. We propose OmniScope, a training-free token compression framework that uses the query as a shared semantic anchor while estimating relevance separately for audio and video. OmniScope allocates modality-specific token budgets, prunes visual tokens with an anchor-delta strategy that preserves both global context and temporal changes, and merges audio tokens within each second to reduce redundancy while maintaining temporal continuity. Across four audio-video benchmarks and two Qwen2.5-Omni model scales, OmniScope achieves the best average accuracy across all compression settings. At 25% overall token retention, it delivers up to 3.53x prefill speedup and more than 15% GPU memory reduction, with only a 0.35-point drop in average accuracy. These results suggest a simple design principle for OmniLLM inference: share the query across modalities, but not the salience estimates. The code is available at https://github.com/MAC-AutoML/OmniScope.

[70] WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation cs.CVPDF

Yuhui Zeng, Wang Chen, Jinfa Huang, Tianyu Xie, Yongdong Luo

TL;DR: 本文提出WaveZip，一种基于离散小波变换的联合信号-频域框架，用于高效视频推理。该方法通过1D DWT分析时间冗余并动态分配帧级token预算，通过2D DWT解耦空间特征，在查询显著区域调制高频细节，无需任务特定训练即可提升现有大视觉语言模型的效率。

Details

Motivation: 现有大视觉语言模型因视觉token的二次计算成本而难以处理长视频理解任务，且现有高效方法仅在空间特征域进行压缩，未能解耦结构上下文与语义细节。

Result: 在长视频理解基准测试中，WaveZip在10倍压缩比下仍能保持99.6%的完整性能，持续优于最先进方法。

Insight: 创新点在于利用小波变换将时空冗余与显著性解耦：时间上通过1D DWT与帧间差异门控动态分配token，空间上通过2D DWT分离低频近似与高频细节并进行查询调制，实现了无需训练的高效视频token压缩。

Abstract: Existing Large Vision-Language Models (LVLMs) struggle with long-form video understanding due to the quadratic computational cost of visual tokens. While recent efficient methods attempt to compress tokens via hard pruning or uniform merging, they operate strictly in the spatial feature domain, where robust structural context and discriminative semantic details are inherently entangled. In this work, we propose WaveZip, a joint signal-frequency-domain framework for efficient video inference. Driven by the insight that temporal redundancy resides in low-pass approximation scales while spatial saliency strongly correlates with high-frequency components, WaveZip leverages Discrete Wavelet Transforms (DWT) to disentangle these signals. Temporally, it employs 1D DWT to analyze query-frame relevance, and the resulting high-frequency coefficients are further gated by inter-frame differences, with both signals jointly driving the dynamic allocation of a precise frame-level token budget. Spatially, a 2D DWT decomposes features into low-frequency approximations and high-frequency detail components, where the high-frequency coefficients are modulated within query-salient regions to regulate spatial reconstruction. Importantly, WaveZip requires no task-specific training and can be seamlessly integrated into off-the-shelf LVLMs to boost inference efficiency. Extensive experiments on long video understanding benchmarks demonstrate that WaveZip retains 99.6% of the full performance under an extreme 10x compression ratio, consistently outperforming state-of-the-art methods.

[71] What CLIP Knows but Cannot Say: Recovering Negation from Frozen Intermediate Features cs.CV | cs.AIPDF

Chen-Yi Lu, Yueh-Shao Chen, Somali Chaterji

TL;DR: 本文针对CLIP等对比视觉-语言模型在处理否定语义时存在的表示崩溃问题，提出了一种轻量级后处理校正系统PeakPatch。该系统通过拦截文本编码器的中间层（即组合性峰值层），提取否定信号并重新注入最终层嵌入空间，从而在不修改预训练权重的情况下恢复否定理解能力。

Details

Motivation: CLIP等模型将语义相反的短语（如“一只狗”与“不是一只狗”）映射到几乎相同的嵌入，导致其对否定不敏感，这源于最终层表示中组合性结构的崩溃（即表示崩溃）。

Result: 在NegBench基准测试中，PeakPatch在COCO MCQ上达到74.3%（比CLIP提升35.1%，比最佳编码器微调方法提升17.8%），在VOC MCQ上达到65.5%，并在完全分布外否定检索任务上超越所有微调基线，同时仅增加3.5%的参数。校正后的嵌入还可迁移到文本到图像生成任务（否定分数提升18.4%）并泛化至不同骨干网络。

Insight: 创新点在于揭示了CLIP中间层保留组合性语法而最终层崩溃的结构性现象，并提出通过轻量级后处理模块（嵌入校正网络和分数校正网络）从冻结模型中提取并重注入丢失的语法信号，实现了高效且通用的否定语义恢复。

Abstract: Contrastive vision-language models such as CLIP map semantically opposite phrases (e.g., “a dog” vs. “not a dog”) to nearly identical embeddings, rendering them insensitive to negation. We attribute this failure to a phenomenon we call Representational Collapse: by tracking compositional divergence and visual alignment across the CLIP text encoder, we show that middle layers build compositional syntax, but the final layers collapse this structure as visual alignment rises, producing a syntax-blind final representation. To recover the lost negation signal without altering pretrained weights, we propose PeakPatch, a lightweight post-hoc correction system that intercepts the encoder at its compositional peak. An Embedding Correction Network (ECN) uses cross-attention to extract a negation-specific signal from the peak layer, anchored to a stable baseline, and predicts a deviation vector that re-injects the lost syntax into the final-layer embedding space. A complementary Score Correction Network (SCN) predicts bounded scalar score offsets for discriminative tasks. Both modules are trained jointly end-to-end while all CLIP parameters remain frozen, adding only 5.2M parameters (3.5% of the backbone) and preserving the standard cosine similarity interface. On NegBench, PeakPatch achieves 74.3% on COCO MCQ (+35.1 over CLIP, +17.8 over the best encoder fine-tuning method) and 65.5% on VOC MCQ, while outperforming all fine-tuning baselines on fully out-of-distribution negation retrieval despite training only 3.5% of the parameters. The corrected embeddings also transfer to text-to-image generation (+18.4 negation score) and generalize across ViT-B/32, ViT-L/14, and SigLIP backbones. Project URL: https://stevencylu.github.io/PeakPatch/.

[72] The Gate Always Closes: On Injecting Auxiliary Signals into Frozen Vision-Language Models cs.CVPDF

Moshiur Farazi, Sameera Ramasinghe, Bekir Sait Ciftler, Mahbub Ahmed Turza, Shafin Rahman

TL;DR: 该论文研究了在冻结视觉语言模型中注入辅助信号时，优化器几乎总是关闭可学习门，导致辅助信号被抑制的现象。作者将此归因于死梯度机制和负效用机制，并提出利用几何辅助损失（基于双曲视觉关系图）来正则化LoRA微调，以在训练时注入几何信号并在推理时丢弃，从而在不损害关系推理准确性的情况下提升属性问题性能。

Details

Motivation: 动机在于理解为何在视觉语言模型中为辅助信号路径添加可学习门时，优化器几乎总是选择关闭这些门（即信号被抑制），并探索如何有效利用这种抑制现象来提升模型性能，特别是在关系推理和属性识别任务上。

Result: 在GQA基准测试上，未使用几何损失推理的配置在关系问题上损失2.85-3.39个百分点，而在属性问题上增益约1个百分点；使用几何损失训练但通过软提示推理的配置在关系问题上损失5.14个百分点，属性问题仅增益0.23个百分点。保留几何路径推理的配置保持了原始关系准确性并匹配了属性增益。在OOD的VSR数据集上，RMS前缀方法保留了空间信号，而剥离几何损失（G2）导致VSR性能下降4.6个百分点。

Insight: 创新点包括：揭示了辅助信号注入中的门抑制现象及其两种机制（死梯度和负效用）；提出利用双曲几何的辅助损失（如IoA驱动的蕴含锥和洛伦兹流形上的角排斥）作为训练时正则化器，在推理时丢弃以实现性能提升；发现嵌入范数对齐对生成安全的前缀注入至关重要，并建议用固定尺度的非可选注入替代可学习门。

Abstract: Auxiliary signal pathways in VLMs are routinely fitted with learnable gates so the optimiser can decide how much of the signal to admit. We find that the optimiser almost always decides on zero: across five injection designs, every gated pathway becomes behaviourally closed, with accuracy invariant to ablating the pathway at inference even when the gate parameter would nominally pass 30-45% of the signal. We attribute this suppression phenomenon to two regimes, a dead-gradient regime formalised through the caption-invariance of image-derived signals, and a negative-utility regime in which the auxiliary signal actively hurts the loss. Rather than fight suppression, we exploit it: we regularise LoRA fine-tuning with geometric auxiliary losses from hyperbolic visual relational graphs (IoA-driven entailment cones and angular repulsion on the Lorentz manifold), coupled only through the forward pass at training time and dropped at inference. Disaggregating GQA by question type exposes a clean dissociation. Three configurations without geometric losses at inference lose 2.85-3.39pp on relational questions while gaining ~1pp on attribute questions; a fourth that trains with the losses but infers through a soft prompt loses 5.14pp on rel for only +0.23pp on attr, so training-time regularisation alone does not protect relational accuracy without a geometric inference pathway. Configurations that keep the geometric pathway at inference preserve vanilla-level relational accuracy and match the attribute gain. Out of distribution on VSR, the RMS-prefix recipe preserves the spatial signal; stripping the geometric losses (G2) collapses VSR by 4.6pp, isolating them as the OOD source. A secondary result: embedding-norm alignment is necessary for generation-safe prefix injection, and learnable gates should be replaced with fixed, non-optional injection at matched scales.

[73] UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models cs.CVPDF

Ioannis Maniadis Metaxas, Adrian Bulat, Alberto Baldrati, Anestis Zaganidis, Yassine Ouali

TL;DR: 本文提出了UltraViT，一种专为在设备端部署而优化的视觉编码器，旨在解决大型视觉语言模型（LVLM）在资源受限边缘设备上因计算量大而难以部署的问题。通过考虑实际设备端延迟，设计了金字塔架构并整合异构空间混合器，同时提出两阶段生成式预训练策略以提升语义理解能力。

Details

Motivation: 现有LVLM压缩方法主要关注视觉令牌缩减或小型语言模型，而视觉编码器通常被视为计算密集的整体特征提取器，缺乏针对设备端延迟的专门优化设计。

Result: 在广泛的实验中，UltraViT在设备端以近1.7倍的速度运行，显著优于现有以编码器为中心的基线方法，在高效LVLM编码方面达到了新的最先进水平（SOTA）。

Insight: 创新点包括基于设备端延迟的系统化金字塔架构设计、异构空间混合器的集成，以及通过密集蒸馏和冻结LLM直接生成监督的两阶段预训练策略，有效提升了视觉编码器的语义基础和多模态对齐能力。

Abstract: Large Vision-Language Models (LVLMs) remain bottlenecked by massive computational footprints, precluding their deployment on resource-constrained edge devices. While efforts to compress LVLMs focus heavily on vision token reduction or smaller language models, the vision encoder is largely overlooked, typically deployed as a monolithic, computationally heavy feature extractor. Moreover, there is no previous effort that designs a vision encoder for LVLMs directly optimized for on-device latency. In this paper, we present UltraViT, a vision encoder for LVLMs, explicitly designed and optimized for on-device performance. Specifically, by taking into account real on-device latencies, we systematically design a pyramidal architecture that strategically integrates and adapts heterogeneous spatial mixers at the macro-block level. Furthermore, to pre-train UltraViT, we propose a novel two-stage generative pre-training strategy: cultivating rich spatial features via dense distillation, followed by direct generative supervision from a capacity-mixed frozen LLM. Compared to standard contrastive and SSL, we show that our pre-training is much more effective for achieving high-level semantic grounding for UltraViT needed for the subsequent generative multimodal alignment of LVLM training. Extensive experiments demonstrate that our on-device latency-informed design combined with our tailored training strategy establishes a new state-of-the-art for efficient LVLM encoding, significantly outperforming existing encoder-centric baselines while operating on-device at nearly 1.7xthe speed.

[74] Omni-Prune: Query-Aware Unified Token Pruning for Efficient Omnimodal Large Language Models cs.CV | cs.CLPDF

Yiming Zhong, Chang Nie, Caifeng Shan

TL;DR: 本文提出了Omni-Prune，一种用于高效全模态大语言模型（OmniLLMs）的、无需训练的、查询感知的统一令牌剪枝框架。该方法旨在解决音频-视频输入序列过长导致推理时预填充延迟高和GPU内存占用大的问题。它通过结合编码器注意力和文本查询相关性来联合评估音频和视频令牌的重要性，并利用K-medoids算法选择代表性令牌，从而在保持模型性能的同时显著提升推理效率。

Details

Motivation: 全模态大语言模型在处理同步音频和视频时会产生极长的令牌序列，导致推理时预填充延迟高和内存占用大。现有的令牌剪枝方法主要针对纯视觉输入，忽略了音频与视频之间的跨模态关联以及决定内容重要性的用户查询。

Result: 大量实验表明，Omni-Prune在性能上优于现有基线方法，在保持超过99%全模型性能的同时，实现了高达3.25倍的预填充加速和1.3倍的内存减少。

Insight: 创新点在于提出了一个查询感知的、跨模态统一的令牌剪枝框架，它结合了编码器注意力与文本查询相关性来联合评估音频和视频令牌，并利用自适应时间窗口和K-medoids算法来保留任务相关的跨模态证据和多样性线索。这为高效处理长序列多模态输入提供了一种无需训练的新思路。

Abstract: Omnimodal large language models (OmniLLMs) are rapidly extending multimodal reasoning to cover synchronized audio and video. However, the resulting audio-video token sequences are long, leading to high prefill latency and GPU memory usage at inference time. Existing token pruning methods, designed mainly for vision-only inputs, miss both the cross-modal links between audio and video and the user query that decides which content matters. To bridge this gap, we present Omni-Prune, a training-free, query-aware audio-visual token pruning framework that jointly removes redundancy from both modalities while keeping task-relevant cross-modal evidence. Specifically, Omni-Prune first splits the token sequence into adaptive time windows placed at audio saliency peaks, then scores audio and video tokens on a single scale that combines encoder attention with text-query relevance, and pairs related audio-video tokens so that they are kept together. Within each window, a final K-medoids step then selects a few representative tokens, adding diverse cues that score-based selection alone would miss. Extensive experiments demonstrate that Omni-Prune outperforms established baseline methods, delivering up to 3.25x prefill speedup and 1.3x memory reduction while retaining over 99% of full-model performance.

[75] Explaining BiomedCLIP with Weighted Banzhaf Interactions Supported by Tree-Gram Parsing cs.CV | cs.AI | cs.CLPDF

Jakub Rymarski, Adam Rempała, Bartłomiej Sobieski, Przemysław Biecek

TL;DR: 本文提出了ParseFIxLIP方法，通过将Tree-Gram Parsing整合到FIxLIP框架的Banzhaf交互博弈中，解决了现代分词器在医学领域Vision-Language Models（如BiomedCLIP）解释中导致的语义碎片化问题，从而生成更连贯、可解释的跨模态归因。

Details

Motivation: 现有解释方法（如FIxLIP）在处理现代分词器时，会将临床概念（如’saddle embolus’）拆分为无意义的子词，导致语义不连贯的归因噪声和交互组合爆炸，难以在临床环境中提供可靠解释。

Result: 在ROCOv2医学图像数据集和通用示例上的定性分析表明，该方法能准确捕捉分组词汇对模型预测的协同影响；定量上，解析方法在长文本的高维场景中保持了统计鲁棒性和语义简洁性，而基线方法则表现不佳。

Insight: 创新点在于利用依赖解析树（如spaCy）将相关文本token分组为语义连贯的单元来定义解释参与者，通过smart_depth分组策略统一复杂医学概念，从而缓解概念碎片化，为医学VLM决策提供直观且临床相关的解释。

Abstract: Vision-Language Models (VLMs) are demonstrating significant capabilities in medical tasks like radiology analysis, yet providing faithful and interpretable explanations remains a key consideration for their responsible deployment in clinical settings. However, existing explanation methods, such as the widely used FIxLIP framework, often struggle with the fine-grained nature of modern tokenizers. The tokenization problem fragments clinical concepts—splitting terms like “saddle embolus” into scattered, meaningless subwords—which leads to noisy, semantically incoherent cross-modal attributions. Such fragmentation also results in a combinatorial explosion of interaction possibilities, obscuring the model’s true reasoning. To address this, we introduce ParseFIxLIP, an extension that incorporates the Tree-Gram Parsing into the Banzhaf interaction game used by FIxLIP. This semantically informed strategy utilizes dependency parsing trees to define explanation players by grouping related text tokens into semantically coherent units. Our smart_depth grouping strategy, merging tokens according to spaCy token dependency tree, successfully mitigates concept fragmentation, yielding substantially more interpretable cross-modal interactions by unifying complex medical concepts. Quantitatively, while baselines struggled with the high dimensionality of long captions, our parsing approach maintained statistical robustness and semantic parsimony. Qualitative analysis on BiomedCLIP, validated on medical imagery (ROCOv2) and general examples, confirms that the approach accurately captures the synergistic influence of grouped words on model predictions. In conclusion, our work offers intuitive and clinically relevant insights into VLM decision-making, fulfilling the critical need for coherent explanations in the medical domain.

Weixiang Zhou, Jiabei Zuo, Yuhao Wang, Cong Wang, Huchuan Lu

TL;DR: 该论文提出了一种名为PRISM的新型多模态物体重识别框架，旨在通过整合来自多个模态的互补信息来检索特定物体。该框架基于Prompt-S6和语义感知知识引导，通过语义驱动的令牌剪枝抑制背景噪声，并通过渐进式融合网络实现三模态对齐，从而在复杂场景下生成更鲁棒的多模态表示。

Details

Motivation: 现有方法未能有效解决背景干扰抑制或实现三模态对齐，且计算复杂度高。论文旨在克服这些限制，提升多模态ReID的性能与效率。

Result: 在四个多模态物体ReID基准测试上进行的广泛实验证明了该方法的有效性和效率。

Insight: 创新点在于将Mamba的线性复杂度和序列建模能力与高效的跨模态交互相结合，并引入了语义驱动的令牌剪枝和渐进式融合网络，以利用分割基础模型的语义先验进行动态剪枝和渐进式特征聚合。

Abstract: Multi-modal object Re-Identification (ReID) aims to retrieve specific objects by integrating complementary information from multiple modalities. However, existing multi-modal ReID methods do not effectively address background interference suppression or achieve tri-modal alignment, instead focusing on pairwise feature fusion. Moreover, many current aggregation approaches suffer from high computational complexity. To address these limitations, we propose PRISM, a novel multi-modal ReID framework built upon Prompt-S6 (PS6) and semantic-aware knowledge guidance. PS6 maintains the linear complexity and strong sequence modeling capability of Mamba while enabling efficient cross-modal interaction. Leveraging these advantages, we design two key components: Semantic-Driven Token Pruning (SDTP) and Progressive Fusion Network (PFN). Parsing semantic priors from the segmentation foundation models, the SDTP then leverages these priors and applies dynamic token pruning to suppress background noise and refine feature representations. The PFN progressively aggregates multi-modal features to achieve tri-modal alignment and fully exploit modality complementarity. With the proposed modules, PRISM generates more robust multi-modal representations under complex scenarios. Extensive experiments on four multi-modal object ReID benchmarks demonstrate the effectiveness and efficiency of our approach. The source code is available at https://github.com/zw-absin/PRISM.

[77] VIPER: Visual In-Context Physics Reasoning for Physically Plausible Video Generation cs.CVPDF

Tianxiao Chen, Hanmo Chen, Huajin Chen, Bo Li, Qi Ye

TL;DR: 论文提出了VIPER框架，用于实现基于视觉上下文的物理推理，以生成物理上合理的视频。该方法利用参考视频作为物理过程的视觉演示，通过多模态大语言模型提取物理线索，并指导预训练的图像到视频生成器，从而在保持基础生成器视觉先验的同时，实现物理行为的迁移。

Details

Motivation: 现有视频生成模型难以通过标准文本和图像条件控制物理行为，因为材质响应、接触交互、变形和运动轨迹等连续且相关的物理线索难以用语言详尽描述，但可以通过视频自然演示。

Result: 在未见过的验证集上，VIPER在参考视频物理相似性和人类偏好方面优于代表性的视频生成和视频作为提示的基线方法，同时保持了有竞争力的通用视频质量。

Insight: 创新点在于将参考视频视为期望物理过程的密集视觉演示而非外观模板，并采用多模态大语言模型提取物理线索进行分层训练，实现了无需精心设计提示的物理行为迁移。

Abstract: Modern video generation models can synthesize visually compelling and temporally coherent clips, yet controlling their physical behavior remains difficult with standard text and image conditions. The core challenge is a conditioning bottleneck: material response, contact interaction, deformation, and motion trajectory are continuous and relational physical cues that are hard to specify exhaustively in language but can be demonstrated naturally by video. We propose VIPER, a Visual In-Context Physics Reasoning framework for reference-guided image-to-video generation. Given a target image, a brief target prompt, and a reference video, VIPER treats the reference as a dense visual demonstration of the desired physical process rather than an appearance template. It uses a Multimodal Large Language Model (MLLM) to extract reference-derived physical cues and guide a pretrained image-to-video generator through a hierarchical training strategy, enabling physical behavior transfer while preserving the visual prior of the base generator. To support this setting, we construct VIPER-19K, a curated dataset with material, trajectory, and physical-impact annotations, together with filtered reference-target pairs. Experiments on an unseen validation set show that VIPER achieves stronger reference-video physical similarity and higher human preference than representative video generation and video-as-prompt baselines, while maintaining competitive general video quality. Qualitative results further demonstrate that VIPER can transfer reference-derived physical behavior to new target scenes without requiring carefully engineered prompts.

[78] PlanCraft: Sketch, Refine, and Furnish for Architect-Inspired Progressive 3D Residential Scene Generation cs.CV | cs.CLPDF

Pengyu Zeng, Yuqin Dai, Jun Yin, Ziyang Han, Ng Cheuk Hei

TL;DR: PlanCraft是一个受建筑师启发的渐进式3D住宅场景生成系统。它通过模拟建筑师从草图到精修的设计流程，先生成2D平面图作为空间契约，再在其基础上进行家具布置，从而解决了现有方法在几何有效性和空间合理性上的不足。

Details

Motivation: 现有自动化住宅平面图生成方法忽视了设计的渐进性本质（建筑师从粗略草图开始逐步细化）以及2D平面图作为不可替代的空间契约的重要性。绕过这一契约会导致房间重叠、比例不合理等几何无效布局。

Result: 在实验中，PlanCraft的FID分数比现有最佳2D方法降低了61.1%，在专家评定的空间合理性上超过现有3D系统15分。即使在草图仅完成25%时，其性能已超过所有基线方法。

Insight: 核心创新在于将设计过程建模为渐进式细化（SketchPlan提供训练信号，PlanCraft-Diff进行粗到细生成），并强调2D平面图作为约束家具布置的“空间契约”（PlanCraft-Agent），这更符合真实设计流程并确保了几何有效性。

Abstract: Two structural insights have been overlooked in automated residential floor plan generation. First, design is inherently progressive. Architects begin with rough strokes and refine them over time, whereas existing methods typically require their conditioning representation to be fully specified before generation, a fundamental mismatch with how design actually works. Second, the 2D floor plan is not an optional intermediate but an irreplaceable spatial contract. Once room boundaries, doors, and windows are fixed, furnishing reduces from open-ended spatial reasoning to bounded constraint satisfaction. Bypassing this contract, as existing 3D systems do by delegating layout to language models, yields overlapping rooms and implausible proportions; directly calling general-purpose language models likewise produces geometrically invalid layouts. Guided by these insights, we present PlanCraft. SketchPlan supplies the missing training signal by replaying the architect’s drawing process on 80K real floor plans, producing partial sketches at every completeness level. PlanCraft-Diff progressively sharpens an incomplete sketch into a geometrically precise, vectorizable floor plan through a coarse-to-fine strategy. With the spatial contract established, PlanCraft-Agent then furnishes the scene within well-defined room boundaries. Experiments show that PlanCraft achieves a 61.1% lower FID than the best existing 2D method and surpasses existing 3D systems by 15 points in expert-rated spatial rationality, with a sketch at only 25% completion already outperforming all fully specified baselines.

[79] Token-Region Guided Cross-Attention Fusion for Multimodal Affect Interpretation cs.CV | cs.AIPDF

Musa Tur Farazi, Nufayer Jahan Reza

TL;DR: 本文提出了一种用于孟加拉语政治意图表情包检测的多模态交叉注意力融合框架。该框架首先利用视觉语言模型从噪声图像中提取高保真OCR文本，然后通过交叉模态多头注意力机制对齐语义标记与视觉区域，融合视觉与文本特征。实验表明该方法在PoliMemeDecode1数据集上显著优于单模态基线和标准拼接方法。

Details

Motivation: 解决在低资源语言（如孟加拉语）中，因视觉线索与嵌入式风格化文本之间复杂交互而导致的互联网表情包分类计算挑战，特别是针对政治意图检测任务。

Result: 在PoliMemeDecode1数据集上的实验评估显示，所提出的基于注意力的融合方法显著优于单模态基线和标准拼接方法，实现了约0.94的Macro-F1分数，达到最先进水平。

Insight: 创新点在于引入交叉模态多头注意力机制来对齐语义标记与视觉区域，实现细粒度特征融合；同时探索了整合领域特定政治词典作为先验知识，增强了模型对政治意图的理解。从客观角度看，将高保真OCR提取与语义-视觉对齐相结合，是针对低资源多模态内容分析的有效策略。

Abstract: Automated analysis of multimodal content on social networks has become a critical task for understanding public sentiment and information diffusion in the digital age. However, classifying internet memes remains computationally challenging due to the intricate interplay between visual cues and embedded, often stylized, text, particularly in low-resource languages like Bengali Language. This paper addresses the detection of political intent in Bengali memes by introducing Multimodal Cross-Attention Fusion framework. We first leverage a Vision-Language Model to extract high-fidelity OCR text from noisy meme images. Subsequently, we encode visual and textual features and synthesize them through a cross-modal multi-head attention mechanism that aligns semantic tokens with visual regions. We also investigate the integration of a domain-specific political lexicon as a knowledge prior. Experimental evaluation on the PoliMemeDecode1 dataset shows that our attention-based fusion significantly outperforms unimodal baselines and standard concatenation methods, achieving a state-of-the-art Macro-F1 of approximately 0.94. Interpretability analyzes further confirm that the model effectively learns to ground textual semantics in visual evidence.

Zhijing Cheng, Xuancheng Zhang, Donglin Di, Lei Fan, Baorui Ma

TL;DR: 论文提出MOJITO，一种基于模态联合学习的端到端自动驾驶统一框架。它通过块级模态联合注意力机制，同时更新动作、图像和激光雷达特征，使规划器在生成动作时能直接访问多模态特征，从而避免了传统级联两阶段流水线中感知信息压缩导致的细节丢失问题。

Details

Motivation: 传统端到端自动驾驶系统采用感知到规划的级联两阶段流水线，迫使传感器输入压缩为紧凑的上下文表示，丢失了对规划至关重要的细粒度细节，且难以利用现代视觉基础模型的丰富表征能力。

Result: 在NAVSIM v1数据集上达到88.9 PDMS，在更具挑战性的NAVSIM v2数据集上达到88.4 EPDMS，创造了新的最先进水平（SOTA）。

Insight: 创新点在于移除了级联接口，引入块级模态联合注意力机制，实现了动作、图像和激光雷达特征的同步更新与交互，使规划器能直接利用多模态原始特征，提升了表征能力和规划性能。

Abstract: End-to-end autonomous driving systems commonly follow a cascaded two-stage pipeline where a perception stage compresses multi-modal sensor inputs into a compact context and a downstream planner predicts trajectories conditioned on this context. We argue that this one-way perception-to-planning interface forces sensor inputs into a compact representation, losing the fine-grained details critical for planning. Moreover, by constraining the planner to this compressed context, it is difficult to leverage the rich representations offered by modern vision foundation models. To address these issues, we propose MOJITO, a unified sensor-to-action framework for end-to-end autonomous driving built on modal joint learning. MOJITO removes the cascaded interface and instead performs block-wise Modal Joint Attention that simultaneously updates action, image, and LiDAR features, allowing the planner to directly access multi-modal features during action generation. MOJITO achieves 88.9 PDMS on the NAVSIM v1 dataset and 88.4 EPDMS on the more challenging NAVSIM v2 dataset, setting a new state-of-the-art. Extensive experiments further demonstrate strong scalability, instruction following, and diverse trajectory generation. Code and models are available at https://github.com/mumucc01/MOJITO.

Yuqi Liu, Shengju Qian, Tianyuan Qu, Mingxian Lin, Zixuan Wang

TL;DR: 本文提出了MemVLN，一种用于连续环境视觉语言导航（VLN-CE）的新框架。它通过引入情节记忆（采用金字塔分辨率管理）和程序记忆（使用原子中级动作词汇），在保持长时视觉历史轨迹一致性的同时，实现了实时推理效率（14 FPS）和最先进的性能。

Details

Motivation: 解决现有基于视频的VLN方法难以同时满足长时视觉历史维护与低延迟动作执行这两个需求的问题。

Result: 在VLN-CE基准测试中，MemVLN-4B在R2R和RxR数据集上的成功率（SR）分别比基线Qwen3-VL-4B架构高出5.8%和9.7%，同时推理延迟加速了7倍，达到实时推理（14 FPS）和SOTA性能。

Insight: 核心创新点是设计了情节记忆（用于高效压缩和管理长时视觉历史）与程序记忆（用于绕过自回归解码延迟、实现快速动作生成）的双记忆机制，在模型架构层面平衡了性能与效率的矛盾。

Abstract: Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to maintain long-horizon visual history for trajectory consistency while executing actions with low latency. Existing video-based VLN approaches typically struggle to satisfy both demands simultaneously. To address these challenges, we propose MemVLN, a novel VLN framework that achieves state-of-the-art performance with real-time inference efficiency (14 FPS). MemVLN utilizes a visual encoder to process continuous observations and a Large Language Model (LLM) to interpret instructions and generate actions. Central to our approach is an Episodic Memory management that applies pyramidal resolutions. This mechanism concentrates computation on immediate percepts while retaining compressed long-term history. Complementing to this design, we introduce Procedural Memory for fast action with a compact vocabulary of atomic mid-level actions to bypass auto-regressive decoding latency. Experiments on VLN-CE show that MemVLN-4B surpasses the baseline Qwen3-VL-4B architecture by 5.8% SR in R2R and 9.7% SR in RxR, while achieving a 7$\times$ speedup in inference latency.

[82] D3O: Dynamic Distribution Distillation for Ordinal Regression cs.CV | cs.AIPDF

Chunlai Dong, Yaojun Hu, Yuyang Xu, Haochao Ying, Jian Wu

TL;DR: 本文提出D3O，一种用于序数回归的动态分布蒸馏框架。该方法通过自蒸馏技术，在训练过程中动态演化序数标签分布，以替代传统的静态监督目标，从而更好地处理由主观标注引起的类别边界模糊和标注噪声问题。

Details

Motivation: 序数回归任务中，标签通常是通过主观判断对连续语义进行离散化得到的，这导致了模糊的边界和标注噪声。现有依赖固定监督目标的方法会强化主观标注下的偏差排序，因此需要一种能处理这种不确定性的新方法。

Result: 在四个通用序数回归任务上的大量实验表明，D3O始终优于现有方法，尤其是在严重的类别不平衡和噪声监督条件下。

Insight: 主要创新点在于：1）引入基于视觉-语言对齐的对比序数感知标签增强模块，以恢复捕捉类间模糊性和实例级不确定性的精细标签分布；2）设计基于累积分布函数（CDF）的跨层交互蒸馏机制，在网络层次结构中传播累积序数结构，确保中间表示具有一致的序数几何特性。这为学习超越固定目标的鲁棒序数表示提供了动态监督的有效范例。

Abstract: Ordinal regression is widely used in scenarios where labels are discrete yet inherently ordered. In practice, however, ordinal labels are often obtained by discretizing underlying continuous semantics through subjective human judgment, resulting in ambiguous boundaries and annotation noise. Such uncertainty challenges existing methods that rely on fixed supervision targets, which may reinforce biased ordering under subjective annotations. To address this limitation, we propose D3O, a dynamic distribution distillation framework that replaces static supervision with training-driven evolution of ordinal label distributions via self-distillation. Specifically, we introduce a contrastive ordinal-aware label enhancement module that leverages vision-language alignment to recover refined label distributions capturing both inter-class ambiguity and instance-level uncertainty. Furthermore, we design a CDF-based cross-layer interaction distillation mechanism to propagate cumulative ordinal structure across network hierarchy, ensuring consistent ordinal geometry in intermediate representations. Extensive experiments on four general ordinal regression tasks demonstrate that our proposed D3O consistently outperforms existing approaches, particularly under severe class imbalance and noisy supervision. These results highlight the effectiveness of dynamic supervision in learning robust ordinal representations beyond fixed targets. The code will be publicly available.

[83] GaitFace: A Multimodal Dataset for Long-Range Person Identification cs.CVPDF

Alain Komaty, Luis S. Luevano, Vidit Vidit, Anjith George, Zeina Al Amine

TL;DR: 论文提出了GaitFace数据集，这是一个包含远距离采集的人脸和步态数据的多模态公开数据集，旨在模拟真实边境管控场景。该数据集包含通过移动设备注册的预登记数据和多角度、多摄像头的远距离自然采集数据，用于评估现有生物识别模型在低分辨率和高视角条件下的性能。

Details

Motivation: 解决边境管控中因拥堵和乘客等待时间长带来的效率挑战，以及现有生物识别技术在远距离监控下因大气条件和图像质量退化而受限的问题。同时，弥补如BRIAR等高质量数据集通常仅限于政府机构使用的不足，提供一个公开可用的基准。

Result: 在GaitFace数据集上对当前最先进的人脸和步态模型进行基准测试，结果显示这些模型在低分辨率和仰视角度下表现失败，尽管它们在光学辅助条件下成功。该数据集暴露了这些关键弱点。

Insight: 创新点在于构建了一个公开的多模态远距离生物识别数据集，结合预登记和自然采集数据以模拟真实场景，为无约束生物识别研究提供了严格的公共基准。从客观角度看，该数据集有助于推动模型在恶劣条件下的鲁棒性研究，并促进生物识别技术的实际应用评估。

Abstract: Efficient border control is becoming a significant global challenge, mainly due to severe congestion and extended passenger waiting times. To mitigate these bottlenecks and facilitate passenger flow, biometric technologies are increasingly deployed to streamline identity verification and enhance crossing efficiency. Technical limitations frequently impede biometric identification, particularly in long-range surveillance, where systems must deal with adverse atmospheric conditions and degraded image quality. While high-quality frameworks like BRIAR exist, they are frequently restricted to specific government agencies. This paper introduces GaitFace, a new public dataset that contains face and gait data captured at long distances. To ensure that the research reflects authentic border scenarios, we use Pre-Enrollment data, where a traveler registers via a mobile device, and “In-the-Wild” captures, which records individuals at a distance across multiple viewing angles and different cameras. Benchmarking SOTA face and gait models reveals that current architectures fail under low-resolution and elevated viewpoints despite success with optical assistance. GaitFace exposes these critical vulnerabilities, providing a rigorous public benchmark to drive more robust, unconstrained biometric research.

[84] Neuromorphic Object Detection: An In-Depth Study and Future Directions cs.CVPDF

Jianing Li, Dianze Li, Arren Glover, Xiaopeng Fan, Guoqi Li

TL;DR: 本文是一篇关于神经形态物体检测的综述与基准测试论文。作者系统性地回顾了该领域的研究现状，包括问题定义、数据集、评估指标以及现有方法，并对代表性模型进行了评估分析。最后，论文讨论了未解决的问题并提出了未来的研究方向。

Details

Motivation: 传统帧式相机在高速运动模糊或低光环境下进行物体检测面临重大挑战。神经形态相机能提供高时间分辨率和宽动态范围的异步视觉流，为这些挑战性条件下的物体检测提供了有前景的解决方案。然而，该领域仍缺乏深入理解和标准化基准来评估进展和应对关键挑战。

Result: 论文对广泛的代表性神经形态物体检测模型进行了评估，并提供了比较结果的详细分析。这为领域内的进展评估建立了一个基准。

Insight: 论文的创新之处在于提供了一个全面的综述和基准测试框架，系统地探索了从事件表示、时序建模到多模态融合、异步处理、低延迟和能效计算等多个技术视角。这为研究人员提供了宝贵的资源，并有望指导神经形态物体检测领域的未来发展。

Abstract: Conventional frame-based cameras face significant challenges in detecting objects under high-speed motion blur or in low-light environments. Neuromorphic cameras provide asynchronous visual streams with high temporal resolution and a wide dynamic range, offering a promising solution for object detection under challenging conditions. Despite the development of numerous models and the emergence of various applications in neuromorphic object detection, there is still a lack of deep understanding and standardized benchmarks to assess progress and address key challenges. In this paper, we provide a comprehensive survey and benchmark of existing neuromorphic object detection algorithms. Specifically, we first present a problem description, review the available datasets, and revisit the evaluation metrics. We then explore existing neuromorphic object detection approaches from various perspectives, including event representation, temporal modeling, multimodal fusion, asynchronous processing, low-latency processing, and energy-efficient computing. Furthermore, we evaluate a wide range of representative neuromorphic object detection models and offer detailed analyses of the comparative results. Finally, we discuss unresolved issues in neuromorphic object detection and propose potential future research directions. We hope this survey and benchmark will be a valuable resource for researchers and provide guidance for future advancements in neuromorphic object detection.

[85] SketchMamba: A Lightweight State-Space Model for Joint Progressive Sketch Classification and Stroke Auto-Completion cs.CVPDF

Kavish Jhaveri, Arya Shah

TL;DR: SketchMamba是一个轻量级的状态空间模型，用于联合渐进式草图分类和笔划自动补全。它通过一个单一的因果序列模型，在草图绘制过程中实时分类任何部分前缀，并同时生成其后续笔划。

Details

Motivation: 现有矢量草图模型将识别和生成视为独立任务，无法满足流式界面在草图绘制过程中实时理解的需求。

Result: 在Quick, Draw!数据集的58类子集上，SketchMamba实现了94.93%的最终步准确率和0.706的渐进准确率AUC，在绘制70%笔划时达到其最终准确率的90%。在同等计算预算下，其155万参数的骨干网络性能与因果Transformer相当，并优于循环和卷积基线。

Insight: 创新点在于将密集的每步分类损失应用于选择性状态空间骨干，使单一因果隐藏状态能够统一渐进式识别和自回归生成，无需辅助编码器或任务特定分支。密集监督机制是驱动其早期预测能力的关键。

Abstract: Existing vector-sketch models treat recognition and generation as separate tasks, leaving a gap for streaming interfaces that must understand a drawing as it is being made. We present SketchMamba, a single causal sequence model that continuously classifies a sketch from any partial prefix while simultaneously generating its continuation. We achieve this by applying a dense per-step classification loss to a selective state-space backbone. Evaluated on a 58-class subset of the Quick, Draw! dataset, SketchMamba yields 94.93% final-step accuracy and a progressive-accuracy Area Under the Curve (AUC) of 0.706, crossing 90% of its final accuracy by the time 70% of the strokes are drawn. In a matched-budget comparison, the 1.55 million-parameter backbone ties a causal Transformer while outperforming recurrent and convolutional baselines. Ablations confirm that the dense supervision regime, rather than the architecture alone, drives the early-prediction capability. The results demonstrate that a single causal hidden state can unify progressive recognition and autoregressive generation without auxiliary encoders or task-specific branching.

[86] ConFusion: Continuous Fusion Space Learning for Fine-Grained Controllable Infrared and Visible Image Fusion cs.CVPDF

Guo Yurong, He Yufei, Li Yonghao, Chang Dongliang, Zhang Ke

TL;DR: 本文提出ConFusion框架，用于实现细粒度可控的红外与可见光图像融合。该方法通过高斯条件空间感知调制学习连续融合空间，利用双分支架构解耦模态不变与模态特定特征，并结合文本引导的语义对齐与基于Grounded SAM的实例掩码，实现实例级的精细调控。推理时，多模态大语言模型将用户意图解析为调制变量以指导融合过程。

Details

Motivation: 现有可控红外-可见光图像融合方法通常依赖预定义的离散控制条件，导致融合空间稀疏，无法满足细粒度调制需求。本文旨在解决这一问题，实现更灵活、精细的实例级可控融合。

Result: 大量实验表明，ConFusion在多个融合质量和下游任务指标上均达到了最先进的性能水平。

Insight: 核心创新在于提出了学习连续融合空间的方法，通过高斯条件实例调制变量与实例掩码的结合实现细粒度控制，并利用多模态大语言模型解析用户意图，增强了融合的灵活性与语义一致性。

Abstract: Controllable infrared-visible image fusion aims to integrate complementary thermal and structural information with flexible region-aware modulation, producing fused images that adapt to diverse user requirements and downstream tasks. However, existing methods typically rely on predefined discrete control conditions, leading to a sparse space that fails to support fine-grained modulation demands. To address this, we propose ConFusion, a novel framework that learns the continuous fusion space via Gaussian-conditioned spatial-aware modulation, enabling instance-level fine-grained controllable infrared and visible image fusion. ConFusion employs a dual-branch architecture to disentangle modality-invariant and modality-specific representations under joint reconstruction and text-guided semantic alignment. Gaussian-conditioned instance modulation variables coupled with Grounded SAM-based instance masks guide instance-level fine-grained modulation through the Mask-Guided Specific Feature Modulator, while the Text-Driven Invariant Feature Enhancer improves semantic consistency and enhances fusion. During inference, the multimodal large language model parses user intents into instance-level modulation variables to guide image fusion. Extensive experiments show that ConFusion achieves state-of-the-art performance across multiple metrics in both fusion quality and downstream tasks, while supporting fine-grained controllable image fusion. Our code is available at https://github.com/HeyufeiAnto/Confusion

[87] JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents cs.CVPDF

Yunlong Lin, Zixu Lin, Zhaohu Xing, Biqiang Li, Chenxin Li

TL;DR: 本文提出了JarvisHub，一个面向画布原生多模态创意代理的开放框架。该框架将可编辑画布作为用户工作空间、代理的外部记忆、动作空间和共享项目状态，通过三层架构（画布状态、协议桥接、代理运行时）支持代理在可检查和可编辑的创意状态下进行长程多模态创作。

Details

Motivation: 现有基于提示、聊天或节点的生成系统仅部分支持创意工作所需的完整项目状态（如草稿、版本、工具动作、反馈等），且近期商业系统多为封闭架构，难以研究代理如何表示上下文、选择工具和维持一致性。

Result: 摘要未提及具体的定量实验结果或基准测试，但指出该设计使创意代理超越了孤立工具使用，实现了持续、人类可引导的创意自动化。

Insight: 核心创新在于将画布作为统一的创意项目状态表示（包含多模态工件、依赖、版本和反馈），并通过开放的三层架构支持代理在可检查、可编辑的状态下进行长程规划和修订，促进了人类与代理在创意过程中的持续协作与干预。

Abstract: Creative AI is moving from single-step asset generation toward long-horizon multimodal production. Although recent generative models can synthesize high-quality images, videos, audio clips, UI elements, storyboards, slides, and other creative assets, real-world creative work requires more than isolated prompt-output interactions. It involves references, drafts, alternatives, edits, failed attempts, version relations, tool actions, evaluation signals, and human feedback, which together form an evolving project state. Existing prompt-based, chat-based, and node-based generation systems only partially support this state, as they often discard intermediate context, rely on linear conversations, or require manually specified workflows. Recent commercial systems indicate a shift toward agent-assisted creative production, but their closed architectures make it difficult to study how agents represent context, choose tools, revise artifacts, recover from failures, and maintain consistency over time. To address this gap, we introduce JarvisHub, a canvas-native creative agent harness for long-horizon multimodal creation. JarvisHub treats an editable canvas as the user workspace, the agent’s external memory, action space, and shared project state, representing multimodal artifacts, dependencies, versions, and feedback as typed canvas nodes and links. Through a three-layer architecture of canvas state, protocol bridge, and agent runtime, JarvisHub enables agents to act within an inspectable and editable creative state. This design moves creative agents beyond isolated tool use toward sustained, human-steerable creative automation, where agents can progressively plan, generate, revise, and organize multimodal projects while users remain able to inspect, guide, and intervene throughout the process.

[88] PathSelect: Sequential Token Selection for Whole Slide Pathology cs.CVPDF

Jingzhi Chen, Landi He, Zehong Chen, Peihang Wu, Lijian Xu

TL;DR: 本文提出PathSelect，一种用于千兆像素全玻片病理图像（WSI）的序列化令牌选择框架，以解决视觉语言模型（VLMs）因序列过长导致的计算瓶颈。该方法将令牌剪枝重新定义为序列选择过程，通过可微分的Soft Top-K操作和去噪器在训练中学习最优路由策略，推理时仅依赖训练好的评分器执行确定性的Hard Top-K操作，实现自适应数据依赖的轨迹终止，显著加速生成处理。

Details

Motivation: 现有方法主要依赖空间采样或无训练剪枝，可能稀释弱但信息丰富的信号，导致关键诊断证据丢失，因为病理线索在空间上分散。本文旨在让模型自主学习最优路由策略，而非依赖静态启发式方法。

Result: 在SlideBench（TCGA）基准测试中，该方法在最大约束K=128下平均仅使用44.86个令牌，实现了74.00%的整体准确率，相对于未压缩基线平均减少了约36.6倍的空间令牌，同时持续优于基于采样的方法。

Insight: 创新点包括将WSI令牌剪枝重新定义为序列选择过程，引入可微分的Soft Top-K操作和方差保持噪声门来调制信息流，以及使用对角线注意力去噪器恢复扰动表示而不泄露语义。从客观角度看，该框架作为插件集成到预训练模型中，保持编码器和语言模型冻结，实现了高效的自适应令牌选择。

Abstract: Gigapixel Whole-Slide Images (WSIs) present a fundamental computational bottleneck for vision-language models (VLMs) due to extreme sequence lengths. Existing approaches predominantly rely on spatial sampling or training-free pruning, which risk diluting weak but informative signals, leading to the loss of critical diagnostic evidence due to the spatially diffuse nature of pathological cues. We reformulate WSI token pruning as a sequential selection process, enabling the model to autonomously learn an optimal routing strategy rather than relying on static heuristics. We herein propose a decoupled routing framework integrated as an active plugin into the fully pre-trained SlideChat base model, leaving both the slide encoder and large language model frozen. To provide continuous gradients for the non-differentiable pruning operation during training, we introduce PathSelect. PathSelect employs a variance-preserving noise gate to modulate each patch’s information flow via a differentiable Soft Top-K operator, paired with a diagonal-attention Denoiser that recovers the perturbed representations without semantic leakage. At inference, the PathSelect module is entirely detached. Relying solely on the trained Scorer, a deterministic Hard Top-K operator executes adaptive, data-dependent trajectory termination, significantly accelerating downstream generative processing with exceptionally low sequential token selection latency. Driven by an empirical average of only 44.86 tokens under a maximum constraint of K = 128, our framework achieves 74.00% overall accuracy on SlideBench (TCGA), representing an approximate 36.6x spatial token reduction relative to the uncompressed baseline average while consistently outperforming sampling-based counterparts.

[89] GRAPE: Graduated Routing for Articulated Portrait mesh Estimation cs.CVPDF

Yunfei Liu, Lijian Lin, Ye Zhu, Yu Li

TL;DR: 本文提出了GRAPE方法，用于解决现有3D肖像网格估计方法在头部姿态、颈部运动、面部表情与下颌关节解耦等方面的局限性。该方法构建了一个包含显式躯干-头部运动链的肖像参数模型（PPM），并设计了一个渐进式解剖对齐网络（PAA），通过多源监督进行训练。实验表明，GRAPE在网格恢复质量、姿态对齐和下颌-表情解耦方面优于现有方法，并能提升音频驱动说话人生成和3D肖像生成等下游任务的效果。

Details

Motivation: 现有方法主要依赖3D形变模型（3DMM），但面部中心模型受限于‘漂浮头部’假设，将头部姿态与全局旋转混为一谈；身体中心模型则缺乏高保真的面部表情能力。此外，当前方法难以将下颌关节运动与表情形变分离，过度依赖表情来模拟张嘴动作，导致单目肖像恢复在表示、监督和解剖参数估计方面存在困难。

Result: 实验表明，GRAPE在肖像网格恢复质量、姿态对齐以及下颌与表情的解耦方面优于现有方法。

Insight: 创新点在于构建了包含显式躯干-头部运动链的肖像参数模型（PPM），并提出了一个遵循肖像解剖先验的、由粗到细的渐进式解剖对齐网络（PAA），结合了多源监督策略，有效整合了面部与身体的表示，解决了姿态、表情与关节运动的解耦问题。

Abstract: Articulated portrait mesh estimation is fundamental to 3D understanding, avatar generation, and immersive interaction. Existing approaches primarily rely on 3D Morphable Models (3DMMs). However, face-centric models suffer from the “floating head” assumption, conflating head pose with global rotation due to the lack of neck kinematics. Conversely, body-centric models lack high-fidelity facial expression capabilities. Furthermore, current methods struggle to disentangle jaw articulation from expression blendshapes, often over-relying on expressions for mouth opening. These limitations make monocular portrait recovery difficult across representation, supervision, and anatomical parameter estimation. To address these limitations, we introduce GRAPE(Graduated Routing for Articulated Portrait mesh Estimation). We build a Portrait Parametric Model (PPM) with an explicit torso-to-head kinematic chain and a canonical injection step to merge FLAME and the SMPL-X torso. We propose a Progressive Anatomical Alignment (PAA) network, which is composed of a pretrained portrait encoder, a Graduated-Mask Router, and coarse-to-fine experts that follow the portrait anatomical prior. We then train this network with multi-source supervision that combines sparse anatomical keypoints, feature distillation, foreground mask constraints, and relative geometry constraints. Experiments show that GRAPE improves portrait mesh recovery quality, pose alignment, and jaw–expression disentanglement over prior methods. We also demonstrate that our method can benefit the downstream tasks of audio-driven talking-head generation and 3D portrait generation.

Yu Zhang, Wenda Zhao, Haojun Tang, Haipeng Wang

TL;DR: 本文提出了一种用于多模态遥感图像生成的对比参数解缠框架，能够从单一文本提示生成光学、红外和合成孔径雷达（SAR）等多种模态的图像，并确保语义一致和结构对齐。该方法通过对比参数解缠模块在正交核心子空间内分离共享语义和模态特定属性，并采用解缠优化策略和查询-键结构转移机制，实现了高质量、语义一致且结构对齐的多模态图像生成。

Details

Motivation: 现有遥感图像生成方法主要局限于单模态合成，未能充分利用多模态图像中固有的互补信息。为了解决这一局限性，本文旨在开发一种能够从单一文本提示生成多种模态遥感图像的方法，以利用多模态数据的互补优势。

Result: 大量实验表明，该方法在生成质量、语义一致性和结构对齐方面优于最先进的遥感图像生成方法，并在下游目标分类任务中取得了优异的性能。

Insight: 创新点在于提出了对比参数解缠模块和策略，在参数层面分离共享语义和模态特定属性，以及设计了查询-键结构转移机制，在推理过程中联合建模多模态采样轨迹以实现结构对齐。这为多模态生成任务提供了一种有效的参数解缠和结构对齐方法。

Abstract: Existing remote sensing image generation methods are largely confined to single-modality synthesis and therefore fail to exploit the complementary information inherent in multimodal imagery. To address this limitation, we propose a contrastive parameter disentanglement framework for multimodal remote sensing image generation, which generates semantically consistent and structurally aligned images across multiple modalities, including optical, infrared, and synthetic aperture radar (SAR), from a single text prompt. Specifically, we introduce a contrastive parameter disentanglement module that disentangles shared semantics from modality-specific attributes at the parameter level within an orthogonal core subspace. Based on this module, we develop a disentangled optimization strategy that first constrains the parameter matrix A of the LoRA adapter to capture modality-invariant semantics through a multimodal contrastive objective and then guides multiple parameter matrices B to learn modality-specific attributes under text conditioning. This strategy enables the simultaneous generation of multimodal images with consistent semantic content and distinct modality characteristics. Furthermore, to ensure structural alignment across the generated images, we devise a query-key structure transfer mechanism that jointly models multimodal sampling trajectories during inference by transferring structural correlation priors from an anchor modality to the remaining modalities. Extensive experiments demonstrate that our method outperforms state-of-the-art remote sensing image generation approaches in terms of generation quality, semantic consistency, and structural alignment, while also achieving superior performance in the downstream object classification task.

[91] RRTrack: Robust and Recoverable Object 6D Pose Tracking for Dynamic Scenes cs.CV | cs.ROPDF

Junyue Li, Ye Zheng, Yifan Chen, Zhe Sun, Xuelong Li

TL;DR: 本文提出了RRTrack，一种鲁棒且可恢复的物体6D位姿跟踪方法，旨在解决动态和遮挡场景下物体快速运动和完全遮挡导致的跟踪失败问题。该方法通过结合基于记忆的视频对象分割（VOS）与6D位姿细化的2D-6D闭环跟踪策略，以及一个基于DINOv2的双库模板匹配模块，实现了高效的目标恢复和实时跟踪。

Details

Motivation: 现有逐帧估计器计算成本高，而当前跟踪器因依赖连续可见性，在快速运动和完全遮挡场景下表现不佳。本文旨在开发一个高效、可恢复的6D位姿跟踪器，以应对这些挑战。

Result: 在作者提出的合成RGB-D基准测试（包含三种具有快速运动和完全遮挡的机器人场景）上，RRTrack相比FoundationPose，将equal-subset mean ADD-S AR提升了66.3%，ADD-S AUC提升了65.7%，同时达到55.2 FPS的实时速度。真实世界实验进一步验证了其在噪声感知条件下的鲁棒性。

Insight: 创新点在于提出了2D-6D闭环跟踪策略，将2D分割与6D几何验证结合以维持跟踪鲁棒性；并设计了基于DINOv2的双库模板匹配模块，联合利用离线合成模板和在线观测锚点，实现了目标丢失后的高效恢复，同时保持实时性能。

Abstract: Robust object 6D pose tracking is critical for robotic systems operating in dynamic and occluded scenes. Per-frame estimators are accurate but computationally expensive, while current trackers struggle with fast motion and complete occlusion due to their reliance on continuous visibility. To address these challenges, we present RRTrack, an efficient, recoverable object 6D pose tracker that enables robust tracking through fast motion and target disappearance–reappearance. RRTrack introduces a 2D–6D closed-loop tracking strategy that integrates memory-based video object segmentation (VOS) with 6D pose refinement. The 2D branch maintains target localization, and the 6D branch verifies geometric consistency before memory updates. In addition, a DINOv2-based dual-bank template matching module is developed to recover lost targets by jointly exploiting offline synthetic templates and online observation anchors while maintaining real-time efficiency. We also introduce a synthetic RGB-D benchmark comprising three robotic scenarios with fast motion and full occlusion. Experimental results on the synthetic benchmark demonstrate that RRTrack improves equal-subset mean ADD-S AR by 66.3% and ADD-S AUC by 65.7% over FoundationPose while achieving 55.2 FPS. Real-world experiments further validate the robustness of RRTrack under noisy sensing conditions. Project page: https://github.com/7kevin24/RRTrack

[92] Perturbation-Aware Diffusion-Guided Hybrid Segmentation for Robust and Annotation-Efficient Plant Stress Phenotyping cs.CVPDF

Gurbhit Chaurakoti, Soumyashree Kar

TL;DR: 本文提出了一种扰动感知的扩散引导混合分割框架，用于农业图像中鲁棒且标注高效的植物胁迫表型分析。该框架结合了U-Net、DeepLabV3+和SegFormer等骨干网络生成粗分割掩码，并利用去噪扩散概率模型、潜在扩散或语义引导扩散进行细化。研究通过架构筛选、边界约束优化、扰动引导重训练、低数据评估和跨域适应等实验，验证了其在PlantSegV3数据集上的有效性。

Details

Motivation: 农业图像语义分割的实际部署需要应对外观扰动、标注有限和跨域偏移等挑战，而现有方法通常在域内协议下评估，缺乏鲁棒性。

Result: 在PlantSegV3数据集上，最佳混合模型实现了71.83%的细化平均交并比和26.10%的细化边界F1分数，且在监督大幅减少时保持稳定，展示了强标注效率；扰动分析识别了灰度转换、雾、粗丢弃和阴影为最具破坏性的外观变化，相应的增强策略显著提升了重训练时的鲁棒性；适应后的模型在有限目标监督下能有效迁移到外部农业数据集。

Insight: 创新点在于将扩散模型作为分割细化器与不同骨干网络配对，并结合边界感知优化和扰动感知重训练，以提供可迁移的结构先验，从而在现实资源与分布约束下提升结构 delineation 和鲁棒性；客观来看，该方法通过系统性的混合架构筛选和针对性增强，为标注稀缺和域偏移场景下的农业图像分割提供了实用解决方案。

Abstract: Semantic segmentation in agricultural imagery is often evaluated under in-domain protocols, yet practical deployment requires robustness to appearance perturbations, limited annotations, and cross domain shift. This paper presents a diffusion-guided hybrid segmentation framework in which U-Net, DeepLabV3+, and SegFormer backbones generate coarse masks that are refined by Denoising Diffusion Probabilistic Models (DDPM), latent diffusion, or semantic-guided diffusion. The framework is evaluated through a 3x3 architectural screening study on PlantSegV3, followed by boundary-constrained optimization, perturbation-guided retraining, low-data evaluation, constrained hyperparameter screening, and controlled cross-domain adaptation. On PlantSegV3, the best selected hybrid model achieves 71.83% refined mean Intersection-over-Union (mIoU) and 26.10% refined Boundary-F1, and the selected models remain stable under substantially reduced supervision, demonstrating strong annotation efficiency. Perturbation analysis identifies grayscale conversion, fog, coarse dropout, and shadow as the most disruptive appearance shifts, and the resulting augmentation policy substantially improves robustness during retraining. The adapted models further show effective transfer to external agricultural datasets under limited target supervision, indicating that diffusion refinement and boundary-aware optimization provide transferable structural priors. Overall, the results show that carefully matched backbone-refiner pairings, combined with perturbation-aware retraining, can improve structural delineation and robustness under realistic resource and distribution constraints.

[93] GNM Head: A Generative aNthropometric Model of the human head cs.CV | cs.GRPDF

Stylianos Ploumpis, Jan Bednarik, Gaspard Zoss, Ruslan Guseinov, Luca Prasso

TL;DR: 本文提出了一种名为GNM Head的新型参数化头部模型，它覆盖了头部、面部、颈部、眼球、牙齿和舌头等完整解剖结构，旨在解决现有模型解剖范围有限和几何质量低的问题。该模型基于高分辨率3D扫描和高质量艺术家制作样本构建，并在拟合3D面部扫描任务上实现了SOTA性能。

Details

Motivation: 现有公开参数化头部模型通常解剖范围有限（仅建模外部几何，忽略口腔内和眼部结构），且因输入数据质量低而导致几何质量下降，这限制了其在计算机视觉、图形学以及作为生成式大视觉模型条件信号的应用。

Result: 论文展示了GNM模型在拟合目标3D面部扫描任务上达到了SOTA（State-of-the-Art）性能。

Insight: 主要创新点在于构建了一个覆盖完整头部解剖结构（包括眼球、牙齿、舌头）的综合性参数化模型，并通过结合高分辨率3D扫描数据与高质量艺术家制作样本来提升几何保真度。其模型架构包含针对眼部和口腔内结构的专门子模型，也是一个值得借鉴的设计思路。

Abstract: Parametric models of the human head are essential tools traditionally used in computer vision and graphics for animation, rendering, and reconstruction. More recently, they serve as crucial conditioning signals within generative large vision models, allowing for tight spatial control of generated imagery. However, existing publicly available models are typically limited in anatomical scope, modeling only outer geometry while ignoring intra-oral and ocular structures, and frequently suffer from reduced geometric quality stemming from low-fidelity input datasets. In this report we introduce a new parametric model dubbed Generative aNthropometric Model (GNM), named as a homophone of the human genome. GNM encompasses the head, face, neck, eyeballs, teeth, and tongue, and it is built on an extensive database of high-resolution 3D scans combined with high-quality anatomy specific artist-made samples. This report details the data provenance, the model architecture including the specialized sub-models for the ocular and intra-oral structures, and shows its SotA performance on fitting target 3D face scans. To foster community innovation, the complete GNM framework is made publicly available.

[94] Parameter-Efficient Adaptation of SAM3 for Prompt-Driven Surgical Concept Segmentation cs.CVPDF

Changjing Liu, Yiming Huang, Beilei Cui, Liangjing Shao, Long Bai

TL;DR: 本文提出了一种参数高效的SAM3（Segment Anything Model 3）适配方法，用于解决SAM3在手术图像分割中因领域差异导致的性能下降问题。该方法通过注入低秩适配器（LoRA）到提示编码器、检测器和跟踪器中，同时完全冻结视觉主干，仅优化总参数的0.98%，实现了在单个消费级GPU上的训练。实验表明，该方法在手术概念分割任务上优于零样本SAM3及其他主流基线，并能直接应用于下游机器人手术场景重建和物理模拟流程。

Details

Motivation: SAM3等提示驱动基础模型在自然图像上分割性能强，但手术数据存在领域差异，导致分割精度下降；现有医学SAM方法需要全参数微调，计算成本高、效率低。

Result: 在手术概念分割任务上，该方法在综合实验中一致优于零样本SAM3和其他主流基线，生成的掩码可直接用于下游机器人手术场景重建和物理模拟。

Insight: 创新点在于将参数高效的LoRA适配技术应用于SAM3，通过仅微调少量参数（0.98%）来适应手术领域，在保持高性能的同时大幅降低计算需求，实现了在消费级硬件上的高效训练和部署。

Abstract: Efficient surgical segmentation empowers clinical diagnosis, intraoperative monitoring, and downstream robotic pipelines for reconstruction and simulation. Although prompt-driven foundation models like Segment Anything Model 3 (SAM3) achieve strong segmentation performance on natural images, surgical data exhibits domain gaps against its pre-training data, resulting in degraded segmentation accuracy. Furthermore, existing medical SAM methods require full-parameter fine-tuning, incurring heavy computational consumption and low efficiency. To address these limitations, this work proposes a parameter-efficient Low-Rank Adaptation (LoRA) adaptation of SAM3 for surgical concept segmentation. We inject low-rank adapters into the prompt encoder, detector and tracker while fully freezing the vision backbone, which only optimizes 0.98% of the total model parameters and supports training on a single consumer GPU. Comprehensive experiments demonstrate that our method consistently outperforms zero-shot SAM3 and other mainstream baselines, and the generated segmentation results can be directly deployed to support downstream robotic surgical scene reconstruction and physical simulation pipelines.

Jianhan Lin, Yuchu Qin, Jiateng Yuan, Wenbo Zhang, Shuai Gao

TL;DR: 本文提出了DAP-Pose，一个用于鲁棒多模态位姿估计的端到端模型。它通过双层次跨模态融合模块整合视觉、惯性和GNSS数据的语义与几何运动线索，并利用深度时序对齐模块处理异步数据流，同时引入物理感知约束来保证运动一致性。

Details

Motivation: 在复杂环境中，为自动驾驶和移动机器人系统实现鲁棒且准确的多传感器位姿估计是核心挑战。现有方法在处理异步传感器数据时存在困难，且容易产生漂移。

Result: 在公开的KITTI基准数据集上，DAP-Pose达到了最先进的性能，平均平移误差为1.31%，旋转误差为0.46度，并且在严重人为注入的时序错位下仍能保持鲁棒性能。

Insight: 创新点在于提出了一个显式在潜在空间对齐异步数据流的深度时序对齐模块，以及结合了流形几何和GNSS引导绝对度量尺度的物理感知约束，从而实现了无需严格硬件同步的连贯运动建模和漂移抑制。

Abstract: Robust and accurate pose estimation with multi-modal sensors is fundamental for autonomous vehicles and mobile robotic systems in complex environments. In this paper, we propose DAP-Pose, a unified end-to-end model for robust multi-modal pose estimation. DAP-Pose introduces a Bi-level Cross-modal Fusion (BCF) module that captures complementary semantic and geometric motion cues from visual, inertial, and GNSS measurements. To handle temporal offsets, we designed a Deep Temporal Alignment (DTA) module that explicitly aligns asynchronous streams in latent space, enabling coherent motion modeling without strict hardware synchronization. Furthermore, we incorporate physics-aware constraints via manifold geometry and GNSS-guided absolute metric scale, enforcing motion consistency and mitigating drift. Experiments upon the public KITTI benchmark dataset were conducted to evaluate the performance of DAP-Pose against existing methods. DAP-Pose achieved the state-of-the-art performance, with the lowest average translation error ($t_{rel}$) of 1.31% and rotation error ($r_{rel}$) of 0.46$^{\circ}$. Furthermore, it accurately estimates poses and maintains robust performance under severe artificially injected temporal misalignment.

[96] PathScale-R1: Cross-scale Reasoning for Pathological Image Analysis cs.CV | cs.AIPDF

Chi Phan, Tianyi Zhang, Yufeng Wu, Qiaochu Xue, Jiajie Zhang

TL;DR: 该论文提出了PathScale-R1，一个用于病理图像分析的跨尺度推理框架，通过构建高质量的PathScale-VQA基准测试集，并采用对抗性文本筛选和结构控制干扰项采样策略来抵抗捷径学习，结合难度驱动的推理蒸馏和尺度感知推理结构的强化学习优化模型，实现了在跨尺度病理视觉问答任务上的最先进性能。

Details

Motivation: 现有病理学基准测试和视觉语言模型主要在单尺度设置下开发，难以学习具有临床意义的多尺度推理，且视觉问答任务易受纯文本或表面视觉捷径影响，导致视觉理解评估不可靠。

Result: PathScale-R1在跨尺度推理任务上取得了最先进的性能，并在传统单尺度病理VQA任务上表现出有效的迁移能力。

Insight: 创新点包括设计抵抗捷径学习的基准构建策略（对抗性文本筛选和结构控制干扰项采样），以及通过难度驱动的推理蒸馏和尺度感知推理结构的强化学习来优化跨尺度推理模型，强调了在病理图像分析中整合多尺度视觉证据的重要性。

Abstract: Pathological diagnosis is inherently multi-scale, requiring the integration of global tissue architecture at low magnification with cellular morphology at higher magnification. However, existing pathology benchmarks and vision-language models (VLMs) are still largely developed under single-scale settings, limiting their ability to learn clinically meaningful multi-magnification reasoning. Moreover, naively constructed visual question answering (VQA) tasks may be susceptible to text-only or superficial visual shortcuts, leading to unreliable assessments of visual understanding. To address these limitations, we introduce a benchmark and training framework for shortcut-resistant cross-scale pathology reasoning. We design an Adversarial Text-only Screening strategy for semantic reasoning questions and a Structure-controlled Distractor Sampling strategy for visual grounding questions, encouraging models to rely on cross-scale visual evidence. Based on this pipeline, we construct PathScale-VQA, a high-quality cross-scale pathology VQA benchmark with 10,373 multiple-choice questions grounded in 1,368 diagnostic paths across multiple magnification levels. Building on the semantic reasoning set, PathScale-R1 is optimized through Difficulty-driven Reasoning Distillation supervised fine-tuning followed by reinforcement learning with a Scale-aware Reasoning Structure reward, which encourages the use of evidence across magnifications. Extensive experiments demonstrate state-of-the-art performance of PathScale-R1 on cross-scale reasoning tasks and effective transfer to conventional single-scale pathology VQA. Our code is available at https://github.com/iMVR-PL/PathScale-R1.

[97] Consistent Evidence, Robust Recognition: Faithful Attribution Regularization under Geometric Transformations cs.CVPDF

Xianghao Jiao, Ruoyu Chen, Wei Wang, Jiazi Hu, Jiawei Liang

TL;DR: 本文提出了一种无需标注的归因正则化框架，旨在通过子模搜索提取紧凑、类别区分的证据作为监督信号，并结合路径一致性和终止对齐损失来正则化归因过程，从而提升模型在几何变换下的归因一致性和证据可靠性。

Details

Motivation: 现有归因方法（如Grad-CAM）在保持标签的几何变换下可能存在不一致性，表明模型可能依赖对变换敏感的证据，这促使了归因正则化的需求；但现有方法因归因的忠实性有限，其一致性监督可能无法保证决策过程的一致性，导致变换鲁棒性问题未解决。

Result: 在ImageNet-100上，该方法显著提升了ViT-B/16的归因稳定性、插入和删除分数，仅导致0.28个百分点的精度下降；在ImageNet-1K上，提升了ResNet-50和ConvNeXt-B在变换输入上的准确率，同时将干净准确率下降限制在0.30个百分点，实现了更一致的证据依赖和最小性能损失。

Insight: 创新点在于通过子模搜索生成忠实、紧凑的证据作为监督，并设计了路径一致性和终止对齐损失来对齐搜索轨迹和终止条件，从而可微地正则化归因和证据选择过程，提高了归因的忠实性和模型对几何变换的鲁棒性。

Abstract: Attribution methods are widely used to characterize the evidence underlying model predictions, yet their potential to improve model behavior remains underexplored. Attribution inconsistency under label-preserving geometric transformations may indicate transformation-sensitive evidence reliance, motivating attribution regularization. However, such supervision is valid only when attribution faithfully reflects the evidence driving predictions. Existing self-supervised methods typically align gradient-based maps such as Grad-CAM, whose limited faithfulness means that attribution consistency need not imply consistency of the underlying decision process, leaving transformation robustness unresolved. We propose an annotation-free attribution regularization framework based on submodular search over image regions. By measuring how candidate subsets affect model outputs, the search extracts compact, class-discriminative evidence as search-derived supervision. We further introduce a submodular ranking loss with path-consistency and termination-alignment terms that respectively align spatially corresponding candidate rankings along paired search trajectories and encourage the transformed trajectory to satisfy the stopping criterion at the target terminal step. The loss provides a differentiable surrogate for regularizing both final attributions and the otherwise discrete evidence-selection process. Experiments on ImageNet-100 show that our method substantially improves attribution stability, Insertion, and Deletion on ViT-B/16 with only a 0.28-point accuracy drop, with similar gains on ViT-L/16. On ImageNet-1K, it improves transformed-input accuracy on ResNet-50 and ConvNeXt-B while limiting the clean-accuracy drop to 0.30 points, demonstrating more consistent evidence reliance with minimal performance loss. Code will be released soon.

[98] STEER: Steerable Dyadic Head Avatars cs.CVPDF

Kartik Teotia, Helge Rhodin, Hyeongwoo Kim, Marc Habermann, Christian Theobalt

TL;DR: STEER提出了一种可控的3D对话头部化身运动先验模型，将对话行为分解为凝视、头部节奏和情感的显式控制，允许用户引导化身如何倾听、反应和参与对话。该方法通过从野外对话视频中恢复行为伪标签来训练一个因果流匹配变换器，并嵌入到照片级真实感的高斯头部化身管道中，实现了高质量、可控的动画生成。

Details

Motivation: 现有语音驱动面部动画方法主要关注唇部同步和音频条件运动生成，但将对话行为视为音频的副产品或仅提供粗略的情感控制，导致凝视、头部节奏和情感等关键非语言通道难以显式控制。

Result: STEER在运动质量、动态性和多样性方面优于最近的对话运动基线，在伙伴耦合方面保持竞争力，并支持凝视、头部节奏和情感编辑以及交互式实时部署。

Insight: 创新点在于将对话行为显式分解为可控制的组件（凝视、节奏、情感），并提出了从野外视频中恢复行为伪标签的追踪与标注流程，以及一个无需重新训练底层化身模型即可驱动高保真高斯头部化身的映射方法。

Abstract: Facial movement and expression are central to face-to-face communication, conveying turn-taking, attention, agreement, and engagement alongside speech. While speech-driven facial animation has made strong progress in lip synchronization and audio-conditioned motion generation, most methods treat conversational behavior as an emergent byproduct of audio, or expose only coarse sequence-level affect control. As a result, key non-verbal channels such as gaze contact and aversion, rhythmic head motion, and emotion remain difficult to explicitly control. We present STEER, a controllable 3D dyadic motion prior for reactive conversational head avatars. STEER factorizes conversational behavior into explicit controls for gaze, head rhythm, and emotion, allowing users to steer how an avatar listens, reacts, and engages with a conversation partner. Since temporally aligned annotations for these behaviors are not available in public dyadic corpora, we introduce a tracking and annotation pipeline that recovers behavioral pseudo-labels from in-the-wild dyadic video. A causal flow-matching transformer then learns partner-aware target motion conditioned on audio, partner motion, emotion and the proposed behavioral controls. We further embed STEER in a photorealistic avatar pipeline by extending a Universal Gaussian Head-Avatar Prior with a learned mapping from tracked parametric motion into its avatar-driving space. This enables controllable animation of high-fidelity Gaussian head avatars without re-training the underlying avatar model. STEER outperforms recent dyadic motion baselines on motion quality, dynamics, and diversity, remains competitive on partner coupling, and enables gaze, head-rhythm, and emotion edits together with an interactive live deployment. We make our code and dataset annotations available at our webpage.

[99] OmniCache: Multidimensional Hierarchical Feature Caching For Diffusion Models cs.CVPDF

Zhaoyuan He, Muhammad Muaz, Lili Qiu

TL;DR: 本文提出了OmniCache，一种多维分层特征缓存框架，用于加速高分辨率图像和视频扩散模型（如SD3、FLUX）的推理过程。该方法通过识别并利用扩散模型中间特征中的四种冗余（帧内、帧间、运动和去噪步冗余），设计了一个统一的缓存系统，在无需重新训练的情况下，显著降低了推理延迟。

Details

Motivation: 当前高分辨率图像和视频扩散模型（如SD3、FLUX）推理成本高昂，主要因为需要在许多采样步骤中重复计算注意力密集的去噪器。本文旨在通过利用中间扩散特征中的冗余，而非修改模型权重或重新训练，来解决这一效率低下的问题。

Result: 在SD3、SVD-XT和Latte等模型上，OmniCache分别将推理延迟降低了高达35%、25%和28%，同时在无需训练的条件下保持了视觉保真度和运动连贯性。

Insight: 创新点在于系统性地识别了扩散模型中的四种互补冗余源，并据此设计了统一的、多维度的分层缓存框架（包含Token Cache、Frame Cache、Block Cache和Layered Cache）。其核心在于通过相似性匹配选择可缓存特征、跳过冗余计算并恢复位置一致的缓存激活，从而保留了特征顺序和时空结构，这与简单平均匹配特征的token合并基线方法不同。

Abstract: High-resolution image and video diffusion models, including SD3, FLUX, and recent video diffusion transformers, have substantially improved generative quality but remain expensive at inference time because they repeatedly evaluate attention-heavy denoisers over many sampling steps. We address this inefficiency by exploiting redundancy in intermediate diffusion features rather than changing model weights or retraining. We identify four complementary redundancy sources in image and video generation: intra-frame, inter-frame, motion, and denoising-step redundancy. Based on this analysis, we propose OmniCache, a unified hierarchical caching framework that performs multidimensional feature reuse through Token Cache, Frame Cache, Block Cache, and Layered Cache. Unlike token-merging baselines that average matched features, OmniCache uses similarity matching to select cacheable features, skips redundant computation, and restores positionally consistent cached activations, preserving feature order and spatial-temporal structure. The resulting framework reuses spatial features in temporal layers and temporal features in spatial layers, while Layered Cache captures cross-step redundancy at the model-layer level. Across SD3, SVD-XT, and Latte, OmniCache reduces inference latency by up to 35%, 25%, and 28%, respectively, while maintaining visual fidelity and motion coherence in a training-free setting.

[100] Head Avatars with Dynamic Explicit Hair cs.CV | cs.GRPDF

Vanessa Sklyarova, Haonan Chen, Berna Kabadayi, Tobias Kirschstein, Zicong Fan

TL;DR: 本文提出了一种名为DynHair的新方法，用于从视频中重建具有动态、显式发丝表示的人头化身。该方法结合了基于3D高斯泼溅的结构化头发表示和面部区域建模，通过一个受头部角速度、加速度和相对重力条件约束的时序网络来模拟头发的动态变形，从而实现了对训练视频中头发运动的跟踪以及生成具有可控头发动态的可动画化头像。

Details

Motivation: 动机在于解决人头化身建模中头发动态运动的挑战。与面部区域不同，头发表现出复杂的动态运动效果，难以用静态或附着于表情头部模型的3D高斯来表示，因此需要专门的方法来捕捉和模拟其物理上合理的动态变形。

Result: 实验表明，该方法在头发动态、时间一致性和跨主体泛化方面达到了最先进的性能（SOTA），能够从训练视频数据中恢复头发跟踪，并生成具有可控头发动态的可动画化头像。

Insight: 创新点包括：使用基于角速度、加速度和相对重力的时序网络（LSTM结合FiLM条件调制）来建模发丝的物理合理位移；联合优化头发动态与外观表示以及基于3D高斯泼溅的面部区域表示；通过可微分高斯泼溅结合光度、几何和物理监督进行端到端训练。从客观角度看，该方法将显式发丝表示与动态建模相结合，为头发动画提供了更可控和逼真的解决方案。

Abstract: We present DynHair, a novel method for tracking and modeling dynamic hair for human head avatars. From video input, we reconstruct a dynamic head avatar with an explicit strand-based hair representation using structured 3D Gaussian Splatting. In contrast to the face region of human head avatars, which can be modeled with 3D Gaussians that are attached or generated with respect to some expressive 3D head model, hair is particularly challenging as it exhibits dynamic motion effects. Therefore, we present a novel method that models the dynamic deformations of the hair strands using a temporal network that is conditioned on angular velocity and acceleration of the head, as well as relative gravity. Specifically, an LSTM encodes the motion history and modulates per-point strand features via FiLM conditioning which further used by MLP to produce physically plausible displacements to canonical hairstyle. We jointly optimize this motion and appearance representation of the hair, with a 3DGS-based representation of the face-region, via differentiable Gaussian splatting with photometric, geometric, and physics-based supervision. As a result of our method, we retrieve hair tracking of the training video data and an animatable head avatar with controllable hair dynamics. In our experiments, we demonstrate state-of-the-art performance in terms of hair dynamics, temporal consistency, and generalization across subjects.

[101] Gaze-to-text Generation: Beyond Categorical Decoding of Human Attention cs.CVPDF

Sounak Mondal, Dimitris Samaras, Gregory Zelinsky, Minh Hoai

TL;DR: 本文提出了一种新颖的‘注视到文本生成’学习问题，旨在将人类注视轨迹解码为描述其目标的自然语言，而非预定义的类别。为此，作者引入了首个基于多模态大语言模型的框架Gazette，并利用大语言模型合成‘出声思考’转录本进行指令微调，以过滤个体差异并学习目标相关的时空动态。该方法在多个任务上实现了最先进的注视解码性能。

Details

Motivation: 现有工作将注视解码视为在预定义类别上的判别任务，这限制了其捕捉人类意图丰富细微差别和开放性质的能力。本文旨在将其重新定义为生成式学习问题，以生成自由形式的描述，从而更全面地推断人类目标。

Result: Gazette框架在多个任务的注视解码上实现了最先进的性能，展示了其泛化能力和多功能性。

Insight: 主要创新点在于将注视解码从判别式分类任务重新定义为生成式任务，并利用大语言模型合成‘出声思考’解释来指导模型学习目标相关的注视动态，从而有效过滤个体差异并提升自然语言描述的准确性。

Abstract: We introduce a novel learning problem: decoding gaze into natural language descriptions of human goals across diverse visual tasks. Unlike prior work, which frames gaze decoding as a discriminative task over predefined categories, we formulate it as a generative learning problem: training a model to produce free-form descriptions that capture the rich nuances and open-ended nature of human intentions beyond fixed labels. To this end, we introduce Gazette, the first gaze-to-text decoding framework. Based on multimodal large language models (MLLMs), Gazette learns to decode gaze scanpaths into natural language for goals that may extend beyond categorical labels and require articulation in natural language. To help Gazette filter out individual differences in gaze behavior and learn the goal-specific spatiotemporal dynamics crucial for generating accurate natural language goal descriptions, we propose a novel strategy that leverages the encyclopedic knowledge and reasoning abilities of a large language model to synthesize natural language explanations of goal-directed attentional behavior called think-aloud transcripts. Instruction tuning on these synthetic narratives allows Gazette to achieve state-of-the-art performance in gaze decoding across multiple tasks, demonstrating its generalizability and versatility, thereby enabling gaze to serve as a powerful, non-intrusive cue for inferring human goals and intentions in diverse scenarios.

[102] What Can I Edit? Open-Ended Strategy Discovery and the Emotion Editability Landscape cs.CVPDF

Qing Li, Zeyu Dong, Yin Cui, Chuan Yan, Xiaojiang Peng

TL;DR: 本文提出EmoScope，一个多智能体框架，用于开放式的情绪图像编辑策略发现。它通过情绪条件化的可供性推理，首先发现图像特定的可编辑空间，然后利用语义层次结构平衡内容一致性与情绪表达，最后执行并验证编辑。该方法将任务从’如何编辑’重构为’可以编辑什么’，并支持用户在计划层面进行交互式优化。

Details

Motivation: 现有情感图像编辑方法（包括最近的智能体变体）大多在基于预定义因素分类、知识库或传统编辑模板的有界策略空间中操作，因此常常错过图像特定、基于上下文的策略。本文旨在解决如何识别特定图像对于目标情绪的可供性，从而实现更有效的编辑。

Result: 在一项大规模人类评估中，覆盖所有八个Mikels情绪类别，包含1824对问题和4693个有效回答，参与者平均以88.1%的比例偏好EmoScope，优于两个竞争基线。归因分析进一步表明EmoScope选择了适应目标情绪的策略，而非应用统一模板。

Insight: 核心创新在于将编辑任务重构为开放式的’可供性’发现，通过多智能体框架进行情绪条件化的可供性推理，生成图像特定的编辑计划。这避免了依赖预定义模板的局限性，并支持用户在语义层次（锚点、变量、上下文）上进行交互式优化。此外，研究揭示了基于分类器的度量标准对非刻板、基于上下文的编辑存在情绪条件性盲点。

Abstract: Emotional image editing requires more than applying affective filters or modifying predefined visual factors: an effective edit must identify what a particular image can afford for a target emotion. Existing affective image manipulation methods, including recent agentic variants, largely operate within bounded strategy spaces based on predefined factor taxonomies, knowledge libraries, or conventional editing templates, and therefore often miss image-specific, context-grounded strategies. We introduce EmoScope, a multi-agent framework that reframes the task from “how should I edit?” to “what can I edit?” EmoScope first discovers an image-specific editable space through emotion-conditioned affordance reasoning, then uses a semantic hierarchy of anchors, variables, and context to balance content consistency and emotional expressiveness before executing and verifying the edit. Because its plans are expressed as image-specific affordances rather than retrieved templates, EmoScope also exposes the editing strategy as an interactive surface for user refinement at the plan level. In a large-scale human evaluation covering all eight Mikels emotion categories, with 4,693 valid responses across 1,824 pairwise questions, participants preferred EmoScope over two competitive baselines by 88.1% on average. Attribution analysis further shows that EmoScope selects target-emotion-adaptive strategies rather than applying a uniform template. The same affordance-level plan also supports lightweight user refinement in an interactive pilot. Finally, we show that classifier-based metrics exhibit emotion-conditional blind spots toward non-stereotypical, context-grounded edits, and present a relative content-emotion preference-affinity landscape showing that EmoScope’s advantage varies systematically across image-emotion combinations.

[103] SimBEV2X: A Large-Scale Dataset and Data Generation Tool for Multi-Task Vehicle-to-Everything Cooperative Perception cs.CV | cs.LG | cs.MA | cs.ROPDF

Goodarz Mehr, Sepideh Gohari, Montasir Abbas, Azim Eskandarian

TL;DR: 本文介绍了SimBEV2X，一个基于CARLA模拟器构建的先进合成数据生成工具，以及由此创建的大规模V2X协同感知数据集。该数据集是目前最大的V2X感知数据集，包含多模态传感器数据和丰富的真值标注。此外，论文还提出了CoBEVFusion这一新颖架构，用于多智能体特征聚合，并在该数据集上建立了强基线。

Details

Motivation: 开发鲁棒的V2X算法，特别是依赖统一空间表示（如鸟瞰图）的算法，受到大规模、多模态、多任务数据集缺乏的阻碍。真实世界多智能体数据的收集与标注成本极高，导致现有V2X数据集在规模和范围上都严重受限。

Result: SimBEV2X数据集在规模上比现有V2X数据集大一个数量级，包含102,200帧、588,520个激光雷达点云、超过300万张图像和超过2700万个边界框。提出的CoBEVFusion架构结合了CoopDet3D与融合轴向注意力，实现了上下文感知的多智能体特征聚合，获得了优越的性能。

Insight: 核心创新在于提供了一个可扩展的合成数据生成工具和迄今为止最大、标注最全面的V2X感知数据集，有效解决了该领域数据稀缺的瓶颈。同时，提出的CoBEVFusion架构通过融合轴向注意力改进了多智能体特征聚合机制，为后续研究提供了新的技术思路和强基线。

Abstract: Cooperative perception through vehicle-to-everything (V2X) communication can overcome the inherent physical limitations of individual autonomous vehicles, such as occlusions and limited sensor range. However, the development of robust V2X algorithms, particularly those relying on unified spatial representations like bird’s-eye view (BEV) representation, is hampered by the lack of large-scale, multi-modal, multi-task datasets. Moreover, collecting and annotating a large set of synchronized, real-world multi-agent data is prohibitively expensive. This has resulted in a landscape where existing V2X datasets are notably limited in both size and scope. To overcome this, we introduce SimBEV2X, an advanced synthetic data generation tool built on the CARLA simulator. SimBEV2X automatically creates randomized driving scenarios to collect multi-modal sensor data alongside various types of ground truth including 3D bounding boxes with unique track IDs, HD map information, BEV segmentation maps, and semantic occupancy voxel grids from both vehicles and RSUs. We also present the SimBEV2X dataset, the largest V2X perception dataset to date. The dataset comprises 258 scenes, each involving up to 8 connected vehicles and up to 4 RSUs across a variety of road networks. The SimBEV2X dataset is an order of magnitude larger than existing V2X datasets and contains 102,200 frames, 588,520 lidar point clouds, more than 3 million images, over 27 million bounding boxes, and a comprehensive set of other annotations. Finally, we establish a strong baseline on the SimBEV2X dataset using CoopDet3D and propose CoBEVFusion, a novel architecture that combines CoopDet3D with fused axial attention (FAX) for context-aware multi-agent feature aggregation, resulting in superior performance. SimBEV2X, the SimBEV2X dataset, and CoBEVFusion are available at https://simbev2x.org and https://github.com/GoodarzMehr/SimBEV2X.

[104] DDVT: Dynamic Dual-level Vision Transformer Fusion Network for Answer Grounding in Visual Question Answering cs.CVPDF

Yue Zhang, Xiangyu Li, Wanshu Fan, Xin Yang, Dongsheng Zhou

TL;DR: 本文提出了一种名为DDVT的动态双层级视觉Transformer融合网络，用于视觉问答中的答案定位任务。该方法通过问题引导的动态区域级模块（QGDR）结合图像上下文和文本内容，实现文本相关视觉内容的精确定位；同时引入跨模态多尺度聚合模块（CMA）增强像素级与区域级特征融合，以有效定位与答案相关的视觉区域。实验表明，DDVT在多个广泛使用的基准测试中超越了现有最先进方法。

Details

Motivation: 解决视觉问答中答案定位任务，即根据自然语言问题从图像中定位与视觉内容相关的区域，该任务因其实际应用价值受到广泛关注。

Result: 在多个广泛使用的基准测试上，DDVT的性能超越了当前最先进（SOTA）的方法。

Insight: 创新点包括：1）问题引导的动态区域级模块（QGDR），通过ROI Align结合图像上下文和文本内容，实现文本相关视觉内容的精确定位；2）跨模态多尺度聚合模块（CMA），促进像素级与区域级特征的有效融合，以定位与答案相关的视觉内容。从客观角度看，该方法通过动态双层级（区域级和像素级）的跨模态融合机制，提升了视觉问答中答案定位的精度和鲁棒性。

Abstract: Answer grounding in visual question answering aims to locate the region from a given natural language question associated with the visual content of an image, which has garnered significant attention due to its practical applications. In this paper, we introduce the Dynamic Dual-level Vision Transformer Fusion Network (DDVT) for answer grounding in visual question answering. Specifically, we propose a question-guided dynamic regional-level module (QGDR) that combines complementary image context through ROI Align and text content, enabling precise localization of text-related visual content. Moreover, we present a cross-modal multi-scale aggregation module (CMA) that enhances feature fusion between pixel-level and region-level features, facilitating the effective localization of visual content associated with grounded answers. Furthermore, we fuse the located visual content with text features to locate the region and provide answers to questions posed about the image. Experimental results demonstrate that our DDVT outperforms state-of-the-art methods on several widely-used benchmarks.

[105] DuoAD: Leveraging [CLS] Dual Characteristics for Training-Free Few-Shot Anomaly Detection cs.CV | cs.AIPDF

Jyun-Ze Tang, Po-Han Huang, Ming-Ching Chang, Chih-Fan Hsu, Jeng-Lin Li

TL;DR: 本文提出DuoAD，一种无需训练的小样本异常检测方法。该方法利用Vision Transformer中[CLS]令牌的双重特性：其嵌入提供异常不变的全局语义表示，其注意力图则隐式突出空间异常区域。通过自动增强选择策略和注意力引导的特征重加权机制，该方法在多个数据集上实现了SOTA性能。

Details

Motivation: 现有基于视觉基础模型的免训练异常检测方法主要依赖独立的局部块特征，未能充分利用Vision Transformer编码的全局上下文信息。

Result: 在单样本设置下，该方法在MVTec-AD、VisA和Real-IAD数据集上分别达到97.7%、93.2%和84.5%的Image-AUC分数。使用跨类别、骨干网络和数据集的单一固定配置，该方法在即插即用、免训练的异常检测中建立了新的最先进水平。

Insight: 创新点在于识别并利用了ViT [CLS]令牌的双重特性（全局语义与空间注意力），并基于此设计了完全自动化的框架，无需手动调参。从客观角度看，将全局上下文与局部特征动态结合的策略，为免训练异常检测提供了新的、鲁棒性强的解决方案。

Abstract: Vision foundation models have enabled strong training-free anomaly detection (AD). However, most existing approaches rely primarily on independent local patch features, leaving the global contextual information encoded by Vision Transformers (ViTs) underexploited. In this work, we identify the dual characteristics of the ViT [CLS] token: its embedding provides anomaly-invariant global semantic representation, while its attention maps implicitly highlight spatially abnormal regions. Building on this observation, we propose a fully automated AD framework leveraging global context to remove manual tunings. Our framework introduces (1) an automatic augmentation selection strategy driven by [CLS]-level semantic consistency, and (2) an attention-guided feature reweighting mechanism that dynamically adjusts patch contributions according to [CLS] attention saliency. By integrating these components over multi-level features, our method achieves stable anomaly scoring and precise localization without training or parameter tuning. Under the one-shot setting, it achieves Image-AUC scores of 97.7%, 93.2%, and 84.5% on MVTec-AD, VisA, and Real-IAD. Using a single fixed configuration across categories, backbones, and datasets, the method establishes a new state-of-the-art for plug-and-play, training-free anomaly detection while maintaining strong robustness and practical scalability.

[106] TimePLE: Rethinking Temporal Representation for Video Temporal Grounding cs.CVPDF

Yuhui Zeng, Xinyu Mao, Xiaokun Liu, Xin Tao, Jinfa Huang

TL;DR: 本文提出TimePLE方法，将视频时序定位任务从端点预测重新定义为原生区间定位，通过预测有效时间区间的联合分布来直接建模事件区间。该方法将每个区间映射到规范的位置-持续时间正方形中，并使用单一潜在令牌生成区间分布，再通过持续时间感知坐标校正转换为连续边界。

Details

Motivation: 现有基于视觉语言模型的视频时序定位方法通常通过两个端点输出间接预测区间，将事件区间作为派生对象处理，而区间有效性、持续时间和区间级相似性仅被隐式处理，这限制了模型对事件区间的直接理解和定位能力。

Result: 在四个视频时序定位基准测试上的实验表明，TimePLE一致优于端点预测基线，平均mIoU达到58.9，在短持续时间和中等持续时间事件上取得了明显增益。

Insight: 创新点在于将视频时序定位重新表述为区间原生定位，通过联合区间分布预测和规范位置-持续时间空间映射，显式建模区间有效性和几何相似性；同时使用单一潜在令牌表示和持续时间感知校正，实现了输入时间锚点与输出区间预测的对齐。

Abstract: Video temporal grounding (VTG) aims to localize the continuous video interval described by a natural-language query. However, current VLM-based methods typically produce this interval indirectly through two endpoint outputs, represented either as discrete timestamp tokens or continuous boundary coordinates. These formulations differ in how endpoints are encoded, but not in what is predicted: the event interval remains a derived object, while interval validity, duration, and interval-level similarity are handled only implicitly. We propose TimePLE, which reformulates VTG from endpoint prediction to interval-native grounding by predicting a single joint distribution over valid temporal intervals. TimePLE maps each interval to a point in a canonical position-duration square, where every support point corresponds to a valid span and neighboring points represent geometrically similar intervals. Given a video and query, the VLM generates a single latent <|TIMESPAN|> token whose hidden state is decoded into a joint interval distribution, refined through duration-aware coordinate correction, and converted into continuous boundaries. The same interval representation is used to encode input temporal anchors, aligning video-side temporal evidence with output-side span prediction. To reliably align the latent span representation with complete event intervals, we curate 90K-scale grounded samples and human-verify 3K-scale benchmark annotations. Experiments across four VTG benchmarks show that TimePLE consistently outperforms endpoint prediction baselines, achieving an average mIoU of 58.9, with clear gains on short-duration and medium-duration events.

Mohammed Aldeen, Muhammad Sami Irfan, Sagar Dasgupta, Long Cheng, Mizanur Rahman

TL;DR: 本文开发了首个基于视觉语言模型（VLM）的GNSS欺骗检测框架，用于自动驾驶汽车。该方法融合前视摄像头视觉数据和车内传感器读数，通过三阶段微调过程在共享语义空间中校准数据，以检测预测机动与GNSS数据之间的差异。在真实世界数据集上，微调模型F1分数达到94%-95%，远超零样本基线，并引入自适应推理策略大幅降低计算开销。

Details

Motivation: 自动驾驶汽车依赖GNSS进行定位和导航，容易受到欺骗攻击，可能导致车辆被秘密重定向或引发不安全机动。现有信号级完整性检查存在不足，需要一种利用车载传感器和视觉数据的互补防御层。

Result: 在自建的独立真实世界数据集（包含三种智能欺骗攻击场景）上验证，微调模型F1分数达到94%-95%，准确率在错误转向和停止攻击中达100%，在超调攻击中达88%-93%，显著优于零样本VLM基线（F1分数23%-32%）。自适应推理策略将VLM调用减少86%，每4秒窗口处理时间为65-73毫秒。

Insight: 创新点在于首次将VLM应用于GNSS欺骗检测，通过三阶段微调（视觉线索接地、传感器数据在共享语义空间校准）实现多模态融合与异常检测。提出的自适应推理策略在保持高检测性能的同时大幅降低计算成本，为实际部署提供了可行性。该方法将感知级检查与信号级完整性检查互补，构成了一个实用的道路防御层。

Abstract: Autonomous vehicles (AVs) depend on Global Navigation Satellite Systems (GNSS) for localization and navigation, making them vulnerable to spoofing attacks that can covertly redirect vehicles or induce unsafe maneuvers. In this paper, we develop the first Vision-Language Model (VLM)-based framework for GNSS spoofing detection for autonomous vehicles by fusing front-camera visual data with in-vehicle sensor readings (e.g., speed, acceleration, yaw rate) against GNSS-derived maneuvers. Our approach introduces a three-stage fine-tuning process that first grounds visual cues, and then calibrates sensor data within a shared semantic space to detect discrepancies between predicted and GNSS-derived maneuvers across three attack scenarios. We also generated an independent real-world dataset by driving an instrumented vehicle on public roads in Tuscaloosa, Alabama, equipped with time-synchronized GNSS, IMU, and camera logs to validate cross-regional generalization of our fine-tuned model on unseen data from training data. On this dataset, we then generated intelligent spoofing attacks, including trajectory mirroring with road-network snapping for wrong-turn attacks, position freezing for overshoot scenarios, and drift generation for stop attacks. On this validation dataset, the zero-shot VLMs baseline F1-score ranges from 23% to 32%, whereas our fine-tuned model achieves an F1-score ranging from 94% to 95%. Results show that our VLM-based approach correctly classified every wrong-turn and stop attacks, and attains 88%-93% accuracy for overshoot attacks. Furthermore, we introduce an adaptive inference policy that reduces VLM invocations to 14% (~86% computational reduction) and yields 65ms-73ms per 4s window. These results point to a practical, on-road layer of defense that complements signal-level integrity checks with the use of VLMs.

[108] Color Fundus Photography Analysis: Co-evolution of Data, Preprocessing, and Modeling toward Multimodal AI cs.CVPDF

Yu Li, Wengan He, Wenhui Xu, Lihong Jiang, Fan Xiao

TL;DR: 这篇综述论文系统回顾了彩色眼底摄影（CFP）人工智能的发展历程，强调数据集、预处理技术和建模框架三者之间的协同演化关系。论文指出，CFP数据集已从单一中心的小型任务特定集合，发展为包含多模态配对和纵向临床记录的大型多中心资源；预处理技术从传统图像增强演进为神经数据工程管道、硬件感知的标记优化以及针对不完整电子健康记录（EHR）的自监督填补；建模框架则从卷积神经网络（CNN）发展到视觉基础模型、状态空间模型（SSM）和多模态专家架构。

Details

Motivation: 现有综述通常独立总结特定任务算法、数据集或预处理技术，缺乏对它们与现代人工智能协同演化的统一视角。本文旨在通过整合数据集演化、预处理范式和建模框架的相互作用，提供一个关于CFP人工智能的综合性概述。

Result: 作为一篇综述性论文，未提及具体的定量实验结果或基准测试排名。它主要总结了领域的发展趋势和范式转变。

Insight: 论文的核心创新点在于提出了一个“协同演化”的整合视角，强调数据集、预处理和建模三者的共同优化是未来进展的关键。从客观角度看，将CFP与EHR及纵向患者信息进行多模态融合，以实现超越孤立图像分析的全面临床推理，是当前研究的前沿方向，为临床部署、跨域泛化和边缘智能提供了路线图。

Abstract: Color Fundus Photography (CFP) is a primary non-invasive imaging modality for large-scale screening of ophthalmic and systemic diseases. Existing surveys mainly summarize task-specific algorithms, datasets, or preprocessing techniques independently, lacking a unified perspective on their co-evolution with modern artificial intelligence. This review provides an integrated overview of CFP AI through the interplay of dataset evolution, preprocessing paradigms, and modeling frameworks. We show that CFP datasets have evolved from small single-center collections with task-specific labels to large multi-center resources featuring multimodal pairings and longitudinal clinical records. Preprocessing has progressed from conventional image enhancement to neural data-engineering pipelines, hardware-aware token optimization, and self-supervised imputation for incomplete electronic health records (EHRs). Meanwhile, modeling has advanced from convolutional neural networks (CNNs) to vision foundation models, state space models (SSMs), and multimodal expert architectures. At the multimodal frontier, CFP is increasingly integrated with EHRs and longitudinal patient information, enabling more comprehensive clinical reasoning beyond isolated image analysis. We conclude that future progress depends on the collaborative optimization of datasets, preprocessing, and multimodal modeling, providing a roadmap toward robust clinical deployment, improved cross-domain generalization, and resource-efficient edge intelligence.

[109] Multimodal Semantic-Probabilistic Objectness for Open World Object Detection cs.CV | cs.AIPDF

Weijun Tian, Rui Liu

TL;DR: 本文提出MSPO，一种轻量级语义校准框架，用于增强开放世界目标检测中的未知物体发现能力。该方法通过融合已知类别的语言先验与视觉目标性概率，校准已知与未知预测边界，无需将开放世界检测转化为开放词汇分类任务。

Details

Motivation: 现有PROB方法仅依赖视觉目标性难以区分已知类硬样本、未知类物体与背景杂波，导致已知-未知决策边界模糊。本文旨在利用已知类别的语义信息校准视觉目标性，提升开放世界检测的判别能力。

Result: 在M-OWODB和S-OWODB基准测试中，MSPO在主要综合指标上优于PROB基线，同时保持有竞争力的未知类召回率。早期未知混淆指标得到改善，PASCAL VOC最终mAP最高提升2.7个百分点。

Insight: 创新点在于将已知类别的扩展文本描述（涵盖属性、外观、场景和功能）作为语义先验，与视觉目标性概率融合，为开放世界检测提供有效的校准信号。该方法保持检测器架构不变，且不依赖未来类别名称，符合标准OWOD设定。

Abstract: Open-world object detection (OWOD) requires a detector to recognize known categories, discover unnamed objects from unseen categories, and incrementally learn newly annotated classes. PROB improves unknown discovery by modeling class-agnostic probabilistic objectness in the decoder-query space. However, visual objectness alone cannot determine whether an object-like query corresponds to a hard known instance, an unseen-category object, or background clutter, resulting in an ambiguous known-unknown decision boundary. We propose MSPO, a lightweight semantic calibration framework that augments PROB with task-aware known-category language priors while preserving its detector architecture and incremental learning protocol. For each currently known category, MSPO constructs an extended text description covering category attributes, visual appearance, typical scenes, and functional usage, and encodes it using a frozen CLIP text encoder. Decoder query features are projected into the same semantic space to estimate their support from the current known-category semantics. This semantic evidence is fused with PROB’s visual objectness to calibrate known and unknown predictions without turning OWOD into open-vocabulary classification. Importantly, MSPO never uses future-category names, and all unseen categories remain unnamed during evaluation. Experiments on M-OWODB and S-OWODB show that MSPO improves the strong PROB baseline on the main aggregate metrics while retaining competitive unknown recall. It also improves early unknown-confusion metrics and raises PASCAL VOC final mAP by up to 2.7 points. These results demonstrate that known-category language semantics provide an effective calibration signal for probabilistic objectness under the standard OWOD setting.

[110] Disentangling Semantic Attention from Structural Bias in the Attention Manifold cs.CV | cs.AIPDF

Pengkun Jiao, Bin Zhu, Jingjing Chen, Yu-gang Jiang

TL;DR: 本文针对多模态大语言模型（MLLMs）中注意力机制存在的‘视觉注意力汇’问题，即模型过度关注语义信息不足的视觉令牌，提出了一种无需训练、即插即用的干预方法SPAR。该方法通过净化结构噪声并重新分配注意力预算到信息丰富的视觉区域，以缓解由普遍存在的文本结构偏差导致的多模态幻觉问题。

Details

Motivation: 动机在于揭示并解决MLLMs中注意力机制的内在缺陷，即模型倾向于过度关注某些语义信息不足的视觉令牌（称为‘寄存器’或‘视觉注意力汇’），这导致文本结构偏差稀释了语义视觉信号，从而引发多模态幻觉。

Result: 在多种幻觉基准测试上的综合评估表明，SPAR方法能够以可忽略的计算开销有效恢复真实的视觉基础，提升了模型性能。

Insight: 创新点在于将‘视觉注意力汇’现象重新定义为一种超越孤立令牌的、对视觉特征施加的广义文本偏差，并提出了SPAR这一训练免费的干预框架，通过净化结构噪声和自适应重分配注意力来缓解偏差，为注意力机制的可解释性和鲁棒性提供了新思路。

Abstract: The empirical success of attention mechanism in Multimodal Large Language Models (MLLMs) often obscures its inherent, subtle flaws. Specifically, MLLMs consistently exhibit disproportionate attention toward certain semantically uninformative visual tokens, a phenomenon termed “register” or “Visual Attention Sinks.” While existing inference intervention methods attempt to identify these sink tokens and redistribute their attention weights, such approaches typically treat these tokens in isolation and suffer from computational inefficiency. Instead, we reframe this phenomenon as a generalized textual bias exerted over visual features that extends beyond isolated sink tokens. From this perspective, a pervasive structural bias leads to the dilution of the semantic visual signal, precipitating multimodal hallucinations as the model prioritizes linguistic priors over valid visual evidence. To address this limitation, we introduce Saliency-guided Purification and Adaptive Redistribution (SPAR), a training-free, plug-and-play intervention. SPAR mitigates this generalized textual bias by purifying structural noise and subsequently redistributing the reclaimed attention budget to the most informative visual regions. Comprehensive evaluations across a diverse spectrum of hallucination benchmarks demonstrate that SPAR effectively restores authentic visual grounding with negligible computational overhead.

[111] AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars cs.CVPDF

Hengyuan Zhang, Jingna Sun, Meiguang Jin, Junfeng Ma

TL;DR: AptAvatar是一个14B参数的长时音频驱动虚拟人生成框架，通过端点锚定分布蒸馏和自生成历史回放技术，实现了仅需2步推理即可生成生动720p长视频，在保持视觉保真度和长时身份一致性的同时获得60倍加速。

Details

Motivation: 解决生产级音频驱动虚拟人生成中，现有加速方法因采用因果注意力、缩短时间跨度或降低模型容量/分辨率而牺牲生成质量或运动表现力的问题，旨在实现高效推理而不妥协。

Result: 实验表明，AptAvatar仅用2次网络前向传播（NFEs）就能生成生动的720p长时虚拟人视频，在保持视觉保真度和长时身份一致性的同时实现了60倍的加速。

Insight: 创新点在于提出了端点锚定分布蒸馏来弥合多步教师模型与两步学生模型之间的差距，以及自生成历史回放技术来提升长时一致性，避免了昂贵的在线展开，为高质量、高效率的长序列生成提供了新思路。

Abstract: Production-ready audio-driven avatar generation requires efficient inference without sacrificing fidelity or motion expressiveness. However, existing acceleration methods often compromise quality through restrictive architectural choices, such as causal attention and short temporal horizons, or by reducing model capacity and resolution. Without such compromises, we propose AptAvatar, a 14B-parameter long-form audio-driven avatar generation framework that delivers fast and expressive inference. For efficiency in production-level applications, AptAvatar addresses the extreme two-step generation challenge. To bridge the gap between the multi-step teacher model and the two-step student model, we introduce Endpoint-Anchored Distribution Distillation. It augments vanilla distribution matching with a dedicated Anchor Score Estimator trained on the trajectory-endpoint distribution defined from a frozen pretrained 4-step bridge generator. This provides an attainable endpoint-level anchor for the evolving two-step student. To improve long-horizon consistency, we further introduce Self-Generated History Replay, which reuses cached outputs from earlier generator checkpoints as history conditions during chunk-wise training. This approximates inference-time conditioning on self-generated histories without costly online rollouts, mitigating quality degradation from accumulated history errors. Extensive experiments demonstrate that AptAvatar generates vivid 720p long-form avatar videos with only 2 NFEs, achieving a 60x speedup while preserving visual fidelity and long-horizon identity. Code is available at https://github.com/TaoLiveAIGC/AptAvatar

[112] A Unified Stereo Geometry Estimation Framework for Disparity and Surface Normal cs.CVPDF

Qizhe Wei, Xianda Guo, Shaocong Xu, Hong Li, Runyi Yang

TL;DR: 本文提出GeoStereo，一个统一的立体几何估计框架，利用扩散先验联合预测视差和表面法线。该框架将前馈立体匹配流程与基于扩散的法线估计分支耦合，通过视差到法线的初始化策略和构建扭曲到左视图的条件，使两个任务有效交互。扩散分支为病态区域提供强结构先验以增强视差估计，而前馈分支为准确的法线预测提供可靠的几何指导。

Details

Motivation: 现有前馈立体方法在挑战性区域（如弱纹理、遮挡）难以产生可靠预测，主要缺乏强几何先验。本文旨在通过联合学习视差和表面法线，利用扩散模型的结构先验来提升立体几何估计的鲁棒性。

Result: 在零样本设置下，GeoStereo在KITTI和NYUv2等多个基准测试中达到Rank-1的视差估计性能，并在iBims-1和ScanNet等真实室内基准上取得了最佳的法线估计精度。实验表明，该方法在低光照、高反射表面和透明物体等挑战性场景中表现可靠。

Insight: 创新点在于将扩散先验引入立体几何估计，通过耦合视差与法线预测任务实现双向增强：扩散分支提供结构先验改善视差，前馈分支提供几何指导改善法线。所提的视差到法线初始化策略和扭曲条件构建有效促进了任务间交互。

Abstract: Stereo matching and surface normal estimation are fundamental tasks in 3D vision. However, existing feed-forward stereo methods still struggle to produce reliable predictions in challenging regions, mainly due to the lack of strong geometric priors. In this paper, we propose $\textbf{GeoStereo}$, a unified stereo geometry estimation framework that leverages powerful diffusion priors to jointly predict disparity and surface normals. Specifically, GeoStereo couples a feed-forward stereo matching pipeline with a diffusion-based normal estimation branch. To enable effective interaction between the two tasks, we introduce a disparity to normal initialization strategy and construct a warp to left-view condition for the diffusion process. This coupled design allows the diffusion branch to provide strong structural priors that enhance disparity estimation in ill-posed regions, while the feed-forward branch offers reliable geometric guidance for accurate normal prediction. Extensive experiments show that GeoStereo performs reliably in challenging scenarios, including low-light environments, highly reflective surfaces, and transparent objects. Under zero-shot settings, it achieves Rank-1 disparity estimation on multiple benchmarks, including KITTI and NYUv2, and delivers the best normal estimation accuracy on many real indoor benchmarks, such as iBims-1 and ScanNet. Project page: https://qz-wei.github.io/GeoStereo.github.io/

[113] MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning cs.CV | cs.AIPDF

Tuan-An To, Yuk-Kwan Wong, Tuan-Anh Vu, Ziqiang Zheng, Sai-Kit Yeung

TL;DR: 该论文提出了首个以事件为中心的海洋视频理解数据集MarineEVT，包含20K个多任务视频级视觉问答对，并设计了基于视觉工具的事件中心推理框架EVT-R1，通过视觉工具定位和解释海洋视频中的关键信息，显著提升了海洋视频理解性能。

Details

Motivation: 现有视觉语言模型在视频理解领域性能下降，尤其海洋视频理解面临领域专业知识需求高、关键事件稀疏且分布不均等挑战，需要专门的数据集和方法来解决。

Result: 在MarineEVT数据集上，EVT-R1框架相比11个SOTA视觉语言模型表现最佳，分别超过顶级开源模型5.22分和顶级商业模型11.09分，实现了SOTA性能。

Insight: 创新点在于构建首个事件中心的海洋视频理解数据集，并提出将视觉工具集成到推理过程中，通过工具驱动的方式定位和解释稀疏事件，为领域特定视频理解提供了可借鉴的框架设计思路。

Abstract: Recent Vision-Language Models (VLMs) have achieved remarkable success in visual understanding, driven by the growing availability of high-quality image-text pairs. However, the performance of VLMs often degrades in the video domain due to the essential need for temporal understanding and the scarcity of large-scale annotated video data. In this work, we focus on marine video understanding, which brings further challenges: first, it requires substantial domain expertise; and video VLMs usually struggle with localizing and interpreting critical information from marine videos, as the informative events are typically sparse, unpredictable, and unevenly distributed. To address these challenges, we carefully curate the first event-centric marine video understanding dataset called MarineEVT, which features 20K multi-task, video-level visual question-answering pairs spanning multiple dimensions of marine understanding and analysis. Meanwhile, based on MarineEVT, we decompose marine video understanding as an Event-centric Visual Tool-integrated Reasoning process EVT-R1 for short, where we leverage powerful visual tools to drive the model to localize and interpret critical information aligned with visual questions and human intent. To demonstrate its effectiveness, we compare EVT-R1 against 11 SOTA VLMs in different settings. EVT-R1 outperforms the top open-source and top commercial models by 5.22 and 11.09, respectively. MarineEVT and EVT-R1 lay the foundation for ecological discovery and marine education, fostering the development of VLMs capable of interpreting marine dynamics, reasoning about ecological interactions, and supporting sustainable ocean video understanding and analysis.

[114] Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification cs.CVPDF

Haopeng Li, Yitong Li, Junsong Chen, Tian Ye, Haozhe Liu

TL;DR: 本文提出了一种名为Sol-Attn的训练时无需修改的动态稀疏注意力方法，旨在加速视频生成模型的推理过程。该方法通过在一次在线softmax计算中，统一完成动态路由、稀疏计算和近似校正，实现了比现有方法更好的精度-效率权衡。

Details

Motivation: 扩散Transformer模型用于高保真视频生成时，长序列导致注意力计算成为推理瓶颈。现有训练无关的动态稀疏注意力方法存在两个问题：1）路由机制（选择计算哪些键值块）成本高且不灵活；2）丢弃未选块导致精度损失。本文旨在设计一种更廉价、动态且能控制精度损失的稀疏注意力方案。

Result: 在图像和视频生成任务上的实验表明，Sol-Attn在保持视觉质量的同时，将训练无关稀疏注意力的质量-效率前沿推向了新水平。具体地，它为视频生成和视频编辑任务分别带来了2.1倍和2.3倍的端到端加速。

Insight: 核心创新在于“在线块阈值化与代理分数重用”机制：在在线softmax过程中，通过将块代理分数与阈值比较来动态选择关键块，无需显式存储代理分数图，从而降低了路由开销。同时，直接重用未选块的代理分数来近似其贡献，以减轻精度损失。该方法将动态路由、稀疏计算和近似校正统一在一个计算过程中。

Abstract: Diffusion transformers are essential for high-fidelity video generation, but long token sequences make attention a dominant inference bottleneck. Training-free dynamic sparse attention alleviates this bottleneck by computing only selected key-value blocks, yet existing methods struggle to sparsify attention both efficiently and accurately for two reasons: (1) Rigid, unpredictable, and costly routing: selecting a fixed fraction of top-ranked blocks by proxy score imposes fixed budgets, whereas retaining blocks to reach a target cumulative proxy probability mass yields dynamic but potentially imbalanced budgets; both incur non-negligible overhead from computing and materializing proxy scores. (2) Lossy keep-or-drop sparsification: unselected blocks are discarded entirely, degrading accuracy under aggressive sparsity. These limitations motivate cheaper dynamic-budget routing while limiting accuracy degradation. In this paper, we introduce training-free Sol-Attn (Sparsifying online attention), which unifies dynamic routing, sparse computation, and approximation correction in a single online-softmax pass, achieving a better accuracy-efficiency trade-off in sparse attention. The core of Sol-Attn is on-the-fly block thresholding with proxy-score reuse, which selects critical blocks by comparing block proxy scores against a threshold during online softmax. This design enables dynamic yet controllable block budgets without materializing the proxy map, while directly reusing the proxy scores of unselected blocks to approximate their contribution. Experiments across image and video generation tasks show that Sol-Attn advances the quality-efficiency frontier of training-free sparse attention, delivering 2.1 times and 2.3 times end-to-end speedups for video generation and editing, respectively, while preserving visual quality.

[115] When Low CER is Not Enough: An Analysis of Hallucinations in Vision-Language OCR Systems on Historical Uruguayan Documents cs.CV | cs.LG | eess.IVPDF

Marina Gardella, Camilo Mari{ñ}o, Diego Belzarena, Ignacio Ram{í}rez, Gregory Randall

TL;DR: 本文评估了传统OCR系统与基于视觉语言模型（VLM）的方法在乌拉圭独裁时期历史文档数据集（Berrutti数据集）上的表现。研究发现，尽管VLM在字符错误率（CER）和词错误率（WER）上优于传统方法，但这些量化指标掩盖了系统性的失败模式，如正字法归一化、虚假内容生成和语义替换，这些错误可能严重扭曲语义但对CER/WER影响很小。

Details

Motivation: 研究动机是评估VLM在历史档案转录中的适用性，尽管VLM在标准基准测试中达到SOTA，但其在实际档案转录中的可靠性尚未得到充分理解。

Result: 在Berrutti数据集上，VLM在CER和WER上持续超越传统OCR方法，但定性分析揭示了标准指标无法捕捉的系统性错误，特别是影响命名实体的语义失真。

Insight: 创新点在于揭示了量化OCR性能（如CER/WER）与实际转录保真度之间的关键差距，强调需要超越字符级准确率的评估框架，以捕捉生成转录的语义可靠性，这对于历史档案等敏感应用至关重要。

Abstract: Optical Character Recognition (OCR) is a key component in the digitization of historical archives. Recently, Vision-Language Models (VLMs) have emerged as strong alternatives to traditional OCR systems, achieving state-of-the-art performance on standard benchmarks. However, their suitability for archival transcription remains insufficiently understood. In this work, we benchmark traditional OCR systems and VLM-based approaches on the Berrutti dataset, a challenging collection of Uruguayan dictatorship-era documents derived from microfilm scans. While VLMs consistently outperform traditional methods in terms of Character Error Rate (CER) and Word Error Rate (WER), we show that these improvements hide a more complex picture. Through a detailed qualitative analysis, we uncover systematic failure modes that are invisible to standard metrics, including orthographic normalization, spurious content generation, and semantic substitutions that preserve fluency while altering meaning. Errors affecting named entities are particularly critical, as they can introduce substantial semantic distortions with minimal impact on CER and WER. These findings reveal a critical gap between quantitative OCR performance and transcription fidelity in real-world archival settings, and highlight the need for evaluation frameworks that go beyond character-level accuracy to capture the semantic reliability of generated transcriptions.

[116] ReflexTrack: A Feedback-Driven Agent for Training-Free Referring Video Object Segmentation cs.CVPDF

Yuanjia Li, Tianyang Xu, Tao Zhou, Zhangyong Tang, Xiao-Jun Wu

TL;DR: 本文提出ReflexTrack，一种无需训练的反馈驱动智能体，用于解决参考视频目标分割（RVOS）任务。该方法通过掩码引导的空间细化和视频级掩码反射机制，在空间和时间层面进行闭环验证，提升了无训练RVOS的可靠性。

Details

Motivation: 现有基于多模态大语言模型和可提示分割模型的免训练RVOS方法通常依赖一次性空间定位和掩码传播，缺乏对初始提示和时间预测的验证，导致可靠性不足。

Result: 在Ref-VPS基准上达到69.7的Q分数，在ReasonVOS基准上达到67.2的J&F分数，表明预测级反馈显著提升了免训练RVOS的性能。

Insight: 创新点在于引入了空间和时间两个层面的反馈循环：通过迭代更新边界框和正负点进行空间细化，以及通过定位不可靠区间、选择修复关键帧和掩码引导重传播进行时间反射。所有组件在推理时保持冻结，实现了无需训练的高效可靠分割。

Abstract: Referring video object segmentation (RVOS) requires segmenting a target specified by natural language throughout a video. Recent agentic approaches combine multimodal large language models with promptable segmentation models to perform RVOS without task-specific training. However, most pipelines rely on one-shot spatial grounding followed by mask propagation, leaving both the initial prompts and temporal predictions largely unverified. We introduce ReflexTrack, a training-free, feedback-driven agent that closes this loop at both spatial and temporal levels. Mask-guided Spatial Refinement evaluates the mask induced by the current keyframe prompt and iteratively updates the bounding box together with positive and negative points, yielding a more reliable initialization. Video-level Mask Reflection assesses the complete mask sequence, localizes unreliable intervals, selects complementary repair keyframes, and generates candidate predictions through mask-guided re-propagation. Only candidates that provide a verified improvement are used to update the affected intervals, preserving reliable predictions elsewhere. All components remain frozen during inference. ReflexTrack achieves an overall $\mathcal{Q}$ score of $69.7$ on Ref-VPS and a $\mathcal{J}&\mathcal{F}$ score of $67.2$ on ReasonVOS. These results demonstrate that prediction-level feedback substantially improves the reliability of training-free RVOS.

[117] Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels cs.CV | cs.CL | cs.IRPDF

Zhuchenyang Liu, Yao Zhang, Yu Xiao

TL;DR: 本文研究了视觉文档理解中的证据归因问题，提出了一种基于语言引用的替代方法，以解决传统坐标接口导致的归因幻觉问题。通过引用-检索管道，模型在无需坐标或区域标注的情况下，显著提升了证据召回率并降低了幻觉率。

Details

Motivation: 现有视觉文档理解模型通过坐标接口进行证据归因时，即使答案正确也常无法准确定位支持区域（即归因幻觉），这限制了模型的可靠性。本文旨在探索是否坐标接口的表达能力限制了归因性能，并寻求无需坐标或昂贵区域标注的改进方案。

Result: 在双语CiteVQA验证子集上，语言接口（引用文本）相比坐标接口将证据召回率从最高8点提升至26-47点，幻觉率约减半，答案质量基本不变；通过GRPO训练方法，8B骨干模型的严格归因准确率从22.4提升至33.8。

Insight: 创新点在于用文本引用替代坐标输出，结合多模态检索器定位证据区域，避免了坐标接口的表达限制；提出的GRPO训练方法利用答案和检索区域作为奖励，无需区域级标注即可优化证据引用，为提升归因可靠性提供了实用路径。

Abstract: Reliable visual document understanding requires a model to attribute each answer to the evidence regions that support it. Recent benchmarks and systems express this step through a coordinate interface: the model outputs the coordinates of bounding boxes that mark the evidence regions in the document. Under this interface, vision-language models often fail to identify the right regions even when the answer is correct, a failure known as Attribution Hallucination. We present a study that investigates whether this failure is partially limited by what the model can express through coordinates. On a verified bilingual CiteVQA subset, we compare the coordinate interface with a language interface in which the model outputs only text, quoting its evidence verbatim, and a multimodal retriever returns the location of each quote as a page region proposed by a layout parser (tables and figures are quoted through their captions or notes); the comparison is repeated over six open vision-language models. Compared with the coordinate interface, evidence recall rises from at most 8 points to between 26 and 47 and the hallucination rate roughly halves, with little change in answer quality. Building on this comparison, we use the same quote-and-retrieve pipeline as a training scaffold: because region-level evidence labels are expensive to collect for long documents, we introduce a GRPO recipe whose reward is a judge’s reading of the gold answer and crops of the retrieved regions, training the model to quote better evidence without any region labels and raising an 8B backbone’s strict attributed accuracy from 22.4 to 33.8. These findings indicate a practical path to improve attribution”without a coordinate interface and without costly region-level supervision.

[118] Bigger or Cheaper? Scale and Quantization Effects on Uncertainty Signals in Vision-Language Models Under Image Degradation cs.CV | cs.CL | cs.LGPDF

M M Asif Ferdous

TL;DR: 本文研究了在图像退化条件下，视觉语言模型（VLMs）的模型规模（scale）和4位量化（quantization）对两种不确定性信号（模型内部置信度和语言化置信度）的影响。研究发现，模型规模提升能显著改善内部不确定性信号，但语言化置信度提升有限；4位量化对准确率影响较小，但会严重损害置信度信号。在固定内存预算下，建议选择更大的量化模型而非更小的全精度模型。

Details

Motivation: 在消费级硬件上部署VLMs时，模型需要决定何时回答、何时拒绝，这依赖于能跟踪正确性的置信度信号。实践者在固定内存预算下面临选择：小模型全精度、小模型量化、或大模型量化到相同内存占用，这三种配置对置信度信号有不同影响，需要评估其权衡。

Result: 在Qwen2-VL系列模型上，对5,700个预测（涵盖六种真实图像退化、三种严重程度）进行测试。结果显示：模型规模从2B增至7B，内部不确定性信号（平均错误检测AUROC）从0.80提升至0.98，但语言化置信度提升微弱（平均0.61至0.69）。4位量化使准确率仅下降1.6个百分点，但内部AUROC从0.95降至0.80，语言化置信度解析率从99%崩溃至64%。在固定内存预算下，7B-4bit配置在三种可行配置中实现了最佳准确率和最佳不确定性信号（内部AUROC 0.98）。

Insight: 创新点在于系统评估了模型规模和量化对VLMs两种置信度信号（内部概率与语言化表达）的影响，揭示了模型“所知”与“所言”之间的差距随规模增大而扩大。客观分析认为，该研究强调了错误检测AUROC（而非校准误差）是区分两种信号的关键指标，并为实际部署提供了基于选择性预测操作点的直接建议，即优先选择更大规模的量化模型以平衡内存、准确性和不确定性估计。

Abstract: Vision-language models (VLMs) deployed on consumer hardware must decide when to answer and when to defer, and that decision depends on having a confidence signal that tracks correctness. A practitioner with a fixed memory budget faces a choice between a small model at full precision, the same small model quantized, and a larger model quantized into the same footprint – three configurations that push the confidence signal in opposing directions. We measure, on identical inputs, how model scale and 4-bit quantization affect two confidence signals in the Qwen2-VL family: the confidence a model states in natural language, and its own mean token probability over the answer it generates. Across 5,700 predictions spanning six realistic photographic degradations at three severities, we find that scale sharply improves the model’s internal uncertainty signal (mean error-detection AUROC 0.80 to 0.98 from 2B to 7B) while its verbalized confidence stays weak and often at chance (mean 0.61 to 0.69): the gap between what the model knows and what it says widens rather than closes with size. We find that 4-bit quantization is nearly free for accuracy (-1.6 points) but expensive for the confidence signal (internal AUROC 0.95 to 0.80, and the verbalized-confidence parse rate collapses from 99% to 64%). For a fixed memory budget the recommendation is therefore to prefer a larger quantized model over a smaller full-precision one: 7B-4bit gives both the best accuracy and the best uncertainty signal (internal AUROC 0.98) of the three configurations that fit. We frame the results as selective-prediction operating points so they translate directly into a deployment recommendation, and we argue that error-detection AUROC, not calibration error, is the metric that exposes the difference between the two signals.

[119] BeyondFusion: Self-Aligned Latent Diffusion for Calibration-Free Infrared Super-Resolution and Infrared-Visible Fusion cs.CV | cs.LG | physics.opticsPDF

Minchong Chen, Xiaoyun Yuan, Minyu Cao, Jianing Zhang, Jun Zhang

TL;DR: 本文提出了BeyondFusion，一个统一的潜在扩散框架，用于解决无需标定的可见光引导红外超分辨率和红外-可见光图像融合任务。该框架通过引入跨模态自对齐模块和不对齐数据增强，在去噪过程中学习内容自适应的跨模态对应关系，从而在传感器未标定或未同步的情况下，有效利用可见光的结构语义线索并保持热一致性。

Details

Motivation: 移动红外-可见光成像系统中，由于光学、视点、视场和曝光时间不同导致的跨传感器不对齐问题，阻碍了实际部署。本文旨在解决无需标定的可见光引导红外超分辨和红外-可见光融合问题。

Result: 在公共基准测试和移动红外-可见光成像系统上的大量实验表明，该方法在对齐输入、低分辨率红外观测、合成不对齐以及传感器未同步的真实移动捕获数据上均表现出色。下游行人检测任务进一步验证了其有效性。

Insight: 创新点在于提出了一个统一的潜在扩散框架，支持任务特定训练和联合训练，并引入了跨模态自对齐模块，在去噪U-Net中隐式学习跨模态对应，避免了显式配准或几何变形，从而实现了在未标定条件下的鲁棒多模态成像。

Abstract: Mobile infrared-visible imaging typically pairs a compact infrared sensor with a high-resolution visible camera for complementary perception. While cross-sensor misalignment caused by different optics, viewpoints, fields of view, and exposure timings hinders practical deployment. In this paper, we propose BeyondFusion, a unified latent diffusion framework for calibration-free visible-guided infrared super-resolution and infrared-visible fusion tasks. The proposed framework supports both task-specific training and joint training where two tasks are optimized and executed as two readouts of the same generative process. Instead of relying on explicit registration or geometric warping, BeyondFusion introduces a cross-modal self-aligning (CMSA) module into the denoising U-Net. CMSA reorganizes infrared and visible latent tokens into a shared attention space to learn content-adaptive cross-modal correspondence during the denoising process. Together with misalignment augmentation module, the model is facilitated to exploit visible structural and semantic cues while preserving thermal consistency, enabling high-frequency infrared reconstruction and informative fused-image generation under uncalibrated conditions. Extensive experiments on public benchmarks and a mobile infrared-visible imaging system show strong performance across aligned inputs, low-resolution infrared observations, synthetic misalignments, and real mobile captures with unsynchronized sensors. Ablation studies, unified training analysis, and downstream pedestrian detection further validate the effectiveness of BeyondFusion for calibration-free multimodal imaging.

[120] ViDS: Video Diffusion Shader using 3D Face Tracking cs.CVPDF

Wenbo Ji, Davide Davoli, Zhe Chen, Liam Schoneveld, Matthias Nießner

TL;DR: ViDS是一种视频扩散着色器，利用3D人脸跟踪实现富有表现力且保持身份特征的肖像动画。该方法首先从参考图像重建身份特定的3DMM网格，然后使用驱动视频的表情和姿态参数进行动画化，借助3DMM法线图的密集几何线索，通过视频扩散模型作为神经着色器合成逼真的肖像动画。

Details

Motivation: 解决现有基于扩散的肖像动画方法依赖基于地标的调节或隐式运动潜在变量，难以实现精细表情控制和身份保持的问题。

Result: 与先前方法相比，实现了更详细、一致的表情和姿态控制，同时忠实地保持了身份和外观，详细的消融研究验证了设计选择的有效性。

Insight: 创新点在于利用3DMM跟踪提供密集几何线索，并引入自回归扩散采样过程以扩展生成长度并减少片段间不连续性，实现了更细粒度的表情控制。

Abstract: We introduce ViDS, a Video Diffusion Shader that leverages 3D face tracking for expressive and identity-preserving portrait animation. We first reconstruct the identity-specific 3DMM mesh from the reference image, and then animate it using expression and pose parameters from a driving video. Leveraging dense geometric cues from 3DMM normal maps, we employ a video diffusion model as a neural shader to synthesize lifelike portrait animations while preserving the appearance and identity of the reference image. We find that more accurate 3DMM tracking enables finer-grained expression control. We also introduce an autoregressive diffusion sampling process that extends generation beyond the model’s native window while reducing discontinuities between adjacent clips. Compared with prior diffusion-based approaches for portrait animation that rely on landmark-based conditioning or implicit motion latents, our method achieves more detailed and consistent expression and pose control while faithfully preserving identity and appearance. Detailed ablation studies validate the effectiveness of our design choices. Project page: https://fusheng-ji.github.io/ViDS/

[121] ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding cs.CV | cs.AI | cs.CLPDF

Hangjie Yuan, Yichen Qian, Zhiwei Tang, Xianzhe Xu, Lirong Wu

TL;DR: 本文提出了ClinFusion，一个以视觉为中心的多模态大语言模型系统，旨在实现全面的医学理解。它通过一个新颖的级联空间感知局部融合编码器架构，统一处理2D和原生3D医学图像，并引入了一个基于视觉的评估框架，包括用于指令遵循评估的MedIF-Bench和用于临床对齐、事实驱动的报告生成评估的感兴趣区域（RoI）方法。

Details

Motivation: 部署多模态大语言模型于医疗领域面临核心挑战：模型需要从异构的2D和3D医学图像中吸收知识，且评估协议必须与放射科医生的临床实践对齐，提供准确、细粒度且基于事实的评估。

Result: ClinFusion在全面的2D和3D多模态医学基准测试（涵盖视觉问答、报告生成和指令遵循）以及文本医学任务上，均取得了新的最先进水平。它在24个基准中的20个上超越了领先的开源医学MLLM（如Hulu-Med, Lingshu），在16个基准中的13个上展示了优于GPT-5.2和Gemini-3-Flash等强大专有模型的多模态能力。经认证放射科医生盲评确认，其生成的报告排名最高，且其RoI评估指标在所有自动评估指标中与专家判断的相关性最强。

Insight: 主要创新点在于提出了一个组合式级联视觉编码器架构（Cascade Spatial-Aware Locality Fusion），统一处理2D和原生3D医学图像，以及一个基于视觉的、临床对齐的评估框架（MedIF-Bench和RoI-grounded方法）。其架构设计和对评估范式的系统性改进，为解决医疗领域多模态模型部署的视觉中心挑战提供了有效方案。

Abstract: Multimodal large language models (MLLMs) hold immense potential to revolutionize clinical practice, yet deploying them in the medical domain is fundamentally a vision-centric challenge: models must absorb knowledge from heterogeneous 2D and 3D medical images, and evaluation protocols must align with radiologists’ clinical practice and provide an accurate, fine-grained and factualness-driven assessment. In this paper, we introduce ClinFusion, a vision-centric MLLM designed for holistic medical understanding that systematically addresses these limitations. We propose a compositional and cascaded vision encoder architecture featuring a Cascade Spatial-Aware Locality Fusion operator that unifies diverse 2D and native 3D medical image understanding within a fused encoder. We further introduce a vision-grounded evaluation framework, including MedIF-Bench for instruction-following assessment and a region-of-interest-grounded method for clinically aligned and factualness-driven report generation evaluation. We show that ClinFusion sets a new state-of-the-art across a comprehensive suite of 2D and 3D multimodal medical benchmarks—spanning visual question answering, report generation, and instruction following—as well as textual medical tasks, outperforming leading open-source medical MLLMs (\textit{e.g.}, Hulu-Med, Lingshu) on 20 out of 24 benchmarks and demonstrating multimodal capabilities better than powerful proprietary models such as GPT-5.2 and Gemini-3-Flash on 13 out of 16 benchmarks, and can be further augmented with agentic tool use for retrieval-augmented and tool-assisted clinical workflows. A blinded evaluation by board-certified radiologists confirms that ClinFusion produces the highest-ranked reports, and validates our RoI-grounded metric as achieving the strongest correlation with expert judgment among all automatic evaluation metrics examined.

[122] UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling cs.CVPDF

Zhipeng Bao, Zhen Zhu, Nupur Kumari, Anurag Bagchi, Yu-Xiong Wang

TL;DR: 本文提出了UniGen-AR框架，旨在通过结合多模态大语言模型（MLLM）和高效的自回归视觉解码器（VAR），解决现有视觉生成系统（如扩散模型）因迭代采样导致的高延迟问题，实现一个统一模型处理超过15种跨四个类别的视觉生成任务。

Details

Motivation: 现代计算机视觉流水线是碎片化的，不同任务（如文生图、编辑、修复）由独立模型处理。基于扩散模型的统一视觉生成（UVG）系统虽质量高、可控性强，但其迭代采样导致推理延迟高，限制了实际部署。

Result: 实验表明，UniGen-AR在保持或提升输出质量的同时，推理延迟比基于扩散模型的基线降低了高达19倍。消融实验进一步揭示了VQ-VAE分词器设计（特别是码本大小和层次结构）是VAR在UVG中可扩展性的关键因素。

Insight: 核心创新点是将MLLM的灵活条件编码能力与VAR解码器的采样效率和潜在统一特性相结合，构建了一个高效、统一的多任务视觉生成框架。这确立了视觉自回归建模作为统一视觉生成的一个有竞争力且高效的骨干网络。

Abstract: Modern computer vision pipelines remain fragmented, with tasks such as text-to-image generation, editing, restoration, and classical perception handled by separate models. We study Unified Visual Generation (UVG), where a single model produces diverse image-valued outputs through a unified multimodal interface. While diffusion-based systems dominate UVG due to strong quality and controllability, their iterative sampling incurs substantial inference latency, limiting practical deployment. To address these limitations, we propose UniGen-AR, a framework that pairs a general-purpose multi-modal language model (MLLM) with an efficient next-scale visual auto-regressive (VAR) decoder. This design retains the flexibility of MLLM-based conditioning while leveraging the sampling efficiency and latent unification properties of VAR models. In our framework, the MLLM encodes free-form instructions and control signals into a unified sequence, which guides the VAR decoder to generate image-valued outputs for over 15 tasks spanning four families. Empirically, UniGen-AR achieves up to $19 \times$ lower inference latency than diffusion-based baselines while maintaining or improving output quality. Our ablations further reveal that VQ-VAE tokenizer design, particularly codebook size and hierarchy, is a critical factor for VAR scalability in UVG. These results establish visual auto-regressive modeling as a compelling and efficient backbone for unified visual generation. Our project page is at https://zpbao.github.io/projects/unigenar.

[123] Reasoning to Regulate: Chain-of-Thought for Traffic Rule Understanding cs.CVPDF

Yueru Luo, Xu Yan, Changqing Zhou, Yiming Yang, Chao Zhan

TL;DR: 本文提出了一种结合思维链（CoT）的视觉语言模型框架，用于自动驾驶中的交通规则理解任务。该方法通过两轮策略从大型语言模型生成推理依据，并利用视觉语言模型验证器过滤错误案例，构建高质量的训练数据。随后采用监督微调和基于答案的细粒度奖励强化学习两阶段训练方案，在MapDR数据集上显著提升了模型的可解释性和准确性。

Details

Motivation: 自动驾驶中理解和遵守交通规则是安全关键需求，但现有方法通常将规则理解视为直接序列预测任务，忽略了连接交通标志语义与地图结构的底层推理过程。

Result: 在MapDR数据集上的大量实验表明，该方法显著提高了可解释性和准确性，建立了首个基于推理的规则感知自动驾驶框架。

Insight: 创新点在于将思维链推理显式引入交通规则理解任务，设计了可扩展的推理依据生成流程和两阶段训练方案，通过答案导向的细粒度奖励强化学习进一步提升性能。

Abstract: Understanding and complying with traffic regulations is a safety-critical requirement for autonomous driving, yet remains challenging due to the diversity and context dependence of traffic signage. Importantly, regulation understanding is not a simple recognition task, but a reasoning problem: whether a rule applies depends on interpreting the sign in relation to the spatial layout of lanes and scene context. To support such reasoning, MapDR provide fine-grained annotations that link each traffic sign’s regulatory rules to the specific lanes they govern. Existing methods, however, largely treat this as direct sequence prediction, ignoring the underlying reasoning that connects sign semantics and map structure. To address this limitation, we explicitly incorporate reasoning into this task and propose a framework that equips vision-language models (VLMs) with chain-of-thought (CoT) capabilities. We first design a scalable CoT curation pipeline that bootstraps rationales from a strong LLM through a two-round strategy and employs a VLM-based verifier to filter out incorrect cases, yielding a high-quality set of (CoT, answer) pairs. Building on this foundation, we adopt a two-stage training scheme: supervised fine-tuning (SFT) to teach rationale-to-answer generation, followed by GRPO reinforcement learning with answer-grounded, fine-grained rewards to further improve final answer accuracy. Extensive experiments on MapDR show that our approach significantly improves both interpretability and accuracy, establishing the first reasoning-based framework for regulation-aware autonomous driving.

[124] Face Age Verification Vulnerabilities Under Simple Appearance Manipulations cs.CVPDF

Ioannis Sarridis, Ioannis Kompatsiaris, Symeon Papadopoulos

TL;DR: 本文系统研究了基于视觉的年龄验证模型对未成年人可能使用的简单外观操纵（如画胡须、涂口红）的鲁棒性。评估了七种模型（包括视觉、视觉语言和多模态大语言模型）在三个数据集和四种操纵类型下的表现，发现胡须操纵可导致高达61%的真阴性翻转为假阳性，并揭示了不同人口群体（如印度人、女性）受影响的差异性。最后探索了在线性探针设置中使用偏缓解方法减轻这些偏差的可能性。

Details

Motivation: 在线平台日益依赖自动年龄估计系统来执行最低年龄政策，但现有模型对未成年人可能用于绕过系统的简单外观改变的鲁棒性存在担忧。

Result: 在模拟的视觉操纵下，胡须茬操纵导致高达61%的真阴性翻转为假阳性；评估显示印度人群体更易受胡须操纵影响，而女性在所有操纵类型中都比男性更受影响。

Insight: 论文的创新点在于首次系统评估了年龄验证模型对简单、易实现的外观操纵的脆弱性，并量化了不同人口统计学群体间的差异性影响；客观来看，其将对抗性测试范式应用于现实世界的年龄验证场景，并探索了轻量级偏缓解方案，对实际部署具有重要警示意义。

Abstract: Online platforms increasingly rely on automated age estimation systems to enforce minimum-age policies. Focusing on vision-based models designed for this task, concerns arise regarding their robustness to simple appearance changes that underage individuals may use to bypass such systems, such as drawing a mustache or applying lipstick. In this work, we present a systematic study of age verification robustness by simulating visual alterations that can be easily achieved by underage individuals. We evaluate seven models, including vision, vision-language, and multimodal large language models, across three datasets and four manipulation types. Interestingly, under drawn beard stubble, up to 61% of True Negatives are flipped into False Positives. Furthermore, we investigate how different demographics are affected by such manipulations, finding that Indians are more affected by beard stubble manipulations, while females are more affected than males across all manipulations. Finally, we explore how these biases can be mitigated using bias mitigation methodologies in lightweight linear probe settings.

[125] MATS: A novel multi-modality multi-task learning framework for 3D perception in autonomous driving cs.CVPDF

Junchen Huo, Wanming Hao, Song Wang, Enqing Chen, Shouyi Yang

TL;DR: 本文提出了一种名为MATS的新型多模态多任务学习框架，用于自动驾驶中的3D感知。该框架通过模态自适应的BEV融合模块和任务特定的专家混合（MoE）模块，解决了现有方法中单一BEV特征图难以同时满足多个感知任务需求的问题，从而显著提升了感知性能。

Details

Motivation: 当前多模态3D感知研究通常采用复杂的融合策略，在统一的鸟瞰图（BEV）特征图上整合信息以进行多任务联合学习，但单一特征图难以承载足够信息来同时满足不同任务的需求，导致感知性能受限。

Result: 在大型基准数据集nuScenes上，使用相机和激光雷达模态输入数据，所提方法显著超越了当前最先进（SOTA）方法。单任务实验结果也显示，该方法显著优于基线模型。

Insight: 创新点在于提出了模态自适应的BEV融合模块（通过建模全局跨模态依赖来自适应地重新校准BEV特征，为不同任务生成多样化的BEV特征图）和任务特定的MoE模块（解耦任务并使网络能自动为每个特定任务选择合适的BEV特征候选），实现了更灵活高效的多模态多任务学习。

Abstract: Multi-modality data from different sensors provides rich complementary information for 3D perception, becoming an essential component in reliable autonomous driving systems. Current research typically designs intricate and complex fusion strategies to integrate information from multimodal data on a unified bird’s-eye-view (BEV) feature map for the joint learning of multiple perception tasks. However, such a single feature map hardly carries sufficient information to simultaneously meet the requirements of various perception tasks, leading to a very limited perception performance. To mitigate this limitation, this paper proposes MATS, a novel multi-modality multi-task learning approach with modality-adaptive BEV fusion and task-specific Mixture-of-Experts (MoE) for 3D perception. Specifically, a simple modality-adaptive BEV fusion module is designed to adaptively recalibrate the BEV features by modeling the global cross-modality dependencies, generating diverse BEV feature maps for various perception tasks. For joint multi-task learning, this paper proposes a task-specific MoE module to decouple the tasks and enable the network to automatically choose the appropriate BEV feature candidates for each specific task. To validate the effectiveness of the proposed approach, we conduct extensive experiments on the large-scale benchmark nuScenes. With the camera- and LiDAR-modality input data, the proposed approach outperforms the state-of-the-art (SOTA) by a significant margin. Furthermore, the experimental results on the single tasks show that the proposed approach significantly outperforms the baselines. The code and trained models will be available upon publication.

[126] FilmBench: A Film-Grade Benchmark for Cinematic Video Generation cs.CV | cs.AIPDF

Shengyi Wang, Niantong Li, Guangzheng Hu, Hong Qi, Fei Ding

TL;DR: 本文提出了FilmBench，一个基于电影专业语言构建的文本到视频（T2V）和参考到视频（R2V）生成基准测试。它从获奖电影片段中逆向工程生成提示词，并采用包含3个轴、12个组件和38个子指标的三级电影学评估分类法，旨在评估视频生成的电影级质量而非基本合理性。

Details

Motivation: 现有视频生成基准测试的提示词多来自网络或LLM模板，评估分类法（如整体视觉质量、粗略文本对齐和时间平滑度）过于基础，无法评估专业的电影级制作工艺，因此需要建立一个基于真实电影制作和评判标准的专业基准。

Result: 在领先的视频生成模型（9个T2V模型，7个R2V模型）上进行基准测试，其内部开发的专家级自动评估代理（FilmOps）在模型级别上复现了人类评估者的排名，Spearman ρ系数达到0.95（T2V）和0.96（R2V）。得分远低于先前的网络风格基准，并揭示了动态美学方面的差距以及单镜头到多镜头性能的显著下降。

Insight: 创新点在于将电影学专业标准系统性地引入AI视频生成评估，通过逆向工程真实电影片段构建高质量、多镜头的提示词集，并开发了可复现人类专家排名的自动评估框架（FilmOps），这为评估生成视频的艺术性和专业性提供了新范式。

Abstract: Progress in video generation keeps narrowing the visual gap between AI-generated and professionally produced footage, yet most benchmarks still draw prompts from web sources or LLM templates and score them with untrained, generic multimodal models. More fundamentally, their evaluation taxonomies remain rudimentary (overall visual quality, coarse text alignment and temporal smoothness) rather than the professional Cinematic Language criteria by which films are actually made and judged, so they assess basic video plausibility rather than film-grade craft. We introduce FilmBench, a text-to-video (T2V) and reference-to-video (R2V) benchmark grounded in the professional Cinematic Language of the film- academy tradition and co-developed with directors and faculty from the Beijing Film Academy and the Hujing Digital Media & Entertainment Group film studio. It rests on three choices. First, prompts are reverse-engineered from clips of award-winning films spanning 20 cinematic genres and chosen by professional directors, so every prompt is anchored to a verified live-action reference; the prompts follow real shot lists, and most script multiple shots (1,056 of the 1,169 prompts are multi-shot), unlike prior single-clip benchmarks. Second, evaluation follows a three-level Cinematic taxonomy of 3 axes, 12 components and 35 (T2V) +3 (R2V-only) sub-metrics. Third, we develop an in-house expert-grade automatic evaluation agent and open-source its core suite of Cinematic Language operators (FilmOps). Benchmarking leading video generation models (9 for T2V, 7 for R2V), the evaluator reproduces the human model ranking at model-level Spearman \r{ho} = 0.95 (T2V) and 0.96 (R2V). Scores fall well below prior web-style benchmarks, with two consistent gaps in dynamic aesthetics and a marked single- to multi-shot performance drop that widens for weaker models.

[127] PRISM: Prompt Refinement via Image-grounded Self-rewarding Mechanism for Text-to-Image Generation cs.CVPDF

Guo Tang, HongJie Luo, Tianxu Wang, Ying Zhang, Hao Wang

TL;DR: 本文提出了PRISM框架，通过基于图像的自我奖励机制来优化文本到图像生成中的提示词。PRISM通过结构化视觉诊断来解释生成的图像，并从语义一致性、美学质量和人类偏好对齐三个维度进行评分，从而形成一个提示-图像-反馈的闭环。该方法首先通过多任务监督微调初始化一个统一的视觉语言模型，然后使用结合理想点和切比雪夫距离的混合奖励进行自我奖励优化，以改进提示策略。

Details

Motivation: 现有的文本到图像生成模型对提示词的表述高度敏感，而现有的提示优化方法主要依赖文本侧重写、提示扩展或外部奖励信号，缺乏基于图像的诊断能力，且难以学习可重用的优化策略。

Result: 大量实验表明，PRISM在提升整体图像质量和细粒度语义对齐方面表现优异，同时能为目标提示词优化提供可解释的反馈。

Insight: 创新点在于构建了一个基于图像的自我奖励闭环机制，通过结构化视觉诊断和多维度评分来优化提示词，并采用混合奖励策略进行策略优化，这为提示工程提供了更可解释和可学习的框架。

Abstract: Text-to-image generation models can synthesize high-quality images from natural language descriptions, but their performance remains highly sensitive to prompt formulation. Existing prompt optimization methods mainly rely on text-side rewriting, prompt expansion, or external reward signals, offering limited image-grounded diagnosis and weak support for learning reusable optimisation policies. In this paper, we propose PRISM, a Prompt Refinement framework via Image-grounded Self-rewarding Mechanism. PRISM closes the prompt-image-feedback loop by interpreting generated images with structured visual diagnosis and scoring them along semantic consistency, aesthetic quality, and human preference alignment. It first initializes a unified VLM through multi-task supervised fine-tuning, and then improves the prompt policy via self-rewarding optimization with a hybrid ideal-point and Chebyshev reward. Extensive experiments show that PRISM improves holistic image quality and fine-grained semantic alignment, while providing interpretable feedback for targeted prompt refinement. The code is available at https://anonymous.4open.science/r/PRISM-FF81.

[128] TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation cs.CVPDF

Qijun Gan, Chenwei Zhang, Meiguang Jin, Junfeng Ma, Qiu Shen

TL;DR: 本文提出了TaoMate，一种用于实时长格式数字人生成的锚点引导持久记忆框架。该框架通过维护不可变的视觉锚点、将已生成的音视频块压缩为固定容量的动态状态，并利用模态特定的残差注意力检索这些状态，从而在保持主体外观和音视频同步的同时，避免了处理完整历史的高计算成本。

Details

Motivation: 实时长格式数字人生成依赖因果模型来扩展音视频内容，但有限的缓存会丢弃旧信息，而关注完整生成历史则计算昂贵且可能传播累积误差。本文旨在解决在保持外观和同步性的同时，高效管理长序列生成上下文的问题。

Result: 在长格式视频延续任务上，评估了外观、时序、同步性、面部和语音等多个诊断指标。结果表明，TaoMate在自回归生成下，能在提示条件片段间保持稳定的外观和强大的音视频同步性。

Insight: 核心创新在于将持久记忆（锚点和压缩的动态状态）与阶段局部去噪依赖分离，这允许跨块的阶段并行执行，从而加速自回归推理而无需针对流水线进行重新训练。锚点保持的因果上下文蒸馏方法也是一个关键设计，它在变化生成视野、前缀来源和缓存历史可靠性的同时，保持视觉锚点不变。

Abstract: Real-time long-form digital-human generation relies on causal models to extend audio-visual content while preserving subject appearance and audio-video synchronization across successive segments. A bounded cache retains local motion and phonetic context but discards older evidence, whereas attending to the complete generated history is computationally expensive and can propagate accumulated errors. We present \method, an anchor-guided persistent-memory framework for few-step joint audio-video generation. The framework preserves an immutable visual anchor, compresses completed video and audio blocks into fixed-capacity dynamic states, and retrieves those states through modality-specific residual attention without extending the active cache. A reference-aware modulation method additionally conditions video features on dynamic and anchor appearance statistics. Anchor-preserving causal-context distillation varies rollout horizon, prefix provenance, and cache-history reliability while keeping the immutable visual anchor unperturbed. By separating persistent memory from stage-local denoising dependencies, \method further admits stage-parallel execution across blocks, accelerating autoregressive inference without pipeline-specific retraining. We evaluate long-form video continuations with appearance, temporal, synchronization, facial, and speech diagnostics. Results show that \method preserves stable appearance across prompt-conditioned segments and strong audio-visual synchronization under autoregressive generation. Our project page is https://taoliveaigc.github.io/TaoMate.

[129] Mixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Grounding cs.CVPDF

Tianyi Gao, Han Fang, Tianyi Ding, Hao Li, Xin Wei

TL;DR: 本文提出了一种名为Mixture-of-Thought-Tokens（Motto）的新型自由形式多模态定位方法，旨在弥合感知与推理之间的差距。该方法通过空间锚定的思维标记化和上下文自适应的标记链，统一了精确的视觉定位和复杂的语言推理能力，并在新构建的PR-Bench基准上实现了最先进的性能。

Details

Motivation: 现有方法在统一精确的视觉定位（感知）和复杂的语言推理方面存在困难：基于文本坐标的方法限制了模型对密集视觉对象的感知能力，而基于潜在标记的方法则因缺乏空间参考和推理步骤而削弱了高级推理能力。

Result: 在广泛的实验中，Motto在多种自由形式的定位任务上实现了最先进的（SOTA）性能，特别是在新构建的用于评估感知-推理差距的PR-Bench指代表达理解基准上表现出色。

Insight: 创新点在于提出了空间锚定的思维标记化，为特殊标记提供明确的空间对应关系以增强感知，以及上下文自适应的标记链，在交错推理链中动态切换定位模式以增强推理。这为多模态大语言模型提供了一个统一的感知与推理框架。

Abstract: Multimodal Large Language Models have made great progress in grounding tasks, yet existing methods still struggle to unify precise localization and complex reasoning. For one thing, text-based methods rely on coordinates or index prediction, severely limiting the perceptual capabilities of the model for dense visual objects. Meanwhile, latent token-based methods employ special tokens without inherent spatial references and use a decoding mechanism that lacks thinking steps, weakening high-level reasoning capabilities. Consequently, developing a unified framework that excels in both perception and reasoning remains challenging. To address this, we propose Mixture-of-Thought-Tokens (Motto), a new free-form multimodal grounding method that bridges the perception-reasoning gap, enabling MLLMs to empower diverse, arbitrary grounding queries. Specifically, we introduce Spatially-Grounded Thought Tokenization to explicitly align special tokens with spatial locations for clear spatial correspondence and visual interpretability. We further design a Context-Adaptive Chain-of-Tokens that dynamically switch grounding modes within an interleaved reasoning chain, achieving robust grounding across tasks of varying complexity. In addition, we construct PR-Bench, a new referring expression comprehension benchmark to evaluate the perception-reasoning gap. Extensive experiments demonstrate that Motto achieves state-of-the-art performance across diverse free-form grounding tasks.

[130] MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning cs.CVPDF

Shaofei Lei

TL;DR: 本文提出了一种名为MAViE的多尺度自适应视觉编码器，旨在解决现有视觉语言模型中仅使用预训练视觉编码器最后一层特征所导致的文本、局部属性和空间关系信息丢失，以及高分辨率输入带来的上下文长度和推理延迟增加的问题。该方法通过位置相关的门控机制融合视觉Transformer的浅层、中层和深层特征，并基于问题相关性、局部信息内容、全局语义和空间覆盖度进行问题条件化的令牌路由，从而在保持全局语义的同时增强边缘、文本和局部结构信息。

Details

Motivation: 现有视觉语言模型通常将预训练视觉编码器产生的所有令牌投影到大型语言模型中，但最终层特征可能丢弃文本、局部属性和空间关系，且高分辨率输入会显著增加上下文长度和推理延迟。

Result: 在统一的7B语言模型框架下的模拟实验中，MAViE将SigLIP-SO400M的平均视觉令牌数从729个减少到146个（约减少80.0%），并在VQAv2、GQA、TextVQA、ScienceQA-IMG和MMBench等基准上平均得分提高了2.2个百分点，同时将单图像首令牌生成时间从228毫秒降低到129毫秒。

Insight: 创新点包括：1）通过多尺度特征融合和位置相关门控保留不同层次视觉信息；2）引入问题条件化的自适应令牌路由机制，根据图像复杂度和任务需求动态调整令牌预算；3）采用全到压缩表示蒸馏和空间多样性正则化来缓解压缩损失。这些设计有望提升细粒度视觉感知和高效多模态推理能力。

Abstract: Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features can discard text, local attributes, and spatial relationships, while high-resolution inputs substantially increase context length and inference latency. We introduce \method, a Multi-scale Adaptive Vision Encoder. \method uses position-dependent gates to fuse shallow, intermediate, and deep features from a vision Transformer, preserving global semantics while enhancing edges, text, and local structure. It then performs question-conditioned token routing according to question relevance, local information content, global semantics, and spatial coverage, with a token budget that adapts to image complexity. To mitigate compression loss, we further introduce full-to-compressed representation distillation and a spatial diversity regularizer. In an illustrative simulation under a unified 7B language-model framework, \method reduces the average number of SigLIP-SO400M visual tokens from 729 to 146 (approximately 80.0%) and improves the mean score on VQAv2, GQA, TextVQA, ScienceQA-IMG, and MMBench by 2.2 percentage points, while reducing single-image time to first token from 228,ms to 129,ms. We provide the full model design and evaluation protocol. All reported numbers currently serve only as placeholders for paper organization and experimental design; formal claims require real training runs, independent replications, and official benchmark evaluation.

[131] Rethinking Expert Training for Model Merging with Prompt Learning cs.CVPDF

Christos Georgakilas, Aniello Panariello, Samir El Karrat Moreno, Simone Calderara, Dimosthenis Karatzas

TL;DR: 本文重新审视了模型合并中的专家训练方法，提出了一种名为双调优专家（DTEs）的两阶段训练策略。该方法首先通过提示学习进行适应，然后微调视觉编码器，以减少任务特定参数更新的幅度并提高专家模型的合并兼容性。

Details

Motivation: 现有模型合并方法主要关注改进合并过程本身，并通常假设专家是通过全参数微调获得的。本文旨在重新思考专家训练阶段，以提升合并后模型的性能。

Result: 在多个CLIP架构、全微调和LoRA专家上的实验表明，DTEs能持续提升标准合并方法的性能，并且在合并异构专家集时仍然有效。

Insight: 创新点在于揭示了基于提示的适应是一个强大的基线，并提出了先学习提示再微调编码器的两阶段策略，这减少了参数干扰并提高了合并兼容性，为模型合并提供了新的专家训练视角。

Abstract: Model merging aims to combine multiple domain-specialized experts trained from a shared foundation model into a single multi-task model. Existing approaches largely focus on improving the merging procedure itself and typically assume experts obtained through full-parameter fine-tuning. In this work, we revisit expert training for model merging. We first show that prompt-based adaptation provides a strong baseline: independently learned prompts can be exploited across tasks while keeping the backbone fixed, avoiding the interference introduced by weight merging. Building on this observation, we introduce Dual-Tuned Experts (DTEs), a two-stage training strategy that first learns prompts and then fine-tunes the vision encoder. This reduces the magnitude of task-specific parameter updates and produces experts with higher merge compatibility. Experiments across multiple CLIP architectures, full fine-tuning, and LoRA experts show that DTEs consistently improve merged performance of standard merging approaches and remain effective even when combining heterogeneous sets of experts.

[132] RP-OPSD: Resolution-Privileged On-Policy Self-Distillation for Multimodal Large Language Models cs.CVPDF

Qihui Zhu, Yuchen Wang, Zijian Wen, Tao Zhang, Mengjie Zhang

TL;DR: 本文提出了RP-OPSD方法，一种用于多模态大语言模型的、基于分辨率差异的策略上自蒸馏方法。该方法利用高分辨率图像作为教师模型的私有信息，对低分辨率输入下的学生模型生成的轨迹进行密集监督，从而提升模型在低分辨率下的能力，并将改进迁移到原始分辨率推理中。

Details

Motivation: 现有策略上自蒸馏方法依赖于已验证的解决方案轨迹、外部模型生成的解释或手动定位的视觉证据，这限制了其在多模态大语言模型中的可扩展应用。本文旨在利用高、低分辨率图像视图之间的信息差，提供一个无需额外人工标注或外部模型的、简单且可扩展的自蒸馏方案。

Result: 在Qwen3.5-9B模型上的实验表明，RP-OPSD在原始分辨率下实现了平均性能5.45%的相对提升，并且相比OPSD获得了1.78倍的训练加速。

Insight: 创新点在于将图像分辨率差异作为私有信息的简单且可扩展的来源，用于策略上自蒸馏。这避免了对外部资源（如人工标注、额外模型）的依赖，仅需图像-问题对即可实现有效的模型能力提升和训练加速。

Abstract: On-Policy Self-Distillation (OPSD) uses privileged information available only to the teacher to provide dense token-level supervision on trajectories generated by the student. However, existing methods often rely on verified solution traces, explanations generated by external models, or manually localized visual evidence, which limits their scalable application to multimodal large language models. To address this issue, we exploit the information gap between high- and low-resolution views of the same image and propose RP-OPSD (Resolution-Privileged On-Policy Self-Distillation for Multimodal Large Language Models). During training, the student policy generates on-policy trajectories from images at one-quarter of the original resolution, while the teacher policy provides supervision using the original-resolution images. By minimizing the divergence between their output distributions along the student trajectories, the student learns the predictive behavior of the teacher under high-resolution inputs, thereby strengthening its low-resolution capability and transferring the learned improvement to original-resolution inference. RP-OPSD requires neither additional human annotations nor external models to generate solution traces but only image–question pairs. Experiments on Qwen3.5-9B show that RP-OPSD achieves a 5.45% relative improvement in average performance at the original resolution and a $1.78\times$ training speedup over OPSD. These results demonstrate that resolution differences can serve as a simple and scalable source of privileged information, providing an effective and efficient approach to on-policy self-distillation for multimodal large language models.

[133] ESRVS: Extreme Semi-Supervised Retinal Vessel Segmentation with a Single Annotated Image cs.CV | cs.AI | cs.LGPDF

Mingzhi Xu, Yizhe Zhang

TL;DR: 本文提出了一种极端半监督视网膜血管分割方法ESRVS，该方法仅需一张标注图像和大量未标注图像。通过选择代表性参考图像进行人工标注，并利用目标域适应的DINOv3特征进行血管线索迁移，构建多粒度血管原型，结合原型相似性映射和物理先验生成初始伪标签，再通过加权伪标签训练和对抗性细化来优化监督信息。

Details

Motivation: 解决医学图像分析中密集专家标注成本高昂的问题，探索在极端半监督设置下（仅一张标注图像）进行视网膜血管分割，以实现最小化人工监督的学习目标。

Result: 在八个公共数据集上，ESRVS在六个数据集上取得了最佳的Dice和clDice分数，在所有八个数据集上均获得了最佳的HD95分数，优于使用10%至20%标注数据的其他半监督方法。结合Mask2Former时，ESRVS平均保留了93.7%的全监督Dice分数和95.1%的全监督clDice分数。

Insight: 创新点包括利用目标域适应的基础模型（DINOv3）特征进行标签传播，构建多粒度血管原型并结合物理先验生成伪标签，以及通过加权训练和对抗性细化来优化监督转移，为高标签效率的视网膜血管分割提供了新思路。

Abstract: Learning from minimal human supervision is a long-standing goal in medical image analysis, where dense expert annotations are costly. We study retinal vessel segmentation in an extreme semi-supervised setting with one annotated image and a pool of unlabeled images. We propose ESRVS, which selects a representative reference image for manual annotation and transfers vessel cues using target-domain-adapted DINOv3 features. ESRVS constructs a multi granular vessel prototype, combines prototype-similarity maps with a physics-inspired prior to generate initial pseudo-labels, and refines the transferred supervision through weighted pseudo-label training and adversarial refinement. Across eight public datasets, ESRVS achieves the best Dice and clDice on six datasets, and the best HD95 on all eight datasets among the compared semi-supervised methods, although those methods use 10 to 20% labeled data. With Mask2Former, ESRVS retains on average 93.7% of fully supervised Dice and 95.1% of fully supervised clDice. These results demonstrate the potential of foundation-model label propagation for highly label-efficient retinal vessel segmentation. Code is available at https://github.com/IAANNH/ESRVS.

[134] NSL-SLAM: High-Fidelity Neural Structured-Light Depth for Practical SLAM and Reconstruction cs.CVPDF

Jiaheng Li, Binsheng Zhang, Xinhai Chang, Wenzheng Chen

TL;DR: NSL-SLAM是一个专为高保真结构光深度设计的实用SLAM系统。它通过将单目深度先验融入神经结构光解码，显著提升了深度感知质量，并构建了一个以深度为中心的SLAM流程，利用密集且度量准确的几何信息进行跟踪，仅辅以稀疏视觉对应和轻量级光束法平差来处理退化情况与长程漂移。

Details

Motivation: 结构光相机广泛应用于深度感知，但现有SLAM系统未能充分利用其高质量的深度信息。本研究旨在通过结合神经解码与单目先验来增强结构光深度，并设计一个深度优先的SLAM流程，以实现更稳定、更精确的跟踪与重建。

Result: 在合成数据集Replica-SL上，NSL-SLAM取得了最佳的跟踪精度，并在共享深度协议下将重建F-score比SOTA基线提升了1.6个点。在包含8个挑战性场景的真实基准测试中，它是唯一在所有序列上都避免灾难性失败的方法，且轨迹偏差比所选基线降低了43.3%。系统在线运行速度为20.9 FPS。

Insight: 创新点在于将单目深度先验整合到神经结构光立体解码中，显著提升了深度估计精度；同时，设计了一个以深度为中心的SLAM架构，将高质量的深度作为主要跟踪信号，简化了系统流程并提升了鲁棒性，实现了深度估计与SLAM设计的相互增强。

Abstract: Structured-light (SL) cameras power depth sensing in millions of devices, and recent neural SL decoding methods have substantially improved their depth quality. SLAM systems can benefit greatly from such strong depth sensing, where reliable geometry enables stable tracking and faithful reconstruction. In this work, we present NSL-SLAM, a practical SLAM system tailored for high-fidelity structured-light depth. We first strengthen SL depth sensing: inspired by the neural structured-light (NSL) method, we further incorporate strong monocular depth priors into the SL stereo decoding, reducing depth RMSE by 35% on Replica-SL compared to NSL. We then build a depth-centric SLAM pipeline with this stronger depth: because structured-light geometry is dense and metrically accurate, we keep it as the primary tracking signal, and add only sparse visual correspondences for geometrically degenerate cases and lightweight bundle adjustment for long-range drift. Our depth estimator and SLAM design reinforce each other: stronger depth makes a simple SLAM pipeline effective, and the depth-centric pipeline ensures this advantage transfers to downstream reconstruction. Experimentally, on the synthetic Replica-SL benchmark, NSL-SLAM achieves the best tracking accuracy and improves reconstruction F-score by 1.6 points over the SOTA baseline under a shared-depth protocol. On a real benchmark of 8 challenging scenes, it is the only method that avoids catastrophic failure on all sequences while achieving 43.3% lower trajectory deviation than selected baselines. The SLAM system runs online at 20.9 FPS, demonstrating that stronger structured-light depth and depth-centric system design together enable practical, robust SLAM.

[135] DecoupleMix: Decoupled Ratio Search and Convex Allocation for Scalable VLM Data Recipes cs.CV | cs.AIPDF

Jiahao Xie, Zhongbin Guo, Qianle Wang, Ruiqi Lu, Dongling Xiao

TL;DR: 本文提出DecoupleMix框架，将视觉语言模型（VLM）预训练数据构建问题系统化为混合优化问题，通过解耦为跨能力类别的比例搜索和类别内数据集的凸优化分配，实现可扩展、可复现的数据配方设计。该方法能指导后续数据收集，并将数据集验证转化为可控实验。

Details

Motivation: 当前VLM数据构建实践多依赖启发式方法，缺乏原则性、可归因的标准来设定跨领域数据比例和纳入新数据，且前沿数据配方未公开。本文旨在将数据构建转化为系统化的混合优化问题，使其成为可复现的工程学科。

Result: 实验表明，DecoupleMix方法在多个基准测试上 consistently 超越启发式基线。此外，在小规模代理上发现的最优比例无需重新调整即可无缝迁移到更大规模。使用额外800亿多模态继续预训练tokens，所训练的VLM与使用更大多模态预算训练的强开源模型具有竞争力。

Insight: 创新点在于将数据混合问题解耦为两个正交子问题（跨类别比例搜索和类别内凸优化分配），并引入基于质量和难度的多维数据集级评估，将数据集验证转化为可控、可归因的实验。这为可扩展的VLM数据配方提供了系统化框架。

Abstract: While data curation for Vision Language Models (VLMs) is increasingly active, public practice for constructing pretraining mixtures remains largely heuristic: practitioners stack datasets that pass quality filters, set cross-domain ratios by intuition, and lack a principled, attributable criterion for admitting new data, while frontier recipes remain undisclosed. We formulate data construction as a systematic mixture-optimization problem and turn it into a reproducible engineering discipline by decoupling the mixture into two orthogonal sub-problems: inter-class ratios across capabilities and intra-class ratios within a category. For inter-class allocation, we use a single-variable iterative search; for intra-class composition, we apply a multidimensional, dataset-level assessment scoring Quality and Difficulty, and formulate selection as a constrained convex optimization with a diversity objective. The DecoupleMix framework delivers two critical capabilities: guiding what data to collect next and rendering dataset validation a controlled, attributable experiment. Experiments show our approach consistently surpasses heuristic baselines. Moreover, optimal ratios discovered on small-scale proxies transfer seamlessly to larger scales without retuning. Using 80B additional multimodal continue-pretraining tokens, our VLM is competitive with strong open-source models trained with substantially larger multimodal budgets.

[136] The Visual Bottleneck: Sparse-Frame Adaptation of MLLMs for Joint Spatial-Temporal Video Grounding cs.CV | cs.AIPDF

Jiameng Zhang, Srikanth Madikeri

TL;DR: 本文针对大规模视频平台中稀疏帧输入（如每视频8-16帧）下多模态大语言模型（MLLMs）在时空视频定位任务上性能严重下降的问题，通过系统实证研究提出训练策略以弥合训练与部署条件间的差距。研究发现视觉特征提取是主要瓶颈，仅微调ViT最后三层即可显著提升性能，且边界感知采样策略进一步改善时序定位精度。

Details

Motivation: 解决大规模视频平台因处理能力限制只能使用稀疏帧输入时，现有基于密集帧预训练的MLLMs在时空视频定位任务上出现的严重性能退化问题，旨在弥合训练与部署条件之间的不匹配。

Result: 在稀疏帧输入（16帧）下，仅微调ViT最后三层（占总参数4%）使Qwen3-VL 8B模型的时序mIoU从22.3%提升至68.8%，超越使用密集帧输入的零样本8B模型12.8个百分点；结合边界感知采样策略Hybrid16，时序mIoU较均匀采样提升26点。微调后的2B模型在有无密集帧访问下均持续优于零样本8B模型。

Insight: 创新点在于揭示稀疏帧视频定位中视觉特征提取是关键瓶颈，而非语言模型微调；提出高效的参数微调策略（仅调整少量视觉层）和边界感知采样方法，证明训练策略优于模型规模，为资源受限场景下的视频理解提供了轻量级适配方案。

Abstract: Large-scale video platforms process millions of uploads hourly, requiring moderation systems that can localize when and where policy violations occur within each video. Processing every frame is infeasible at scale, so systems are constrained to sparse inputs of 8 to 16 frames per video. Yet state-of-the-art multimodal large language models (MLLMs) are pretrained on dense sequences of hundreds of frames, creating a fundamental mismatch between training and deployment conditions. This mismatch causes severe performance collapse: the Qwen3-VL 8B model drops from 56.0% to 22.3% temporal mIoU when frames are reduced to 16, a 60.2% relative degradation. We present a systematic empirical study of training strategies to close this gap for spatial-temporal video grounding. Our results suggest that visual feature extraction is the dominant bottleneck under sparse-frame inputs. Adapting only the final three ViT layers, 4% of total parameters, achieves 68.8% temporal mIoU and surpasses a zero-shot 8B model using dense inputs by 12.8 points. Language model fine-tuning, by contrast, offers negligible or negative returns. A boundary-aware sampling strategy, Hybrid16, further improves temporal mIoU by 26 points over uniform sampling when temporal boundaries are available. We conclude that for sparse-frame video grounding, training strategy dominates model scale: a fine-tuned 2B model consistently outperforms a zero-shot 8B model, with or without dense frame access.

[137] CameraAnything: Refilming Videos with Arbitrary Camera Control cs.CVPDF

Yixuan Li, Yanhong Zeng, Ka Leong Cheng, Jiayi Zhu, Hanlin Wang

TL;DR: CameraAnything是一个统一的相机控制视频编辑框架，首次实现了对相机内参（如焦距、分辨率）和外参（如位置）的联合控制。它通过引入逐像素普吕克射线注入和分辨率感知的3D RoPE来建模相机参数对视频外观的耦合影响，并利用可扩展的合成数据管道解决训练数据稀缺问题。该框架支持在单次生成过程中进行任意视点控制、焦距调整、分辨率适应和多镜头过渡，为影视编辑和跨平台内容适配提供了实用价值。

Details

Motivation: 现有方法要么依赖昂贵的3D重建来实现完整相机控制，要么仅限于外参编辑，且内参与外参的耦合影响使得解耦建模极具挑战。

Result: 论文未在摘要中提及具体定量结果或基准测试，但强调框架能够实现富有表现力的视频重拍，具有强大的实用价值。

Insight: 创新点包括采用逐像素普吕克射线注入与分辨率感知3D RoPE进行联合相机参数编码，以及开发结构化多相机录制合成数据管道来克服数据稀缺，实现了无需裁剪或外绘的端到端相机控制视频编辑。

Abstract: We introduce CameraAnything, the first unified framework for camera controlled video editing that enables joint control of both intrinsic and extrinsic camera parameters. Existing approaches either rely on expensive 3D reconstruction to achieve full camera functionality or restrict editing to extrinsic parameter manipulation. Moreover, the coupled influence of intrinsic and extrinsic parameters on video appearance makes disentangled modeling particularly challenging. To address this, we adopt per-pixel Plücker ray injection alongside resolution-aware 3D RoPE in self-attention, building both camera conditioning and spatial positional encoding on the target latent to jointly control camera position, focal length, and native resolution editing without cropping or outpainting. To overcome the scarcity of paired training data, we further develop a scalable synthetic pipeline that constructs diverse dynamic scenes through structured multi-camera recording and generates synchronized videos with varied camera configurations. With a tailored orthogonal training strategy, CameraAnything enables expressive video reshooting with arbitrary viewpoint control, focal length adjustment, resolution adaptation, and multi-shot transitions within a single generation process, offering strong practical value for cinematic video editing and cross-platform content adaptation in video production.

[138] CADER: Confidence-Aware Dynamic Evidence Reasoning for Long-Video Understanding cs.CV | cs.AIPDF

Jinlong Yang, Wenhao Zhang, Kuanwei Lin, Sijie Cheng

TL;DR: CADER是一个无需训练的自适应长视频理解框架，通过置信度感知的动态证据推理机制，根据问题难度灵活调整推理过程。它首先对均匀采样的帧进行全局推理并评估答案置信度，高置信度样本直接输出结果；对于不确定样本，则激活第二阶段工具增强循环，结合时间裁剪、轻量语义验证和相关性引导重采样来逐步定位问题相关证据。

Details

Motivation: 现有长视频理解系统通常对所有样本采用统一的推理流程，导致简单问题调用不必要的工具处理，而困难问题又缺乏细粒度时间证据的精细控制。论文旨在解决这种一刀切策略的效率与可靠性问题。

Result: 在多个VideoQA基准测试上的实验表明，CADER提升了长视频推理性能，同时为高置信度样本绕过了第二阶段处理。此外，当应用于仅使用无工具思维链监督训练的主干模型时，CADER取得了与专用工具增强框架相当的性能。

Insight: 核心创新在于将工具使用视为样本级决策，通过置信度估计实现自适应推理流程，在无需额外训练的情况下，为困难样本动态引入细粒度证据定位机制，提供了一种实用的推理时自适应长视频理解路径。

Abstract: Long-video understanding increasingly relies on large vision-language models and tool-augmented reasoning, but most systems apply the same inference procedure to every example regardless of difficulty. This uniform strategy invokes unnecessary tool-assisted processing for easy questions and provides limited control when difficult questions require fine-grained temporal evidence. We propose CADER (Confidence-Aware Dynamic Evidence Reasoning), a training-free framework for adaptive and reliable long-video reasoning. CADER first performs global reasoning over uniformly sampled frames and estimates answer confidence with a logit-margin signal, allowing high-confidence examples to exit early. For uncertain examples, CADER activates a second-stage tool-augmented loop that combines temporal cropping, lightweight semantic verification, and Relevance-Guided Resampling to progressively localize question-relevant evidence. This design treats tool use as a sample-level decision: a single global pass handles easy cases, while additional reasoning is reserved for examples where uncertainty suggests that more evidence is needed. Experiments on multiple VideoQA benchmarks show that CADER improves long-video reasoning while bypassing Stage~2 for high-confidence samples. Moreover, when applied to a backbone trained only with tool-free chain-of-thought supervision, CADER achieves competitive performance against specialized tool-augmented frameworks, suggesting a practical inference-time route for adaptive long-video reasoning.

[139] EgoPlay: Event-Triggered Video Editing for Egocentric Streams cs.CV | cs.AIPDF

Jinjie Mai, Gordon Guocheng Qian, Willi Menapace, Arpit Sahni, Chaoyang Wang

TL;DR: 本文提出了EgoPlay，一种用于第一人称视角视频流的事件触发视频编辑方法。该方法通过在基于Ego4D构建的事件条件数据上微调预训练的V2V扩散Transformer，实现了对单目视频的编辑。给定视频和形如‘当X发生时，执行Y’的提示，EgoPlay能推断事件X是否发生及发生时间，保留事件前帧，并仅对事件后的视频部分应用编辑Y。

Details

Motivation: 解决现有方法需要级联独立的事件检测器和编辑器，无法端到端联合处理事件识别、时序约束和像素级编辑的问题，旨在为第一人称视频流提供更高效、统一的事件触发编辑方案。

Result: 在Ego4D基准测试中，EgoPlay大幅超越了基于指令的第一人称视频编辑SOTA基线EgoEdit，在编辑质量、视觉质量和背景一致性上分别获得17.7%、16.9%和16.4%的相对提升；同时，在相同指标上超越了VLM引导的检测器-编辑器基线15.7%、14.5%和13.5%，且GPU内存使用量减半。

Insight: 主要创新点在于提出了一个端到端模型，联合学习事件识别、时序约束和像素编辑，并引入了事件感知评估协议。客观来看，其构建的大规模事件触发剪辑-提示对数据集，以及用于流式推理的因果变体设计，是推动性能提升的关键因素。

Abstract: We introduce EgoPlay, an event-triggered video-to-video editor for egocentric streams, obtained by fine-tuning a pretrained V2V diffusion transformer on event-conditioned data built primarily from Ego4D. Given a monocular video and an event-triggered prompt of the form “when X happens, do Y,” EgoPlay infers whether and when event X occurs, preserves pre-event frames, and applies edit Y only to the post-event continuation. Rather than cascading a separate event detector with an editor, EgoPlay learns event recognition, temporal restraint, and pixel-level editing jointly in a single end-to-end model, while also handling negative and multi-event prompts. To support this, we construct a large-scale dataset of 106K event-triggered clip-prompt pairs spanning positive triggers, fabricated-trigger negatives, and multi-event prompts. We then train a bidirectional video diffusion editor with event-triggered supervision and derive a causal variant for chunk-by-chunk streamable inference. We further introduce an event-aware evaluation protocol that separately measures post-trigger editing quality, pre-trigger preservation, and false-trigger robustness. On the Ego4D benchmark, EgoPlay substantially outperforms EgoEdit, the state-of-the-art instruction-based egocentric video editing baseline, with relative gains of 17.7%, 16.9%, and 16.4% in editing quality, visual quality, and background consistency. It also surpasses a VLM-guided detector-editor baseline by 15.7%, 14.5%, and 13.5% on the same metrics, while using less than half the GPU memory.

[140] Test-Time Adaptation via Dual Distillation for Videos Under Severe Distribution Shifts cs.CVPDF

André Sacilotti, Samuel Felipe dos Santos, Jurandy Almeida

TL;DR: 本文提出了一种名为TADD（Test-time Adaptation via Dual Distillation）的视频测试时自适应框架，旨在解决深度学习模型在面临严重分布偏移的视频数据时性能下降的问题。该方法基于冻结的CLIP骨干网络，仅更新一个轻量级的投影适配器，通过零样本蒸馏和目标蒸馏两种互补损失，在推理时在线适应目标域数据，并在多个视频动作识别基准上取得了优于现有方法的性能。

Details

Motivation: 深度学习模型在真实世界视频场景中常因未预料到的分布偏移而导致性能严重下降，而现有的测试时自适应（TTA）方法在处理连续、时间相关的视频数据及严重域偏移时仍面临挑战，相关研究较少。

Result: 在UCF-HMDB、Daily-DA和Sports-DA三个视频动作识别基准的闭集场景评估中，该方法均优于现有SOTA TTA基线，分别提升了最高+3.81%、+2.63%和+3.03%。

Insight: 创新点在于提出了一个双蒸馏框架，结合了零样本蒸馏（利用预训练视觉语言模型的域无关特征）和目标蒸馏（保留预训练适配器中的源域判别知识），并仅通过一个轻量级投影适配器进行在线更新，实现了高效且鲁棒的视频测试时自适应。

Abstract: Deep learning models have achieved state-of-the-art performance in several computer vision tasks. However, they experience severe performance degradation when applied to real-world scenarios due to unanticipated distribution shifts. Test-Time Adaptation (TTA) attempts to solve this problem by using unlabeled data from the target domain to dynamically adapt to the test distribution at inference time, without access to the source data. However, TTA remains a challenging problem when adapting to continuous, temporally correlated data, such as videos, and in scenarios where the target domain contains severe domain shifts. For this reason, few works in the literature explore TTA for videos under such extreme conditions. To overcome these limitations, we propose Test-time Adaptation via Dual Distillation (TADD), an online adaptation framework that relies on a lightweight projection adapter to bridge the domain gap. The adapter module is pre-trained on the source domain and then adapted to the target using our proposed complementary losses: (i) zero-shot distillation, which encourages alignment with the domain-agnostic features from a pre-trained vision-language model (VLM); and (ii) target distillation, which retains the source domain discriminative knowledge encoded in the pre-trained adapter. Built upon a frozen CLIP backbone, our method introduces this lightweight projection adapter as the sole updatable component during inference. We conducted extensive evaluations on three well-known video action recognition benchmarks: UCF-HMDB, Daily-DA, and Sports-DA. Our experiments in the closed-set scenario demonstrate that our method consistently outperforms state-of-the-art TTA baselines. Notably, our TTA approach improves upon previous methods by up to +3.81% on UCF-HMDB, +2.63% on Daily-DA, and +3.03% on Sports-DA.

Francisco Mena, Dino Ienco, Roberto Interdonato, Cassio F. Dantas, Simon Besnard

TL;DR: 本文提出了一种多模态协同学习框架，用于处理训练与推理阶段模态任意缺失的分类问题。该方法通过特征级和决策级两种策略促进模态间协作，而非传统多模态融合，在两种基准测试中显著提升了缺失模态条件下的鲁棒性。

Details

Motivation: 现实场景中由于传感器故障或隐私限制，多模态数据在训练和推理时可能出现模态不一致性，而现有方法多局限于双模态且依赖预定义缺失模式，无法处理任意模态缺失的挑战。

Result: 在两个多模态分类基准测试中，所提方法在多种缺失条件下均取得显著鲁棒性提升：第一种方法在仅缺失单一模态时表现更稳健，第二种方法在极端缺失（仅保留一个模态）时性能更优。

Insight: 创新点在于将焦点从多模态融合转向模态间协同学习，并首次系统处理任意模态缺失场景；通过特征级与决策级信息互补机制，为动态缺失环境提供了灵活解决方案。

Abstract: Multi-modal classification leverages complementary information across diverse data sources to enhance predictive performance. However, real-world scenarios subject to operational constraints, such as sensor failures or privacy restrictions, lead to inconsistent modality availability between training and inference times. To handle missing modalities, prior studies have mainly covered bimodal data setups and focused on designing robust fusion processes. Instead, we adopt a multi-modal co-learning framework that prioritizes inter-modal collaboration rather than multi-modal fusion. Specifically, we consider that any subset of modalities may be absent, without assuming predefined missing-modality patterns, an inference scenario we refer to as missing arbitrary modalities. To address this challenge, we introduce two alternative approaches that leverage information at both feature- and decision-level. Experiments on two multi-modal classification benchmarks demonstrate significant robustness gains in various missing modality conditions. The first method shows more robust behavior under minimal missing conditions, where a single modality is absent, whereas the second performs better under extreme missing conditions, where all-but-one modalities are missing. Our code is available at https://github.com/fmenat/Co4Miss.

[142] QueenVIS: Rethinking Image-Only Training for Video Instance Segmentation via Query Enrichment cs.CVPDF

Arian Kheirandish, Fardin Ayar, Ehsan Javanmardi, Manabu Tsukada, Mahdi Javanmardi

TL;DR: QueenVIS提出了一种基于查询增强的图像级训练视频实例分割方法，通过增强Mask2Former查询的判别能力和时空稳定性，在不使用视频级监督的情况下实现了与视频训练方法相竞争的性能。该方法在训练时引入特征预测损失和中心预测损失两个辅助头来对齐查询与实例特征，并在推理时丢弃这些头，结合无训练的查询传播和记忆库方案来维持时序一致性。

Details

Motivation: 当前视频实例分割方法大多依赖成本高昂的视频级监督和复杂的视频训练跟踪器，而图像级训练方法（如MinVIS）虽然避免了视频训练，但其查询质量不足导致跨帧关联不稳定。本文旨在通过增强图像训练查询的判别能力和时空稳定性，探索图像级训练在视频实例分割中的潜力。

Result: 在YouTube-VIS和OVIS数据集上，使用ResNet-50骨干网络，QueenVIS相比MinVIS在YouTube-VIS上提升高达6.7 AP，在OVIS上提升4.8 AP，在长序列YouTube-VIS分割上提升10.3 AP，最终在YouTube-VIS上达到50.9 AP，与近期视频监督的SOTA方法保持竞争力。

Insight: 论文的创新点在于提出了一种查询中心的框架，通过特征预测损失和中心预测损失在单帧训练中增强查询的判别能力和空间结构，并在推理时采用无训练的查询传播和记忆库来维持时序身份，这为不依赖视频监督的VIS方法提供了新的研究方向，强调了提升查询质量的重要性。

Abstract: Video instance segmentation (VIS) requires models to detect, segment, and track object identities across frames, and most methods enforce temporal consistency through video-level supervision. Image-only training approaches, with MinVIS as one prominent example, have challenged this assumption, reaching competitive VIS without video training by treating frames as independent images and associating instances only at inference. The field has nonetheless moved toward ever more elaborate video-trained trackers, which depend on costly identity-consistent annotations, leaving the image-only direction under-explored. A diagnostic analysis identifies object query quality as the bottleneck: queries trained only to localize objects within a frame drift apart across frames and destabilize association. QueenVIS introduces a query-centric framework for strengthening image-trained VIS. During single-frame training, we enrich Mask2Former queries with two auxiliary heads: a feature-prediction loss that aligns each query with the pooled backbone descriptor of its instance, and a center-prediction loss that injects spatial structure. Both heads are discarded at inference, adding zero parameters, and temporal identity is maintained by a training-free query-propagation and memory-bank scheme. On YouTube-VIS and OVIS with a ResNet-50 backbone, QueenVIS improves over MinVIS, up to +6.7 AP on YouTube-VIS, +4.8 AP on OVIS, and +10.3 AP on the long-sequence YouTube-VIS split. QueenVIS achieves 50.9 AP on YouTube-VIS and remains competitive with recent video-supervised state-of-the-art, without processing a single video clip during training. Our findings suggest that strengthening the discriminative power and temporal stability of object queries is an important, underexplored axis for VIS. Code and models: https://github.com/ArianKheir/QueenVIS

[143] Spatio-Temporal Conditional Denoising Transformer for Modality-Missing RGBT Tracking cs.CVPDF

Andong Lu, Ziyi Zha, Jiandong Jin, Shihao Li, Chenglong Li

TL;DR: 本文提出了一种时空条件去噪变换器（SCDT），用于解决RGBT跟踪中模态缺失导致的多模态特征表示不完整和不稳定的问题。该方法通过整合空间线索和长短期时间上下文，在统一框架中自适应地重建缺失模态信息并增强弱模态特征，从而提升跟踪的鲁棒性。

Details

Motivation: 现有方法在RGBT跟踪中处理模态缺失时，通常尝试从可用模态中恢复缺失数据，但在复杂场景下生成的数据质量可能不佳，且现有方法在处理完整和缺失数据时灵活性有限。

Result: 在三个公开基准数据集上的大量实验表明，该方法在RGBT跟踪任务中一致性地超越了当前最先进的方法。

Insight: 创新点在于提出了一个统一的时空条件去噪变换器框架，它联合利用长短期时间上下文作为条件来引导噪声特征学习，并引入了噪声调制适应机制，能够根据模态可用性动态调整行为，从而无需改变架构或参数即可同时处理模态缺失和完整场景。

Abstract: Missing modalities in RGBT tracking often lead to incomplete and unstable multimodal feature representations that greatly degrade the performance. Existing methods typically attempt to recover missing modalities from available ones, but the quality of data generated in challenging scenarios might be unsatisfactory. In addition, current approaches exhibit limited flexibility in processing both missing and complete data. To overcome these limitations, we propose a Spatio-temporal Conditional Denoising Transformer (SCDT), which integrates the spatial cues and the temporal context to adaptively perform information reconstruction of missing modalities and feature enhancement of weak modalities in a unified framework, for robust modality-missing RGBT tracking. In particular, SCDT leverages the short-term temporal cues from recent historical frames to capture the fine-grained temporal correlations and the long-term temporal cues encoding modality evolution to capture the global context. By jointly exploiting long short-term temporal contexts as the conditions, SCDT progressively guides noisy features of available modalities to learn reliable and temporally consistent multimodal representations. Furthermore, SCDT introduces a noisemodulated adaptation mechanism that dynamically adjusts its behavior according to the modal availability, enabling a single framework to unify feature learning under both modality-missing and complete scenarios without changing the architecture or parameters. Extensive experiments on three public benchmark datasets demonstrate that our method consistently outperforms state-of-the-art methods. The code is available here.

[144] Panda: Unsupervised Pelvic Anomaly Detection for Real-Time MR Imaging cs.CVPDF

Anika Knupfer, Maximilian Lindholz, Johanna Paula Müller, Jordina Aviles Verdera, Smiti Tripathy

TL;DR: 本文提出了一种名为Panda的无监督异常检测框架，专门用于实时盆腔MRI成像中的异常检测。该框架利用DINOv3视觉Transformer编码器结合噪声MLP瓶颈和线性注意力解码器，从健康病例中学习正常表示，无需标注即可标记异常。在子宫肌瘤数据集上的评估显示，该方法在像素级AUROC达到88.06%，帧级特异性为95.45%，处理速度为40.5切片/秒，满足实时临床部署需求。

Details

Motivation: 女性盆腔疾病诊断常被延迟，盆腔MRI虽提供优越的软组织对比度，但实时异常检测因生理运动、组织变形和仪器伪影而具挑战性。现有监督方法不实用，因为不良事件罕见、异质且难以标注。

Result: 在子宫肌瘤数据集的精选子集上评估，框架达到像素级AUROC 88.06%，帧级特异性95.45%（处理速度40.5切片/秒），满足实时临床部署要求，提供空间异常图和帧级分数以支持放射科医生决策。

Insight: 创新点包括将Dinomaly框架适配于盆腔MRI，结合冻结DINOv3编码器、噪声MLP瓶颈和线性注意力解码器防止恒等映射，并通过编码器-解码器表示间的每令牌余弦距离实现异常定位，提供实时空间反馈。

Abstract: Female pelvic diseases remain an under researched area characterized by often delayed diagnosis. While pelvic MRI offers superior soft-tissue contrast for diagnosis and image-guided procedures, real-time anomaly detection remains challenging due to physiological motion, tissue deformation, and instrument artifacts. Existing supervised approaches are impractical, as adverse events are rare, heterogeneous, and difficult to annotate. We present a Dinomaly-based unsupervised anomaly detection framework adapted for pelvic MRI that learns normative representations from healthy cases and flags deviations without requiring labels. Our approach leverages a frozen DINOv3 Vision Transformer encoder combined with a noisy MLP bottleneck and Linear Attention decoder to prevent identity mapping while maintaining computational efficiency. Anomalies are localized via per-token cosine distance between encoder and decoder representations, yielding spatial anomaly maps that provide immediate feedback at the scanner to support radiologist decision-making and adaptive protocol adjustment. Evaluated on a curated subset of the Uterine Myoma Dataset, the framework achieves a pixel-level AUROC of 88.06% and high specificity (95.45%) at frame level at 40.5 slices/s, meeting real-time clinical deployment requirements. The spatial anomaly maps and frame-level scores provide immediate, localized feedback at the scanner to support radiologist decision-making and adaptive protocol adjustment during active procedures.

[145] DreamStyle3D: Efficient 3D Stylized Asset Generation via Dual-Attention Disentanglement cs.CVPDF

Kai Wang, Ziheng Ouyang, Xuying Zhang, Ming-Ming Cheng, Qibin Hou

TL;DR: 本文提出了DreamStyle3D，一个基于解耦双交叉注意力机制的高效3D风格化资产生成框架。该方法通过显式分离几何与风格特征，实现了高效的风格注入并保持结构一致性，同时采用轻量级训练策略提升风格一致性与模型泛化能力。实验表明，该方法能在10秒内生成高保真、几何一致的风格化3D资产，显著提升了效率。

Details

Motivation: 随着游戏、动画和虚拟现实行业的发展，对高效生成风格化3D资产的需求迅速增长。现有方法大多依赖间接的2D到3D风格化流程，难以同时保证风格保真度、几何一致性和生成效率，因此需要一种能显式解耦风格与几何的原生3D风格化框架。

Result: 广泛的实验表明，DreamStyle3D能在10秒内生成高保真且几何一致的风格化3D资产，在效率上取得显著提升的同时保持了优异的风格质量。

Insight: 主要创新点在于提出了解耦双交叉注意力机制，显式分离几何与风格特征以实现高效风格注入；同时采用了轻量级训练策略以增强风格一致性和泛化能力。此外，构建了一个包含约1.5万个内容-风格-风格化三元组的自动化数据管道和数据集，用于训练和评估。

Abstract: With the growth of gaming, animation, and virtual reality industries, the demand for efficient generation of stylized 3D assets is rapidly increasing. However, existing approaches still struggle to jointly preserve style fidelity, geometric consistency, and generation efficiency, as most of them still rely on indirect 2D-to-3D stylization pipelines. This motivates a native 3D stylization framework that can explicitly disentangle style from geometry while remaining efficient. To this end, we propose DreamStyle3D, an efficient framework for stylized 3D asset generation built on a Decoupled Dual Cross-Attention mechanism. Our method explicitly separates geometric and stylistic features to enable efficient style injection while preserving structural consistency, and further adopts a lightweight training strategy to enhance style consistency and model generalization. In addition, we build an automated data pipeline and construct a dataset of about 15K content-style-stylized triplets for training and evaluation. Extensive experiments demonstrate that our DreamStyle3D can generate high-fidelity, geometrically consistent stylized 3D assets within 10 seconds, substantially improving efficiency while maintaining superior style quality and offering a new solution for 3D content creation. The code and data are available at https://github.com/HVision-NKU/DreamStyle3D.

[146] SADe: Sparse-Atom Support Decontamination for Few-Shot Segmentation with Weak Support Annotations cs.CVPDF

Hang Xing, Guangjun Liu, Yan Xia, Xueming Ding

TL;DR: 本文提出SADe方法，用于解决少样本分割中弱标注支持样本（如边界框、涂鸦、粗糙掩码或伪掩码）可能包含纹理相似干扰物和背景上下文，从而污染类别原型或视觉提示的问题。SADe是一种与预测器无关的支持去污染层，通过稀疏自编码器原子证据和密集相似性结合来估计支持补丁的可靠性，生成清洁的支持掩码，可独立使用或作为插件集成到现有少样本分割模型中。

Details

Motivation: 少样本分割通常假设支持掩码是干净的像素级标注，但实际应用中支持监督常使用弱标注（如边界框、涂鸦等），这些标注可能包含目标之外的纹理相似干扰物和背景，在查询预测前污染类别原型或视觉提示，影响分割性能。

Result: 在匹配的弱支持协议下，SADe在九个独立提示-样本组合中的六个实现了最高的查询mIoU；使用相同ProMi查询头时，在紧密边界框下与SAM3衍生掩码的mIoU差距在0.03以内，在box-r2和box-r4下分别超过它们11.17和19.49点；作为插件，在四个冻结下游模型和两个数据集的72个匹配边界框族比较中，SADe在70个案例中优于原始支持。

Insight: 创新点在于引入稀疏自编码器原子证据，通过对比弱支持区域内外的原子激活提供因子级可靠性线索，超越了密集相似性的局限性；方法具有预测器无关性，训练一次后冻结，可灵活集成到异构少样本分割模型中而不改变查询端推理，提升了弱标注下的分割鲁棒性。

Abstract: Few-shot segmentation (FSS) commonly assumes clean pixel-level support masks, yet practical support supervision often uses boxes, scribbles, coarse masks, or pseudo-masks. These weak annotations may include texture-similar distractors and background context alongside the target, contaminating class prototypes or visual prompts before query prediction. We introduce SADe, a predictor-agnostic support decontamination layer that estimates the reliability of selected support patches without query information. Central to SADe is sparse autoencoder (SAE) atom evidence: dense similarity may respond to both target and texture-similar context, whereas contrasting atom activations inside and outside the weak-support region provides factor-level reliability cues. A lightweight router combines atom evidence with dense similarity and episode statistics to predict patch reliability and generate a cleaned support mask. Trained once on synthetic weak-support episodes from FSS-1000, the router is frozen for all target evaluations. The resulting mask supports standalone prediction or can be supplied to heterogeneous FSS models through native support interfaces without altering query-side inference. Under a matched weak-support protocol, SADe achieves the highest query mIoU in six of nine standalone prompt-shot combinations. With the same ProMi query head, it is within 0.03 mIoU of SAM3-derived masks under tight boxes and surpasses them by 11.17 and 19.49 points under box-r2 and box-r4, respectively. As a plug-in, SADe improves over raw support in 70 of 72 matched box-family comparisons across four frozen downstream models and two datasets. On point and scribble prompts, its average performance remains close to the corresponding raw-support baseline. Ablations and atom-removal controls show that atom evidence contributes reliability information beyond dense similarity.

[147] Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation cs.CV | cs.AI | cs.LGPDF

Bingnan Li, Haozhe Wang, Haozhong Xiong, Fangtai Wu, Jinpeng Yu

TL;DR: 本文重新思考了在策略蒸馏（OPD）中应用无分类器引导（CFG）的方法。研究发现，现有方法直接匹配教师和学生的引导速度，但在分支层面存在识别不足的问题，可能导致正负分支误差相互抵消。论文提出了负分支不对称性（NBA）这一失效模式，并引入了正方向匹配（PDM）这一分支感知的OPD目标来解决该问题。

Details

Motivation: 动机在于理解并改进在策略蒸馏（OPD）中如何有效利用无分类器引导（CFG），因为现有方法在CFG下的行为机制尚不明确，可能导致知识转移效率低下。

Result: 在密集到稀疏视频控制任务中，实验表明，朴素的引导匹配对推理引导尺度高度敏感，而提出的PDM方法能够实现更鲁棒和有效的知识转移。

Insight: 创新点在于揭示了CFG下OPD的分支层面识别不足问题（NBA），并提出了分支感知的正方向匹配（PDM）目标，通过分别约束正预测和CFG条件方向来改进蒸馏效果。

Abstract: On-policy distillation (OPD) adapts diffusion models by querying a teacher along trajectories generated by the current student, but how it should behave under classifier-free guidance (CFG), a default component of modern diffusion systems, remains poorly understood. Existing OPD methods naturally extend velocity matching to the CFG-composed prediction, directly matching teacher and student guided velocities. We show that this objective is under-identified at the branch level: positive- and negative-branch errors can compensate in the guided prediction. Through two contrasting cases, we find that naive matching remains effective under shared negative conditioning, where both branch errors decrease jointly. When the model’s native CFG schema retains privileged information in the teacher’s negative branch that is unavailable to the student, however, this joint reduction breaks down and the composed objective induces antagonistic branch-error dynamics, reducing the positive-branch error while increasing the negative-branch error. We term this failure mode Negative Branch Asymmetry (NBA). To address NBA, we introduce Positive–Direction Matching (PDM), a branch-aware OPD objective that separately constrains the positive prediction and the CFG conditional direction. We apply PDM to dense-to-sparse video control, where naive guided matching is highly sensitive to inference guidance scales, while branch-aware supervision enables more robust and effective knowledge transfer.

[148] KANEx: Translating Kolmogorov-Arnold Networks’ Interpretability to Medical Explainability cs.CV | cs.AIPDF

Krithi Shailya, Ananya Lakshmi Ravi, Venkatanathan K. V., Sowmya S. Sundaram, Gokul S. Krishnan

TL;DR: 本文提出了KANEx框架，首次利用Kolmogorov-Arnold Networks (KANs) 的符号透明性来增强视觉语言模型（VLMs）在医学影像（如胸片）分类中的可解释性。该框架通过KANs固有的可解释功能单元，为VLM的推理提供基础，并衍生出KAN-Map热图生成方法，从而生成更可信的文本解释和显著图。

Details

Motivation: 医学影像计算机视觉模型（如胸片分类器）虽然有效，但其黑盒性质削弱了临床医生的信任。现有方法常结合视觉语言模型生成自然语言解释，但未能解决视觉模型本身的不透明问题。

Result: 在MIMIC-CXR数据集上的实验表明，基于KAN的架构（以ResNet/ViT为基线）在语义相似性上有所提升，并能生成显著更忠实（faithful）的显著图。KAN架构将视觉定位和下游推理质量提高了10%。

Insight: 核心创新点在于首次将KANs的数学可解释性（基于样条的可解释功能单元）与VLM结合，提出了KANEx框架和直接从KAN模型推导热图的KAN-Map方法。这为构建可信赖的医学AI提供了一条新路径，即通过数学上可解释的单元来同时支撑语言解释和视觉归因。

Abstract: Computer vision models have become highly effective for medical applications, yet their black-box nature continues to undermine clinician trust. In clinical workflows, chest X-ray classifiers are increasingly paired with Vision-Language Models (VLMs) to generate natural-language explanations. However, these systems add linguistic fluency without addressing the underlying opacity of the visual model. With the emergence of Kolmogorov-Arnold Networks (KANs), whose spline-based components provide inherently interpretable functional units, we investigate whether this architectural transparency can be leveraged to produce more trustworthy textual explanations. We introduce KANEx, the first ever framework that leverages the symbolic transparency of KANs to ground VLM reasoning. This interpretability also made it possible to design KAN-Map, a novel heatmap generation method derived directly from KAN models rather than gradient approximations. We feed these grounded contexts into downstream VLMs for enhanced explainability. Benchmarked on the MIMIC-CXR dataset, we demonstrate that KAN-based architectures with ResNet/ViT baselines demonstrate improved semantic similarity while producing significantly more faithful saliency maps. KAN architectures improve visual localization and downstream reasoning quality by 10%. Our findings suggest that grounding linguistic explanations and visual attributions in mathematically interpretable units is a necessary step toward trustworthy medical AI.

cs.RO [Back]

[149] DeVA: Decoupled Video-Action Model with physical guidance for robot policy learning cs.RO | cs.CVPDF

Mengqi Zhang, Sahil Khose, Simar Kareer, Yuchen Song, Unnat Jain

TL;DR: 本文提出了DeVA模型，一种用于机器人策略学习的解耦视频-动作模型。该模型通过专门的视频和动作专家、多级特征迁移以及物理显著性引导，解决了现有视频-动作模型在策略学习中存在的耦合优化困难和视频信息利用不足的问题。

Details

Motivation: 动机在于当前通用的机器人操作需要能够预测视觉场景演变的策略，而现有的视觉-语言-动作模型在动态物理和时序因果关系方面的监督有限，视频生成模型虽能编码时空先验，但现有视频-动作模型要么耦合视频与动作预测导致策略适应难以优化，要么在引导动作分支时未能充分利用视频信息。

Result: 在仿真基准测试和真实世界部署中的实验表明，DeVA在有限数据下表现出色，比统一架构收敛更快，并且物理引导（可供性/深度）带来了明显的性能提升。

Insight: 创新点在于提出了一个解耦的架构，通过专门的专家模块、多级特征迁移以及物理显著性引导来增强视频信息向动作策略的传递，使得策略学习更易于优化，并有效利用了物理先验知识。

Abstract: Generalizable robot manipulation requires policies that can anticipate how visual scenes evolve while executing language instructions. While recent Vision-Language-Action models benefit from large-scale pretraining, their predominantly static pretraining objectives provide limited supervision for physical dynamics and temporal causality, leaving control-relevant knowledge to be learned from downstream robot demonstrations. Video generative models offer a promising foundation by encoding rich spatiotemporal priors through future predictions. However, existing Video-Action Models either couple video and action prediction in a shared backbone, making policy adaptation harder to optimize, or under-utilize video information when guiding the action branch. In this work, we introduce DeVA, a Decoupled Video-Action model with specialized video and action experts, multi-level feature transfer, and physically salient guidance. DeVA transfers representations from multiple video layers to the action expert, enabling rich information exchange while making policy learning more tractable. It further supervises intermediate video features and the action stream with physically salient guidance (affordance/depth). Experiments on both simulation benchmarks and real-world deployment demonstrate strong performance with limited data, faster convergence than a unified architecture, and clear performance gains from physical guidance.

[150] LabRobFail: A Benchmark for Robotic Failure Analysis in Chemical Self-driving Laboratories cs.RO | cs.CVPDF

Haobo Wang, Baoli Sun, Anqi Zou, Dongsheng Huang, Zelin Lv

TL;DR: 本文介绍了LabRobFail，一个用于化学自驱实验室中机器人故障分析的基准框架。该框架包含模拟故障注入的LabRobFail-Sim、包含超过20,000条轨迹的LabRobFail-Data数据集，以及评估六种故障分析能力的LabRobFail-Bench。作者还开发了领域专用的视觉语言模型LabRobFail-VLM，用于生成结构化故障诊断和恢复指令，显著提升了故障检测和定位的准确性，并作为实时监督器提高了下游任务的成功率。

Details

Motivation: 在自驱实验室中部署具身智能体可以加速科学发现，但其可靠性受到化学实验不可逆和安全关键性质的限制，且进展因缺乏细粒度评估协议和稀缺的故障数据而受阻。

Result: 在已知环境中，LabRobFail-VLM实现了92.58%的故障检测准确率和85.58%的时间定位准确率，显著优于通用VLM。当作为实时监督器集成时，它将下游VLA任务的成功率提高了10-20个百分点。

Insight: 论文的创新点在于提出了一个专注于故障分析的综合性基准框架，通过可控的多层次故障注入（控制、物理、语义）构建大规模数据集，并开发了领域专用的VLM来提供结构化诊断和恢复指令，这为闭环恢复和可靠的实验室自主性提供了细粒度故障理解的价值。

Abstract: The deployment of embodied agents in self-driving laboratories could accelerate scientific discovery, yet their reliability is constrained by the irreversible and safety-critical nature of chemical experiments. Progress is further hindered by scarce failure data and the lack of fine-grained evaluation protocols. To address these challenges, we introduce LabRobFail, a failure-centric framework for learning and evaluating robotic failure analysis in chemical laboratories. LabRobFail-Sim injects controllable failures at the control, physics, and semantic levels, enabling the construction of LabRobFail-Data, which contains over 20,000 trajectories across 70+ task scenarios, five failure categories, and 11 fine-grained failure types. LabRobFail-Bench evaluates six capabilities spanning task understanding, failure detection, temporal localization, severity assessment, failure classification, and actionable correction. We further develop LabRobFail-VLM, a domain-specialized vision-language model that generates structured failure diagnoses and recovery instructions. On seen environments, it achieves 92.58% failure-detection accuracy and 85.58% temporal-localization accuracy, substantially outperforming general-purpose VLMs. When integrated as a real-time supervisor, it improves downstream VLA task success rates by 10-20 percentage points, demonstrating the value of fine-grained failure understanding for closed-loop recovery and reliable laboratory autonomy. Our code and data are available at https://github.com/Su-ISE-2001/SciRobo

[151] Surgical Re-enactment for Operating Room Workflow Datasets cs.RO | cs.CVPDF

Jana Nina Friedrich, Andrea Karin Maria Ross, Angelo Henriques, Mario Peter Martin Weisser, Ling Zhang

TL;DR: 本文提出了一种在重建手术室中逐步重演完整手术流程的方法论，旨在生成可重复、可标注的工作流数据集，以支持活动识别模型训练、场景图生成和手术过程模型形式化。该方法专为机器人辅助眼科手术设计，结合了专家咨询、结构化工作流形式化、手术室重建、角色培训、真实手术室观察以及带后采简报的迭代录制。

Details

Motivation: 实现智能手术室的愿景需要深入理解手术工作流，这依赖于从全室和手术视野角度捕捉所有人员动作的真实数据集。然而，在真实手术室中获取此类数据面临伦理审批、摄像机安装空间有限以及无菌规定禁止使用追踪标记等巨大挑战。

Result: 论文未在摘要中明确提及具体的定量实验结果或基准测试，但强调该方法能够创建用于训练活动识别模型、生成场景图和形式化手术过程模型的数据集，并为其他研究小组提供了可无缝采纳的具体建议。

Insight: 创新点在于提出了一种系统化的手术重演方法论，以可控、可重复的方式生成高质量的手术工作流数据集，从而规避了真实手术室数据采集的诸多限制。该方法结合了多学科专业知识与结构化流程，具有可推广到其他外科领域的潜力。

Abstract: The introduction of new technologies, such as surgical robots, is driving the vision of a connected, smart operating room (OR). However, realizing this vision requires a deep understanding of surgical workflows, which relies on realistic datasets capturing the actions of all OR personnel from both full room and surgical field perspectives. Acquiring such data in real ORs is prohibitively challenging due to factors such as ethics committee approvals, limited space for camera installation, and sterility regulations preventing the use of tracking markers. We present a step-by-step methodology for re-enacting complete surgical procedures in a reconstructed OR. This approach enables the creation of repeatable and annotatable workflow datasets for training activity recognition models, generating scene graphs, and formalizing surgical process models. Developed for robot-assisted ophthalmic surgery, our methodology combines expert consultation, structured workflow formalization, OR reconstruction, role-based training, real OR observation, and iterative recording with post-take debriefing. We provide concrete recommendations to allow other research groups to seamlessly adopt this methodology for their own surgical domains.

[152] Data Pyramid for Embodied Manipulation cs.RO | cs.CVPDF

Yifan Ye, Yankai Fu, Yaoxu Lv, Bohan Hou, Jun Cen

TL;DR: 本文提出了一个‘数据金字塔’框架，用于组织和分析具身智能体（如机器人）训练所需的多模态数据来源。该金字塔包含五种互补数据源：真实机器人数据、UMI风格数据、第一人称和第三人称数据、仿真数据以及通用视觉语言数据，并围绕可扩展性与机器人对齐之间的权衡进行组织。论文通过此框架分析了现有具身基础模型的数据配方，并讨论了未来挑战。

Details

Motivation: 具身智能体（如机器人）的训练缺乏像互联网规模的多模态数据，需要将观察与物理状态、动作耦合的数据。现有数据来源多样但分散，缺乏系统性分析，因此需要建立一个框架来理解不同数据源在构建具身基础模型中的作用。

Result: 论文未报告具体的定量实验结果，而是提供了一个概念性分析框架。它通过提出的数据金字塔框架，定性分析了近期具身基础模型（如具身大脑模型、视觉语言动作模型、世界动作模型）的数据配方如何影响其在感知、推理、规划、动作生成和世界预测等方面的能力。

Insight: 创新点在于提出了一个系统性的‘数据金字塔’框架，用于分类和评估具身学习的数据生态系统，强调了不同数据源在可扩展性、机器人对齐、数据质量、多样性、可重用性和物理保真度等方面的权衡。该框架为设计和评估下一代具身系统的数据策略提供了理论基础，并指出了未来关键挑战，如构建大规模触觉数据集、收集失败与恢复数据等。

Abstract: Multimodal foundation models learned to see and to speak by consuming the whole internet. Embodied agents admit no such shortcut, since they require data that couple observations with physical states and actions. These signals can be provided, to varying degrees, by multiple data sources. In this work, we organize the embodied data ecosystem as a “pyramid” spanning five complementary sources: real-robot data, UMI-style data, egocentric and exocentric data, simulation data, and general vision-language data. We organize the pyramid around the tension between scalability and robot alignment, and further characterize each source in terms of data quality, diversity, reusability, and physical fidelity. We then analyze recent embodied foundation models through the lens of their data recipes, examining how different sources are selected, aligned, and mixed during pretraining. For embodied brain models, vision-language-action models, and world-action models alike, we relate data composition to capabilities in perception, reasoning, planning, action generation, and world prediction. We close by discussing six open challenges: building large-scale tactile datasets, collecting failure and recovery data, developing scalable data-collection pipelines, aligning actions across embodiments, leveraging egocentric data for dexterous manipulation, and designing principled data recipes for robot learning. We hope this work paves the foundation for the design of next-generation embodied systems.

cs.CY [Back]

[153] Revitalizing Public Urban Places through Cultural and Political Memory: A Technological Approach with LLMs and Augmented Reality cs.CY | cs.AI | cs.CLPDF

Lara Vartziotis, Tina Vartziotis, Valentin Keckeisen, Frank Beutenmueller, Martin Obstbaum

TL;DR: 本文探讨了记忆、场所与身份的交叉点，研究如何利用苹果Vision Pro等新技术，通过数字孪生和虚拟现实技术，将文化记忆融入具有历史意义的城市景观，以视觉元素唤起记忆与遗产。

Details

Motivation: 旨在通过技术手段，特别是增强现实和大型语言模型，激活公共城市空间的文化与政治记忆，解决城市空间文化认同感缺失和历史传承断裂的问题。

Result: 论文未提及具体定量结果或基准测试，但提出了一种整合数字孪生、虚拟现实和语义图像搜索的方法论，以支持城市转型和文化遗产保护的研究。

Insight: 创新点在于将苹果Vision Pro等沉浸式技术应用于城市记忆的可视化叙事，通过数字孪生连接虚拟环境与真实景观，动态展示历史演变，促进文化可持续性。

Abstract: This paper explores the intersection of memory, place, and identity, examining how new technologies, particularly Apple Vision Pro, can illuminate this nexus. Leveraging digital twins and virtual reality, it investigates how memory is woven into landscapes and urban environments of cultural and historical significance, identifying visual elements that evoke memory and heritage. Applications such as Apple Vision Pro can facilitate image extension to define place identity, informing viewers about cultural and political entities across timelines. Visual storytelling can showcase the evolution of landscapes and the preservation of cultural heritage, while Virtual Reality (VR) enables the recreation of historical landscapes and urban-scapes. This immersive approach invites users to transcend temporal boundaries and experience the past dynamically. Semantic Image Search can support research by uncovering images related to monuments, tradition, or cultural identity. This research introduces a methodology to connect digital twins and virtual environments with urban and non-urban landscapes to illustrate cultural, historical, and environmental sustainability. Central to this approach is defining the resilience of the current state, its future evolution, and the significance of the past. These technologies facilitate a historical and cultural embrace while evoking the feeling of returning to a specific place years later. The methodology outlines the integration of technologies needed to revitalize public urban places through cultural and political memory. Through these applications, this paper contributes to research on digital twins of spaces, urban transformation, and cultural heritage preservation. By offering insights into the relationship between memory, place, and identity in the digital age, it supports a deeper understanding of our collective past and its impact on the present.

cs.LG [Back]

[154] Multimodal Domain Generalization for Depression Detection: An Attention-Based BiLSTM Network with Domain-Adversarial Training cs.LG | cs.AI | cs.CL | cs.SDPDF

Ali Tabaraei, Federico Simonetta, Stavros Ntalampiras

TL;DR: 本文提出了一种用于抑郁症检测的多模态领域泛化框架，通过结合声学和文本模态，并利用带有注意力的BiLSTM网络以及领域对抗训练，来提升模型在跨说话人场景下的泛化能力。

Details

Motivation: 解决基于深度学习的自动抑郁症检测方法因说话人差异导致的领域偏移问题，从而提升模型在患者无关场景下的泛化性能。

Result: 在Androids-Corpus数据集上使用5折交叉验证进行评估，最优基线模型（MelSpec和ItalianBERT特征提取器，30秒片段）加入领域泛化后，准确率提升2.5%，F1分数提升3.3%，达到93.2%的准确率和94.2%的F1分数，超越了现有所有基准。

Insight: 创新点在于首次将领域泛化引入患者无关的多模态抑郁症检测，通过领域对抗训练抑制说话人身份信息以学习领域不变表示，并结合模态内与跨模态注意力机制进行决策融合，增强了模型的鲁棒性和泛化性。

Abstract: Automatic depression detection with deep learning has shown promise but often suffers from limited generalization due to domain shift arising from inter-speaker variability. To address this critical issue, we present the first patient-independent multimodal depression detection framework that incorporates domain generalization (DG), jointly leveraging both acoustic and textual modalities. The proposed model integrates bidirectional Long Short-Term Memory (BiLSTM) with intra- and cross-modal attention mechanisms, accompanied by segment-level fusion for decision-making. Generalization is further enhanced by applying a gradient reversal layer inspired by Domain-Adversarial Training of Neural Networks (DANN), which promotes domain-invariant representations by adversarially limiting the model’s ability to identify individual speakers, effectively reducing patient-specific bias. Conducting experiments on the Androids-Corpus dataset with a 5-fold cross-validation (CV) protocol, various pairings of audio and text feature extractors were evaluated over different segment durations, determining MelSpec and ItalianBERT as the optimal baseline at a 30-second segment duration. The addition of DG to this baseline yields a 2.5% increase in accuracy and 3.3% in F1-score, achieving 93.2% accuracy, 93.2% precision, 96.2% recall, and 94.2% F1-score, surpassing all existing benchmarks. Extensive ablation studies assess the impact of multimodal fusion, deep architectural choices, and DG, highlighting their combined contribution to robust and generalizable depression detection.

[155] CausalGate: Causal Importance Distillation for Transformer Module Pruning cs.LG | cs.CL | cs.CV | stat.MLPDF

Kiran Nair, Smriti Regmi, Rodrigue Rizk

TL;DR: 本文提出了CausalGate，一种基于因果干预的Transformer模块剪枝框架，用于实现计算高效的大语言模型推理。该方法通过校准阶段干预并测量Attention和MLP子层输出的因果重要性，将重要性层次蒸馏为静态的轻量级标量门，从而在推理时无需路由开销即可动态跳过冗余模块。

Details

Motivation: 现有的大语言模型自适应推理方法依赖隐藏状态相似性或激活幅度等观测性启发式指标来丢弃冗余模块，但这些基于相关性的度量往往无法捕捉对语义准确性至关重要的细微非线性结构计算。

Result: 在TinyLlama-1.1B、Qwen2.5-3B和Llama-3.1-8B模型上，于语言建模和常识推理基准测试中，CausalGate持续优于主流的动态路由和层跳过基线方法，将理论计算节省转化为具体的硬件延迟降低，且无运行时开销。

Insight: 核心创新在于采用因果干预（通过零化子层输出并测量最终logit分布的KL散度）来精确量化模块的结构重要性，而非依赖相关性指标；并通过可微排序损失将动态重要性蒸馏为静态门，消除了运行时决策开销，实现了理论效率与实际部署效率的统一。

Abstract: Existing adaptive inference methods for Large Language Models rely on observational heuristics, such as hidden-state similarity or activation magnitudes, to drop redundant modules. However, these correlation-based metrics often fail to capture subtle, non-linear structural computations vital for semantic accuracy. We introduce CausalGate, an intervention-guided framework for compute-efficient transformer inference. During a calibration phase, CausalGate isolates individual Attention and MLP sub-layers, zeros out their respective outputs, and measures the exact semantic damage via the Kullback-Leibler divergence of the final logit distribution. To eliminate runtime routing overhead, this structural importance hierarchy is distilled into a global set of static, lightweight scalar gates using an Exponential Moving Average smoothing objective paired with a differentiable pairwise ranking loss. Evaluated on TinyLlama-1.1B, Qwen2.5-3B, and Llama-3.1-8B across language modeling and commonsense reasoning benchmarks, CausalGate consistently outperforms prominent dynamic routing and layer-skipping baselines, translating theoretical compute savings into concrete hardware latency reductions with zero operational overhead.

[156] Frustratingly Simple Black-Box Adaptation of Language Models via Logit Bias cs.LG | cs.AI | cs.CLPDF

Ofek I. Cohen, Lior Shani, Aviv Rosenberg, Ankur Samanta, Tal Wagner

TL;DR: 本文研究了一种基于logit偏置的黑盒语言模型适配方法，该方法通过在解码时添加一个固定的上下文无关logit偏置向量来改进模型在特定领域的性能，无需修改模型权重或计算梯度。

Details

Motivation: 为了解决组织在适配语言模型时面临的微调操作复杂、提示工程繁琐以及数据隐私问题，本文探索了一种仅需API级别控制的轻量级替代方案。

Result: 实验表明，该方法在数学和推理基准测试上优于基础模型，且所需可训练参数远少于传统微调方法。

Insight: 创新点在于将KL正则化强化学习目标转化为一个固定的logit偏置向量学习问题，并推导出闭式逆倾向估计器，提供了一种在最小访问需求下适配语言模型的轻量级机制。

Abstract: Many organizations aim to adapt language models for internal use, both to improve performance on domain-specific tasks and to address privacy concerns around sensitive data. However, such adaptation remains non-trivial: it often requires operationally challenging fine-tuning of open-source models or ad hoc prompt optimization. We study a minimal alternative based on a simple API-level control: allowing users to bias the model’s logits with a user-defined vector. We develop a black-box method for learning a single context-independent logit-bias vector, added at every decoding step, without modifying model weights or requiring gradients. Starting from a KL-regularized reinforcement learning (RL) objective, we characterize when such a fixed logit-bias vector can approximate the optimal prefix-dependent correction and derive a closed-form inverse-propensity estimator from rollouts, rewards, and token probabilities. Empirically, this simple decoding-time intervention improves over base models on mathematical and reasoning benchmarks while using far fewer trainable parameters than conventional fine-tuning. Our results suggest that learned logit bias is a lightweight mechanism for adapting language models under minimal access requirements.

[157] Through the Bottleneck: How Multi-head Latent Attention Separates Content from Position in Language Models cs.LG | cs.AI | cs.CLPDF

Dhruvil S, Fenil Sojitra, Ravirajsinh Chauhan

TL;DR: 本文首次对DeepSeek-V2中引入的多头潜在注意力机制进行了全面的机制可解释性研究，发现其低秩瓶颈KV压缩不仅减少了缓存，还主动重塑了Transformer的内部电路结构，实现了内容与位置信息的分离。

Details

Motivation: 尽管MLA已被大规模生产模型采用，但尚无研究探讨其低秩瓶颈保留或丢弃了何种信息，以及它如何重塑Transformer的内部电路，本文旨在填补这一空白。

Result: 在预训练于网络/代码/数学混合数据并微调于TinyStories的1.14亿参数Transformer上，通过SVD、注意力头分类、线性探测和干扰归因分析，发现cKV瓶颈保留了98%的实体身份信息，同时丢弃了位置信息。

Insight: MLA的低秩瓶颈主动学习纯内容表示，实现了内容与位置的有效分离；它改变了归纳头等电路结构的形成方式，集中出现在特定层；并揭示出存在一个同时具有最高SVD有效秩和最强干扰归因分数的“语义枢纽”层。

Abstract: Multi-head Latent Attention (MLA), introduced in DeepSeek-V2, compresses key-value pairs through a shared low-rank bottleneck (cKV), achieving 81% KV-cache reduction during inference. Despite its adoption in massive production models, no prior work has studied what information this bottleneck preserves or discards, nor how it reshapes internal transformer circuits. We present the first comprehensive mechanistic interpretability study of MLA, training a 114M-parameter transformer (pretrained on a web/code/math mixture, fine-tuned on TinyStories) and analyzing its representations through SVD, attention head taxonomy, linear probing, and a disruption-attribution analysis. Our key findings are: (1) the cKV bottleneck learns a pure content representation, preserving entity identity (98% retention) while discarding positional information, validating MLA’s separation of content from position via RoPE; (2) induction heads co-locate at a single layer (Layer 12), unlike their distributed formation in standard MHA; (3) a single “semantic hub” layer (Layer 15) simultaneously exhibits the highest SVD effective rank and strongest disruption-attribution score; and (4) the bottleneck is globally over-provisioned, using only 46% of its capacity on average. These findings suggest MLA does not merely compress attention passively, but reshapes how the model organizes content, position, and circuit structure. We view this as an initial data point and detail scope limitations in Section 5.

[158] In-Context Learning as Implicit Policy Gradient cs.LG | cs.AI | cs.CLPDF

Masahiro Kaneko, Timothy Baldwin

TL;DR: 本文揭示了大型语言模型（LLM）中基于分数的上下文学习（ICL）与策略梯度优化之间的结构对应关系。通过理论证明，在特定权重配置下，自注意力机制可以实现类似于REINFORCE算法的奖励加权聚合，并推导了注意力更新引起的分布偏移的上界，建立了与KL约束策略优化的类比。实验验证了LLM能有效利用分数信息将输出分布向高分示例偏移，且注意力权重与示例分数高度相关。

Details

Motivation: 尽管实证研究表明LLM可以通过将生成的样本及其评估分数作为上下文示例来迭代改进输出，但其背后的理论基础仍不清楚。本文旨在从理论上解释这一现象。

Result: 在多个LLM上的广泛实验验证了理论：LLM能有效利用分数信息使输出分布向高分示例偏移，且注意力权重与示例分数呈现强相关性。

Insight: 核心创新点在于建立了分数条件ICL与策略梯度优化（特别是REINFORCE算法）之间的形式化对应关系，并推导了注意力更新的分布偏移上界，为理解LLM的上下文学习机制提供了新的理论视角和与强化学习的类比框架。

Abstract: Recent work has shown that large language models (LLMs) can iteratively improve their outputs by incorporating generated samples and their corresponding evaluation scores as in-context examples. Despite these empirical findings, the theoretical foundations underlying this phenomenon remain poorly understood. In this paper, we show that score-conditioned In-Context Learning (ICL) admits a structural correspondence to policy gradient optimization. We first provide a constructive proof that self-attention mechanisms can implement reward-weighted aggregation analogous to the REINFORCE algorithm under specific weight matrix configurations, and discuss the relationship between this construction and the behavior of pretrained transformers. The correspondence is directional in hidden-state space and holds exactly only under the stated simplifying conditions; we quantify its strength empirically. Within our simplified hidden-state model, we furthermore derive an exact upper bound on the distribution shift induced by a bounded attention update, yielding a trust-region-like analogy to KL-constrained policy optimization. We validate our theory through extensive experiments across multiple LLMs, demonstrating that LLMs effectively utilize score information to shift output distributions toward high-scoring exemplars, and that attention weights exhibit a strong correlation with example scores.

[159] ACRL: Adaptive Control of Training-Inference Discrepancy for Stable Reinforcement Learning cs.LG | cs.AI | cs.CLPDF

Wenwu Fan, Qihong Lin, Zhijie Xia, Zhuo Zheng, Sihao Wang

TL;DR: 本文提出了一种名为自适应控制强化学习（ACRL）的方法，旨在解决大语言模型（LLM）强化学习训练中因训练与推理阶段差异（如架构分离和精度量化不同）导致的不稳定问题。ACRL通过自适应地将训练-推理差异维持在合理范围内来稳定训练过程，并同时增加策略熵以促进探索并提升准确性。

Details

Motivation: 动机在于解决大语言模型强化学习训练中的不稳定性，这种不稳定性主要源于训练与推理阶段在架构和计算精度（如训练使用高精度而推理使用低精度量化）上的差异。

Result: 实验结果表明，当推理引擎使用FP8量化时，ACRL能持续将训练-推理差异控制在合理范围内，稳定了RL训练；其准确性不仅与BF16基线相当，还优于重要性采样（IS）修正方法。

Insight: 创新点在于提出了自适应控制训练-推理差异的原则和方法，通过维持差异在合理范围来稳定训练，并自然地增加策略熵以增强探索，这为解决RL训练中的精度不匹配问题提供了新思路。

Abstract: Reinforcement Learning (RL) training for Large Language Models (LLMs) often suffers from instability due to the discrepancy between training and inference. This training-inference discrepancy stems from two primary factors: an architectural separation between training and inference engines, and the use of low-precision quantization in inference versus higher-precision computation in training. To address training instability issues caused by high training-inference discrepancy, we present the principles and methods for its adaptive control. We propose Adaptive Control Reinforcement Learning (ACRL), which adaptively maintains the training-inference discrepancy within a reasonable range to ensure stable RL training. Beyond stabilization, ACRL inherently increases policy entropy, thereby enhancing exploration and improving accuracy. The experimental results show that when the inference engine utilizes FP8 quantization, ACRL consistently maintains the training-inference discrepancy within a reasonable range and stabilizes RL training. Furthermore, ACRL not only matches the accuracy of the BF16 baseline but also outperforms importance sampling (IS) fixes.

[160] QFedPolyp: A Communication- and Inference-Efficient Federated Learning Framework for Polyp Segmentation cs.LG | cs.AI | cs.CVPDF

Madan Baduwal, Priyanka Paudel

TL;DR: 本文提出了一种名为QFedPolyp的通信和推理高效的联邦学习框架，用于协作式息肉分割。该框架结合了量化感知训练和低精度模型通信，在保护医疗数据隐私的同时，显著降低了通信开销并提升了模型推理速度。

Details

Motivation: 解决自动息肉分割中，集中式深度学习需要共享敏感医疗数据带来的隐私风险，以及传统联邦学习因反复传输全精度模型参数而产生的高通信成本问题。

Result: 在Kvasir-SEG和CVC-ClinicVideoDB数据集上，全精度联邦训练分别达到0.910和0.930的Dice分数；采用8位均匀量化通信可将传输成本降低约4倍，同时保持有竞争力的分割精度，量化模型推理速度比全精度模型快达1.5倍。

Insight: 创新点在于将量化感知训练与联邦学习相结合，在训练阶段模拟量化以提升模型对低精度通信的鲁棒性，从而在保护隐私的前提下，实现通信效率和推理速度的同步优化，为实时临床部署提供了轻量级解决方案。

Abstract: Background and Objective: Automatic polyp segmentation supports computer-aided diagnosis and early colorectal cancer detec- tion. Centralized deep learning requires hospitals to share sensitive medical data, while federated learning preserves privacy but introduces high communication costs through repeated transmission of full-precision model parameters. We propose QFedPolyp, a communication- and inference-efficient federated learning framework for collaborative polyp segmentation. Methods: QFedPolyp combines quantization-aware training with low-precision model communication. Each hospital locally trains a lightweight U-Net on private data while simulating quantization during training. Clients transmit quantized model parameters to a central server, where they are reconstructed and aggregated using Federated Averaging. Evaluation is performed on Kvasir-SEG, CVC-ClinicVideoDB, PolypGen, and BKAI-IGH NeoPolyp. Results: Full-precision federated training achieves Dice scores of 0.910 on Kvasir-SEG and 0.930 on CVC-ClinicVideoDB. Uni- form 8-bit communication reduces transmission cost by approximately 4 times while preserving competitive segmentation accuracy. Quantized models also achieve up to 1.5 times faster inference than full-precision models. Conclusions: QFedPolyp enables privacy-preserving collaborative polyp segmentation with reduced communication overhead and faster inference. The resulting lightweight models are suitable for real-time clinical deployment.

[161] Same Predictions, Different Reasons: The Effect of Quantization on Model Explanations cs.LG | cs.CVPDF

Kazi Kamruzzaman Rabbi, Md. Zami Al Zunaed Farabe, M. Sohel Rahman

TL;DR: 本文系统评估了后训练量化（PTQ）对五种广泛使用的CNN架构（VGG19、ResNet18、EfficientNet-B0、DenseNet161和MobileNetV2）在INT8和INT4精度下模型可解释性的影响。研究发现，分类精度并不能可靠地指示量化后模型解释的稳定性，不同架构的表现差异显著，这对高可解释性要求应用中量化模型的可靠部署具有直接意义。

Details

Motivation: 后训练量化（PTQ）是资源受限边缘设备上部署深度学习模型的实用方案，虽然已知能保持分类精度，但其是否也保持模型的内部推理过程（即可解释性）仍是一个开放问题。

Result: 在两种二分类数据集上，使用结合Grad-CAM和LIME的双重可解释性框架及多种评估指标（皮尔逊相关系数、结构相似性指数、top-20% IoU及忠实性分析）进行系统比较。结果显示，DenseNet161在两种精度下均保持较强的特征一致性，而EfficientNet-B0在INT8精度下虽能达到有竞争力的空间注意力和分类精度，但在输入级特征归因方面表现出显著退化。

Insight: 论文的创新点在于系统性地将量化对模型内部推理（可解释性）的影响作为一个独立于分类精度的评估维度，并揭示了模型架构选择与量化策略同等重要。从客观角度看，其提出的双重可解释性评估框架和互补的量化评估指标，为量化模型的可信部署提供了更全面的分析工具。

Abstract: Post-training quantization (PTQ) has become a practical solution for deploying deep learning models on resource-constrained edge devices by compressing high-precision floating-point weights into low-precision representations without requiring retraining. Past research has demonstrated that quantization largely preserves classification accuracy; however, whether it also preserves the model’s internal reasoning remains an open question. This study presents a systematic evaluation on how static PTQ affects the interpretability / explainability of five widely used CNN architectures: VGG19, ResNet18, EfficientNet-B0, DenseNet161, and MobileNetV2 at INT8 and INT4 precision. We employ a dual interpretability framework that combines Grad-CAM for spatial attention analysis with LIME for input-level feature attribution, and systematically compare full-precision and quantized models on two binary classification datasets. Interpretability is evaluated using three complementary metrics: the Pearson correlation coefficient, structural similarity index, and top-20% IoU to capture distributional and structural variations in model explanations, supplemented by deletion/insertion faithfulness analysis. The results show that classification accuracy is not a reliable indicator of interpretability stability under reduced precision. DenseNet161 maintains strong feature consistency across both precision levels, whereas EfficientNet-B0, despite achieving competitive spatial attention and classification accuracy at INT8 precision, exhibits a substantial degradation in input-level feature attribution. These findings have direct implications for the trustworthy deployment of quantized models in applications with high interpretability requirements, demonstrating that architecture selection is as important as the quantization strategy.

[162] Learning Sampling Parameters for Diffusion Models cs.LG | cs.CVPDF

Arisrei Lim, Yossi Gandelsman

TL;DR: 本文提出了LeSAMP框架，用于学习文本到图像扩散模型中与提示词相关且随时间步变化的采样参数（如负提示、分类器引导尺度、噪声调度）。该方法将参数选择建模为强化学习问题，利用大语言模型生成参数调度，并通过人类偏好模型和VLM评判器进行优化。实验表明，该方法在Flux.1和Stable Diffusion 3.5上显著优于固定参数基线。

Details

Motivation: 当前扩散模型的推理采样参数（如提示词、负提示、引导尺度、噪声调度）通常被手动设定为固定值，并在不同提示词和去噪时间步中保持不变。然而，不同的生成阶段和提示词可能受益于不同的参数值，因此需要一种动态、自适应的参数选择方法。

Result: 在Flux.1 [dev]和Stable Diffusion 3.5上的评估显示，LeSAMP相比基线方法，在使用人类偏好分数时胜率高达68.12%，使用VLM评判器时胜率达73.37%。用户研究进一步验证了其有效性，胜率最高达59.46%。

Insight: 创新点在于将扩散模型采样参数的选择形式化为一个强化学习问题，并利用大语言模型来生成条件于提示词和时间步的动态参数调度。这为改善扩散模型输出提供了一种与现有后训练方法互补的新途径，实现了参数的自适应优化。

Abstract: Text-to-image diffusion models expose many inference-time sampling parameters, including prompts, negative prompts, classifier-free guidance scales, and noise schedules. These parameters are typically manually chosen once and then held fixed across prompts and denoising timesteps, even though different prompts and stages of generation can benefit from different parameter values. We introduce LeSAMP, a framework for learning prompt-conditioned, timestep-varying sampling parameters. We formulate parameter selection as a reinforcement learning problem: Given a user prompt, a large language model is trained to emit schedules for the chosen sampling parameters. We optimize our model using rewards from human preference models and VLM-as-a-judge. We evaluate our model on Flux.1 [dev] and Stable Diffusion 3.5, and find that compared to baselines, LeSAMP has a win rate of up to 68.12% using human preference scores and 73.37% using VLM-as-a-judge. These gains are validated in a user study where we achieve win rates of up to 59.46% over previous baselines. Our results suggest that learned sampling-parameter policies provide a complementary approach to existing post-training methods for improving diffusion model outputs.

[163] FlowCTS: On-policy Continuous Trajectory Supervision of Flow Models cs.LG | cs.CVPDF

Kaiyang Ye, Yuan Ge, Junxiang Zhang, Bei Li, Ziming Zhu

TL;DR: 本文提出了一种名为FlowCTS的连续轨迹监督方法，用于流模型的在线策略蒸馏。该方法通过匹配从同一学生访问状态初始化的后续学生轨迹和参考轨迹，利用轨迹与速度场之间的积分关系，推导出时间加权的速度匹配上界，并将其离散化为由监督步数参数化的实用目标。在多个基准测试中，FlowCTS-OPD在收敛速度和性能上均优于传统的基于KL散度的在线策略蒸馏方法。

Details

Motivation: 在线策略蒸馏在大型语言模型后训练中有效解决了稀疏奖励和暴露偏差问题，但其在流模型中的应用尚未充分探索。本文旨在将在线策略蒸馏扩展到流模型，以克服传统方法中存在的监督不匹配问题。

Result: 在GenEval基准上，FlowCTS-OPD将性能从0.90提升至0.93；在OCR基准上，从0.90提升至0.92；在PickScore基准上，从22.75提升至23.06。该方法在所有目标指标上均优于混合奖励的强化学习基线，并展现出比传统基于KL散度的在线策略蒸馏更快的收敛速度。

Insight: 创新点在于提出了连续轨迹监督框架，通过匹配轨迹而非仅匹配状态分布，解决了传统KL散度方法中因辅助SDE转移核引起的时间监督不匹配问题。此外，研究还揭示了监督步数增加在提供更丰富轨迹信息与增加优化难度之间的权衡关系。

Abstract: While on-policy distillation (OPD) effectively addresses sparse rewards and exposure bias in large language model post-training, its extension to flow models remains underexplored. To this end, we propose Flow Continuous Trajectory Supervision (FlowCTS), which matches subsequent student and reference trajectories initialized from the same student-visited state. Using the integral relation between trajectories and velocity fields, we derive a temporally weighted velocity-matching upper bound and discretize it into practical objectives parameterized by the number of supervision steps. Under a multi-reference setup, single-state FlowCTS-OPD outperforms vanilla KL-based OPD with faster convergence. FlowCTS-OPD improves GenEval from 0.90 to 0.93, OCR from 0.90 to 0.92, and PickScore from 22.75 to 23.06, while outperforming a mixed-reward RL baseline across all target metrics. Further analysis reveals a clear temporal supervision mismatch in vanilla KL-based OPD arising from its auxiliary SDE transition kernels. Beyond on-policy setting,FlowCTS also consistently outperforms vanilla SFT , particularly on OCR, while increasing supervision steps exhibit a trade-off between richer trajectory information and greater optimization difficulty.

[164] MXAttention: Data-Free Optimal Scaling and Pre-Normalization Quantization for MXFP4 Attention cs.LG | cs.AI | cs.CVPDF

Jianlin Yu, Jing Lin, Linghui Kong, Aiyue Chen, Weiyi Sun

TL;DR: 本文提出MXAttention，一种无需数据的后训练量化框架，用于解决MXFP4注意力机制在扩散视频生成模型中的数值问题。通过引入通用最优缩放（UOS）和预归一化量化（PNQ）两个组件，有效缓解了缩放与下溢的权衡以及softmax循环中的行归一化误差，从而在保持生成质量的同时实现高效推理。

Details

Motivation: 注意力机制的二次计算成本是扩散视频生成模型的主要瓶颈，而MXFP4量化因幂二缩放导致的裁剪-下溢权衡以及softmax循环中的行归一化误差，会降低生成质量，因此需要一种无需数据的量化方法来优化MXFP4注意力。

Result: 在Wan2.2和HunyuanVideo上的实验表明，MXAttention将OCP MXFP4与FP16之间的VBench图像质量差距缩小了至少95%，显著提升了帧级相似度，并在所有报告的VBench指标上保持FP16级别的生成质量（绝对退化小于0.01），同时与基于NVFP4的基线方法性能相当且开销可忽略。

Insight: 创新点包括：通用最优缩放（UOS）利用幂二微缩放的周期性结构，无需校准或搜索即可推导出分布无关的最优缩放边界Qmax=7.25；预归一化量化（PNQ）在行求和前量化未归一化的softmax指数，通过构造保留归一化，从而减少量化误差。这些方法为低精度注意力量化提供了高效且数据无关的解决方案。

Abstract: The quadratic cost of attention is a major bottleneck in diffusion-based video generation models. MXFP4 attention provides a promising path toward efficient inference, but direct MXFP4 quantization often degrades generation quality due to two numerical issues: the clipping-underflow trade-off from power-of-two scaling and the row-wise normalization error introduced in the softmax loop. We propose MXAttention, a data-free post-training quantization framework for MXFP4 attention. MXAttention introduces two components: Universal Optimal Scaling (UOS), which exploits the periodic structure of power-of-two microscaling to derive a distribution-independent optimal scaling boundary Qmax=7.25 without calibration or search, and Pre-Normalization Quantization (PNQ), which quantizes unnormalized softmax exponentials before row-wise summation to preserve normalization by construction. Experiments on Wan2.2 and HunyuanVideo show that MXAttention closes at least 95% of the VBench Imaging Quality gap between OCP MXFP4 and FP16, substantially improves frame-level similarity, and preserves FP16-level generation quality with less than 0.01 absolute degradation on all reported VBench metrics. MXAttention also achieves performance competitive with strong NVFP4-based baselines with negligible overhead when fused into the attention pipeline. The implementation is publicly available in MindIE-SD.

cs.CE [Back]

[165] A Scale-adaptive Vision Model Links C. elegans Neuronal Morphology to Behavior for Neurotoxicity Assessment cs.CE | cs.CVPDF

Haochao Ying, Shenchong Lv, Yutao Sun, Zijian Tu, Xufeng Jin

TL;DR: 本文提出了一种用于秀丽隐杆线虫多巴胺能神经元的专用自监督视觉模型，以及一个包含27,117张标注图像的共聚焦显微镜基准数据集CeNeuMorph。该模型采用了一种尺度自适应的掩码图像建模策略，能够在固定计算预算下跨分辨率和补丁大小联合学习表征，有效解析从精细树突串珠到整体胞体萎缩的全谱神经退行性病变。模型在分类、分割和检测任务上超越了通用和生物医学基础模型，并将视觉特征与形态描述符融合，成功预测了多巴胺依赖的行为缺陷。

Details

Motivation: 神经毒性评估目前依赖于主观且难以预测行为结果的人工形态学评分。虽然秀丽隐杆线虫提供了一个遗传可控且符合3R原则的替代模型，但从共聚焦显微镜图像中大规模量化神经元表型在计算上仍具挑战性，因为现有的视觉基础模型无法解析神经元成像中的稀疏信号和多尺度病变。

Result: 在分类、分割和检测任务上，该模型超越了通用模型（如MAE）和生物医学基础模型（如MedSAM）。通过融合视觉特征与形态描述符，模型能够预测多巴胺依赖的行为缺陷（R²=0.498）。在筛选180种农用化学品后，识别出苯并咪唑基团是先前未被认识的多巴胺能神经毒性决定因素。

Insight: 核心创新点是提出了一种尺度自适应的掩码图像建模策略，通过将结构语义学习与刚性网格约束解耦，在可处理的计算框架内有效解析全谱神经退行性病变。这为连接形态学到功能、构建可扩展的神经毒性评估和药物发现替代模型提供了新方法。

Abstract: Neurological disorders are a leading cause of global disability and are increasingly linked to environmental chemical exposures. Yet neurotoxicity assessment still relies on hand-scored morphological readouts that are subjective and poorly predictive of behavioral outcomes. Caenorhabditis elegans provides a genetically tractable, 3R-compliant alternative, but quantifying neuronal phenotypes from confocal microscopy at scale remains computationally challenging: existing vision foundation models, trained on natural or radiological images, cannot resolve the sparse signals and multi-scale lesions of neuronal imaging. Here, we introduce a dedicated self-supervised vision model for C. elegans dopaminergic neurons, together with CeNeuMorph, a multi-grained confocal benchmark of 27,117 annotated images. Specifically, moving beyond standard Masked Autoencoders, we propose a scale-adaptive masked image modeling strategy that jointly learns representations across resolutions and patch sizes under a fixed token budget. By decoupling structural semantic learning from rigid grid constraints, the model effectively resolves the full spectrum of neurodegenerative lesions - ranging from fine dendritic beading to gross soma shrinkage - within a tractable computational framework. Finally, our model surpasses both generalist and biomedical foundation models across classification, segmentation and detection tasks. Fusing visual features with morphological descriptors enables prediction of dopamine-dependent behavioral deficits ($R^2=0.498$). Screening 180 agrochemicals, we identify the benzimidazole moiety as a previously unrecognized determinant of dopaminergic neurotoxicity. Together, the work demonstrates how scale-adaptive self-supervised learning can connect morphology to function for a scalable alternative to mammalian in vivo models for neurotoxicity assessment and drug discovery.

cs.SD [Back]

Jun Zhan, Chen Yang, Yitian Gong, Donghua Yu, Kuangwei Chen

TL;DR: 本文提出了OmniVAE，一种联合训练的音频-视频变分自编码器，旨在解决现有方法在联合生成同步音频和视频时面临的跨模态对齐挑战。通过引入片段级音频-视频对比目标和从预训练模态特定编码器蒸馏特征，该方法学习细粒度语义对齐的潜在表示，从而提升下游生成任务的质量和同步准确性。

Details

Motivation: 现有方法通常使用单独训练的音频和视频VAE，导致潜在空间缺乏跨模态对齐，使得下游生成模型需要从头学习跨模态同步，这限制了联合音频-视频生成的细粒度对应关系。

Result: 实验表明，所提出的对比目标和特征蒸馏目标一致提升了潜在空间的可学习性，在下游文本到音频-视频生成任务中实现了更高的生成质量和更准确的跨模态同步。

Insight: 创新点在于通过联合训练和跨模态对齐机制（如片段级对比学习）来学习统一的音频-视频表示，这为全模态建模提供了基础，强调了学习统一表示的重要性。

Abstract: Recent generative models are moving beyond silent video or standalone audio synthesis toward the joint generation of synchronized audio and video. Despite this progress, jointly generating audio and video with fine-grained cross-modal correspondence remains challenging due to their fundamental structural differences. Most existing methods use audio and video VAEs trained separately. As a result, the two latent spaces lack cross-modal alignment, leaving the downstream generative model to learn cross-modal synchronization from scratch. We present OmniVAE, a jointly trained audio-video VAE that learns fine-grained semantic alignment between audio and video latent representations. Beyond reconstruction, OmniVAE uses a segment-level audio-video contrastive objective to capture temporal-semantic correspondence and align the two latent spaces. In parallel, it distills features from pretrained modality-specific semantic encoders into each modality, improving the downstream learnability of both latent spaces. Extensive experiments show that both objectives consistently improve the learnability of the latent spaces, translating into higher generation quality and more accurate cross-modal synchronization in downstream text-to-audio-video generation. These findings underscore the importance of learning unified representations as a foundation for omnimodal modeling.1

cs.HC [Back]

[167] Let Me Look at You: Advanced Facial Expression Modeling for Conversational Speech Synthesis cs.HC | cs.CL | eess.ASPDF

Yifan Hu, Shuwei He, Rui Liu, Haizhou Li

TL;DR: 本文提出FacialTalker，一个基于大语言模型的面部表情感知对话语音合成框架。为了解决现有方法忽视面部表情模态以及缺乏大规模多模态对话数据集的问题，作者引入了AUTokenizer将面部表情离散化为紧凑token，并提出了双直接偏好优化策略来联合优化视觉和语音token序列。此外，还构建了VSDD-1K大规模多模态对话数据集。实验表明，FacialTalker在面部表情感知和语音合成质量上均优于基线模型。

Details

Motivation: 现有对话语音合成方法往往忽视面部表情这一传递丰富情感线索的重要模态，且缺乏大规模、自然的包含语音和视觉模态的对话数据集，这限制了对话场景下视觉情感理解的发展。

Result: 在构建的VSDD-1K数据集上进行的广泛主客观实验表明，FacialTalker在面部表情感知和语音合成质量上持续优于强基线模型，生成的语音更自然、更具表现力且与对话上下文更一致。

Insight: 创新点包括：1）提出AUTokenizer，一个基于面部动作单元监督训练的单码本视觉分词器，用于高效编码面部表情；2）提出双直接偏好优化策略，将DPO扩展到同时对视觉和语音token序列施加偏好约束，以增强多模态对话上下文理解；3）通过全自动流程构建了大规模、高质量的多模态对话数据集VSDD-1K。

Abstract: Conversational Speech Synthesis is a fundamental component of human-computer interaction, aiming to generate contextually appropriate, expressive, and empathetic speech. However, facial expressions encode subtle and rich affective cues that are crucial for empathetic speech interaction, whereas existing approaches often overlook this important modality. In addition, the lack of large-scale natural conversational datasets with both speech and visual modalities also limits the development of visual affect understanding in conversational settings.To address these limitations, we propose FacialTalker, a facial-expression-aware CSS framework built upon a large language model backbone. To efficiently encode facial expressions, we propose AUTokenizer, a single-codebook visual tokenizer that discretizes each frame-level facial expression into a compact token, trained with supervision from combinations of facial Action Units. We further introduce a dual direct preference optimization (DualDPO) strategy, which extends the DPO by jointly imposing preference constraints on both visual and speech token sequences, to enhance the model’s understanding of facial expressions and speech semantics in multimodal conversational contexts. Moreover, we construct VSDD-1K, a large-scale multimodal dialogue dataset collected through a fully automated pipeline from real-world Internet conversations, comprising over 1,033 hours of synchronized speaker videos and speech, with more than 85% of frames containing valid faces. Extensive objective and subjective experiments demonstrate that FacialTalker consistently outperforms strong baselines in facial-expression perception and speech synthesis quality, generating speech that is more natural, expressive, and better aligned with the conversational context. The results also validate the effectiveness of our training strategy and dataset construction pipeline.

cs.IR [Back]

[168] Language-Routed RAG and Direct Option Scoring for Multilingual Financial QA: DS@GT at FinMMEval cs.IR | cs.AI | cs.CLPDF

Justice Ayela, Kabir Sahni

TL;DR: 本文介绍了DS@GT团队为FinMMEval 2026 Task 1多语言金融问答评测提交的系统。该系统构建了一个基于LangGraph的检索增强管道，通过语言检测、多语言知识库检索（使用BGE-M3嵌入和FAISS索引）以及检索增强直接评分（RADS）来回答金融认证考试问题。系统还采用了基于语言的路由策略来选择不同的大语言模型，并针对低资源语言融合了跨语言检索索引。

Details

Motivation: 解决多语言金融认证考试（如CFA、EFPA、CPA）问答的挑战。这些考试需要结构化领域推理，而标准NLP基准无法捕捉，且跨语言检索和表示基础设施不完善，使得问题在多种语言（英语、西班牙语、希腊语、中文、印地语）中更加复杂。

Result: 在FinMMEval 2026 Task 1多语言金融问答基准上进行了评估。结果表明，思维链提示会显著降低希腊语准确率（从90.7%降至20.9%），启用Qwen3的默认思考模式会使阿拉伯语的RADS性能降至接近随机水平。系统通过语言感知检索、模型路由和评分策略选择实现了有效的多语言金融推理。

Insight: 创新点包括：1）语言路由的检索增强生成（RAG）管道，根据查询语言动态选择不同的大语言模型（如Qwen3-14B用于阿拉伯语、中文和印地语）；2）检索增强直接评分（RADS），通过读取候选选项字母的下一个词元对数概率来评分，而非生成自由形式的输出；3）针对低资源语言，使用加权倒数排名融合（RRF）融合单语言和跨语言检索索引。客观来看，其核心洞察是，有效的多语言金融推理需要精细的语言感知组件（检索、模型选择、评分策略）组合，而非统一的通用方法，这揭示了当前大模型在多语言、专业领域任务中存在的显著性能不对称性。

Abstract: We present DS@GT’s submission to FinMMEval 2026 Task 1, a multilingual financial exam question answering benchmark spanning English, Spanish, Greek, Chinese, and Hindi. Financial certification exams such as the CFA, EFPA, and CPA demand structured domain reasoning that standard NLP benchmarks do not capture, and this challenge compounds across languages where retrieval and representation infrastructure is underdeveloped. We build a retrieval-augmented pipeline on LangGraph that detects query language and retrieves semantically relevant exemplars from a 30,209-entry multilingual knowledge base using BGE-M3 embeddings and FAISS indexing. The system then scores answers via Retrieval-Augmented Direct Scoring (RADS), reading next-token log-probabilities over candidate option letters rather than generating free-form output. For low-resource languages, we fuse per-language and cross-lingual retrieval indices using weighted Reciprocal Rank Fusion. Model selection is language-routed: Qwen3-14B for Arabic, Chinese, and Hindi; Qwen2.5-14B for English; and Llama-3.1-8B for Greek, a routing derived from empirical ablations that reveal substantial language-asymmetric performance gaps. Notably, chain-of-thought prompting significantly degrades Greek accuracy (90.7% to 20.9%), and enabling Qwen3’s default thinking mode collapses Arabic RADS performance to near-chance levels. Our results indicate that effective multilingual financial reasoning requires language-aware retrieval, model routing, and deliberate scoring strategy selection.

[169] Energy Constrained Hierarchical Underwater Monitoring via Local Multi-Agent RAG cs.IR | cs.CV | cs.MA | cs.ROPDF

Mohamed Amine Janati, Laurent Gautier, Stéphane Barbot

TL;DR: 本文提出了一种面向严格能量约束的海洋生物监测分层架构，结合了持续工作的边缘传感与选择性启动的高性能本地推理。系统采用主-从卫星设计，超低功耗微控制器持续监测视觉和声学信号，而高性能Jetson设备仅在需要时激活，执行包含数据摄取、视觉目标提取、嵌入索引、物种识别、检索增强推理和自动报告的全本地多模态处理流程。

Details

Motivation: 解决海洋生物监测中存在的严格能量限制、水下连接性差以及从远程部署传输原始多模态数据成本高昂的问题。

Result: 通过视觉和声学监测案例研究进行评估，该系统将超低功耗连续传感与本地多模态智能相结合，使水下监测站能够生成结构化的、可供研究人员直接使用的知识，同时压缩本地数据以支持灵活的声学、光学或卫星传输。

Insight: 创新点在于分层主-从架构与本地多模态智能管道的结合，利用嵌入索引和检索增强生成（RAG）技术组织多源数据，并采用基于LangChain的多智能体框架协调查询路由、结构分析和能源管理，在本地实现高效推理以减少通信开销。

Abstract: Marine life monitoring is limited by strict energy constraints, poor underwater connectivity, and the high cost of transmitting raw multimodal data from remote deployments. This paper proposes a low-consumption underwater monitoring architecture that combines always-on edge sensing with selective high-performance local reasoning. The system follows a hierarchical master–satellite design in which ultra-low-power MAX78000/MAX78002 microcontrollers continuously monitor visual and acoustic signals, while an NVIDIA Jetson Orin NX is activated only for scheduled processing, event-driven analysis, or researcher interaction. Once active, the Jetson executes a fully local multimodal pipeline for data ingestion, visual target extraction, embedding-based indexing, species identification, retrieval-augmented reasoning, and automated reporting. BioCLIP/OpenCLIP embeddings are used to organize mission data, marine taxonomic references, scientific documents, and operational metadata in local ChromaDB collections. A dedicated identification layer combines visual similarity search, centroid-based classification, and supervised classifiers to support adaptive species recognition. A LangChain-based multi-agent framework coordinates query routing, structured analysis, energy management, hardware reconfiguration, and report generation. The architecture is evaluated through visual and acoustic monitoring case studies. The proposed system bridges ultra-low-power continuous sensing with local multimodal intelligence, enabling underwater stations to produce structured, researcher-ready knowledge while compressing local data for flexible acoustic, optical, or satellite transmission, minimizing both energy use and communication overhead.

[170] DeCoRAG: Cognitive Decoupling and Semantic-Aware Cropping for Complex Document Understanding cs.IR | cs.CVPDF

Shuo Wang, Kai Zhang, Wenyuan Huang, Yizheng Yu, Xia Liao

TL;DR: 本文提出DeCoRAG，一种用于复杂文档理解的多模态检索增强生成（RAG）框架。它通过认知解耦和语义感知裁剪，解决了现有图RAG方法在处理视觉密集但结构稀疏的文档时面临的准确性与效率双重困境。该方法在多个复杂文档基准测试上显著提升了语义通过率，并大幅降低了计算开销。

Details

Motivation: 现有基于图的多模态RAG流程在处理视觉密集、结构稀疏的复杂文档（如从金融图表中提取微小数据标记）时，面临计算开销巨大和严重幻觉问题。其核心假设是视觉语言模型（VLM）能够解析高密度布局中的稀疏语义，但作者挑战了这一假设，发现这会导致‘视觉注意力沉没’和灾难性语义丢失。

Result: 在多个复杂文档基准测试上，DeCoRAG相比最强基线将语义通过率提升了高达12.5个百分点，并能泛化至DocVQA任务。其提出的RAP-Crop机制在不牺牲端到端准确性的前提下，将离线图构建阶段的提示词元减少了40.8%。

Insight: 主要创新点在于提出了‘认知解耦’范式，将知识处理从耦合的视觉-语义推理中分离出来。具体通过建立宏观语义锚点来中和注意力沉没，并驱动区域感知剪枝与裁剪（RAP-Crop）机制，将推理空间从嘈杂的背景转移到纯净的、意图驱动的语义簇，从而在根本上修正了现有方法的缺陷。

Abstract: Advancing multimodal retrieval-augmented generation (RAG) for complex document understanding presents a formidable dual dilemma of accuracy and efficiency, particularly in graph RAG. Processing structurally sparse yet visually dense layouts, such as extracting a tiny data marker from a financial chart, often incurs computationally prohibitive token overhead while still triggering catastrophic hallucination. However, multimodal Graph RAG pipelines rely on graph-construction stages that assume Vision-Language Models (VLMs) can resolve sparse semantics within high-density layouts. We challenge this assumption, revealing that forcing VLMs to localize visual evidence, interpret semantics, and extract relations triggers a “Visual Attention Sink,” a mechanism driving catastrophic semantic loss, while full-page processing incurs massive computational overhead. Controlled interventions verify that this failure is boundary-driven rather than content-specific and that semantic anchoring mitigates it. To fundamentally correct this flawed paradigm, we introduce DeCoRAG, a multimodal Graph RAG pipeline that shifts knowledge processing from coupled visual-semantic reasoning to “Cognitive Decoupling.” Rather than passively processing raw pixels, its graph-construction stage establishes a macroscopic Semantic Anchor to neutralize the attention sink. This anchor subsequently drives our Region-Aware Pruning and Cropping (RAP-Crop) mechanism, shifting the reasoning space from dense, noisy backgrounds to purified, intent-driven semantic clusters. The resulting graph supports hybrid retrieval and answer generation. Across complex document benchmarks, DeCoRAG improves the semantic pass rate by up to 12.5 percentage points over the strongest baseline and generalizes to DocVQA. RAP-Crop reduces offline graph-construction prompt tokens by 40.8% without sacrificing end-to-end accuracy.

cs.GR [Back]

[171] Text-based Tactile Graphics Generation for the Visually Impaired cs.GR | cs.CV | cs.HCPDF

Ruihan Gao, Joonghyuk Shin, Ava Pun, Jaesik Park, Wenzhen Yuan

TL;DR: 本文提出了一种从自然语言提示直接生成可3D打印的2.5D触觉图形的集成生成系统，该系统联合生成全局基础几何、细粒度触觉表面纹理和符合标准的盲文，并通过用户研究验证了其优于基线方法。

Details

Motivation: 触觉图形是盲人和低视力（BLV）人群获取非文本信息的主要媒介，但难以扩展或个性化；现有生成模型专注于视觉真实感，无法满足触觉感知和物理制造约束。

Result: 广泛的评估以及BLV参与者和蒙眼视力正常参与者使用3D打印输出的用户研究表明，参与者一致认为本文结果优于基线。

Insight: 创新点在于首次提出了一个集成生成系统，引入了制造感知技术（如模板引导的浮雕生成、基于快速扩散的文本到纹理模块、严格的基础平坦化），将生成图形从屏幕扩展到可触摸的浮雕，拓宽了BLV社区对生成AI的访问。

Abstract: Tactile graphics are a primary medium for blind and low-vision (BLV) individuals to access non-textual information. However, they are difficult to scale or personalize. While recent generative models have revolutionized visual content creation, they are optimized for screen-based visual realism and fail to satisfy the haptic perceptual and physical fabrication constraints required for touch. We present the first integrated generative system that produces fabrication-ready 2.5D tactile graphics directly from natural language prompts, jointly generating global base geometry, fine-grained tactile surface textures, and standard-compliant braille within a unified 3D-printable representation. Our approach introduces fabrication-aware techniques, including template-guided relief generation, a fast diffusion-based text-to-texture module for high-resolution tileable normal maps, and strict base flattening to ensure tactile readability and printability, while supporting both automatic generation and interactive texture control. Extensive evaluations, together with in-person user studies with BLV participants and blindfolded sighted participants using physically 3D-printed outputs, show that participants consistently prefer our results over baselines. By extending generative graphics beyond screens to touchable reliefs, our work broadens access to generative AI for the BLV community and beyond.

cs.AI [Back]

[172] Same Question, Different Answers: Evaluating LLM Reliability Beyond Accuracy cs.AI | cs.CL | cs.LGPDF

Kazem Faghih, Yize Cheng, Shoumik Saha, Mobina Pournemat, Armin Gerami

TL;DR: 这篇论文研究了大型语言模型在相同问题不同表述下的可靠性，发现模型输出对提示词的具体措辞高度敏感，即使整体准确率变化不大，实例层面的答案稳定性却很差。通过四个基准测试和13个模型，作者揭示了标准准确率指标可能掩盖了模型的不稳定性，并提出了一种简单的自释义策略来部分恢复潜在知识。

Details

Motivation: 尽管LLM在基准测试中表现出高准确率，但其在语义相同但措辞不同的等价问题上的可靠性尚不明确。论文旨在探究模型在事实问答和数学推理任务中，面对保持语义的释义时答案的一致性，以评估其真实可靠性。

Result: 在四个基准测试和13个模型上的实验表明，模型输出频繁依赖于提示词的精确措辞。虽然整体准确率在释义间变化不大，但实例层面的不稳定性很高：对于许多问题，模型会根据措辞在正确和错误答案之间切换，不匹配率超过23%。通过简单的自释义策略，可以在推理时部分恢复潜在知识并提升性能。

Insight: 论文的创新点在于提出了超越标准准确率的可靠性评估框架，强调通过等价输入的一致性来更清晰地衡量LLM的可靠性。客观分析认为，其揭示了模型知识检索的不一致性，并展示了通过自释义利用这种不一致性来提升性能的实用方法，这对模型评估和推理优化具有借鉴意义。

Abstract: Large language models (LLMs) often achieve strong accuracy on benchmarks, yet it remains unclear how reliably they apply this knowledge when the same question is phrased in different but equivalent ways. In this work, we study how model answers change under meaning-preserving paraphrases across factual question answering and mathematical reasoning tasks. Across four benchmarks and 13 models, we find that model outputs frequently depend on the exact wording of the prompt. While overall accuracy typically changes only modestly across paraphrases, instance-level behavior is far less stable: for many questions, models alternate between correct and incorrect answers depending on phrasing, with mismatch rates reaching more than 23%. Conditioning on questions that are answered correctly in their original form reveals even larger failures measured by answer flip rates, showing that single-prompt correctness is often a poor indicator of reliability. At the same time, we find that models often produce a correct answer for at least one paraphrase of a question, suggesting that the underlying knowledge is present but inconsistently retrieved. Building on this observation, we show that a simple self-paraphrasing strategy can partially recover this latent knowledge and improve performance at inference time. Together, these findings suggest that standard accuracy metrics can mask substantial instability, and that evaluating consistency across equivalent inputs provides a clearer picture of LLM reliability.

[173] MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models cs.AI | cs.CLPDF

Jinsong Shu, Chenyang Wu, Zhongle Xie, Baokun Wang, Lidan Shou

TL;DR: 本文提出MM-ShiftKV，一种用于多模态大语言模型（MLLM）的、无需训练的解码感知型预填充阶段KV缓存选择方法。该方法通过构建方差扩展的查询代理来近似解码时的查询行为，并基于聚合注意力权重估计提示KV的重要性，以解决现有方法因预填充与解码阶段查询分布差异导致的KV重要性估计不稳定问题。

Details

Motivation: 现有预填充阶段KV选择方法假设预填充时的查询能代表解码时的查询，但在多模态推理中，解码时查询的方差远大于预填充阶段表示，导致在严格的缓存预算下KV重要性估计不稳定，可能错误丢弃关键的视觉token，损害模型的指代和推理性能。

Result: 在多模态基准测试上的实验表明，在严格的KV缓存预算下，MM-ShiftKV持续优于现有方法。

Insight: 创新点在于提出了一种训练免费、解码感知且严格仅需预填充的KV选择机制，其核心洞察是构造方差扩展的查询代理来弥合预填充与解码阶段查询分布的差异，从而更稳定地估计KV重要性，提升缓存效率下的模型性能。

Abstract: Key-Value (KV) caching is essential for efficient inference in multimodal large language models (MLLMs), yet its memory footprint grows linearly with context length and becomes a major bottleneck due to the large number of visual tokens. Recent prefill-stage KV selection methods estimate KV importance from prefilling statistics, implicitly assuming that prefilling-time queries are representative of those encountered during decoding. We show that this assumption breaks down in multimodal inference, where decoding-time queries exhibit substantially larger variance than prefilling-stage representations, leading to unstable KV importance estimation under tight cache budgets. As a result, small ranking errors can disproportionately discard semantically critical visual tokens and degrade grounding and reasoning performance. We propose MM-ShiftKV, a training-free, decode-aware and strictly prefill-only KV selection method. MM-ShiftKV approximates decoding-time query behavior during prefilling by constructing variance-expanded query proxies and estimates prompt KV importance based on their aggregated attention mass. Experiments on multimodal benchmarks demonstrate that MM-ShiftKV consistently outperforms existing methods under strict KV-cache budgets. Our code is available at https://github.com/zjuDBxAI/MM-ShiftKV.

[174] Structure Over Scale: Schema-Constrained Causal Graphs for RAG cs.AI | cs.CL | cs.IRPDF

Marc Saouda, Rajprakash Bale, Eren Aldis, Cloves Almeida

TL;DR: 本文提出HCG-RAG（分层因果图RAG），一种基于模式约束因果图的检索增强生成方法，以解决现有GraphRAG系统因无限制提取实体和关系导致图规模过大、构建成本高的问题。该方法通过自动化流程将语料库提炼为固定的因果变量词汇表，并构建紧凑的两层图，在保持答案质量的同时大幅降低了计算开销。

Details

Motivation: 现有基于图的检索增强生成系统通常进行无限制的实体和关系提取，导致图的大小和构建成本随语料库长度线性增长，而非根据查询所需的推理需求进行优化，这带来了效率低下和可审计性差的问题。

Result: 在医学和临床基准测试（包括神经学家验证的癫痫数据集）上，HCG-RAG匹配或超越了最佳实体-关系系统。与最耗LLM调用的基线（MS-GraphRAG）相比，节点数减少3-20倍，构建时的LLM调用减少8-135倍。消融实验表明，因果图作为结构化检索过滤器，比仅使用嵌入检索的准确率提升了6个百分点。

Insight: 核心创新在于用模式约束的因果图替代开放式提取，通过固定类型化的因果变量词汇表和分层图结构，实现了图规模的紧凑化和构建成本的大幅降低。研究还表明，对于具有可发现分层因果结构的领域，图内容的组织方式（高层次结构）比节点数量更重要，这为高效RAG系统设计提供了新思路。

Abstract: Graph-based retrieval-augmented generation (GraphRAG) grounds answers in structured knowledge, but current systems extract entities and relationships exhaustively, producing graphs whose size and construction cost scale with corpus length rather than with the reasoning a query requires. We introduce HCG-RAG (Hierarchical Causal Graph RAG), which replaces open-ended extraction with schema-constrained causal graphs: an automated pipeline distills a corpus into a fixed, typed vocabulary of causal variables and materializes a compact two-tier graph over it. Our schema-constrained graphs match entity-relation baselines on answer quality at a fraction of the cost: 3-20x fewer nodes, 8x-135x fewer build-time LLM calls than the most LLM-intensive baseline (MS-GraphRAG), and graphs compact enough for a domain expert to audit, correct, and extend. On medical and clinical benchmarks, including a neurologist-validated epilepsy dataset, HCG-RAG matches or exceeds the best entity-relation systems. An ablation isolates the causal graph as a structured retrieval filter, contributing +6 percentage points (pp) over embedding-only retrieval. Across all domains with discoverable hierarchical causal structure, only methods imposing higher-level organization outperform flat entity-relation retrieval, indicating that what is placed in the graph matters more than how many nodes it contains.

[175] Tokengeist: Multi-Turn Attribution Tracing in Agentic Conversations cs.AI | cs.CLPDF

Jessica Tang, Shraddha Barke, Sharad Agarwal

TL;DR: 本文提出了多轮上下文归因（MTCA）任务，旨在追踪多轮对话中模型响应的来源，识别哪些先前轮次的token影响了当前回答，以及这些依赖关系如何在对话中传播。作者提出了Tokengeist框架，通过将归因问题转化为对话轮次有向无环图（DAG）的递归遍历，来恢复完整的依赖路径。同时，作者发布了MTCABench基准，包含665个多轮对话的3,845个目标跨度，并标注了深度达14的黄金来源图。实验表明，现有单次归因方法在多跳依赖上表现不佳（源召回率低于20%），而Tokengeist达到90%，揭示了单次归因的系统性失败模式（称为来源崩溃）。

Details

Motivation: 现有上下文归因方法通常单次处理完整上下文，只能恢复表面依赖，无法捕捉现实世界对话和多步推理任务中的分层、非线性结构，因此需要解决多轮对话中依赖关系的传播问题。

Result: 在四个开放权重模型上，平坦归因方法在多跳依赖恢复上源召回率低于20%，而Tokengeist达到90%，在MTCABench基准上表现出色，揭示了单次归因的’来源崩溃’失败模式。

Insight: 创新点在于将多轮归因任务形式化为递归遍历DAG，提出了与归因方法无关的Tokengeist框架，并通过MTCABench基准提供了黄金标注，强调了递归跨轮推理的重要性，为理解模型对话依赖提供了新视角。

Abstract: When a language model produces a response in a multi-turn conversation, which tokens from prior turns shaped that answer, and how did those dependencies propagate across prior turns? Existing context attribution methods process the full context in a single pass, recovering surface-level dependencies but missing the layered, non-linear structure of real-world dialogues and multi-step reasoning tasks. We introduce multi-turn context attribution (MTCA): given a target span in a model response, the task of tracing attribution backward across turns to identify not only which prior turns were directly relevant, but also how those turns themselves depended on earlier context. We propose Tokengeist, an attribution-method-agnostic and scalable framework that recovers full dependency paths by casting attribution as a recursive traversal of a directed acyclic graph (DAG) over conversation turns. We will release MTCABench, a benchmark of 3,845 target spans across 665 multi-turn conversations, annotated with gold provenance graphs reaching depths of up to 14, across four dependency types. Across four open-weight models, flat attribution methods fail to recover multi-hop dependencies, achieving under 20% source recall, while Tokengeist reaches 90%. Our results reveal systematic failure modes of single-pass attribution – which we term provenance collapse – and motivate attribution methods that reason recursively across turns.

[176] Masked Distillation: Internalizing the Chain-of-Thought in Language Models cs.AI | cs.CLPDF

Durgesh Kalwar, Vardhan Palod, Subbarao Kambhampati

TL;DR: 本文提出了一种名为“掩码蒸馏”的知识蒸馏框架，旨在将大型推理模型（LRMs）生成的长链式思维（CoT）中间步骤内部化到语言模型的参数中，从而减少推理时的延迟、内存使用和服务成本。该框架通过教师模型（在思考模式下生成CoT）监督学生模型（在非思考模式下直接预测答案）来实现，并在GSM8K和Countdown两个推理任务上进行了实验评估。

Details

Motivation: 大型推理模型在推理时生成的长链式思维中间步骤会显著增加延迟、内存和成本，但这些步骤的正确性与最终答案的正确性并无因果关联，且步骤长度不能可靠反映问题复杂度。因此，研究如何将这些中间计算内部化到模型参数中，使模型能直接（或以更短中间步骤）生成答案。

Result: 在GSM8K（小学数学算术）和Countdown（数字谜题搜索任务）两个推理领域的控制实验中，通过掩码蒸馏框架训练的学生模型能够内部化链式思维，实现直接生成答案或缩短中间步骤，具体定量结果未在摘要中明确给出，但暗示了该方法在减少推理开销方面的潜力。

Insight: 创新点在于将链式思维中间步骤视为“脚手架”，通过知识蒸馏将其内部化到模型参数中，允许模型在推理时跳过或缩短这些步骤。这包括自蒸馏（同一模型作为教师和学生）和双模型（大教师监督小学生）两种设置，以及通过调整监督中间令牌长度来探索内部化程度，为优化推理效率提供了新思路。

Abstract: Large Reasoning Models (LRMs) produce long, explicit chains of intermediate steps before generating a final answer at inference time. These intermediate traces dominate latency, memory usage, and serving cost, even though the final answer correctness is not causally related to the trace correctness and the trace length is not a reliable indicator of the problem complexity. This raises a natural question: can the computation expressed in these intermediate tokens be internalized into the parameters of a language model, enabling it to produce answers directly (or with much shorter intermediate traces)? We introduce \textit{masked distillation}, a knowledge-distillation framework in which a student LLM is trained to predict only the solution tokens conditioned on the question, while a reasoning teacher provides feedback on the student’s responses after conditioning on the question and its own CoT trace. We instantiate this framework in two settings: (i) a \textit{self-distillation} setting, in which the same model serves as the teacher in thinking mode and as the student in non-thinking mode, and (ii) a \textit{dual-model} setting, in which a larger reasoning teacher supervises a separate smaller non-thinking student over the solution tokens. By treating intermediate tokens as a scaffold which reasoning models use to fit over the solution tokens, We additionally vary the length of intermediate-token scaffolding the student is supervised on, interpolating between full internalization (the student emits only the solution) and no internalization (the student emits the full trace before the answer). We evaluate the framework through controlled experiments on two reasoning domains: GSM8K (grade-school arithmetic) and Countdown (a number-puzzle search task).

[177] CRAFT: Learn the Schema, Execute the Plan cs.AI | cs.CL | cs.LG | cs.MA | cs.SEPDF

Aakash Kolekar, Sahika Genc, Shahriar Shariat, Bunyamin Sisman, Tibor Mezi

TL;DR: CRAFT提出了一种两阶段后训练方法，用于构建基于企业数据模式的编码智能体。该方法首先通过模式剥离的监督微调学习领域结构化的计划和可执行行为，然后通过执行导向的强化学习对齐工具选择、代码质量、计划-代码一致性和执行失败恢复策略。

Details

Motivation: 解决企业编码智能体在将自然语言分析请求转换为可执行代码时，因需要将详尽模式文档注入每个提示而导致的推理开销大、模式演进复杂以及多轮分析可靠性差的问题。

Result: 在企业广告分析环境中评估，相对于基线，CRAFT将综合智能体得分提升了9.6个百分点，一致性提升4.1个百分点，多轮连贯性提升4.2个百分点，同时将输入令牌负担减少约9倍，模式发现循环减少高达5倍。

Insight: 创新点在于将稳定的模式知识和工具使用行为通过后训练习得，而非依赖提示注入，并通过结合执行验证、数据完整性检查和LLM判断推理审计的三重门过滤器来筛选训练轨迹，以保障生产环境所需的一致性。

Abstract: Enterprise coding agents translate natural-language analytical requests into executable code over proprietary APIs, schemas, and metric definitions. Yet the prevailing deployment pattern injecting exhaustive schema and tool documentation into each prompt increases inference overhead, complicates schema evolution, and undermines reliability in multi-turn analysis. We investigate whether stable schema knowledge and tool-use behavior can instead be acquired through post-training while preserving the consistency required for production-facing analytics. We present CRAFT, a two-stage post-training recipe for schema-grounded coding agents. First, schema-stripped PLAN supervised fine-tuning learns domain-structured plans and executable behaviors from validated trajectories without exhaustive prompt-time schema injection. Second, execution-shaped reinforcement learning aligns the policy for tool selection, code quality, plan-code consistency, and recovery from failed executions. Training trajectories are curated through a Tri-Gate filter combining execution validation, data-integrity checks, and LLM-judge reasoning audit. We evaluate CRAFT for planned rollout in advertising analytics, covering campaign performance analysis, metric drill-downs, entity-level performance analysis, and multi-turn analytical refinement. The enterprise evaluation environment incorporates beta APIs as the agent-facing tool surface and spans 25 schema-linked core entities and 30 agentic workflows. Relative to a schema-stuffed baseline, CRAFT improves composite Agent Score by +9.6 pp, consistency by +4.1 pp, and multi-turn coherence by +4.2 pp, while reducing input-token burden by approximately 9x and schema-discovery loops by up to 5x. We further report deployment tradeoffs, reward-shaping limitations, and training-infrastructure extensions required for multi-turn tool-use reinforcement learning in enterprise settings.

[178] STAIF: A Stage-wise Optimization for Complex Instruction Following cs.AI | cs.CLPDF

Jian Hong, Chen Cheng, Quan Liu, Yuhao Chen, Enhong Chen

TL;DR: 本文提出STAIF，一种分阶段优化框架，旨在提升大语言模型遵循复杂多约束指令的能力。该框架将主观（软）约束的对齐与客观可验证（硬）约束的优化解耦，第一阶段使用带多个负样本的偏好优化来增强对软约束的敏感性，第二阶段使用带可验证奖励的强化学习来强制满足硬约束。为支持该方法，作者构建了一个高质量的双语数据集STAINSTRUCT。

Details

Motivation: 现有对齐方法（如DPO）优化的整体奖励信号往往未能充分强调对单个约束的严格满足，尤其是在分布外或多约束场景下，导致大语言模型遵循复杂指令的能力存在根本性挑战。

Result: 广泛的实验分析验证了STAIF设计的有效性，并在代表性基准测试上取得了最先进的性能，超越了强基线模型，并展现出真正的泛化能力。

Insight: 核心创新在于将复杂指令遵循问题解耦为软、硬约束的分阶段优化，并引入了带多个负样本的偏好优化和带可验证奖励的强化学习。从客观角度看，构建高质量双语多约束指令数据集STAINSTRUCT也为该领域研究提供了有价值的资源。

Abstract: Following complex instructions with multiple explicit constraints remains a fundamental challenge for large language models (LLMs). Existing alignment methods, such as DPO, optimize holistic reward signals that often underemphasize strict satisfaction of individual constraints, particularly under out-of-distribution or multi-constraint settings. In this paper, we propose STAIF, a stage-wise optimization framework that decouples the alignment of subjective (soft) constraints from the optimization of objectively verifiable (hard) constraints. Stage 1 applies preference optimization with multiple negative samples to sharpen sensitivity to soft constraints, while Stage 2 applies Reinforcement Learning with Verifiable Rewards (RLVR) to enforce strict compliance with hard constraints. To support this method, we construct STAINSTRUCT, a high-quality bilingual (English, Chinese) dataset of approximately 31,000 complex multi-constraint instructions. Extensive analyses validate the design of STAIF and show state-of-the-art performance on representative benchmarks against strong baselines, as well as genuine generalization.

[179] How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift cs.AI | cs.CLPDF

James Elcock, William F. Shen, Xinchi Qiu, Nicholas D. Lane

TL;DR: 本文系统评估了三种主流任务适应方法（SFT、KL-SFT、RLVR）对LLM在安全、事实性、立场稳定性、社会危害、可控性和指令遵循等六个领域15个对齐维度的影响。研究发现，任务适应并非均匀改变对齐属性：RLVR在提升任务性能的同时引发较小但非零的对齐漂移，SFT导致显著更大的跨领域对齐漂移，而KL正则化能通过锚定参考模型缓解该现象。表征分析进一步验证了行为漂移与对齐相关表征变化的关联。

Details

Motivation: 现有研究指出任务适应可能破坏LLM的预训练对齐（尤其是安全性），但对其在多维度对齐领域的系统性影响缺乏深入理解。本文旨在填补这一空白，探究不同任务适应方法如何重塑模型的多维对齐属性。

Result: 在涵盖六个关键领域的15个对齐维度评估中，RLVR在保持任务性能提升的同时引发的对齐漂移最小（但仍非零），SFT导致的对齐漂移最大，KL-SFT通过增强参考模型锚定减轻了漂移但效果仍不及RLVR。表征分析显示对齐相关表征的变化与行为漂移模式一致。

Insight: 论文创新性地将任务适应重新定义为一种对齐干预行为，而非单纯的能力提升步骤，并提出了多维对齐评估应作为后训练流程标准组件的观点。方法上通过系统性的多维度行为与表征分析，揭示了不同适应方法对对齐属性差异化影响的内在机制，为设计更安全的模型适应方案提供了实证依据。

Abstract: Post-training is a key mechanism for adapting large language models to downstream tasks. While prior work suggests that task adaptation can alter a model’s pre-existing alignment, especially its safety behavior, its broader effects across alignment domains remain poorly understood. We address this gap through a systematic evaluation of representative task-adaptation methods, including supervised fine-tuning (SFT), KL-regularized SFT, and reinforcement learning with verifiable rewards (RLVR) across 15 alignment aspects spanning six key domains: safety, factuality, stance stability, social harm, controllability, and instructability. Our results reveal that post-training does not reshape alignment uniformly. RLVR improves task performance while inducing comparatively small, but non-zero, metric-specific shifts, while SFT leads to substantially larger alignment drift across domains. KL regularization mitigates this effect: stronger reference-model anchoring reduces alignment drift from the baseline, although KL-SFT still falls short of RLVR in preserving alignment. Representation-level analysis further supports this pattern, with shifts in alignment-relevant representations tracking behavioral drift. Together, these results show that task adaptation is not merely a capability-improving step, but an alignment intervention in its own right, motivating multi-dimensional alignment evaluation as a standard component of post-training pipelines.

[180] ConsistencyGate: Preventing Memory Contamination in LLM Agents via Self-Consistency Admission Control cs.AI | cs.CLPDF

Yan Zhang, Shibo Li

TL;DR: 本文提出ConsistencyGate方法，用于防止LLM智能体在多轮交互中的记忆污染问题。该方法通过在写入外部记忆前对候选事实进行多次查询验证，仅当平均支持分数超过阈值时才允许写入，从而减少幻觉事实的持久性影响。

Details

Motivation: 解决LLM智能体在长期操作中因幻觉事实写入外部记忆而导致的记忆污染问题，现有方法未能在写入时确保正确性。

Result: 在四个LLM骨干模型上，ConsistencyGate在LoCoMo-Contam、MSC-Contam和MemContam三个基准测试中均降低了污染率，相比全写入基线有显著改进。

Insight: 创新点在于提出了一种无需微调、模型无关的写入时准入控制机制，通过自一致性验证来过滤候选事实，特别针对隐式陈述的事实进行成本集中处理。

Abstract: LLM agents that operate over many turns accumulate facts in an external memory store and reuse them as premises for downstream reasoning. A hallucinated fact written at one step therefore persists as a false premise for every subsequent step, a failure mode we call memory contamination. Existing memory management addresses retrieval and capacity but not write-time correctness; this admission problem cannot be solved by utility- or recency-based criteria, and uncontrolled contamination compounds across long trajectories. We propose ConsistencyGate, a write-time admission gate that, before committing a candidate fact m extracted from context c, queries the LLM K times for a soft support score and admits m only when the average exceeds a threshold. The mechanism is model-agnostic, requires no fine-tuning, and reduces to a single forward pass in a log-probability variant for latency-sensitive deployments. To measure the effect on natural data, we construct two real-conversation benchmarks (LoCoMo-Contam and MSC-Contam) by planting controlled single-detail corruptions in long-term conversations from LoCoMo and MSC, and complement them with a structured synthetic corpus (MemContam) that isolates a near-oracle upper bound. Across four LLM backbones, ConsistencyGate reduces contamination on every benchmark relative to a write-everything baseline, with the cost concentrated on facts that are stated only implicitly in the source context. We release all three benchmarks together with the gate implementation.

[181] AgentOmnia: Scaling Agentic Models for Full-Scenario Applications cs.AI | cs.CLPDF

Hao Jiang, Gangtao Xin, Yingdi Huang, Guojie Zhu, Jiangshan Zhang

TL;DR: 本文提出了AgentOmnia框架，旨在解决智能体模型在跨领域、能力、任务难度和交互场景中的碎片化进展问题。该框架通过协调任务空间定义、数据合成、后训练、评估和改进流程，支持ToC、ToB和ToE全场景应用。基于可扩展的领域×能力×原子难度分类法，结合双向环境-任务合成与多种流水线，构建了大规模状态化环境与任务集，并通过后训练和基于评估失败的自进化机制显著提升了模型性能。

Details

Motivation: 当前大语言模型智能体的发展在领域、能力、任务难度和交互设置上呈现碎片化，缺乏统一的全场景扩展框架。本文旨在通过一个协调的框架（AgentOmnia）来系统化地定义、合成、训练、评估和改进智能体，以应对消费者、商业和员工应用中的多样化需求。

Result: 在OmniaBench挑战性子集上，模型通过率从9.16%提升至37.11%；在OmniaBench、τ²-Bench、DeepPlanning和VitaBench四个基准上的宏平均从22.86%提升至41.69%。在统一协议下，AgentOmnia在OmniaBench上领先于其他后训练基线，并保持了最高的四基准宏平均成绩，甚至超越了更大规模的Qwen3-235B-A22B-Thinking-2507和Qwen3.5-35B-A3B模型。改进广泛覆盖了三个应用划分、十个能力维度、八个原子难度因子和90个一级领域中的76个。

Insight: 创新点包括：1) 提出了全场景智能体扩展的统一框架，协调了从任务定义到自进化的全流程；2) 引入了可扩展的领域×能力×原子难度分类法（OmniaBench），支持细粒度诊断；3) 结合了双向环境-任务合成、工具依赖、程序结构和基于求解器的多种数据合成流水线；4) 利用后训练（监督微调、在线智能体强化学习、回滚课程学习）和基于评估失败生成产品需求文档（PRD）的针对性自进化机制，实现了性能的广泛提升。

Abstract: Large language model agents have advanced rapidly, yet progress remains fragmented across domains, capabilities, task difficulty, and interaction settings. We frame this as full-scenario agentic scaling and present AgentOmnia, a framework coordinating task-space definition, data synthesis, post-training, evaluation, and improvement across To-Consumer (ToC), To-Business (ToB), and To-Employee (ToE) applications. An extensible Domain x Capability x Atomic Difficulty taxonomy aligns these stages and enables fine-grained diagnosis with OmniaBench. AgentOmnia combines bidirectional environment-task synthesis with tool-dependency, program-structured, and solver-based pipelines, constructing 5,018 stateful environments with 255,375 tools and 52,361 tasks. Programs, solvers, and verifiers provide correctness signals, while supervised fine-tuning, online agentic reinforcement learning, and a rollback curriculum support post-training. Evaluation failures translate into Product Requirement Documents (PRDs) for targeted self-evolution. Starting from Qwen3-30B-A3B-Thinking-2507, AgentOmnia raises the pass rate on the OmniaBench challenging subset from 9.16% to 37.11% and the macro-average across OmniaBench, $τ^2$-Bench, DeepPlanning, and VitaBench from 22.86% to 41.69%. Under a unified protocol,it leads the evaluated agentic post-trained baselines on OmniaBench and retains the highest four-benchmark macro-average. It also surpasses Qwen3-235B-A22B-Thinking-2507 on all four benchmarks and exceeds Qwen3.5-35B-A3B on the macro-average. Gains span three application splits, ten capability dimensions, eight atomic-difficulty factors, and 76 of 90 level-1 domains, indicating broad rather than category-specific improvement. A one-round study provides initial evidence for PRD-guided self-evolution, motivating validation at larger scales and in industrial settings.

[182] Concept-based Visual Counterfactual Explanations with Diffusion Models cs.AI | cs.CVPDF

Yassine Oueslati, Daniil Kirilenko, Martin Gjoreski, Marc Langheinrich

TL;DR: 本文提出了C-VCE，一种基于扩散模型的概念视觉反事实解释框架。它通过将概念瓶颈层直接集成到生成模型中，利用人类可解释的概念而非外部噪声鲁棒分类器来指导反事实图像的生成，从而在保持图像真实性的同时，最小化并控制对相关区域的编辑，以翻转模型的预测。

Details

Motivation: 现有基于扩散模型的反事实解释方法依赖外部分类器处理含噪图像，这导致方法脆弱且难以部署。本文旨在解决这一问题，通过构建一个内置分类器的生成模型，直接利用可解释概念来生成更可靠、更可控的反事实解释。

Result: 在CelebA等基准测试中，C-VCE在翻转率上达到或优于基线方法，同时生成的反事实图像在视觉上更接近原始输入，失真更少，优于依赖外部噪声图像分类器的基线方法。

Insight: 主要创新点是将概念瓶颈层直接集成到扩散模型中，通过可解释概念指导生成过程，并结合概率正则化和基于梯度的掩码来控制编辑范围。这为构建更易理解、更安全使用的生成模型提供了一种有前景的途径，即通过暴露和控制内部概念层来实现。

Abstract: Visual counterfactual explanations aim to answer “what minimal change to this image would flip the model’s prediction?”, and are increasingly important as vision models are deployed in safety-critical domains (e.g., medicine). Existing diffusion-based methods can produce realistic edits, but they rely on external classifiers that must work reliably on noisy images, which makes them fragile and hard to deploy for robust explanations. We introduce C-VCE, a new diffusion framework that builds the classifier directly into the generative model via a concept bottleneck layer, so that counterfactuals are guided by human-interpretable features (concepts) instead of a separate noise robust classifier that works with pixel-level edits. Our model lets users to toggle on/off semantic concepts during sampling, then minimally adjusts relevant image regions, while preserving the rest of the image, respecting feature correlations. To keep edits small and controlled, we add a simple probabilistic regularizer that balances “change the prediction” against “stay close to the original”, plus a gradient-based mask that confines modifications to the most relevant regions. On benchmarks such as CelebA, C-VCE matches or improves flip rates while producing counterfactuals that are visually closer to the input and less distorted than baselines that depend on separate noisy-image classifiers. These properties make C-VCE a practical tool for vision systems where users need concrete “what-if” images without having to trust an additional, noise-robust classifier. More broadly, our results suggest that exposing and controlling an internal concept layer is a promising way to make powerful generative models easier to understand and safer to use.

Tianyu Yang, Shir Simon, Zhenzhen Li, Minhao Cheng, Xiangliang Zhang

TL;DR: 本文提出MM-R2，一种多模态代理检索框架，通过先推理后检索的方式解决多模态检索增强生成（mRAG）中的两大挑战：检索目标不明确和搜索空间弱结构化。该方法首先从图像-问题对构建意图接地的检索状态，然后在结构化的知识图谱上进行检索，代理先选择相关检索单元再在其中执行查询。

Details

Motivation: 现有mRAG系统直接从原始多模态输入在扁平证据空间中检索，面临检索目标不明确（问题意图需与正确视觉指称物对齐）和搜索空间弱结构化（语义不同的证据在单一全局排序中竞争）两大挑战。

Result: 在Infoseek和Encyclopedic VQA数据集上的实验表明，MM-R2在答案准确率上显著优于强基线，同时产生更可解释和可验证的检索轨迹。

Insight: 创新点在于提出’先推理后检索’的代理规划范式，通过显式建模’检索什么’和’在哪里搜索’来构建意图接地的检索状态，并利用结构化的知识图谱进行分层检索。此外，构建大规模多步检索轨迹数据集MM-R2-Traj并采用两阶段后训练策略（监督微调和GRPO）来赋能该能力。

Abstract: Multimodal retrieval-augmented generation (mRAG) aims to answer image-text queries with external knowledge, but most existing systems still retrieve directly from raw multimodal input over a flat evidence space. This design often struggles with two key challenges: the retrieval target is under-specified because the question intent must be grounded to the correct visual referent, and the search space is weakly structured, forcing semantically distinct evidence to compete in a single global ranking step. We propose MM-R2, a multimodal agentic retrieval framework that reasons before retrieval by explicitly modeling both what to retrieve and where to search. MM-R2 first constructs an intent-grounded retrieval state from the image-question pair, capturing the information need, grounded referent, and retrieval constraints. It then performs retrieval over a structured KnowledgeMap, where the agent selects relevant retrieval units before issuing grounded queries within them. To enable this capability, we build MM-R2-Traj, a large-scale trajectory dataset of multi-step retrieval processes, and adopt a two-stage post-training strategy with supervised fine-tuning and GRPO. Experiments on Infoseek and Encyclopedic VQA datasets show that MM-R2 substantially outperforms strong baselines on answer accuracy while also yielding more interpretable and verifiable retrieval trajectories.

[184] DocHRL: A Hierarchical Reinforcement Learning Framework for Cost-Optimised Document Classification cs.AI | cs.CV | cs.LGPDF

Mohammed Yousif, Prabhjot Singh, Arjun Pankajakshan, Madhu Reddiboina

TL;DR: 本文提出了DocHRL，一种用于文档分类的分层强化学习框架。该框架能够根据每个文档的复杂程度，动态地选择最具成本效益的分类策略（如视觉分类器、LLM、OCR或人工审核），从而优化计算和人力资源的使用。

Details

Motivation: 现实世界的文档分类流程通常对所有文档应用相同的模型序列，这导致简单文档被过度处理，而复杂文档可能得不到充分审查，造成资源使用效率低下。本文旨在解决这种计算和人力资源的浪费问题。

Result: 在RVL-CDIP基准测试上，使用近端策略优化进行训练后，DocHRL在16个文档类别上实现了0.973的宏观F1分数，同时将每个文档的平均成本降低至2.74个归一化单位，远低于固定独立分类器的成本。

Insight: 主要创新点在于将文档分类建模为一个具有两层策略层次结构的顺序决策问题，并通过负的总期望成本作为奖励信号，统一优化推理成本、误分类成本和人工标注成本。这为构建成本感知的、自适应的文档理解系统提供了新思路。

Abstract: Real-world document classification pipelines typically apply the same sequence of models to every incoming document, regardless of its complexity or type. This leads to inefficient use of compute and human resources: simple documents are over-processed while difficult ones may not receive enough scrutiny. We introduce DocHRL, a hierarchical reinforcement learning framework that learns to adaptively and dynamically select the most cost-effective classification policy on a per-document basis. DocHRL formulates document classification as a sequential decision problem with a two-level policy hierarchy: a top-level policy selects among broad options (vision classifiers, LLMs, OCR, and human-in-the-loop review), while option-specific sub-policies choose the concrete model or tool to invoke. The reward signal is the negative total expected cost, which captures inference cost, cost of misclassification, and cost of human labelling. Trained with Proximal Policy Optimisation on the RVL-CDIP benchmark, DocHRL achieves a macro F1 of 0.973 across 16 document classes while reducing average per-document cost to 2.74 normalised units compared to substantially higher costs incurred by fixed standalone classifiers. Our results demonstrate that cost-aware reinforcement learning can simultaneously improve classification performance and operational efficiency in document understanding systems.

[185] CachedSearch: Training-Free Cached Exploration for Test-Time Search in Video Diffusion cs.AI | cs.CV | cs.MMPDF

Shreshth Saini, Neil Birkbeck, Yilin Wang, Balu Adsumilli, Alan C. Bovik

TL;DR: 本文提出了一种名为CachedSearch的训练免费缓存探索方法，用于视频扩散模型的测试时搜索。该方法通过使用有损缓存来加速候选视频的生成过程，在几乎无损质量的情况下使每次生成速度提升2-3倍，并验证了缓存不会显著破坏候选视频的排名，从而在降低计算成本的同时保留了大部分最佳候选搜索的收益。

Details

Motivation: 测试时搜索能让小型视频扩散模型媲美大型模型，但计算成本高出2-10倍，且所有候选视频都需要完全去噪，尽管大多数最终被丢弃。因此，需要一种训练免费的方法来减少计算开销，同时保持搜索效果。

Result: 在Wan2.1-T2V-1.3B模型上，使用自适应缓存包装器（每候选加速约2倍），ImageReward评分显示缓存生成与完全生成的候选视频排名高度相关（中位数Spearman等级相关系数为0.905），在VBench套件上top-1一致率达到72%。在N=8时，CachedSearch以63%的成本捕获了best-of-N收益的94.7%，且随着搜索宽度增加，捕获率上升。该方法在1.3B至14B的六个模型和四个家族（Wan、LTX、CogVideoX、Hunyuan）上均有效，其中Wan2.1-14B模型匹配了1.3B模型的保真度。

Insight: 创新点在于首次研究了缓存是否破坏视频测试时搜索中的候选排名，发现错误主要集中在排名相近的候选之间，使得破坏自限。CachedSearch通过激进缓存探索所有候选，然后仅完全重新生成获胜者，实现了计算效率提升。该方法训练免费、验证器无关且与搜索算法正交，可作为测试时扩展的即插即用倍增器。

Abstract: Test-time search lets small video diffusion models rival larger ones, but costs 2-10x more. All candidates are fully denoised, although most are discarded. Training-free caching makes each rollout 2-3x faster at near-lossless quality. Composition is safe only if lossy caching preserves verifier rankings. We present the first study of whether caching corrupts candidate ranking in video test-time search. On Wan2.1-T2V-1.3B with an adaptive caching wrapper (~2x per-candidate speedup), ImageReward scores seed-matched cached and full rollouts. Median per-prompt Spearman rank correlation is 0.905, with 72% top-1 agreement on the VBench suite. VBench-2.0 replicates this result on a harder suite. Recomputing the cached winner at full compute retains 90-94% of the full-search gain. Errors cluster among near-tied candidates, making corruption self-limiting. This finding leads to CachedSearch. It explores every candidate with aggressive caching, then re-generates only the winner at full compute. At N=8, it captures 94.7% of best-of-N’s gain at 63% of the cost. Capture rises with width. At matched budget, it searches twice as wide for 38% more gain. The result holds from 1.3B-14B across six models and four families: Wan, LTX, CogVideoX, and Hunyuan. Wan2.1-14B matches the 1.3B model’s fidelity. Mid-trajectory pruning multiplies the exploration saving to 3.11x at 88.6% capture. Ports to other model families require recalibrating a single parameter, showing that fidelity tracks architecture rather than parameter count. CachedSearch is training-free, verifier-agnostic, and orthogonal to the search algorithm, making it a plug-in multiplier for test-time scaling.

[186] ERUnderstand: Evaluating Vision-Language Models on Structured ER Diagrams cs.AI | cs.CV | cs.DBPDF

Ali Ansari, Yasmin Mohammadi, Farnoush Nili, Parsa Esmaeilkhani, Longin Jan Latecki

TL;DR: 论文提出了首个用于评估视觉语言模型对结构化实体关系图理解能力的大规模基准ERUnderstand，包含来自教育、真实世界和合成来源的2960个ER图，并提供了标准化的机器可读表示用于细粒度评估。研究发现，现有先进模型对常见ER图元素识别较好，但对弱实体、多值属性和N元关系等复杂结构识别性能显著下降，而推理增强模型虽有提升但仍受语言先验和复杂度影响。

Details

Motivation: 实体关系图是概念数据库设计的核心，但通常仅以渲染图像形式存在，缺乏机器可读模式，这限制了AI辅助数据库工程的发展，因此需要建立评估模型理解ER图能力的基准。

Result: 在ERUnderstand基准上评估，先进视觉语言模型对常见ERD元素恢复可靠（F1 > 0.74），但对弱实体（低至0.28 F1）、多值属性（0.14 F1）和N元关系（0.07 F1）性能骤降；推理增强模型整体性能提升15-25%，但仍对语言先验和图表复杂度敏感。

Insight: 创新点在于构建了首个大规模、多来源、覆盖多样领域和复杂度的ER图理解基准，并揭示了当前模型在复杂ER结构理解上的显著短板；客观来看，该工作为多模态数据库模式理解提供了标准化评估框架和细粒度分析工具。

Abstract: Entity-Relationship Diagrams (ERDs) are central to conceptual database design, yet they are typically available only as rendered images rather than machine-readable schemas, limiting AI-assisted database engineering. We introduce ERUnderstand, the first large-scale benchmark for structured understanding of ER diagrams, comprising 2,960 diagrams collected from curated educational sources, real-world schemas, and synthetically generated examples spanning diverse domains, notations, complexity levels, and Extended Entity-Relationship (EER) constructs. Each diagram is paired with a standardized machine-readable representation for fine-grained evaluation of schema elements. Evaluating state-of-the-art Vision-Language Models (VLMs), we find that while common ERD elements are recovered reliably (F1 > 0.74), performance drops sharply on weak entities (as low as 0.28 F1), multivalued attributes (0.14 F1), and N-ary relationships (0.07 F1). Reasoning-augmented models improve overall performance by 15-25% but remain sensitive to linguistic priors and increasing diagram complexity. ERUnderstand provides a standardized benchmark for evaluating multimodal understanding of conceptual database schemas. The benchmark, dataset, evaluation toolkit, and generation code are publicly available at https://github.com/salinaria/ERUnderstand.

eess.IV [Back]

[187] Generative Video Compression with Adaptive Score Distillation eess.IV | cs.CVPDF

Naifu Xue, Zhaoyang Jia, Haosen Li, Zihan Zheng, Jiahao Li

TL;DR: 本文提出了首个专为压缩设计的生成式视频编解码器GenVC，它基于从头训练的像素空间视频扩散模型，并采用全局到局部层次结构恢复时空细节。为加速推理，论文提出自适应分数蒸馏方法，解决了直接应用分布匹配蒸馏导致运动停滞的问题，实现了高质量的单步生成重建。

Details

Motivation: 现有基于扩散模型的视频编解码器通常改编自文本条件生成的基础模型，缺乏专门为压缩任务设计和训练的扩散模型，这限制了其在超低码率下的性能。

Result: 在超低码率下，GenVC在感知质量上达到了最先进水平，与GLVC相比，在匹配LPIPS和FID时平均码率分别节省了62.5%和71.3%。模型参数量仅为4.78亿，在A100 GPU上能以单步15.1 fps的速度解码1080p视频。

Insight: 创新点在于首次构建了面向压缩的视频扩散模型，并提出了自适应分数蒸馏来解决教师模型引导失效导致的运动停滞问题，实现了高效的单步高质量生成重建，避免了依赖大规模预训练骨干网络。

Abstract: Diffusion models provide strong generative capabilities for video compression at ultra-low bitrates. Existing diffusion-based video codecs adapt base models originally developed for text-conditioned generation, whereas diffusion models designed and trained specifically for compression remain unexplored. To fill this gap, we introduce our Generative Video Codec (GenVC), built on a video diffusion model trained from scratch for compression. To our knowledge, this is the first compression-oriented video diffusion model. We realize this model directly in pixel space with a global-to-local hierarchy that recovers fine spatio-temporal details, enabling high-quality generative reconstruction from compressed representations. To accelerate inference, we distill the multi-step model into one step using distribution matching distillation (DMD). Applying DMD directly, however, drives the student toward motion-stalled reconstructions. We trace this to a teacher-side guidance failure: once student-induced perturbations leave the frozen teacher’s training region, its guidance can become misleading, causing DMD updates to reinforce rather than correct the student drift. To break the resulting feedback loop, we propose Adaptive Score Distillation, which gates DMD updates according to their alignment with the ground-truth direction, enabling high-quality reconstruction with coherent motion. Experimental results show that GenVC achieves state-of-the-art perceptual quality at ultra-low bitrates, with average bitrate savings of 62.5% at matched LPIPS and 71.3% at matched FID over GLVC. Unlike prior codecs that inherit billion-scale pretrained backbones, our diffusion model has only 478.0M parameters and decodes 1080p video in a single step at 15.1 fps on an A100 GPU.

[188] Metric Surface Reconstruction of Neurosurgical Scenes from Monocular Operating Microscope Images and Microscope Pose eess.IV | cs.CV | physics.med-phPDF

Thomas Bucher, Didier Neuenschwander, Thomas Petutschnigg, Michael Murek, David Bervini

TL;DR: 本研究评估了结合标准单目手术显微镜图像与显微镜位姿数据，重建神经外科手术暴露区域三维几何形状的可行性。通过使用预训练的Depth Anything 3模型进行深度估计，并结合泊松表面重建生成网格，在体模实验中实现了毫米级的重建精度。

Details

Motivation: 旨在探索是否可以利用常规的单目手术显微镜图像和导航系统提供的位姿数据，客观、定量地重建神经外科手术暴露区域的真实三维几何结构，以支持手术暴露量化、图像融合和未来手术器械工作空间表征。

Result: 在体模实验中，对于模拟较深手术通道的体模A，重建精度为1.95±1.70 mm至2.33±2.15 mm；对于模拟直接暴露表面的体模B，精度为1.02±0.93 mm至1.52±1.21 mm。重建结果与结构光扫描和薄层CT的参考表面进行了比较。

Insight: 创新点在于提出了一种基于基础模型（Depth Anything 3）且无需任务特定微调的三维重建流程，将标准手术显微镜的单目视频输出与导航位姿数据相结合，证明了在受控环境下实现毫米级精度三维表面重建的技术可行性，为手术场景的客观量化提供了新思路。

Abstract: Objective: We evaluated whether metric 3D geometry of neurosurgical operative exposure can be recovered from standard monocular operating-microscope images combined with microscope pose data. Methods: In a phantom-based laboratory study, two aneurysm training phantoms were imaged with a ZEISS Pentero 800 microscope integrated with Brainlab Cranial Navigation. Microscope images from the standard composite video output were stored with synchronous microscope poses. After intrinsic and extrinsic calibration, depth was estimated with the pretrained Depth Anything 3 model without task-specific fine-tuning. Fused point clouds were converted to meshes using Poisson surface reconstruction. Reconstructions were compared with reference surfaces from structured-light scanning and fine-slice CT. Results: For phantom A, representing a deeper surgical corridor, reconstruction accuracy ranged from 1.95 $\pm$ 1.70 mm to 2.33 $\pm$ 2.15 mm. For phantom B, representing a directly exposed surface, accuracy ranged from 1.02 $\pm$ 0.93 mm to 1.52 $\pm$ 1.21 mm. Larger image sets mainly improved completeness, while accuracy remained within a narrower range. Corridor analysis showed preservation of overall geometry with local deviations in incompletely reconstructed regions. Conclusions: Standard monocular microscope images combined with navigation-derived pose data can reconstruct millimeter-range 3D surfaces using a foundation-model-based pipeline. These results show technical feasibility in a controlled phantom setting and support further development toward objective quantification of operative exposure, image fusion, and characterization of working spaces for future surgical instrumentation.

[189] JPEG AIC2026: A large-scale dataset for fine-grained assessment of image coding eess.IV | cs.CVPDF

Mohsen Jenadeleh, Jon Sneyers, João Ascenso, Thomas Richter, Alexander Karabutov

TL;DR: 本文介绍了AIC2026数据集，这是一个用于高保真图像压缩评估的大规模数据集。该数据集包含70张源图像，通过语义聚类、客观图像质量评估（IQA）方法间的度量不一致性以及人工检查筛选而来，并覆盖了8种传统和4种基于学习的编解码器在17种配置下产生的压缩伪影。每张源图像被7种编解码器编码，并在20个感知均匀的失真级别上提供解码图像，总计9,618张失真图像，以支持对细微质量差异的分析。

Details

Motivation: 随着传统和基于学习的图像编码技术的进步，业界需要能够支持对压缩图像质量进行细粒度评估的基准数据集，特别是针对基于学习的图像压缩方法。

Result: 论文使用24种传统和12种基于学习的IQA方法进行了广泛的客观分析。结果表明，当前IQA方法在评估细粒度质量差异（尤其是基于学习编解码器引入的伪影）时存在显著分歧。

Insight: 创新点在于构建了一个大规模、细粒度的图像压缩评估数据集，通过语义聚类和IQA度量不一致性筛选源图像，并采用感知均匀的失真级别（基于CVVDP度量以JND单位定义）进行采样，这有助于更精确地分析率失真行为和评估客观度量指标在细微质量差异上的表现。

Abstract: Recent advances in conventional and learning-based image coding have increased the demand for benchmark datasets that support fine-grained assessment of compressed image quality, particularly for learning-based image compression methods. This paper introduces Assessment of Image Coding 2026 (AIC2026), a large-scale dataset for high-fidelity image compression containing 70 source images selected from 2,787 candidates using semantic clustering, inter-metric disagreement among objective image quality assessment (IQA) methods, and manual inspection and refinement. The dataset covers a wide range of compression artifacts produced by eight conventional and four learning-based codecs across 17 coding configurations. Each source image is encoded using seven codecs. For each source-codec pair, decoded images are provided at 20 perceptually spaced distortion levels, corresponding approximately to 0.2-4.0 just-noticeable difference (JND) units using the ColorVideoVDP (CVVDP) metric for distortion estimation, yielding 9,618 distorted images. This fine-grained sampling enables analysis of rate-distortion behavior and objective metric evaluation for subtle quality differences across a wide range of compression artifacts. We report an extensive objective analysis using 24 conventional and 12 learning-based IQA methods. The results show substantial disagreement among current IQA methods for fine-grained quality differences, particularly for artifacts introduced by learning-based codecs. The complete dataset is publicly available at https://doi.org/10.18419/DARUS-6156.

[190] Codebook Capacity Governs Perceptual Quality Across Resolutions in Hierarchical Discrete Video Compression eess.IV | cs.CV | cs.MMPDF

Manikanta Kotthapalli, Banafsheh Rekabdar

TL;DR: 本文研究了基于分层离散潜在表示的学得视频编解码器在不同空间分辨率下的性能表现。通过控制实验，发现感知质量（LPIPS）主要受码本容量影响，而几乎不受空间分辨率影响。在所有测试分辨率下，所提模型在相同或更低码率下均优于H.264和H.265。

Details

Motivation: 基于连续潜在表示的学得视频编解码器在扩展到新空间分辨率时，通常需要针对特定分辨率重新训练或调整率失真权衡，因为熵模型和拉格朗日权重与操作点紧密耦合。本文旨在探究分层离散潜在编解码器是否表现出同样的敏感性。

Result: 在UCF101数据集上，对码本大小K∈{128,256,512,1024}和分辨率64×64、128×128、256×256进行了实验。感知质量（LPIPS）强烈依赖于码本容量，而几乎不依赖于空间分辨率。在所有分辨率和码本大小下，模型在相同或更低码率下均优于H.264（在128×128分辨率下提升25-52%）和H.265（在256×256分辨率下提升21-37%）。

Insight: 核心创新点在于揭示了在分层离散视频编解码器中，码本容量（而非空间分辨率）是主导感知压缩质量的设计变量。这一特性可能简化多分辨率部署，并为生成式视频模型的可扩展离散分词器设计提供参考。同时，底层熵效率随分辨率提高而保持稳定或改善，表明更大的空间网格被更有效地利用。

Abstract: Learned video codecs based on continuous latent representations typically require resolution-specific retraining or rate-distortion (RD) recalibration when scaling to new spatial resolutions, because entropy models and Lagrangian weights are tightly coupled to the operating point. We investigate whether hierarchical discrete latent codecs exhibit the same sensitivity. Using a controlled empirical study of MS-VQ-VAE video compression across codebook sizes $K \in {128,256,512,1024}$ and resolutions $64\times64$, $128\times128$, and $256\times256$ on UCF101, we show that perceptual quality (LPIPS) depends strongly on codebook capacity but only negligibly on spatial resolution. Fitting a log-linear model $Q(K,r) = α\log_2 K + β\log_2 r + γ$ to all 12 operating points yields $α=-0.0094$ ($t=-6.6$, $p<0.001$) and $β=-0.0009$ ($t=-0.43$, $p=0.68$, not significant), with $R^2=0.82$. Codebook capacity is therefore roughly $10\times$ more influential than spatial resolution per log-unit increase. In parallel, bottom-level entropy efficiency $η=H(z)/\log_2 K$ remains stable or improves with resolution (84-87% at $64\times64$; 92-94% at $256\times256$), confirming that larger spatial grids are utilized more efficiently rather than less. Across all resolutions and codebook sizes, our models outperform H.264 on LPIPS at matched or lower bitrate, with gains of 25-52% at $128\times128$ and 21-37% over H.265 at $256\times256$. These findings suggest that codebook size $K$, not spatial resolution, is the dominant design variable governing perceptual compression quality in hierarchical discrete video codecs – a property that may simplify multi-resolution deployment and inform the design of scalable discrete tokenizers for generative video models.

[191] A Controlled Visual-Backbone Benchmark for Multimodal Short-Term Solar Irradiance Forecasting eess.IV | cs.CVPDF

Oshadha Samarakoon, Dushan Herath, Ishara Ranmandala, Dilshara Herath, Roshan Godaliyadda

TL;DR: 本文提出了一种受控的视觉骨干网络基准测试方法，用于多模态短期太阳辐照度预测。研究固定了除视觉骨干外的整个预测流程，比较了ConvNeXt、Swin Transformer、VMamba等多种视觉骨干在Folsom和NREL数据集上对未来10分钟辐照度的预测性能。结果表明，在Folsom数据集上，所有视觉骨干均优于智能持续性基准，其中VMamba Small和Swin Base表现最佳；但在NREL数据集上，智能持续性基准仍是最强的。

Details

Motivation: 现有天空图像辐照度研究通常同时改变多个系统组件（如图像编码器、时序模型等），导致难以公平比较不同视觉骨干的性能。本文旨在通过固定除视觉骨干外的所有其他模块，建立一个受控的基准测试环境，以更清晰地评估和比较不同视觉架构在预测任务中的效果。

Result: 在Folsom严格划分数据集上，所有评估的视觉骨干均优于智能持续性基准（clear-sky-index smart persistence），其中VMamba Small和Swin Base的RMSE分别为65.39 W/m²和65.50 W/m²，而纯时序诊断模型的RMSE为69.51 W/m²。在NREL严格划分数据集上，智能持续性基准的RMSE为17.48 W/m²，仍是最优的，而最佳视觉骨干Swin Tiny的RMSE为23.76 W/m²。

Insight: 论文的创新点在于设计了一个高度受控的基准测试协议，通过固定多模态预测流程中的其他所有组件，仅改变视觉骨干，从而实现了对视觉架构性能的公平比较。从客观角度看，这种方法为评估视觉骨干在特定任务（如太阳辐照度预测）中的有效性提供了一个可复现和标准化的框架，有助于未来研究更准确地识别架构优势，而非仅追求整体性能优化。

Abstract: Sky-image irradiance studies often compare forecasting systems in which the image encoder, temporal model, fusion block, target definition, and training recipe all change together. We use a narrower protocol: the multimodal forecasting pipeline is fixed, and only the visual backbone is varied. The shared setup keeps preprocessing, clear-sky-index normalization, weather-history encoding, fusion, regression head, loss, optimizer schedule, seed, and chronological split policy unchanged. We compare ConvNeXt, Swin Transformer, VMamba, Spatial Mamba, and MambaVision backbones for 10min-ahead forecasting on Folsom and a strict matched NREL split. Forecast skill is measured against clear-sky-index smart persistence, and temporal-only rows are reported as weather-history diagnostics rather than as the main ranking criterion. On the Folsom strict split, all evaluated visual-backbone runs improve over smart persistence. In the evaluated single-seed strict runs, VMamba Small and Swin Base reach matched Folsom RMSE values of 65.39 W/m^2 and 65.50 W/m^2; the temporal-only diagnostic reaches 69.51 W/m^2. On the 313-sample NREL strict split, smart persistence remains strongest at 17.48 W/m^2, while the lowest visual RMSE is obtained by Swin Tiny at 23.76 W/m^2. These results provide a reproducible encoder comparison under one fixed multimodal operating point rather than establishing architecture-level dominance, statistically resolved ranking, or fully optimized forecasting performance. Code available here: https://github.com/Oshadha345/irradiance_benchmark

cs.SE [Back]

[192] AssumptionMiner: Extracting, Tracing, and Revising Implicit Assumptions in LLM Code Generation cs.SE | cs.CLPDF

Jie “JW” Wu

TL;DR: 本文提出了AssumptionMiner框架，旨在解决LLM代码生成中因自然语言提示不完整而产生的隐式假设问题。该框架通过生成一个结构化的显式假设层，使开发者能够检查、确认或修改这些假设，并基于AST依赖图实现针对性的代码再生。

Details

Motivation: 解决LLM代码生成中，由于真实世界提示不完整，模型会填充隐式假设，导致生成的代码可能满足测试但违背开发者意图的问题。

Result: 在包含676个标注假设的180个模糊编程任务基准上，置信度加权集成方法在假设提取上达到0.816的F1分数，比最强离线基线提升3.6倍；在人工验证的定位基准上，AST引导的定位比基于关键词和全文件的基线更精确。

Insight: 将隐式假设作为代码生成的一等公民进行显式提取和结构化表示，并利用AST依赖图实现假设驱动的精准代码再生，提高了LLM代码生成的透明度和可控性。

Abstract: Large language models (LLMs) generate code from natural-language prompts, yet real-world prompts rarely provide complete specifications. When prompts leave input formats, error handling, or design decisions unspecified, LLMs fill these gaps with implicit assumptions that shape the generated code’s behavior and correctness. Because these assumptions remain hidden, generated code may satisfy tests while violating developer intent. We present AssumptionMiner, a framework that makes implicit assumptions a first-class artifact of LLM-based code generation. In addition to code, AssumptionMiner produces an explicit assumption layer, a structured representation of inferred constraints and design decisions that developers can inspect, confirm, or revise. An AST-based dependency graph enables targeted regeneration of only the code affected by a revised assumption. We also introduce a benchmark of 180 ambiguous programming tasks with 676 annotated assumptions, including a human-verified subset for evaluating code localization. We evaluate assumption extraction, code localization, and assumption-guided regeneration. Across open-source LLMs, a confidence-weighted ensemble achieves an F1 score of 0.816 for assumption extraction, improving on the strongest offline baseline by 3.6x. On the human-verified localization benchmark, AST-guided localization identifies more precise code regions than keyword-based and whole-file baselines. During assumption revision, targeted regeneration modifies less code than non-targeted alternatives while exposing challenges in handling cascading edits. These results demonstrate that making assumptions explicit improves the transparency and controllability of LLM-based code generation.

[193] StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agents cs.SE | cs.CVPDF

Yan Yang, Xiangru Jian, Ziyang Luo, Zirui Zhao, Yutong Dai

TL;DR: StateAct是一种以代码为先的多智能体框架，旨在提升计算机使用智能体在长视野任务中的性能。它通过直接访问底层程序状态（如文件、应用后端和DOM）来执行任务，而非依赖有损的屏幕截图像素信息。该框架包含一个主智能体（通过代码操作状态）和一个专用的GUI子智能体（仅在少数需要时处理截图和点击交互），并引入了基于状态的验证机制以确保任务完成质量。

Details

Motivation: 当前计算机使用智能体主要依赖屏幕截图进行感知，但截图仅是底层程序状态的有损渲染，不同状态可能产生相同像素，限制了智能体的准确性和效率。论文旨在通过直接利用程序状态来克服这一瓶颈，将主要挑战从感知转向推理。

Result: 在OSWorld 2.0基准测试中，StateAct将Claude Opus 4.8的二元成功率从20.6%提升至26.9%，部分成功率从54.8%提升至61.6%，且每任务成本比仅基于截图的相同模型低约9倍。仅使用代码而无GUI子智能体的变体部分成功率仅为45.9%，低于基于截图的基线（54.8%）。

Insight: 创新点在于提出了’状态接地’概念，将动作、验证和记忆直接锚定在程序状态上，而非像素。这通过代码优先的多智能体架构实现，主智能体专注状态操作，GUI子智能体按需处理交互，并引入独立验证门检查结构性错误，从而显著降低感知瓶颈，提升长视野任务中的推理效率和可靠性。

Abstract: Computer-use agents are usually improved by strengthening perception: better models for reading a screenshot and choosing where to click. Yet a screenshot is only a lossy rendering of the underlying program state, e.g., the files, application backends, and DOM that hold the task data. Different states can produce the same pixels, while code can inspect and modify that state directly. StateAct is a code-first, multi-agent harness built around this distinction. Its main agent works directly with program state by using code, while a dedicated GUI subagent handles screenshot-and-click interaction on the few subgoals that need it, just 28 of 108 tasks and 1.1% of main-agent steps. The same direct access to program state also supports verification: an independent finish gate double-checks the saved result for structural failures, e.g., output that is missing, unsaved, or written to the wrong path. To stay on track over hundreds of steps, the main agent hands subgoals to fresh subagents, keeping its own context focused. On OSWorld 2.0, StateAct lifts Claude Opus 4.8 from 20.6% to 26.9% on binary success, and from 54.8% to 61.6% on partial success, at ~ 9x lower cost per task than the same model driven by screenshots alone; a code-only variant with no GUI subagent reaches only 45.9% partial, below that screenshot-based baseline’s 54.8%. In general, grounding action, verification, and memory in state, what we call state-grounding, shifts the main bottleneck from perception toward reasoning: failures depend more on what the agent thinks than on what it sees.

cond-mat.mtrl-sci [Back]

[194] Harnessing X-ray Absorption Spectroscopy Data through Multimodal Mining of Battery Literature cond-mat.mtrl-sci | cs.AI | cs.CL | cs.DL | cs.IRPDF

Tanjin He, Aikaterini Vriza, Logan Ward, Xu Huang, Yiming Chen

TL;DR: 该论文提出了一种多模态（图像和文本）文献挖掘方法，用于从电池研究文献中提取和数字化X射线吸收光谱（XAS）数据。通过开发一个可扩展的数据处理流程，从全文文章中识别XAS图表、数字化光谱曲线，并将其与测量边缘和材料等元数据关联，构建了一个包含13,740条光谱的开放数据集。该数据集为大规模XAS分析、跨实验室比较、高通量表征和先进材料的自主发现提供了基础。

Details

Motivation: XAS是理解材料局部电子和原子结构的关键技术，但大多数已发表的光谱数据因嵌入在文献图表中且文本描述零散，难以用于数据驱动分析。论文旨在解决这些分散知识难以被AI利用的问题。

Result: 应用该流程处理电池文献后，生成了一个包含13,740条XAS光谱的开放数据集，涵盖66种吸收元素和多种电池化学体系。专家验证确认了光谱和元数据信息提取的准确性。

Insight: 创新点在于开发了一个结合图像识别和文本挖掘的多模态文献挖掘流程，将非结构化的文献图表数据转化为结构化、AI可读的实验数据资源，为材料科学领域的数据驱动研究提供了新范式。

Abstract: X-ray absorption spectroscopy (XAS) is central to understanding the local electronic and atomic structure of materials, yet most published spectra remain inaccessible to data-driven analysis because they are embedded in figures and described through fragmented textual context in the literature. Here, we use multimodal (image and text) literature mining to transform this dispersed knowledge into an AI-ready experimental data resource. We developed a scalable spectroscopy data digitization pipeline that identifies XAS figures in full-text articles, digitizes spectral curves, and links each spectrum to accompanying metadata on the measured edge and material. Applying this pipeline to the battery literature produced an open dataset of 13,740 XAS spectra, spanning 66 absorbing elements and diverse battery chemistries, with expert validation confirming accurate extraction of spectral and metadata information. By converting literature-embedded spectra into structured numerical data, this dataset provides a foundation for large-scale XAS analysis, cross-laboratory comparison, high-throughput characterization, and autonomous discovery of advanced materials.

Table of Contents

cs.CL [Back]

[1] PatiGonit22K: A Comprehensive Dataset for Solving Complex Bengali MWPs cs.CLPDF

[2] Learning When to Reason for Text-to-SQL via SFT and DPO cs.CL | cs.AIPDF

[3] Not All LLM Reasoning is Visible in the Chain-of-Thought cs.CL | cs.AI | cs.LGPDF

[4] Toward Automated Detection of Documentation Inconsistencies in Electronic Health Records cs.CL | stat.APPDF

[5] Beyond Direct Answering: Aligning Educational LLMs as Socratic Guides via Heuristic Reinforcement Learning cs.CL | cs.AIPDF

[6] ADAGE: A Language-Agnostic Pipeline for Analogical Reasoning Evaluation cs.CL | cs.AIPDF

[7] Co-Evolving Graph and Text Memory for Training-Free Multi-Hop Question Answering cs.CL | cs.MAPDF

[8] LA-RL: Label-Aware Self-Reflection for Reinforcement Learning in Information Extraction cs.CLPDF

[9] Reasoning or Memorization: Can LLMs Understand and Generate Chinese Xiehouyu Riddles? cs.CL | cs.AIPDF

[10] Do Small Models Use the Law You Give Them? Context-Injected Fine-Tuning for Legal QA in Bangladesh cs.CL | cs.AIPDF

[11] Two Regimes of Chain-of-Thought Unfaithfulness: Behavioral Detection Fails Where Models Are Wrong cs.CLPDF

[12] Do Diagrams Help Large Language Models Reason? Evidence from Syllogistic Reasoning cs.CL | cs.AIPDF

[13] Guiding Language Models to Be More Empathetic: Culturally Sensitive Mental Health Advice Generation Through Human-LLM Collaboration cs.CLPDF

[14] The JEPA Paradox in Language: The Geometry of Linguistic Alternatives cs.CLPDF

[15] Novel Claim or Déjà Vu? Rethinking “Contamination-Free’’ Dynamic Evaluation for Multimodal Automated Fact-Checking cs.CL | cs.AI | cs.MMPDF

[16] A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever cs.CL | cs.AI | cs.IR | cs.LG | cs.PFPDF

[17] Zing: Social Mind for LLMs cs.CLPDF

[18] Indic DiarBench: A Multilingual Joint Diarization and ASR Benchmark for Indian Languages cs.CL | cs.AIPDF

[19] Understanding Tone-Dependent Inference Cost in Large Language Models cs.CL | cs.AIPDF

[20] StanceFlip: A Comprehensive Multi-Dimensional Benchmark for Multimodal Conversational Stance Flipping Forecasting cs.CL | cs.AIPDF

[21] A New Role for Relevance: Guiding Corpus Interaction in Agentic Search cs.CLPDF

[22] CAGE: Cognitive Attribution Graphs for Faithful Inline Citation Generation in Long-Form Question Answering cs.CLPDF

[23] INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models cs.CLPDF

[24] CONSISTRE: A Unified Consistency-Aware Framework for Document-Level Relation Extraction with Large Language Models cs.CLPDF

[25] Closed-Loop Validation-Repair for Healthcare Interoperability: A Multi-Model Study of Schema Compliance in Clinical LLMs cs.CL | cs.AIPDF

[26] Grounding latent algorithm routing in transformer reasoning cs.CLPDF

[27] Looping Is Not Reliability: State-Bound Evidence and Typed Revision Contracts for Agentic Code Repair cs.CL | cs.AIPDF

[28] Kimi K3: Open Frontier Intelligence cs.CL | cs.LGPDF

cs.CV [Back]

[29] DINOv3-MIL: Per-Kidney Multi-Label Tumour and Cyst Detection from Foundation-Model Patch Tokens on KiTS23 cs.CVPDF

[30] FogDrive: A Multi-Modal Synthetic Driving Dataset for Perception under Graded Fog cs.CVPDF

[31] MegaSlide-DiT: Memory-Centric Adaptation and Deformable Local Attention for Efficient Video Diffusion cs.CV | cs.AIPDF

[32] MIME: Multimodal Interactive Motion Encoder cs.CV | cs.LGPDF

[33] Histopathological Spectrum-Guided Prostate Stratification via Segmentation-Assisted Diagnostic Transformer cs.CVPDF

[34] Visible-Light Imaging Diagnosis of Neutral Particle Emission Tomography in the Tokamak Divertor: An Efficient Transformer-based Surrogate Model cs.CV | cs.AI | cs.LGPDF

[35] LowAux-RDNet: Low-Pass Residual Supervision with Scene-Balanced Real-World Training for Single-Image Reflection Removal cs.CVPDF

[36] RMS@CC-MMD 2026: Multimodal Misogyny Detection via Geometric Interaction and Multi-View Consensus cs.CV | cs.AI | cs.CLPDF

[37] StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design cs.CVPDF

[38] Child-Oriented AIGC Video Risk Reviewing: A Benchmark and Knowledge-Supported Iterative Reasoning Framework cs.CV | cs.MMPDF

[39] scMIR: a vision-language foundation model for single-cell light microscopy image representation cs.CV | cs.AI | physics.opticsPDF

[40] Visual Token Compression Enhances Robustness of MLLMs cs.CV | cs.LGPDF

[41] TOM-GS: Editable Video Representation via Temporal Opacity Modulation of Static 3D Gaussians cs.CVPDF

[42] An Interactive Vision Language Platform for Cognitive Remediation in Schizophrenia cs.CV | cs.AIPDF

[43] DAMamba-UNet3D: A Parameter-Efficient Mamba State Space U-Net with Dynamic Adaptive Scan for 3D Medical Image Segmentation cs.CV | cs.AIPDF

[44] Visual Information Extraction from Documents via Classification-Guided Large Vision-Language Models cs.CVPDF

[45] PCA: Persistence-Aware Compression and Aggregation for Fast Video Large Language Models cs.CVPDF

[46] Open Your Model’s Eyes: Video and Context-Aware Multimodal Backchannel Prediction cs.CV | cs.CLPDF

[47] Trustworthy Medical Segmentation: Uncertainty-Aware U-Net Evaluation Under Clinical Image Degradation cs.CV | cs.LGPDF

[48] Calibration-Free 3D Multi-Camera People Tracking for Indoor Environment cs.CVPDF

[49] Benchmarking the Domain Gap: Model Selection Instability Under Domain Shift in Video Capsule Endoscopy cs.CVPDF

[50] Fast Fourier Convolutional GAN for 30 m Clear-Sky Land Surface Temperature Gap-Free Reconstruction cs.CV | eess.IV | physics.geo-phPDF

[51] Cortex: Compact Behavior Cloning for Quake with Frozen Visual Features cs.CV | cs.AI | cs.LGPDF

[52] Cheap Probes Predict Expensive Training in 3D-CT Vision–Language Models cs.CV | cs.AIPDF

[53] AI-generated Images Challenge Visual Trust in High-risk Scenarios cs.CV | cs.AIPDF

[54] ID-V2V: Identity-Preserving Video Restylization cs.CVPDF

[55] Gaze-Anchored Social Net: Decoding Implicit Relations via Joint Modeling cs.CVPDF

[56] Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests cs.CV | cs.AIPDF

[57] Controlling Embedding Spaces with Text-Conditioned Transformations cs.CV | cs.AIPDF

[58] Small-Pollinator Detection in Cluttered Field Video cs.CVPDF

[59] HALLELUAI: A Hallucination-Aware AI System for Ultra-Realistic Image-to-Video Generation at Scale cs.CV | cs.AIPDF

[60] Layering Virtual Try-On cs.CVPDF

[61] mmSimPrior: Learning Simulation Priors for Data-Efficient Real-World Generalizable Radar-Based Human Motion Reconstruction cs.CVPDF

[62] Structured Redundancy Modeling for Efficient Visual Token Pruning in High-Resolution MLLMs cs.CVPDF

[63] DispatchRAG: Grounding Emergency Dispatch Decisions in Real-World Protocols from Traffic Accident Video cs.CVPDF

[64] DishSeg24k: A Large-Scale Benchmark for Food Segmentation with Stochastic Expert Decoding cs.CVPDF

[65] Similarity Is Not Logic: Factored Inference for Dual-Encoder Vision-Language Models cs.CV | cs.CL | cs.LGPDF

[66] Towards Dual-Brain Minimal Sufficient Representation for Vision-Language Navigation cs.CVPDF

[67] A Reconstruction-Based Framework for Caption Evaluation Beyond Reference Captions cs.CVPDF

[68] SARATR-X-v2: Scale-Aware Structural Pre-Training for SAR Foundation Models cs.CVPDF

[69] OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models cs.CVPDF

[70] WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation cs.CVPDF

[71] What CLIP Knows but Cannot Say: Recovering Negation from Frozen Intermediate Features cs.CV | cs.AIPDF

[72] The Gate Always Closes: On Injecting Auxiliary Signals into Frozen Vision-Language Models cs.CVPDF

[73] UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models cs.CVPDF

[74] Omni-Prune: Query-Aware Unified Token Pruning for Efficient Omnimodal Large Language Models cs.CV | cs.CLPDF

[75] Explaining BiomedCLIP with Weighted Banzhaf Interactions Supported by Tree-Gram Parsing cs.CV | cs.AI | cs.CLPDF

[76] Multi-Modal Object Re-Identification with Prompt-S6 and Semantic-Aware Knowledge Guidance cs.CVPDF

[77] VIPER: Visual In-Context Physics Reasoning for Physically Plausible Video Generation cs.CVPDF