cs.CL [Total: 12]
cs.CV [Total: 48]
cs.LG [Total: 1]
q-bio.NC [Total: 1]
eess.AS [Total: 1]
cs.AI [Total: 1]
cs.RO [Total: 3]
cs.MM [Total: 1]
cs.CR [Total: 2]

cs.CL [Back]

[1] Verbalizable Representations Form a Global Workspace in Language Models cs.CL | cs.AI | cs.LGPDF

Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, Isaac Kauvar

TL;DR: 本文提出了一种名为Jacobian lens的新颖可解释性技术，用于识别大型语言模型中准备被言语化的表征，这些表征构成了一个称为J-space的全局工作空间。研究发现J-space具有类似于人类意识访问的功能特性，如可报告性、可控制性和灵活性，并且其结构特征与全局工作空间理论相符。该技术为揭示模型未言说的内部认知过程（如战略审议和评估意识）提供了实用窗口，并引入了反事实反思训练以改进模型行为。

Details

Motivation: 受人类大脑中仅有小部分信息可被意识访问（即可言语报告、控制与灵活推理）的启发，研究旨在探索大型语言模型中是否存在类似的功能区分，以理解模型的内部认知过程。

Result: 通过Jacobian lens技术识别出的J-space表征在功能上表现出全局工作空间特性，并在结构上显示出中间层携带连贯内容、同时容纳约数十个概念以及权重广播更广泛等特征。在对齐审计中，该技术揭示了模型输出中未出现的战略审议和评估意识等内部过程。

Insight: 创新点在于提出了Jacobian lens这一可解释性方法，将模型内部表征与言语化能力关联起来，从而构建了一个功能性的全局工作空间（J-space），为理解模型认知提供了新视角；同时，反事实反思训练通过干预模型的内部反思过程来改进行为，是一种新颖的对齐方法。

Abstract: Out of everything the human brain processes, only a small fraction is consciously accessible, in the sense of being available for verbal report, deliberate control, and flexible reasoning. In this paper, we present evidence that an analogous functional distinction has emerged in large language models. Using a new interpretability technique, the Jacobian lens, we identify the representations a model is poised to verbalize at any point in its processing. These representations, which we collectively call the J-space, exhibit the functional properties characteristic of a global workspace: their contents can be reported, deliberately summoned and held, used to carry the intermediate steps of silent reasoning, and passed as arguments to arbitrary downstream computations, while automatic processing such as text parsing and routine inference proceeds without them. The J-space also has structural signatures that global workspace theory associates with conscious access: it carries coherent content only in an intermediate band of layers, holds on the order of tens of concepts at a time, and is broadcast by the model’s weights more widely than other representations. These properties make it a practical window into a model’s unspoken thinking. In alignment audits, it reveals strategic deliberation, evaluation awareness, and trained-in misaligned dispositions that never appear in the model’s outputs. We find that post-training installs the Assistant’s point of view in the workspace, and we introduce counterfactual reflection training, which improves behavior by training only what a model would say if interrupted and asked to reflect. These results indicate that language models maintain a small, privileged set of representations bearing some of the functional hallmarks of conscious access, and that decoding these representations sheds light on ongoing cognitive processes.

[2] Large Language Models as Unified Multimodal Learners for Clinical Prediction cs.CL | cs.AIPDF

Ajay Madhavan Ravichandran, Bilgin Osmandoja, Klemens Budde, Klaus Netter, Tobias Strapatsas

TL;DR: 这篇论文提出了一种简化的多模态临床预测方法：将所有患者数据（包括文本和结构化数据）统一转换为自然语言序列，然后直接对预训练语言模型进行端到端微调，无需为多模态融合设计特定架构。该方法在三个不同的临床预测任务（院内死亡率、移植失败预测、急诊分诊分类）上进行了评估，结果表明其性能与或优于传统的任务特定多模态基线模型，并在移植失败预测上超越了临床实践中部署的梯度提升系统。

Details

Motivation: 目前大多数临床预测系统依赖于为每个任务和临床环境重新设计的、任务特定的多模态融合架构，这增加了系统复杂性。论文旨在探索一种更简单的替代方案，即利用大语言模型作为统一的多模态学习器，避免复杂的定制化融合设计。

Result: 在MIMIC-III院内死亡率预测、德国移植中心的移植失败预测以及救护车记录的急诊分诊分类三个任务上，基于文本序列化的方法（使用ModernBERT、Llama 3.1、Gemma、DeepSeek-R1-Qwen、Qwen3等模型微调）均达到或超越了既有的多模态基线模型。特别是在移植失败预测任务上，该方法超越了临床实践中部署的梯度提升模型。

Insight: 核心创新点在于将所有模态的临床数据统一序列化为单一的自然语言流，从而可以直接利用预训练语言模型的能力进行端到端学习，无需为多模态融合设计复杂的专用架构。这为多模态临床预测提供了一个简化且有效的通用范式，显著降低了系统复杂性。

Abstract: Electronic health records combine free-text clinical narratives with structured measurements such as vital signs, laboratory values, and comorbidities. Yet most clinical prediction systems still rely on task-specific fusion architectures, pairing dedicated encoders for each modality with learned combination mechanisms that must be re-engineered for every new task and clinical setting. We propose a simpler alternative: convert all patient data, regardless of modality, into a single natural language sequence and fine-tune a pretrained language model end-to-end, with no architectural modification for fusion. We evaluate this approach across three clinically distinct prediction tasks: in-hospital mortality on MIMIC-III, graft failure prediction using longitudinal data from a German transplant center, and emergency triage classification from ambulance records - comparing encoder-based (ModernBERT) and decoder-based (Llama 3.1, Gemma, DeepSeek-R1-Qwen, Qwen3) fine-tuning against established multimodal baselines and, for graft failure, a gradient boosting model currently used in clinical practice for post-transplant patient management. Across all three tasks, unified textual serialization matches or exceeds task-specific multimodal baselines, and outperforms the clinically deployed gradient boosting system on graft failure prediction. These results indicate that a single serialization-based paradigm, without bespoke fusion architectures, is sufficient for multimodal clinical prediction - substantially reducing system complexity while matching or exceeding specialized designs.

[3] EpiNarrate: Agentic Generation of Grounded Narratives from Epidemiological Scenario Projections cs.CLPDF

Rituparna Datta, Srini Venkatramanan, Bryan L. Lewis, Yiqi Su, Harry Hochheiser

TL;DR: EpiNarrate是一个用于从流行病学情景预测数据中生成有根据的公共卫生叙述的智能体框架。它通过将结构化数值推理与自然语言生成分离，解决了直接使用大语言模型（LLM）总结此类复杂数据时产生的信息不一致、遗漏和脆弱性问题。该框架首先提取情景维度并组织成偏序模式，然后构建增强数据集并通过强制语义和算术一致性的比较语法推导有效的定量陈述，最后基于最大熵原理进行有趣性驱动的选择以平衡覆盖范围和非冗余性。

Details

Motivation: 将复杂的流行病学预测清晰、易懂地传达给政策制定者和公众至关重要，这需要超越简单的数字报告，进行多维度、定量且有背景的叙述。然而，直接使用大语言模型来总结和情境化来自大型集成数据集（包含干预假设、地理和人口分层、结果、时间范围和不确定性分位数）的预测，往往会导致不一致、遗漏和脆弱的行为。

Result: 在COVID-19情景建模中心（COVID-19 Scenario Modeling Hub）上的实验表明，该模型生成的叙述在事实依据和显著流行病学模式的覆盖范围方面都有所改进，同时保持了专家撰写报告的风格。

Insight: 论文的核心创新在于提出了一个将结构化数值推理与自然语言生成解耦的智能体框架，通过引入偏序模式系统遍历多维空间、使用强制一致性的比较语法推导定量陈述，以及基于最大熵原理的有趣性选择机制，确保了生成叙述的定量准确性、语义一致性和信息覆盖的平衡。这为处理复杂、多维数据集并生成可靠、全面的叙述提供了一种系统化方法，超越了直接依赖LLM的局限性。

Abstract: Generation of clear and accessible public health narratives is critical for communicating complex epidemiological projections to policymakers and the general public at large. Such narratives require more than simply reporting numbers: projections must be contextualized and quantitatively grounded across multiple dimensions. Further, projections are often derived from large ensemble datasets which combine intervention assumptions, geographic and demographic strata, outcomes, time horizons, and uncertainty quantiles. However, directly using large language models (LLMs) to summarize and contextualize such data often leads to inconsistencies, omissions, and fragile behavior. We introduce an agentic framework (EpiNarrate) for public health report generation that separates structured numerical reasoning from natural-language generation. The framework first extracts scenario axes and organizes them into a partial-order schema, enabling systematic traversal of the underlying multidimensional space. It then constructs an augmented dataset and derives valid quantitative statements through a comparison grammar that enforces semantic and arithmetic consistency. To balance coverage and non-redundancy, we introduce an interestingness-driven selection mechanism based on maximum-entropy principles. Experiments on the COVID-19 Scenario Modeling Hub demonstrate that our model produces narratives with improved factual grounding and broader coverage of salient epidemiological patterns, while preserving the style of expert-written reports.

[4] Process Reward Informed Tree Rollout for Effective Multi-Turn RL cs.CL | cs.AI | cs.LGPDF

Xintong Li, Sha Li, Yuwei Zhang, Changlong Yu, Rongmei Lin

TL;DR: 本文提出了一种名为PATR（Process-Scorer Guided Adaptive Tree Rollout）的质量感知树状展开框架，用于解决多轮次强化学习（RL）中探索效率低下的问题。该方法利用任务相关的过程反馈对部分轨迹进行评分，选择性地从有希望的状态进行分支探索，并重用共享前缀、保守地停止退化路径，从而在相同训练预算下实现更高效的探索。

Details

Motivation: 当前流行的GRPO/RLOO等RL方法在训练LLM智能体时，依赖于对多个独立采样的完整轨迹进行优势估计。在长视野的智能体任务中，这种均匀的展开策略会浪费预算在无信息的死胡同尝试上，而有望的中间状态却得不到充分探索。

Result: 在FrozenLake和具有挑战性的SWE-Bench基准测试上进行评估。实验表明，PATR在SWE-Bench上将性能提升了高达+5.0分，在FrozenLake上提升了+9.3分，突显了过程引导的树状展开作为可扩展多轮次RL的有效策略。

Insight: 核心创新点在于将多轮次智能体轨迹的树状结构视角形式化，并将有效探索问题重新定义为决定在何处分支。PATR框架通过过程评分器引导自适应分支，实现了采样预算的优化分配，同时保持与标准策略优化方法的兼容性，为长视野任务中的RL探索提供了新思路。

Abstract: Reinforcement learning (RL) has become a key approach for training LLM agents, yet popular methods such as GRPO/RLOO rely on multiple independently sampled complete trajectories for advantage estimation. In long-horizon agentic tasks, such a uniform rollout strategy can waste budget on uninformative dead-end attempts, while promising intermediate states do not receive sufficient exploration. The multi-turn structure of agentic trajectories, with interleaved actions and observations, naturally supports organizing a trajectory group as a tree, where each turn serves as a decision point for exploration. This perspective reframes effective exploration as the problem of deciding where to branch. We propose Process-Scorer Guided Adaptive Tree Rollout (PATR), a quality-aware rollout framework for multi-turn agent RL. PATR uses task-appropriate process feedback to score partial trajectories, selectively branches from promising states, reuses shared prefixes, and conservatively stops degenerate paths to reduce wasted sampling. The resulting rollout groups remain compatible with standard policy optimization while providing more efficient exploration under the same training budget. We evaluate PATR on FrozenLake and the challenging SWE-Bench, which is largely unexplored by prior tree-rollout agent RL methods. Experiments show that PATR improves performance by up to +5.0 points on SWE-Bench and +9.3 points on FrozenLake, highlighting process-guided tree rollouts as an effective strategy for scalable multi-turn RL.

[5] Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning cs.CLPDF

Leichao Dong, Dongxu Zhang, Yiding Sun, Qirui Wang, Yuhan Wang

TL;DR: 本文提出了BIRD（Bootstrapped Iterative Self-Reasoning Distillation），一种两阶段的自推理蒸馏方法，旨在提升大型推理模型的效率。该方法首先通过简洁指令从基础模型中采样并筛选出正确答案的轨迹，进行轻量级的提示切换监督微调，然后将这种简洁性内化为默认推理行为。接着，在更干净、信息更丰富的前缀上，使用简洁的自教师模型进行策略上的反向KL蒸馏。

Details

Motivation: 现有策略自蒸馏方法存在初始化瓶颈，即从冗长的基础模型开始训练时，KL损失施加在嘈杂、冗余或已偏离正确轨道的上下文上，导致学生模型继续探索低效的推理轨迹。BIRD旨在通过改进训练前的轨迹分布来克服这一瓶颈，实现更高效的推理蒸馏。

Result: 在Qwen3系列模型上，BIRD在MATH-500和AIME基准测试上取得了比提示方法和冷启动策略蒸馏更好的准确率-效率权衡。具体而言，在Qwen3-8B上，它将MATH-500的准确率从86.2%提升至92.0%，同时将平均响应长度从3099个token减少至1115个token。

Insight: 论文的核心创新点在于两阶段蒸馏框架，特别是通过“提示切换”的监督微调步骤，将指令诱导的简洁性固化为模型的默认推理行为，从而为后续的策略蒸馏提供了更优的初始轨迹分布。这揭示了前缀支持（即训练数据的上下文质量）是高效推理蒸馏的关键因素。

Abstract: Large reasoning models often solve problems through long chain-of-thought (CoT) traces, yet much of this computation is spent on redundant derivations, repeated self-verification, and detours that do not improve the final answer. Existing on-policy self-distillation methods reduce this cost by matching a student model to a concise copy of itself on prefixes sampled from the student’s own rollouts. We show that this objective has an initialization bottleneck. Since supervision is applied only to visited prefixes, training from a verbose base model places the KL loss on contexts that are often noisy, redundant, or already off track. In such regions, a concise teacher can provide only local corrections, while the student continues to explore trajectories that an efficient reasoner should avoid. In this paper, we propose BIRD(Bootstrapped Iterative Self-Reasoning Distillation), a two-stage self-reasoning distillation method that improves the rollout distribution before on-policy training. BIRD first samples concise solutions from the base model under a brevity instruction, keeps only answer-correct traces, and performs a lightweight prompt-switch SFT step. The traces are generated with the brevity instruction but learned under the original task prompt, turning instruction-induced conciseness into a default reasoning behavior. Starting from this warm model, BIRD then applies on-policy reverse-KL distillation with a concise self-teacher, now on cleaner and more informative prefixes. Across Qwen3 series models, BIRD achieves a stronger accuracy-efficiency trade-off than prompting and cold-start on-policy distillation on MATH-500 and AIME benchmarks. On Qwen3-8B, it improves MATH-500 accuracy from 86.2% to 92.0% while reducing the average response length from 3,099 to 1,115 tokens. These results highlight prefix support as a central factor in efficient reasoning distillation.

[6] CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations cs.CL | cs.AIPDF

Suzan Awinat, Alfonso Ortega del Puente

TL;DR: CAMMAR是一个针对阿拉伯语隐喻表示的文化感知嵌套表示学习框架，通过分阶段的语义课程将意义组织为嵌套的词汇、文化和隐喻嵌入子空间，以解决现有阿拉伯语模型中词汇、文化和隐喻信息在单一表示空间中混淆的‘语义模糊’问题。

Details

Motivation: 阿拉伯语隐喻是一种基于文化的意义构建机制，但当前阿拉伯语模型通常将词汇、文化和隐喻信息混合在单一表示空间中，导致‘语义模糊’，无法有效捕捉隐喻的文化根基。

Result: 在基于词匹配的比喻/字面对阿拉伯语隐喻数据集上评估，CAMMAR的几何读出方法在配对监督下能有效检测隐喻（AUC高达0.84，82.6%的对比中比喻得分高于字面对应词），但在无监督域对比下表现仅随机水平，显示了监督对几何结构形成的关键作用。

Insight: 创新点包括：基于Al-Jurjani的nazum理论，将比喻意义建模为基于先验语义关系的组合；采用嵌套子空间结构分离不同语义层次；提供无需训练的基于词汇与隐喻表示距离的隐喻性几何度量；实验表明在词汇层基于形态词根能带来小幅但一致的增益，反映了其作为测量锚点的质量。

Abstract: Metaphor in Arabic is a culturally grounded mechanism for constructing meaning, encoding cultural knowledge that shapes interpretation. Yet current Arabic language models typically collapse lexical, cultural, and metaphorical information into a single representational space, a phenomenon we term “semantic smearing”. We introduce CAMMAR (Culture-Aware Matryoshka for Metaphorical Arabic Representations), a representation learning framework that organizes meaning into nested lexical, cultural, and metaphorical embedding subspaces through a staged semantic curriculum. The design implements compositional principles of Al-Jurjani’s theory of nazum, modeling figurative meaning as compositionally grounded in prior semantic relations, and yields a training-free geometric measure of metaphoricity based on the distance between lexical and metaphorical representations. Evaluated on a new span-annotated Arabic metaphor set as word-matched figurative/literal pairs, the geometric readout detects metaphor well above chance when the inter-layer geometry is shaped by paired supervision (AUC up to 0.84; figurative outscores its literal counterpart for the same word in 82.6% of pairs), but sits at chance under an unsupervised domain contrast alone, a clean separation between a legible-under-supervision regime and a non-emergent one. A controlled ablation shows that grounding the lexical layer in morphological roots gives a small but consistent gain, an effect absent from direct probing that reflects the layer’s quality as a measurement anchor. We will release the datasets, cultural concept inventory, and code upon acceptance.

[7] DECODEM: Data Extraction from Corporate Organizational Documents via Enhanced Methods cs.CL | cs.AI | cs.CYPDF

Jens Frankenreiter

TL;DR: 本文介绍了DECODEM，一套用于评估从公司章程和细则等公司组织文件中自动提取公司治理变量的基准数据集。论文利用这些数据集评估了多种基于大语言模型的提取流程，发现自动化提取对许多治理条款具有高可行性，中位性能接近方法上限，但性能因变量而异，少数条款导致了大部分错误。

Details

Motivation: 传统上，公司治理研究依赖人工编码非结构化文本（如章程和细则），成本高、难以扩展且不透明。本文旨在通过创建标准化基准和系统评估方法，推动自动化特征提取在公司治理数据集构建中的应用。

Result: 在DECODEM基准数据集上的评估表明，自动化提取对许多治理条款能达到高准确率，中位性能接近各种方法的上界。更复杂的提示策略和级联流程虽未持续提升前沿模型的性能，但在某些场景下显著缩小了前沿模型与效率导向模型之间的差距。

Insight: 创新点在于创建了首个用于公司治理变量提取的标准化基准数据集（DECODEM），并系统评估了不同提示设计、任务分解和文档处理流程的影响。客观分析表明，流程设计可以在一定程度上替代模型能力，为资源受限场景提供了优化方向。

Abstract: Much empirical legal research depends on translating unstructured text into structured variables. In corporate governance research as elsewhere, this translation has traditionally relied on human coding of documents such as charters and bylaws, a process that is costly, difficult to scale, and often opaque. This paper introduces DECODEM, a set of benchmark datasets for evaluating the automated extraction of corporate governance variables from organizational documents. The benchmarks pair randomly sampled corporate charters and bylaws with high-quality human annotations covering a range of governance provisions commonly studied in empirical work. Using these datasets, the paper evaluates several large-language-model extraction pipelines that vary in prompt design, task decomposition, and document handling. The underlying task consists of a set of document-level binary classification problems, one for each governance variable. The results show that automated extraction is feasible at a high level of accuracy for many provisions, with median performance near the upper bound across approaches. At the same time, performance varies systematically across variables, with a small number of provisions accounting for most of the remaining errors. More elaborate prompting strategies and cascading pipelines do not consistently improve performance for frontier models, but substantially narrow the gap between frontier and efficiency-oriented models in some settings, suggesting that pipeline design can partly substitute for model capability. By providing a standardized benchmark and a systematic evaluation of extraction methods, the paper demonstrates that current frontier models can extract legally meaningful information from complex corporate documents with high accuracy and suggests an important future role for automated feature extraction in constructing corporate governance datasets.

[8] From Plausible to Actionable: A Position on LLM Self-Explanations cs.CLPDF

Elize Herrewijnen, Benedetta Muscato, Gizem Gezici, Fosca Giannotti

TL;DR: 本文探讨了大型语言模型（LLM）生成的自我解释（self-explanations）的特性，指出这些解释虽然看似合理（plausible），但其忠实性（faithfulness）存疑，然而仍具有高度可操作性（actionable）。作者从可解释人工智能（XAI）的传统视角出发，分析了现有评估协议的局限性，并提出了评估合理性、忠实性和可操作性的实用指南。

Details

Motivation: 尽管LLM生成的自我解释在可解释人工智能（XAI）中展现出潜力，但其是否真实反映模型内部推理过程（即忠实性）仍是一个开放问题。本文旨在批判性地审视这些解释的局限性，并推动评估标准超越表面的合理性。

Result: 本文是一篇观点论文（opinion paper），未报告具体的定量实验结果或基准测试。它主要提出了评估LLM自我解释的框架和指南，强调将评估标准扩展到可操作性（actionability）。

Insight: 核心创新在于将“可操作性”确立为评估LLM自我解释的关键新维度，主张即使解释的忠实性不确定，其支持不同利益相关者做出明智决策和采取适当行动的能力也具有重要价值。这为XAI评估提供了更实用和面向应用的新视角。

Abstract: Large Language Models (LLMs) can generate natural language explanations that rationalize their own decisions, a phenomenon commonly referred to as self-explanations.Such explanations have emerged as a promising direction for explainable artificial intelligence (XAI), particularly for interpreting LLM behavior.However, while self-explanations often appear plausible, whether they faithfully reflect a model’s underlying reasoning process remains an open question. In this opinion paper, we argue that self-explanations can be highly plausible, questionably faithful, and yet highly actionable. From a traditional XAI perspective, we identify the limitations of standard evaluation protocols for LLM-generated self-explanations and propose practical guidelines for assessing their plausibility and faithfulness. Moreover, we argue that evaluation should extend beyond these criteria to actionability, highlighting applications of LLM rationalization capabilities that support informed decision-making and appropriate action across diverse stakeholders.

[9] Loop the Loopies! cs.CL | cs.AIPDF

Zitian Gao, Yilong Chen, Yihao Xiao, Xinyu Yang, Ran Tao

TL;DR: 本文提出了Loopie，一种目前最强大的循环Transformer模型。该系列包含两个MoE模型：20B参数（2B激活）和6B参数（0.6B激活）。通过新颖的后训练流程，Loopie在2025年IMO和IPhO竞赛中实现了无需工具的黄金奖牌级性能。

Details

Motivation: 解决循环Transformer长期面临的挑战：在预训练计算量增加N倍时，通常增加N倍参数比循环模型N次效果更好，Loopie旨在突破这一性能瓶颈。

Result: 在相同计算预算下，Loopie显著优于普通Transformer基线（如30B-A3B模型）；在2025年国际数学奥林匹克（IMO）和国际物理奥林匹克（IPhO）上达到黄金奖牌水平。

Insight: 创新点在于结合了MoE架构与循环Transformer设计，并引入了增强推理能力的后训练流程，有效提升了模型性能与计算效率的平衡。

Abstract: We present Loopie, the most powerful looped Transformer to date. The Loopie series consists of two Mixture-of-Experts (MoE) models: a 20B-parameter model with 2B active parameters and a 6Bparameter model with 0.6B active parameters. Looped Transformers have long faced a challenge: given an N-fold increase in pre-training compute, increasing the parameter count by a factor of N usually outperforms looping a model N times. Loopie addresses this challenge. Extensive ablation studies, including comparisons with a vanilla 30B-A3B model, show that Loopie substantially outperforms vanilla Transformer baselines trained with the same compute budget. Our novel post-training pipeline equips Loopie with strong reasoning abilities. At the 2025 IMO and IPhO, Loopie achieves gold-medal performance without tools.

[10] Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning cs.CL | cs.AIPDF

Ajay Patel, Kartik Hosanagar, Ramayya Krishnan, Chris Callison-Burch, Karim Lakhani

TL;DR: 该论文构建了BusinessCaseBench基准，用于评估前沿AI模型在商业案例分析等知识工作与推理任务上的表现，发现当前模型已能取得较高分数且能力快速提升。

Details

Motivation: 现有AI基准主要测试事实回忆、数学解题等能力，缺乏对白领专业人士日常分析性知识工作（如综合复杂信息、不确定性下判断）的衡量，论文旨在填补这一测量空白。

Result: 在涵盖18个学科数百个商业案例问题的BusinessCaseBench上，前沿AI模型已能根据专家评分标准获得高分，且同一模型系列在两年内能力显著提升。

Insight: 创新点在于利用商学院案例教学法构建基准，将主观性分析任务结构化评估；客观来看，这为衡量AI在复杂、开放式知识工作上的进展提供了新方法。

Abstract: Large language models (LLMs) are improving rapidly as reflected in benchmark scores, yet these AI benchmarks largely test capabilities such as factual recall, narrow question answering, mathematical problem-solving, and coding and agentic tool-use. What remains poorly measured is AI progress on the analytical knowledge work white-collar professionals perform daily, including synthesizing complex information, exercising judgment under uncertainty and incomplete information, applying strategic and adversarial thinking in multi-stakeholder settings, weighing trade-offs, and producing defensible, structured analyses. This gap is even more pronounced for subjective components of such work, where success can be challenging to define. The “case method” form of education practiced by top business schools provides a natural foundation for addressing this measurement gap, and we construct BusinessCaseBench, a benchmark spanning hundreds of questions drawn from business cases across eighteen disciplines, each paired with a grading rubric derived from the expert-written instructor case solution. On BusinessCaseBench, frontier AI models already score highly against instructor rubrics, and capability within one model family improves substantially over two years. These results provide strong evidence that AI performance on this class of work is already high and rapidly improving, with implications for business schools, where case pedagogy trains undergraduates and MBAs in this kind of analytical reasoning, and for entry-level professional roles, where such skills have historically anchored early-career work.

[11] Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D cs.CLPDF

Haodong Wen, Yiran Zhang, Yingfa Chen, Kaifeng Lyu

TL;DR: 本文指出前沿大语言模型在简单复制任务上存在困难，并归因于Transformer位置编码的归纳偏置。作者提出2D-RoPE方法，将文本组织为二维网格，使复制任务简化为固定列偏移检索。实验表明，使用2D-RoPE的浅层Transformer在合成复制任务上表现优异，且在大规模预训练中保持优势。

Details

Motivation: 解决前沿大语言模型无法精确复制输入字符串的问题，该问题源于Transformer位置编码的局部上下文匹配捷径，而非精确定位输入位置。

Result: 在合成复制实验中，2D-RoPE使浅层Transformer在训练未见的长输入上实现完美复制，远超标准位置编码；在DCLM上对高达1.4B参数模型的大规模预训练中，2D-RoPE在复制任务上持续保持优势。

Insight: 创新点在于将文本视为二维网格而非一维序列，通过行列ID分配简化复制任务；客观分析认为，二维位置编码能更有效地建模长距离依赖，为语言建模提供新视角。

Abstract: While large language models (LLMs) can solve advanced reasoning problems in seconds, we show that even frontier models fail to perform a much simpler operation: exactly copying an input string that lies well within their context windows. We attribute this failure to positional encodings in Transformer architectures, whose inductive bias favors copying through a shortcut based on matching local contexts rather than carefully locating the corresponding input positions. To address this issue, we introduce 2D-RoPE, which organizes text into a 2D grid rather than a 1D sequence and assigns each token a row ID and a column ID. Under this view, copying becomes simply retrieving input tokens at a fixed column offset, which makes the task easy to learn. In synthetic copy experiments, shallow Transformers with 2D-RoPE achieve perfect copying at input lengths hundreds of times longer than those seen during training, whereas standard positional encodings fall far behind. We further show that the advantage of 2D-RoPE language models on copy tasks consistently holds in large-scale pretraining on DCLM with model sizes up to 1.4B parameters. Overall, our results suggest that viewing text in 2D can benefit language modeling, and we hope this encourages future work to further explore the potential of 2D positional encodings.

[12] ToolSciVer: Multimodal Scientific Claim Verification with Visual Tool Augmented Reinforcement Learning cs.CL | cs.AIPDF

Binglin Zhou, Peng Shi, Ryo Kamoi, Nan Zhang, Rui Zhang

TL;DR: ToolSciVer是一个用于多模态科学声明验证（MSCV）的工具增强强化学习框架。它通过为视觉语言模型（VLM）配备三种类型感知的视觉工具（表格行列聚焦、图表结构解析、高分辨率区域缩放），将密集的科学视觉信息转化为明确的证据，并使用GRPO策略优化方法进行训练。在SciVer和MuSciClaims数据集上的实验表明，该方法在多个VLM上优于基于提示和基于RL的工具使用方法。

Details

Motivation: 现有的多模态科学声明验证方法在定位关键视觉证据、准确解读结构化科学图表以及整合多模态观察进行可靠推理方面存在困难。

Result: 在SciVer和MuSciClaims数据集上，对来自Qwen、InternVL、Gemma三个家族的五个VLM进行实验，ToolSciVer的性能优于包括基于提示和基于RL的工具使用方法在内的四个强基线，达到了SOTA水平。

Insight: 创新点在于首次为MSCV任务引入工具增强框架，通过类型感知的专用视觉工具将复杂视觉信息显式化，并设计了包含答案正确性、格式有效性、长度控制、工具使用效率和工具有效性惩罚的复合奖励函数，利用GRPO进行策略优化，有效提升了科学声明的验证能力。

Abstract: Multimodal Scientific Claim Verification (MSCV) requires models to verify scientific claims using visually grounded evidence from papers, including figures, tables, charts, and textual context. However, existing methods often fail because they struggle to locate decisive visual evidence, accurately read structured scientific visuals, and integrate multimodal observations into reliable reasoning. We introduce ToolSciVer, the first tool-augmented framework for MSCV to our knowledge. ToolSciVer equips a VLM with three type-aware visual tools, table row/column focus, chart-to-structure parsing, and high-resolution region zoom, which convert dense scientific visuals into explicit, claim-facing evidence, and trains the policy with Group Relative Policy Optimization (GRPO) under a composite reward of answer correctness, format validity, length control, tool-use efficiency, and tool-validity penalties. Experiments on SciVer and MuSciClaims datasets on five VLMs from three model families (Qwen, InternVL, Gemma) demonstrate that our method achieves superior performance compared to four competitive baselines including prompting-based and RL-based tool-use methods, highlighting the effectiveness of learned, type-aware tool use for scientific claim verification.

cs.CV [Back]

[13] An Empirical Study of Handcrafted Feature Learning and Convolutional Neural Networks for Facial Expression Recognition cs.CV | cs.LGPDF

Chethiya Galkaduwa

TL;DR: 本文通过实证研究比较了手工特征（HOG+SVM、LBP+逻辑回归）与轻量级CNN在面部表情识别任务上的性能。实验在FER-2013、CK+和KDEF三个数据集上进行，结果表明CNN在复杂数据上整体表现最佳，而HOG在受控环境中表现强劲，LBP则普遍表现不佳。研究强调了数据集复杂性对性能的重要影响以及鲁棒特征学习对于实际应用的关键性。

Details

Motivation: 尽管卷积神经网络主导了现代面部表情识别，但手工特征描述符（如HOG和LBP）仍是重要的经典基线。本研究旨在通过实证比较，评估这些方法在不同复杂度数据集上的表现，以明确各自的适用场景和局限性。

Result: 在FER-2013、CK+和KDEF三个基准数据集上的实验表明，轻量级CNN取得了最佳的整体性能，尤其是在更复杂的数据上；HOG+SVM在受控环境（如实验室数据集）中表现强劲；而LBP+逻辑回归在所有数据集上均表现不佳。

Insight: 论文的创新点在于对经典手工特征与轻量CNN进行了系统的实证对比，揭示了数据集复杂度是影响方法性能的关键因素。从客观角度看，这强调了针对实际应用场景（数据复杂度）选择或设计特征学习方法的必要性，而非盲目依赖单一技术路线。

Abstract: Facial expression recognition is an important computer vision task with applications in human–computer interaction, mental health monitoring, driver alert systems, and behavioral analysis. While convolutional neural networks (CNNs) dominate modern facial expression recognition, handcrafted feature descriptors such as Histogram of Oriented Gradients (HOG) and Local Binary Patterns (LBP) remain useful classical baselines. This study compares HOG with Support Vector Machine (SVM), LBP with Logistic Regression, and a lightweight CNN across three facial expression datasets: FER-2013, CK+, and KDEF. The results show that CNNs achieve the best overall performance, particularly on more complex data, while HOG performs strongly in controlled environments. LBP performs poorly across all datasets. The study highlights that dataset complexity significantly affects performance and that robust feature learning is essential for real-world facial expression recognition.

[14] Rethinking the Readout: Unlocking Video Backbones for AI-Generated Video Detection cs.CVPDF

Manni Cui, Ziheng Qin, ZiAn Wang, Ruiqi Liu, Dianyuan Zou

TL;DR: 本文提出了一种名为Velocity Gated Patch Velocity Profiling (V-PVP)的轻量级读出模块，用于改进AI生成视频（AIGV）的检测。该方法通过替换视频骨干网络中的标准全局聚合层，专注于捕捉帧间不一致性所导致的细微时序伪影，从而显著提升了多种视频骨干网络在AIGV检测任务上的性能。

Details

Motivation: 现有基于视频预训练骨干网络的检测器在AIGV检测基准上表现不佳，作者认为这是由于标准的全局读出操作过度聚合了时空信息，抑制了局部块级别的时序动态和块间关系，而这些正是检测AIGV伪影最可靠的线索。

Result: 在AIGVDBench基准测试中，该方法在保持骨干网络完全冻结的情况下达到了95.28的AUC值，显著优于仅使用图像预训练探针的强基线模型，展示了其有效性。

Insight: 论文的核心创新在于揭示了视频骨干网络在AIGV检测中潜力未被充分挖掘的原因，并提出了一种轻量、即插即用的读出模块V-PVP，通过并行处理块速度场来重新激活冻结骨干网络的时序潜力，这一设计思路可迁移到其他需要精细时序建模的视频理解任务中。

Abstract: AI-generated videos (AIGVs) typically contain subtle temporal artifacts that arise from inter-frame inconsistencies rather than within individual frames. A detector that captures such artifacts should therefore benefit from video pretrained backbones over image only ones. In practice, however, video backbones with standard global readouts often fail to outperform strong image pretrained probes on AIGV benchmarks. We attribute this gap to excessive spatiotemporal aggregation in the readout. Video pretrained backbones tend to compress each frame into a single global descriptor. This compression suppresses local patch level temporal dynamics and discards inter patch relations, which are precisely the cues that AIGV detection most reliably depends on. Based on this, we propose Velocity Gated Patch Velocity Profiling (V-PVP), a lightweight readout that replaces only the aggregation layer with two parallel streams over the patch velocity field, adding only about $0.5$M trainable parameters. V-PVP serves as a general plug-and-play module that consistently improves performance across diverse video backbones under both end-to-end fine-tuning and linear probing settings. Our method reaches \textbf{95.28} AUC on AIGVDBench while keeping the backbone fully frozen. The results show that simply replacing the aggregation layer reactivates the temporal potential of frozen video backbones, restoring their advantage on AIGV detection. Code is available at https://anonymous.4open.science/r/PVP-81B3/.

[15] Reasoning-Guided Part-Level Visual Grounding via Reinforcement Learning cs.CVPDF

Kazi Sajeed Mehrab, Hani Alomari, Najibul Haque Sarker, Chia-Wei Tang, Zaber Ibn Abdul Hakim

TL;DR: 本文提出了一种名为OP-HRG的分层反射式视觉定位方法，旨在解决多模态大语言模型在根据自由形式语言查询定位物体部分而非整个物体时表现不佳的问题。该方法采用从粗到细的推理策略，先定位父物体，再定位其内部部件，并通过自检和重新编码预测区域进行修正。

Details

Motivation: 动机在于多模态大语言模型虽然能很好地根据语言查询定位整个物体，但由于缺乏物体-部件的层次结构，在查询指定物体部件时表现不佳，因为部件定位与物体定位在同一单一步骤中进行。

Result: 在PascalPart、PartImageNet和InstructPart等基准测试中，使用该方法训练的4B模型性能优于7B的视觉定位大语言模型和SAM3，并且能够迁移到推理分割任务上。

Insight: 创新点在于引入了物体-部件层次化反射式定位的从粗到细推理策略，以及一个包含阶段奖励的、部件感知的GRPO训练框架，通过先定位父物体再定位部件并自检修正，有效提升了部件级视觉定位的精度。

Abstract: Multimodal large language models (MLLMs) ground whole objects well from free-form language queries, but they struggle when the query names a part rather than the object. We trace this to a missing object-part hierarchy, since parts are localized in the same single step used for objects. We propose Object-Part Hierarchical Reflective Grounding (OP-HRG), a coarse-to-fine reasoning-guided grounding strategy that first localizes the parent object and then the part within it. A self-check then reflects on the result, with an extension to re-encode the predicted crop to inspect the region it is correcting. We introduce a part-aware GRPO framework to train our pipeline with stage-wise rewards. A 4B model trained this way outperforms 7B grounding LLMs and SAM3 across PascalPart, PartImageNet, and InstructPart, and transfers to reasoning segmentation.

[16] Training-Free Open-Vocabulary 3D Point-Cloud Segmentation on the Generalized Few-Shot Benchmark cs.CVPDF

Silas kwabla Gah, Ebenezer Owusu

TL;DR: 本文提出了一种无需训练、开放词汇的3D点云分割方法，用于解决广义少样本3D点云分割（GFS-PCS）问题。该方法通过结合冻结的3D视觉语言模型（RegionPLC）和冻结的可提示概念分割器（SAM3），仅使用新颖类别的名称和RGB视图，并利用跨视图一致性进行融合，实现了无需3D标注、无需逐场景训练、甚至无需少样本支持样本的分割。

Details

Motivation: 现有GFS-PCS方法依赖密集但嘈杂的3D视觉语言先验与少样本支持样本的融合，但需要3D标注、逐场景训练和支持样本标注。本文旨在探索在不使用这些监督信息的情况下，仅通过跨视图一致性融合两个冻结模型，能在多大程度上解决开放词汇的3D分割问题。

Result: 在ScanNet200 GFS-PCS基准测试中，该无需训练的方法将新颖类别的mIoU提升了+2.6，同时基础类别精度保持在0.5以内，并恢复了与使用更多监督的SOTA方法之间33%的新颖类别性能差距。在更难的ScanNet++基准上，新颖类别mIoU几乎翻倍（从16.2提升至31.9，+15.7），谐波均值从21.5提升至31.1。

Insight: 创新点在于提出了一种完全无需训练的开放词汇3D分割框架，通过跨视图一致性融合两个预训练模型，无需3D标注、逐场景训练或少样本支持。客观分析表明，该方法揭示了在GFS-PCS任务中，仅依靠跨视图一致性可能比融合少样本支持更有效，从而简化了分割流程并减少了监督依赖。

Abstract: Generalized few-shot 3D point-cloud segmentation (GFS-PCS) asks a model to segment a scene into many base classes seen at training time and a set of novel classes. The state of the art reaches novel classes by reconciling a dense but noisy 3D vision-language prior with the few-shot support, but it pays for this with base 3D labels, per-episode training, and the support annotations themselves. We ask how far the same reconciliation can go with none of these: no training, no 3D labels, and not even the few-shot support. We pair a frozen 3D vision-language model (RegionPLC) as a dense prior with a frozen promptable concept segmenter (SAM3), prompted by the bare novel class names and lifted from posed RGB views, and reconcile the two by cross-view consistency: a point becomes novel only when enough of the views that see it agree. On the ScanNet200 GFS-PCS benchmark this fully training-free, open-vocabulary pipeline improves novel mIoU by +2.6 over the training-free dense prior while holding base accuracy within 0.5, and recovers a third (33%) of the novel-class gap to the trained state of the art that uses far more supervision. We further show that injecting the few-shot support into the pipeline, as a fusion gate and as a prototypical dense classifier, adds nothing over consistency alone and in fact degrades it through the classifier, which is why the method needs no support at all. On the harder ScanNet++ benchmark, where the dense prior is far weaker on novel classes, the same pipeline nearly doubles novel mIoU (+15.7, from 16.2 to 31.9) at a 1.7 base cost, lifting the harmonic mean from 21.5 to 31.1

[17] Explicit Over Implicit: Enhancing CNNs Via Complex Structure Tensor Representations for Periocular Recognition cs.CVPDF

Kevin Hernandez-Diaz, Josef Bigun, Fernando Alonso-Fernandez

TL;DR: 该论文提出使用包含方向特征及其置信度的复数结构张量作为CNN输入，以增强CNN在虹膜周边识别任务中的方向特征提取能力。实验表明，该方法结合小型复数卷积网络和精简的CNN架构，在公开数据集上相比传统灰度输入显著提升了识别准确率。

Details

Motivation: CNN在有效提取方向特征方面存在困难，而哺乳动物视觉系统利用显式方向先验的策略可借鉴。

Result: 在Cross-Eyed和PolyU虹膜周边图像数据集上，使用六种CNN架构进行生物识别验证，该方法在闭集和开集场景下使等错误率降低5-26%，性能优于主流完整CNN架构。

Insight: 通过复数结构张量显式提供方向先验特征，能缓解CNN的表征限制，同时提升模型可解释性并适用于轻量化客户端场景。

Abstract: Our study provides evidence that CNNs struggle to extract orientation features effectively. We show that using the Complex Structure Tensor, which contains compact orientation features with certainties, as input to CNNs consistently improves identification accuracy compared to grayscale inputs alone. Experiments also demonstrated that our inputs, provided by mini-complex convnets, combined with reduced CNN sizes, outperformed full-fledged, prevailing CNN architectures. This suggests that the upfront use of orientation features in CNNs, a strategy seen in mammalian vision, not only mitigates their limitations but also enhances their explainability and relevance to thin-clients. Experiments were conducted on publicly available datasets comprising periocular images (Cross-Eyed and PolyU) for biometric identification and verification in both Close-World and Open-World Scenarios using six CNN architectures. Our experiments on the Cross-Eyed and PolyU datasets yield a 5-26% reduction in EER, providing strong empirical evidence that explicit orientation priors mitigate CNN representational limits in Open-World and Close-World scenarios.

[18] Unsupervised Keypoints for Real-Time Fall Detection: Comparative Analysis Under Real-world Conditions with Predictive Bandwidth Reduction cs.CV | cs.LGPDF

Tasmiah Haque, Jacob Kosinski, Sumit Mohan, Srinjoy Das, Mohammad Abdullah Al-Mamun

TL;DR: 本文提出了一种基于无监督关键点和预测性时间建模的隐私保护框架，用于实时跌倒检测。该框架通过局部处理进行分割和关键点提取，并利用变分循环预测和序列分类从观测和预测的运动中检测跌倒。在UR Fall Detection和Human Fall数据集上，通过随机、受试者独立和基于遮挡的划分进行评估，结果表明无监督关键点在遮挡和部分身体可见的情况下更具鲁棒性。

Details

Motivation: 解决老年人跌倒这一重大安全挑战，同时应对连续视频监控在隐私、计算和带宽方面的部署限制。

Result: 在随机划分下，无监督和监督关键点表现相当；在受试者独立评估中，监督关键点具有统计显著优势，但对遮挡敏感；在基于遮挡的评估中，无监督关键点保持高灵敏度，显著优于监督关键点（后者漏检近一半跌倒）。在带宽受限条件下，无监督关键点的优势进一步扩大。

Insight: 创新点在于利用无监督关键点生成紧凑的运动表示以保护隐私并减少带宽，其解剖独立性使其在身体可见性受损时（如遮挡）比监督关键点更鲁棒。研究强调表示选择应反映预期的视觉条件，为实际部署提供了重要指导。

Abstract: Falls among older adults are a major safety challenge, but continuous monitoring is difficult to sustain. Video captures fall-related posture and motion, yet deployment is limited by privacy, computation, and bandwidth. Supervised pose estimation is anatomically interpretable but vulnerable to occlusion and partial body visibility. We propose a privacy-preserving framework that replaces RGB transmission with compact motion representations based on unsupervised keypoints and predictive temporal modeling. Local processing performs segmentation and keypoint extraction; variational recurrent prediction and sequence classification then detect falls from observed and forecasted motion. We evaluate the framework on the UR Fall Detection and Human Fall datasets using random, subject-disjoint, and occlusion-based splits. Under random splits, neither representation consistently dominates, suggesting that standard protocols may hide meaningful differences. Under subject-disjoint evaluation, supervised keypoints show a statistically significant advantage, but performance varies by subject: they perform better when anatomical landmarks are visible, whereas unsupervised keypoints are more robust to occlusion and partial visibility, though they produce more false positives for complex activities. Under occlusion-based evaluation, supervised keypoints miss nearly half of all falls, while unsupervised keypoints retain strong sensitivity and substantially outperform them. Their anatomical independence allows spatial anchors to adapt to visible body structure rather than fail on absent landmarks. The gap widens under bandwidth constraints, where supervised localization errors compound through the temporal model. These findings show that representation choice should reflect expected visual conditions and that unsupervised keypoints offer an advantage when body visibility is compromised.

[19] Intentional Electromagnetic Interference Attacks on Facial Recognition cs.CV | cs.CR | cs.LGPDF

Tyler Fitzsimmons, Adam Czajka

TL;DR: 本文研究了针对面部识别系统的故意电磁干扰攻击，揭示了生物识别传感器在物理到数字转换过程中的关键漏洞。论文提出了一种标准化的测试方法，用于评估面部识别系统对硬件攻击的鲁棒性，并创建了一个包含电磁干扰下采集的人脸图像数据集作为新的基准。

Details

Motivation: 动机在于揭示生物识别传感器（尤其是面部识别系统）在物理到数字转换管道中的脆弱性，超越传统的呈现攻击（如ISO/IEC 30107标准所定义），探索硬件层面的电磁干扰攻击风险。

Result: 实验表明，使用常见射频设备即可实施故意电磁干扰攻击，并评估了最先进面部识别方法对此类攻击的鲁棒性；同时提供了一个包含干扰与非干扰条件下人脸图像的数据集，作为测试现代人脸匹配器的新基准。

Insight: 创新点在于将攻击从数字域扩展到物理硬件层面，通过电磁干扰揭示面部识别系统的潜在漏洞，并提供标准化测试框架和基准数据集，为生物识别安全研究提供了新的方向。

Abstract: Attacks on general computer vision algorithms are often relegated to the digital domain, with the optimization performed purely in the digital world and then translated to physical mediums for implementation. In the field of biometrics, including facial recognition, physical presentation attacks targeting biometric sensors are dominant and present significant opportunity and risk. This paper highlights a critical vulnerability in the physical-to-digital pipeline of biometric sensors and provides a standardized approach for testing facial recognition system robustness against hardware attacks, going beyond and potentially complementing presentation attacks (as defined in ISO/IEC 30107 standard series). Specifically, in this work we (a) demonstrate that intentional electromagnetic interference is possible to be conducted with commonly accessible radio frequency (RF) equipment, (b) assess the robustness of state-of-the-art face recognition methods against RF-based attacks, and (c) provide a dataset composed of face images captured with and without electromagnetic interference to serve as a new benchmark for testing modern face matchers against RF-sourced interference.

[20] Trajectory-aware Cross-view Geo-localization with Sequential Observations cs.CVPDF

Tianyi Gao, Jiayu Lin, Danielle Beaulieu, Nathan Jacobs

TL;DR: 本文提出SeqGeo-VL数据集和TrajLoc统一框架，用于跨视角地理定位，同时处理视频片段和路线描述文本，通过轨迹感知模块增强匹配性能。

Details

Motivation: 现有方法主要关注视频序列，忽略了路线描述文本这一互补的序列模态，而后者在自动驾驶等场景中往往是唯一可用输入，因此需要统一处理这两种模态以提升地理定位能力。

Result: 在SeqGeo-VL数据集上，TrajLoc在视频和文本地理定位任务上均显著优于现有最先进方法，实现了SOTA性能。

Insight: 创新点包括引入视频-文本-卫星三元组数据集，以及TrajMod轻量级模块，该模块通过轨迹几何条件化查询嵌入，生成空间感知表示，使视觉和语言模态能够相互增强跨视图匹配。

Abstract: Cross-view geo-localization matches ground-level observations against geo-tagged satellite imagery. Recent methods show that sequential queries such as video clips yield richer spatiotemporal cues than single images, yet they overlook a complementary sequential modality: route descriptions – which capture the same trajectory at a higher level of abstraction and are often the only input available (e.g., a user directing an autonomous vehicle to a pickup point). To bridge this gap, we introduce SeqGeo-VL, a dataset of $\sim$39K video-text-satellite triplets, and TrajLoc, a unified framework capable of processing both video clips and route descriptions. By leveraging both dense visual and abstract linguistic semantics, TrajLoc enables these modalities to mutually reinforce cross-view matching. We further propose TrajMod, a lightweight module that conditions query embeddings on trajectory geometry, yielding spatially-aware representations. Experiments show that TrajLoc achieves substantial gains over state-of-the-art methods on both video and text geo-localization. The project page is available at https://humblegamer.github.io/trajloc/.

[21] E3DGS: Unified Geometric-Photometric Equivariance for 3D Gaussian Splatting via Color-as-Geometry Embedding cs.CV | cs.LGPDF

Chankyo Kim, Maani Ghaffari

TL;DR: 本文提出E3DGS，一种用于3D高斯泼溅（3DGS）的刚性SE(3)等变架构。核心创新在于通过表示理论，将颜色（球谐系数）视为几何张量，并引入统一矩阵嵌入将所有高斯属性映射到统一的载体空间gl(3)，从而解决了几何与外观属性在相机坐标系变化下对称性不统一的问题。

Details

Motivation: 现有3DGS方法将颜色仅视为信号而非几何实体，导致几何（位置、协方差）与外观（球谐函数）属性在相机旋转下的变换规则不一致（如旋转对位置、协方差和球谐系数的作用方式不同），这破坏了严格的等变性，使得构建SE(3)等变架构存在根本性的表示瓶颈。

Result: 在物体视觉和动作条件高斯世界建模任务上的评估表明，该方法在相机坐标系变化下展现出更强的鲁棒性，并提高了数据效率。

Insight: 主要创新点在于从表示理论出发，证明了低阶（ℓ≤2）球谐系数在代数上同构于二阶几何张量，从而可将颜色视为几何（Color-as-Geometry），并通过统一矩阵嵌入在gl(3)空间中统一处理所有属性，避免了复杂的Clebsch-Gordan张量积运算，实现了简洁的SE(3)等变架构。

Abstract: 3D Gaussian Splatting (3DGS) captures scenes by coupling explicit geometry (position, covariance) with view-dependent photometry (Spherical Harmonics). However, building $\mathrm{SE}(3)$-equivariant architectures on these primitives presents a fundamental representation bottleneck. Color has been treated as a signal rather than a geometric entity, making it nontrivial to unify symmetry across geometry and appearance as the camera frame changes. While translations are handled by relative coordinates, rotations act heterogeneously across attributes: $μ\mapsto Rμ$, $Σ\mapsto RΣR^\top$, and $f_\ell\mapsto D^\ell(R)f_\ell$. This mismatch complicates strict equivariance, leading existing methods to either discard or flatten SH coefficients, thereby breaking symmetry. We propose a unified solution rooted in representation theory: for SH degrees $\ell\le2$, photometry is algebraically isomorphic to a rank-2 geometric tensor. We prove that the Wigner-$D$ action on these SH coefficients can be exactly reformulated as the conjugation action on $3\times3$ matrices. Leveraging this, we introduce the Unified Matrix Embedding, a lifting that maps all Gaussian attributes into a unified carrier space, $\mathfrak{gl}(3)$. Building on the “Color-as-Geometry” formulation, we present E3DGS, a rigid-body ($\mathrm{SE}(3)$) equivariant architecture that processes 3D Gaussians without Clebsch-Gordan tensor products. Evaluations on object vision and action-conditioned Gaussian world modeling demonstrate that our unified approach yields strong robustness under camera-frame changes and improved data efficiency.

[22] SLAPBench: Benchmarking Multimodal Large Language Models for Four-Finger SLAP Fingerprint Verification cs.CV | cs.AIPDF

Bibesh Pyakurel, M. G. Sarwar Murshed

TL;DR: 本文提出了首个用于评估多模态大语言模型在四指SLAP指纹验证任务中性能的基准测试SLAPBench。该基准基于NIST SD302b数据集构建，包含7,832对指纹图像。研究评估了包括开源模型和闭源模型在内的五种MLLM在零样本、任务描述和相似度评分三种提示策略下的表现，发现提示策略影响模型崩溃行为，而模型能力决定其判别性能。

Details

Motivation: 目前缺乏评估多模态大语言模型能否从四指SLAP指纹图像中进行身份验证的基准，SLAP指纹在边境管控和执法中应用广泛，因此需要建立专门的评估标准。

Result: 在NIST SD302b基准上，不同模型和提示策略结果差异显著：Claude Opus 4.8在二元提示下表现最佳（FAR=20.2%），Qwen3-VL-8B在相似度评分下达到完美分离（AUC=1.000），而其他模型如InternVL3-8B和Qwen2.5-VL-7B表现接近随机。公平性探测显示，随着判别能力减弱，模型在不同人口属性上的差异会增大。

Insight: 论文的创新点在于构建了首个SLAP指纹验证的MLLM基准，并揭示了提示策略对模型崩溃行为的关键影响以及模型内在能力对判别性能的决定作用。同时，研究指出完美性能可能源于数据集的特定结构（如近重复检测），为未来基准设计提供了诊断视角。

Abstract: Four-finger SLAP fingerprints are flat live-scan impressions of the index, middle, ring, and little fingers of one hand, used for identity verification in border control and law enforcement. No benchmark has evaluated whether multimodal large language models (MLLMs) can verify identity from SLAP images. We introduce SLAPBench, the first benchmark for MLLM-based four-finger SLAP fingerprint verification, built from NIST SD302b with 7,832 pairs (176 mated, 7,656 non-mated). We evaluate four open-source MLLMs (InternVL3-8B, Qwen2.5-VL-7B, Qwen3-VL-8B, Gemma-3-12B) and the proprietary Claude Opus 4.8 under zero-shot, task-description, and similarity-scoring prompts. Prompting governs verification behavior. Task-description prompting collapses all four open-source models to near-100% False Accept Rate (FAR), and Gemma-3-12B collapses under zero-shot as well; Claude Opus 4.8 alone resists collapse under both binary prompts, giving the best binary result (FAR = 20.2%). Similarity scoring removes collapse across the open-source models and exposes wide capability gaps: Claude reaches AUC = 0.953 and Gemma-3-12B 0.837, while InternVL3-8B is inverted (AUC = 0.590) and Qwen2.5-VL-7B near random (0.567). Qwen3-VL-8B attains perfect separation (AUC = 1.000), which we treat as a diagnostic rather than as capability: SD302b holds one SLAP capture per finger position, so mated pairs are cross-resolution. A matched-resolution control leaves the perfect score intact, ruling out the resolution shortcut; what cannot be excluded within SD302b is near-duplicate detection, since a mated pair is one capture rendered twice. A fairness probe over gender, race, and age suggests disparity grows as discrimination weakens. SLAPBench establishes the first SLAP-specific MLLM baseline and shows that prompting governs collapse while model capability governs discrimination.

[23] When Can Test-Time Adaptation Help Zero-Shot CT Vision-Language Models? cs.CVPDF

Ailar Mahdizadeh, Puria Azadi Moghadam, Xiangteng He, Leonid Sigal

TL;DR: 本文研究了测试时适应（TTA）在何种条件下能提升零样本3D CT视觉语言模型（VLM）的性能。论文通过诊断分析指出TTA的有效性是有条件的，并针对基础模型已具备区分能力的情况，提出了首个用于此场景的TTA方法CARVE，该方法通过估计正标签基数并进行高效的多视图适应来改进模型。

Details

Motivation: 零样本3D CT VLM能在标签稀缺、临床任务快速变化时实现跨机构部署，但其在分布偏移下的多标签预测可靠性未知，且现有TTA方法不适用于零样本3D CT VLM的无监督多标签适应问题。

Result: 在对比模型CT-CLIP和解剖感知模型fVLM上，当基础模型已具备判别能力时，CARVE在多标签、三分类和二元CT任务上均能提供最一致的性能提升。

Insight: 论文的创新点在于首次明确了零样本3D CT VLM的多标签TTA是一个独特问题，并提出了CARVE这一基数感知的解决方案，其核心是通过估计样本特定正标签基数并优化top-k目标来保留共现异常，同时采用无需梯度的弱3D视图评分以实现内存高效的多视图适应。

Abstract: 3D CT vision-language models (VLMs) classify abnormalities from text prompts in a zero-shot manner, enabling cross-institution deployment where labels are scarce and clinical tasks shift faster than supervised models can be retrained. A real CT scan, however, typically contains several co-occurring abnormalities, and the reliability of zero-shot multi-label prediction under distribution shift remains poorly understood. Test-time adaptation (TTA) updates a model on unlabeled target scans without source data or target annotations, yet existing TTA methods target multi-class softmax prediction on natural images or 2D medical segmentation, and none addresses unsupervised multi-label adaptation for zero-shot 3D CT VLMs. We study when TTA helps zero-shot 3D CT VLMs. A controlled diagnostic analysis shows that TTA is conditional: the volumetric input must preserve the encoder’s depth structure, and the base representation must transfer to the target cohort, with depth reduction alone lowering internal AUROC by more than 0.12. We then focus on the regime where the base model already separates present from absent abnormalities. We introduce CARVE (Cardinality-Aware Retained-View Entropy), the first TTA method for this setting. CARVE estimates a sample-specific positive-label cardinality $\hat{k}$, optimizes a top-$\hat{k}$ objective to preserve co-occurring abnormalities, and performs memory-efficient multi-view adaptation by scoring weak 3D views without gradients before updating on a retained subset. Across contrastive CT-CLIP and anatomy-aware fVLM, CARVE provides the most consistent improvements across multi-label, three-class, and binary CT tasks when the base model is already discriminative. These results establish multi-label TTA for zero-shot 3D CT VLMs as a distinct problem and CARVE as a cardinality-aware solution.

[24] Are All Tokens Necessary for Visual Place Recognition? An Empirical Study of Token Reduction for Efficient Inference cs.CVPDF

Tong Jin, Yunpeng Liu, Shuyu Hu, Qinghua Zhang, Ruize Han

TL;DR: 本文对视觉位置识别（VPR）中基于视觉Transformer的模型进行了首次系统的令牌缩减基准测试，旨在提升推理效率。研究评估了令牌剪枝、令牌合并及混合方法在多种SOTA VPR模型和多样化数据集上的表现，揭示了令牌缩减在计算成本、推理速度和精度之间的权衡。

Details

Motivation: 基于视觉Transformer的VPR方法虽然性能卓越，但处理所有视觉令牌会带来巨大的计算开销，阻碍其在实时和资源受限场景中的部署，因此需要探究是否所有令牌都是必要的。

Result: 实验表明，令牌缩减可将计算成本降低高达29%，吞吐量提升高达44%，同时识别精度下降不到1%，在多个基准数据集上保持了接近原始模型的性能。

Insight: 创新点在于首次为VPR建立了系统的令牌缩减基准，并从多角度（如不同缩减配置、计算复杂度、边缘设备部署）提供了实用见解，为未来高效视觉检索系统的研究奠定了基础。

Abstract: Recent visual place recognition (VPR) methods based on vision transformers, particularly foundation models, have achieved remarkable recognition performance. However, these models process all visual tokens throughout the entire network, resulting in substantial computational overhead, which hinders their deployment in real-time and resource-constrained scenarios. A natural question thus arises: are all visual tokens necessary for VPR? To answer this question, we present the first systematic benchmark of token reduction for efficient visual place recognition. Our benchmark comprehensively evaluates representative token pruning, token merging, and hybrid pruning-merging methods across multiple state-of-the-art VPR models and diverse benchmark datasets covering urban, suburban, and natural environments. We further investigate token reduction from multiple perspectives, including recognition performance under different reduction configurations, computational complexity, inference speed, qualitative visualization, and deployment efficiency on edge devices. Through extensive experiments and in-depth analysis, our benchmark reveals multiple important characteristics of token reduction in VPR and provides several practical insights into the trade-offs between accuracy and inference efficiency. For example, token reduction can reduce computational cost by up to 29% and improve throughput by up to 44%, while incurring less than 1% degradation in recognition accuracy. Overall, this work establishes a comprehensive foundation for future research on token-efficient VPR and efficient visual retrieval systems. Our codes and models will be available at https://github.com/Tong-Jin01/TokenReduction4VPR

[25] Ask Twice, Look Twice: Prompt Echoing Resolves the Question-First Paradox in Vision-Language Models cs.CV | cs.AI | cs.LG | eess.IVPDF

Rakshanda Hassan Abhinandan, John Galeotti, Deva Ramanan, Gautam Rajendrakumar Gare

TL;DR: 本文研究了视觉语言模型（VLM）提示中问题与图像顺序的悖论：直觉上问题在前能引导模型关注，但实际基准测试中图像在前性能更优。作者发现这是由于感知引导与答案读取两个阶段存在冲突，并提出了一种无需训练的解决方案——‘提示回响’，即在图像前后都重复问题（或图像），从而同时实现有效引导和答案读取。该方法在多个VLM和基准测试上显著提升了性能。

Details

Motivation: 动机是解决视觉语言模型提示中的‘问题优先悖论’：直觉上问题在前应能引导模型关注相关图像区域，但实际基准测试中图像在前的提示顺序性能更好，作者旨在探究这一矛盾的根本原因并找到解决方案。

Result: 在五个开源VLM上，该悖论导致性能下降高达17.5个组准确率点。提出的‘回响提示’方法在NaturalBench、POPE、Winoground和开放式VQAv2等基准上超越了最佳单次排序的性能，在Winoground上提升高达19个组准确率点，且无需训练、微调或架构更改。

Insight: 核心创新点是揭示了VLM中感知引导与答案读取的权衡冲突，并借鉴人类阅读中‘附加问题’的心理学发现，提出通过纯提示设计（即‘回响’）来同时满足两个阶段的需求。这是一种无需模型修改即可显著提升性能的简单有效方法。

Abstract: Where should the question go in a vision-language model (VLM) prompt: before the image or after it? Intuition says before: knowing what is asked should tell the model where to look. Yet across visual question answering benchmarks, question-first prompting consistently underperforms the image-first ordering recommended for frontier VLMs, a phenomenon we term the question-first paradox. We trace the paradox to a conflict between two stages of VLM computation. Logit-lens and attention probes show the intuition is half right: a question placed before the image genuinely steers perception, moving image patch representations toward question-relevant concepts. The failure lies downstream. Stranded behind hundreds of image tokens, the question is barely attended by the answer token, which instead commits to image-driven (often wrong) answers; a causal attention knockout confirms that the answer reads the question only when the question follows the image. The diagnosis yields a training-free fix: question echoing, restating the question on both sides of the image so that one copy steers perception while the other is read out at answer time. The same division of labor appears in a fifty-year-old finding on human ``adjunct questions’’, where repeating a question before and after a passage aids comprehension more than either position alone. Echoing the image as well brings further gains, restoring the whole-image view a causal decoder otherwise loses. The paradox holds across five open VLMs, costing up to 17.5 group-accuracy points. Echoed prompts close it and surpass the best single-pass ordering on NaturalBench, POPE, Winoground, and open-ended VQAv2, by up to 19 Winoground group-accuracy points, with no training, fine-tuning, or architecture change. The paradox reveals a trade-off between steering perception and preserving question access; echoing resolves it through prompt design alone.

[26] Region-Grounded Vision-Language Learning for Detection-Guided Mammographic Lesion Classification cs.CVPDF

Zhengbo Zhou, Jiren Li, Dooman Arefan, Margarita Zuley, Shandong Wu

TL;DR: 本文提出了一种区域锚定的视觉语言学习方法，用于检测引导的乳腺X光病灶分类。该方法通过区域-文本对比预训练对齐病灶特征与临床描述，并引入多组件目标来缓解语义坍缩和背景偏差。同时，联合优化辅助病灶检测头以保持空间敏感性，实现定位感知的恶性分类。

Details

Motivation: 传统全局图像-文本对齐方法不适用于乳腺X光分析，因为诊断相关的病灶空间局部化且仅占图像一小部分，全图像级别的表示学习会稀释关键的形态学线索。

Result: 在CBIS-DDSM和VinDr-Mammo两个独立数据集上的广泛实验表明，该方法在域内、跨数据集和迁移学习设置下均优于相关方法。

Insight: 创新点包括：1）区域-文本对比预训练模拟放射科医生的诊断范式；2）多组件目标（正对齐、细粒度语义硬负样本和背景抑制）缓解低词汇量设置下的语义坍缩；3）联合优化检测头实现定位感知分类，提升空间敏感性。

Abstract: Vision-language models trained with contrastive objectives have shown promise in medical image analysis. However, conventional global image-text alignment is ill-suited for mammography, where diagnostically relevant lesions are spatially localized and occupy only a small fraction of the image. Subtle morphological cues critical for malignancy assessment can be diluted when representations are learned at the whole-image level. In this work, we propose a novel region-grounded vision-language learning method for detection-guided mammographic lesion classification. The method mirrors radiologists’ diagnostic paradigm. First, a region-text contrastive pretraining stage aligns lesion-specific features with structured clinical descriptors derived from radiology metadata. To mitigate semantic collapse and background bias in low-vocabulary settings, we introduce a multi-component objective incorporating positive alignment, fine-grained semantic hard negatives, and background suppression. Second, an auxiliary lesion detection head is jointly optimized with contrastive classification to preserve spatial sensitivity and enable localization-aware malignancy classification. Extensive experiments on two independent datasets, CBIS-DDSM and VinDr-Mammo, show superior performance of our method compared to related methods under in-domain, cross-dataset, and transfer learning settings.

[27] Benchmarking MRI Representations for Deep Learning-Based Focal Cortical Dysplasia Segmentation cs.CVPDF

Soumen Ghosh, John Phamnguyen, Amit Soni Arya, Subhojit Mandal, Tilottama Goswami

TL;DR: 本研究系统性地评估了不同MRI表征（包括T1加权、FLAIR图像、比值衍生表征及其多模态组合）在基于nnU-Net框架的局灶性皮质发育不良（FCD）自动分割任务中的表现。研究发现，FLAIR单模态表现最佳，而比值衍生表征单独使用效果不佳；将比值衍生表征与常规T1w和FLAIR图像结合能持续提升分割性能，其中四通道多模态配置取得了最高的Dice分数（0.376）。

Details

Motivation: 局灶性皮质发育不良（FCD）是耐药性局灶性癫痫的主要结构性病因之一，但其在常规MRI上的成像特征细微且异质，使得准确识别和勾画具有挑战性。尽管T1w和FLAIR图像在术前评估中常规采集，但不同MRI表征对基于深度学习的FCD分割的贡献尚不清楚。

Result: 在公开的包含85名FCD患者和25名健康对照的术前MRI数据集上，使用nnU-Net框架和五折交叉验证进行评估。四通道多模态配置（结合T1w、FLAIR和比值衍生表征）取得了最高的整体Dice分数（0.376），相对于传统的T1w+FLAIR表征有5.0%的相对提升。

Insight: 论文的创新点在于首次系统性地对MRI表征在深度学习FCD分割中的作用进行了基准测试，并强调了MRI表征设计是与网络架构同等重要的、有待充分探索的组成部分。客观来看，其通过控制变量（固定预处理、网络和优化策略）来孤立评估表征影响的方法具有借鉴意义，同时证明了多模态融合（尤其是结合比值衍生信息）能有效提升对细微病变的勾画能力。

Abstract: Focal cortical dysplasia (FCD) is one of the leading structural causes of drug-resistant focal epilepsy, yet its subtle and heterogeneous imaging characteristics make accurate identification and delineation challenging on conventional magnetic resonance imaging (MRI). Although T1-weighted (T1w) and fluid-attenuated inversion recovery (FLAIR) images are routinely acquired for presurgical evaluation, the contribution of different MRI representations to deep learning-based FCD segmentation remains poorly understood. In this study, we present a systematic benchmark of MRI representations for automated FCD segmentation using the nnU-Net framework. A publicly available presurgical MRI dataset comprising 85 FCD subjects and 25 healthy controls was used to evaluate eight input configurations, including conventional MRI contrasts (T1w and FLAIR), ratio-derived representations, and their multimodal combinations. To isolate the effect of MRI representation, all experiments employed identical preprocessing, network architecture, optimization strategy, and five-fold cross-validation. Among the evaluated single-modality representations, FLAIR achieved the strongest overall performance, whereas ratio-derived representations alone were insufficient for reliable identification of subtle FCD. Incorporating ratio-derived representations with conventional T1w and FLAIR images consistently improved lesion delineation, with the four-channel multimodal configuration achieving the highest overall Dice score (0.376), representing a 5.0% relative improvement over the conventional T1w+FLAIR representation. These findings demonstrate that MRI representation design is an important yet underexplored component of deep learning-based FCD segmentation and should be optimized alongside network architecture.

[28] Model Merging for Medical LVLMs: A Benchmark and a Winner-Take-All Approach cs.CVPDF

Lichao Mou, Shilan Zhang, Chunlei Li, Bingcong Yan, Jingliang Hu

TL;DR: 本文首次系统研究了医学大视觉语言模型（LVLM）的模型合并问题，提出了一个名为MergeMedBench的综合性基准测试，涵盖八种成像模态和多样化的临床任务类型，并基于两种主流架构构建了16个LoRA微调模型。论文评估了现有合并方法，并提出了一种名为’赢家通吃’的简单且无需超参数的方法，该方法通过保留专家模型中最具主导性的参数来避免信息稀释，在基准测试中表现优于现有方法。

Details

Motivation: 动机在于解决医学领域部署多个针对特定成像模态和临床场景的专家LVLM模型所带来的巨大计算和运营开销问题，模型合并提供了一种无需重新训练即可将多个专家模型整合为单一模型的解决方案，但该领域在医学领域尚未得到充分探索。

Result: 在提出的MergeMedBench基准上进行广泛评估，结果表明，所提出的’赢家通吃’方法在无需超参数的情况下，持续优于现有的基于平均或对齐的策略，为未来研究提供了一个强大的实用基线。

Insight: 创新点在于首次系统性地为医学LVLM建立模型合并基准，并提出了一种新颖的’赢家通吃’合并策略，其核心洞察是保留主导参数、丢弃较弱参数可以避免信息稀释，这为LoRA合并提供了新视角，方法简单有效。

Abstract: Large vision-language models (LVLMs) can be adapted to specialized medical imaging tasks via parameter-efficient fine-tuning approaches such as low-rank adaptation (LoRA), leading to a growing ecosystem of expert models tailored to specific imaging modalities and clinical scenarios. However, deploying multiple expert LVLMs in practice incurs substantial computational and operational overhead. Model merging provides a promising solution by consolidating multiple experts into a single model without retraining, yet it remains largely unexplored in the medical domain. In this work, we present the first systematic study of model merging for medical LVLMs. We introduce MergeMedBench, a comprehensive benchmark spanning eight imaging modalities and diverse clinical task types, comprising 16 LoRA fine-tuned models built upon two mainstream architectures. We conduct an extensive evaluation of existing merging methods and further propose winner-take-all, a simple and hyperparameter-free approach that retains only the most dominant parameters across expert models. By preserving the critical parameters that govern model behavior and discarding weaker ones, our method avoids the information dilution inherent in averaging- or alignment-based strategies. Despite its simplicity, winner-take-all consistently outperforms existing approaches, offering both a new perspective on LoRA merging and a strong practical baseline for future research.

[29] Efficient Frame Selection for Long Videos at Test Time with Attention-Based MLLM Selectors cs.CVPDF

Yilin Wang, Xiangxi Zheng, Dongxing Mao, Linjie Li, Zhengyuan Yang

TL;DR: 本文提出了一种无需训练的帧选择方法DAFS，利用MLLM中跨模态注意力机制提取查询相关帧证据，通过动态规划优化候选池大小和每帧token预算分配，在32帧预算下显著提升长视频理解性能。

Details

Motivation: 解决长视频理解中MLLM需要从数千帧中高效选择关键帧的循环依赖问题，即选择帧需要先理解视频内容，但理解又依赖于帧选择。

Result: 在Video-MME基准测试中，相比均匀采样提升高达6.4个点，在相同帧预算下优于基于训练的选择器，且无需重新训练即可跨骨干网络和任务泛化。

Insight: 创新点在于发现MLLM验证层注意力可直接提供查询相关证据，无需自回归生成；通过注意力聚合实现跨帧比较，并首次将候选池大小和token预算联合建模为离散优化问题。

Abstract: Understanding long videos with multimodal large language models (MLLMs) requires selecting a compact set of frames from thousands of candidates, yet identifying the right frames seemingly requires understanding the video first. We resolve this circular dependency with a simple observation: cross-modal attention at validation-selected extraction layers in MLLMs already provides query-relevant frame evidence without requiring autoregressive generation. We exploit this property to build DAFS (Dynamic Attention-based Budget-aware Frame Selection), a training-free frame selector. A lightweight MLLM selector, even with only 2B parameters, can extract frame-level evidence by converting selected-layer attention into relevance scores through query-conditioned aggregation. This enables cross-frame comparison without autoregressive decoding. To handle the selector’s own context constraint, we formulate the joint allocation of candidate pool size and per-frame token budget as a discrete optimization problem solved by dynamic programming. Under a 32-frame budget, our selector improves over uniform sampling by up to 6.4 points on Video-MME and outperforms prior training-based selectors under matched frame budgets, while generalizing across selector and answerer backbones, and across tasks, without retraining.

[30] PE-Field 4D: Video Generation Models as Canvas cs.CVPDF

Yunpeng Bai, Haoxiang Li, Qixing Huang

TL;DR: 本文提出PE-Field 4D，一种基于扩散Transformer的视频生成框架，通过引入几何感知的交叉注意力机制和投影位置编码，实现了在视角变化和相机运动下对场景几何的精确控制。该方法在相机轨迹重规划、新视角视频合成和几何感知视频编辑等任务中提升了空间可控性，同时保持了基础视频扩散模型的生成先验。

Details

Motivation: 当前扩散Transformer在视频生成中难以控制视角变化和相机运动下的场景几何，本文旨在通过重新审视位置编码的作用，利用其提供的空间偏置来实现几何感知的控制。

Result: 该方法在视角相关的编辑任务（如相机轨迹重规划、新视角视频合成和几何感知视频编辑）中提高了空间可控性，同时保持了底层视频扩散模型的生成能力，代码已开源。

Insight: 创新点在于提出了几何感知交叉注意力机制和投影位置编码方案，使目标视频潜在令牌能够关注来自参考图像或帧的结构化上下文令牌，并通过目标视图的2D重投影和深度感知消歧建立对应关系，从而在保持视频模型原生潜在结构一致性的同时注入几何引导。

Abstract: Diffusion Transformers have recently achieved strong performance in video generation, yet controlling scene geometry under viewpoint changes and camera motion remains challenging. In this work, we revisit the role of positional encoding in video diffusion transformers and show that it provides a useful spatial bias for geometry-aware control. Specifically, if reference tokens are encoded according to their projected locations in the target view, the denoising model is encouraged to retrieve content from position aligned regions of the input video. Building on this observation, we introduce a geometry-aware cross-attention mechanism that enables target video latent tokens to attend to structured context tokens derived from reference images or frames. To establish correspondence between the reference content and the target camera trajectory, we equip the context tokens with a projected positional encoding scheme that combines target-view 2D reprojection with depth-aware disambiguation. At the same time, we preserve the original spatiotemporal positional encoding of the generated video latent, allowing geometric guidance to be injected while maintaining consistency with the video model’s native latent structure. The resulting framework provides a simple and effective approach for controllable video generation. It improves spatial controllability in viewpoint-dependent editing tasks, including camera re-trajectory, novel-view video synthesis, and geometry-aware video editing, while preserving the generative prior of the underlying video diffusion model. The code is available at: https://github.com/MTLab/PE-Field.

Bhavana Verma, Priyanka Meel, Dinesh Kumar Vishwakarma

TL;DR: 本文提出HCIG（分层跨模态不一致图网络）和GCCN（基于图的跨模态矛盾网络）两种模型，用于检测多模态讽刺和网络欺凌。HCIG通过图注意力网络在token、短语和全局三个层次建模跨模态不一致性，并利用分层注意力机制自适应整合这些表示。GCCN则采用基于图的推理和矛盾感知池化进行高效的多模态交互学习。

Details

Motivation: 现有多模态方法主要依赖特征融合或跨模态注意力，难以有效捕捉不同表示层次间的语义不一致性，而讽刺和网络欺凌的意图往往源于文本与视觉信息之间的不一致。

Result: 在MMSD讽刺检测基准上，HCIG取得最佳性能（准确率85.74%，宏F1 85.29%）；在MultiBully网络欺凌数据集上，GCCN获得最高宏F1（68.66%），HCIG取得最高准确率（69.62%）和欺凌类F1（74.90%）。

Insight: 创新点在于分层多粒度不一致性建模，通过图注意力网络在不同语义层次捕获跨模态不一致，并引入分层注意力机制进行自适应整合，相比传统融合策略能提供更有效的多模态推理。

Abstract: Multimodal sarcasm and cyberbullying detection remain challenging because the intended meaning often emerges from incongruity between textual and visual information rather than from either modality alone. Existing multimodal approaches primarily rely on feature fusion or cross-modal attention, which may not effectively capture hierarchical semantic inconsistencies across different levels of representation. To address this limitation, this paper proposes HCIG (Hierarchical Cross-modal Incongruity Graph Network), a novel framework that models cross-modal incongruity at token, phrase, and global levels using graph attention networks and adaptively integrates these representations through a learned hierarchical attention mechanism. As a complementary architecture, we also introduce GCCN (Graph-based Cross-modal Contradiction Network), which performs graph-based reasoning using contradiction-aware pooling for efficient multimodal interaction learning. The proposed models are evaluated on the MMSD sarcasm benchmark and the MultiBully cyberbullying dataset, together with comprehensive ablation studies and cross-task transfer experiments. Experimental results demonstrate that HCIG achieves the best performance on MMSD with 85.74% accuracy and 85.29% macro-F1, while GCCN attains the highest macro-F1 (68.66%) on MultiBully and HCIG achieves the highest accuracy (69.62%) and bullying-class F1 (74.90%). The findings demonstrate that hierarchical multi-granularity incongruity modeling provides more effective multimodal reasoning than conventional fusion strategies, offering a robust framework for sarcasm and cyberbullying detection in social media.

[32] GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Tracking cs.CVPDF

Youngho Kim, Hoonhee Cho, Jae-Young Kang, Kuk-Jin Yoon

TL;DR: 本文提出GoStop：一种基于强化学习的自适应时间聚合方法，用于事件相机的特征跟踪。该方法将事件累积建模为序列决策问题，训练强化学习智能体根据运动线索动态决定是否继续累积事件或执行跟踪推理，从而适应不同的运动模式。

Details

Motivation: 现有基于事件的特征跟踪方法依赖固定启发式规则进行事件累积，无法适应多样化的运动动态，在突变运动或低运动场景下性能下降。

Result: 在提出的动态事件跟踪数据集上进行的广泛实验表明，将该即插即用框架集成到现有特征跟踪方法中，始终优于基于启发式的方法，在动态运动下提高了鲁棒性，并在跟踪精度和效率之间实现了更好的平衡。

Insight: 创新点在于将事件累积过程建模为强化学习问题，实现了自适应的时间聚合，摆脱了对手工规则的依赖；同时引入了具有动态运动分布的数据集来评估跟踪鲁棒性。

Abstract: Feature tracking plays a fundamental role in understanding scene motion and supports various downstream tasks. Event cameras, with their high temporal resolution and asynchronous sensing, enable low-latency and motion-robust perception, making them well-suited for feature tracking under fast and non-linear motion. However, existing event-based feature tracking methods rely on fixed heuristic rules based on hand-tuning for event accumulation. Such strategies fail to adapt to diverse motion dynamics, leading to degraded performance under abrupt motion changes or low-motion scenarios. In this paper, we model event accumulation as a sequential decision-making problem and introduce reinforcement learning (RL) framework to adaptively control the accumulation process for online event-based feature tracking. Our approach trains a RL agent that decides whether to continue accumulating events or to perform tracking inference based on motion cues. The proposed adaptive temporal agent enables dynamic adaptation to varying motion patterns without relying on hand-crafted rules. Furthermore, we introduce a Dynamic Event-based Tracking (DEFT) dataset with dynamic motion distributions to evaluate the robustness of the feature tracking. Extensive experiments demonstrate that integrating our plug-and-play framework to existing feature tracking methods consistently outperforms heuristic-based approaches, improving robustness under dynamic motion while offering a better balance between tracking accuracy and efficiency. Our project codes and datasets are available at https://github.com/kmax2001/GoSTOP

[33] Event3R: Asynchronous-to-Global 3D Reconstruction from Event Camera via Spatial-Temporal Feature Aggregation cs.CVPDF

Jian Huang, Haotian Shen, Xinhao Lou, Chengrui Dong, Wenpu Li

TL;DR: 本文提出了Event3R，一种前馈框架，用于直接从异步事件流生成全局一致的3D点云。该方法通过时空体素表示事件，并利用时间注意力模块和掩码箱建模策略进行特征学习，在合成和真实世界基准测试中实现了鲁棒、时间一致且全局对齐的3D重建。

Details

Motivation: 解决从事件相机进行密集3D重建的挑战，因为事件数据具有异步、稀疏和高动态特性，且缺乏大规模标注数据集，现有基于RGB的方法难以直接应用。

Result: 在合成和真实世界基准测试中，Event3R显著优于现有的基于事件的方法，实现了鲁棒、时间一致且全局对齐的3D重建。

Insight: 创新点包括将事件表示为时空体素以进行时间感知特征集成，提出掩码箱建模策略进行自监督预训练以减少对标注数据的依赖，以及在微调中引入对比对齐和一致性正则化损失以增强跨视图的结构对应和时间连贯性。

Abstract: Robust 3D reconstruction is essential for robotics and embodied perception. Recent feed-forward approaches such as DUSt3R have demonstrated impressive progress in dense 3D reconstruction from RGB images, achieving global geometric consistency and strong generalization. However, extending such dense 3D reconstruction to event cameras remains challenging due to their asynchronous, sparse, and highly dynamic nature, as well as the lack of large-scale, well-labeled datasets. In this work, we introduce Event3R, a feed-forward framework that directly maps asynchronous event streams to globally consistent 3D point clouds. Event3R represents incoming events as spatial-temporal voxels, enabling time-aware feature integration through a temporal attention module that enhances the module’s temporal feature learning. To further strengthen temporal representation learning and reduce reliance on labeled data, we propose a Masked Bin Modeling (MBM) strategy for self-supervised pre-training, enabling robust temporal representation learning with minimal labeled data, and retain it as an auxiliary fine-tuning objective. In addition, contrastive alignment and consistency regularization losses are incorporated during fine-tuning to reinforce structural correspondence and temporal coherence across views. Extensive experiments on both synthetic and real-world benchmarks demonstrate that Event3R achieves robust, temporally consistent, and globally aligned 3D reconstructions, significantly outperforming existing event-based methods.

[34] An Exam for Active Observers cs.CV | cs.AI | cs.CL | cs.LGPDF

Jiarui Zhang, Muzi Tao, Shangshang Wang, Ollie Liu, Xuezhe Ma

TL;DR: 该论文提出了一个名为ActiveVision的基准测试，旨在评估多模态大语言模型（MLLMs）是否具备主动观察能力。该基准包含3个类别共17项任务，要求模型进行重复的视觉感知而非单次静态描述。实验发现，前沿MLLMs（如GPT-5.5和Claude Fable 5）在该基准上表现极差，远低于人类水平，表明当前模型缺乏稳健的主动视觉观察能力。

Details

Motivation: 人类视觉是一个闭环系统，依赖于中间假设不断调整注视点进行主动观察，这对多种任务至关重要。然而，当前视觉-语言基准无法衡量MLLMs是否具备这种能力，因此需要一个新的基准来实证检验。

Result: 在ActiveVision基准上，表现最佳的GPT-5.5模型仅解决了10.6%的任务，在17项任务中有11项得分为零；Claude Fable 5仅解决3.5%。相比之下，三名人类参与者平均达到96.1%。即使让模型编写并运行自己的视觉代码，差距依然显著，因为代码在真实图像上不可靠，且检测其错误本身需要模型所缺乏的主动感知能力。

Insight: 论文的创新点在于首次提出了一个系统性的基准（ActiveVision）来量化MLLMs的主动观察能力，揭示了当前模型在感知-推理闭环上的根本缺陷。这为未来设计支持主动观察的模型架构和训练目标提供了明确动机。

Abstract: Human vision is a closed loop: gaze is continuously redirected by intermediate hypotheses rather than a single snapshot. Decades of psychophysics and cognitive science have argued that this active observation is essential for a wide range of tasks. Whether today’s multimodal large language models (MLLMs) exercise active observation is an empirical question that current vision-language benchmarks do not answer. We introduce ActiveVision, a benchmark that makes active observation measurable for MLLMs, comprising 17 tasks across 3 categories. Tasks are designed to force repeated visual perception rather than a single static description. Frontier MLLMs collapse on ActiveVision: the highest-scoring model we evaluate, GPT-5.5 at the highest exposed reasoning-effort tier, solves only 10.6% of items and scores zero on 11 of the 17 tasks, and even Claude Fable 5, despite topping most reasoning and coding leaderboards, solves just 3.5%, far behind three human participants who average 96.1%. Furthermore, much of the gap persists even when models write and run their own vision code: such code is unreliable on realistic imagery, and catching its failures itself requires the active perception the models lack. Together, these results indicate that current MLLMs lack robust active visual observation, motivating architectures and training objectives that close the perception-reasoning loop.

[35] IoUPD: IoU-Aware Privileged Distillation for Visual Grounding with Multimodal Large Language Models cs.CVPDF

Xiuyuan Zhu, Ke Lu, Hao Wu, Zijin Du, Dongming Zhang

TL;DR: 本文提出IoUPD方法，一种用于坐标生成型多模态大语言模型的IoU感知特权蒸馏方法，旨在解决视觉定位任务中训练与评估目标不匹配的问题。该方法利用真实标注框作为特权训练指导，通过教师-学生蒸馏框架提升模型性能，无需在推理时引入额外模块。

Details

Motivation: 当前基于多模态大语言模型的视觉定位方法通常采用自回归坐标生成方式，这导致训练时优化的是坐标字符串的token级似然，而评估时却使用几何重叠（IoU）作为衡量标准，存在训练与评估目标不一致的问题。

Result: 在标准指代表达式定位基准测试中，IoUPD方法相比强坐标生成基线模型取得了持续的区域级性能提升，表明真实标注框能提供超越坐标标签的有用特权指导。

Insight: 创新点在于将真实标注框作为特权信息引入训练过程，通过设计反映几何重要性和教师可靠性的token权重蒸馏损失，在无需推理时额外开销的情况下，有效对齐了训练目标与评估指标。

Abstract: Visual grounding with multimodal large language models is commonly formulated as autoregressive coordinate generation, where a model outputs bounding-box coordinates as text given an image and a referring-expression prompt. While this interface is simple and compatible with instruction following, it introduces a mismatch between training and evaluation: training optimizes token-level likelihood over coordinate strings, whereas grounding quality is measured by geometric overlap. We propose IoUPD, an IoU-aware privileged distillation method for coordinate-generating multimodal large language models. IoUPD uses ground-truth boxes not only as coordinate targets, but also as privileged training-time guidance. During training, the student receives the original image and prompt, while a frozen teacher receives a box-marked image and an augmented prompt that indicates the marked region. The student is trained with a supervised fine-tuning anchor and a privileged distillation loss whose token weights reflect both geometric importance and teacher reliability. At inference time, IoUPD requires no box overlay, privileged hint, teacher branch, or additional prediction module. Experiments on standard referring-expression grounding benchmarks show consistent region-level improvements over strong coordinate-generating baselines, demonstrating that ground-truth boxes can provide useful privileged guidance beyond serving as coordinate labels.

[36] Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Merging cs.CVPDF

Zhichao Yang, Tianjiao Gu, Zhixianhe Zhang, Xiangfei Sheng, Pengfei Chen

TL;DR: 本文提出了一种基于多模态大语言模型（MLLM）的个性化图像美学评估方法PRAC，该方法通过挖掘偏好丰富的样本和合并美学共鸣的群体来建模个体美学偏好。

Details

Motivation: 个性化图像美学评估（PIAA）旨在预测因人而异的图像美学评分，而美学偏好在不同视觉刺激中表现程度不同且呈现群体特定模式。

Result: 在四个基准PIAA数据库上的大量实验和比较表明，所提出的PRAC模型优于现有最先进方法（SOTA）。

Insight: 创新点在于通过分析图像的集体争议性和个性化偏差来识别偏好丰富样本，并基于偏好嵌入比较测量跨用户偏好相似性，进而提出通过聚合美学共鸣用户的偏好模式进行群体模型合并的策略，以增强对目标个体的个性化建模。

Abstract: Personalized Image Aesthetic Assessment (PIAA) aims to predict aesthetic ratings of images that vary across individuals. The aesthetic preferences manifest to different extents across distinct visual stimuli and exhibit cohort-specific patterns. Motivated by the above fact, this paper presents a Multimodal Large Language Model (MLLM)-based approach, which models individual aesthetic preferences by Preference-Rich sample mining and Aesthetically-resonant Cohort merging (PRAC). Specifically, PRAC first identifies preference-rich samples by analyzing both Collective Controversy and Personalized Deviation of images, maximizing the utility of limited user data. Based upon the preference-rich samples, cross-user preference similarities are measured by comparing preference embeddings. Then, a cohort-based model merging strategy, is proposed by aggregating preference patterns from aesthetically-resonant users, which further enhances the personalization for the target individual. Extensive experiments and comparisons on four benchmark PIAA databases demonstrate the superiority of the proposed PRAC model over the state-of-the-arts. The code and model will be public at https://github.com/yzc-ippl/PRAC.

[37] Debiasing Text-to-Image Evaluation via Implicit Cultural Alignment Reward Modeling cs.CV | cs.AI | cs.LG | cs.MMPDF

Bo-An Chang, Yu-Chih Chen

TL;DR: 本文提出了一种基于轻量级多模态大语言模型的隐式文化对齐奖励模型，用于评估文本到图像生成系统的文化真实性。该方法通过集成隐式文化探测器和跳跃连接交叉注意力机制，有效捕捉文化细节，并在CulturalFrames基准上超越了现有评估指标，同时实现了10倍的速度提升。

Details

Motivation: 现有文本到图像评估指标依赖于视觉语义表示，往往忽略隐式文化规范，导致偏好判断偏差，且基于视觉问答的评估器受自回归文本生成限制，难以实时扩展。

Result: 在CulturalFrames基准的3323个图像对上，该方法达到了80.54%的成对准确率，皮尔逊和肯德尔相关系数分别为0.546和0.377，优于代表性视觉语言指标和基于MLLM的评估器；推理速度达0.21秒/评估，比标准VQA评估器快10倍。

Insight: 创新点包括隐式文化探测器与跳跃连接交叉注意力机制的结合，使后期语义特征直接关注早期视觉表示，以保留文化细节；同时绕过自回归文本生成，提升了评估效率，为偏好优化提供了高效的文化感知标量信号。

Abstract: As Text-to-Image (T2I) systems rapidly advance, evaluating the cultural authenticity of synthesized content has become increasingly important for fair and trustworthy generative AI. Existing T2I evaluation metrics and multimodal judges often rely on visual-semantic representations that underrepresent implicit cultural norms, leading to biased preference judgments and the omission of fine-grained cultural cues. In addition, visual question answering (VQA)-based evaluators typically depend on autoregressive text generation, which limits their scalability for real-time reward modeling. To address these limitations, we introduce an Implicit Cultural Alignment Reward Model built upon a lightweight 4.2-billion-parameter Multimodal Large Language Model (MLLM). Our framework integrates an Implicit Cultural Probe with a Skip-connection Cross-Attention (SkipCA) mechanism, enabling late-stage semantic features to directly attend to early-stage visual representations and better preserve culturally salient details. Evaluations on 3,323 challenging and carefully curated image pairs from the CulturalFrames benchmark show that our approach achieves 80.54% pairwise accuracy, with Pearson and Kendall correlation coefficients of 0.546 and 0.377, respectively, outperforming representative vision-language metrics and MLLM-based evaluators. Moreover, by bypassing autoregressive text generation, our model processes each evaluation in 0.21 seconds under our local inference setup, achieving a $10\times$ speedup over standard VQA-based evaluators. These results suggest that the proposed reward model can provide an efficient and culturally aware scalar signal for preference optimization pipelines such as Reinforcement Learning from Human Feedback and Direct Preference Optimization.

[38] SlotMem: Character-Addressable Internal Memory for Narrative Long Video Generation cs.CVPDF

Yilai Liu, Xin Zhang, Shiyuan Zhang, Hongyang Du

TL;DR: 本文提出SlotMem框架，用于解决叙事长视频生成中角色身份跨场景和长时间跨度的一致性保持问题。该框架通过角色语义探针定位角色相关视觉标记，并使用记忆编码器将DiT标记压缩为紧凑的角色槽记忆，在生成过程中动态更新和检索角色记忆以增强一致性。

Details

Motivation: 现有方法在保持叙事长视频中角色身份一致性方面存在不足：全局一致性方法使用的检索线索与角色身份保持不对齐，而近期以角色为中心的方法仍依赖粗粒度的帧级kv记忆，将身份与偶然视觉因素纠缠，且在有限记忆容量下缺乏连续更新机制。

Result: 在多个叙事长视频生成基准测试上的实验表明，SlotMem在长距离角色一致性方面优于现有基线方法，同时保持了可比的视频质量。

Insight: 创新点在于提出了一个角色可寻址的内部记忆框架，通过角色语义探针精确定位角色相关视觉标记，并采用保守更新策略动态管理角色槽记忆，实现了更精准和持续的角色身份保持，而非依赖全局或帧级记忆。

Abstract: Maintaining recurring character identities across scene transitions and long temporal gaps is a central challenge in narrative long video generation. Methods targeting global consistency often retrieve memory using cues that are not aligned with character identity preservation, while recent character-centric variants still rely on coarse frame-level kv memory that entangles identity with incidental visual factors and lacks a continuous update mechanism under limited memory capacity. To address these limitations, we propose \textbf{SlotMem}, a character-addressable internal memory framework for multi-character narrative long video generation. Specifically, SlotMem uses a Character-Semantic Probe to localize character-relevant visual tokens from cross-attention responses, and a Memory Encoder to compress DiT tokens into compact role-wise slot memory. As generation proceeds, a Memory Writer conservatively updates each character’s memory with new observations, while Character-Wise Cross-Attention retrieves the role memory and injects it only into localized tokens of the same character. Experiments on multiple narrative long video generation benchmarks show that SlotMem improves long-range character consistency over existing baselines, while maintaining comparable video quality. Our code is available at https://github.com/YilaiLiu-HKU/SlotMem.

[39] GeoChrono: Benchmarking and Rethinking Long-Term Temporal Understanding in Remote Sensing cs.CV | cs.AIPDF

Yujie Li, Jiancheng Pan, Zhiwei Wei, Jiuniu Wang, Mugen Peng

TL;DR: 该论文提出了GeoChrono模型，旨在解决遥感领域中长期时空理解能力评估的不足。作者首先构建了ChronoBench基准，将任务分解为四个认知层次，并揭示了现有MLLM模型的瓶颈。基于此，他们设计了GeoChrono模型，通过引入时空轨迹编码器和粗到细的令牌压缩器来增强对长期地理演变的追踪、记忆和推理能力，并在新基准上取得了SOTA性能。

Details

Motivation: 现有遥感研究缺乏对模型长期时空理解能力的系统性评估，无法区分感知、识别、记忆和推理等不同认知层次。为了填补这一空白，需要构建一个多维基准来分解和评估这些能力，并针对发现的瓶颈设计更强大的模型。

Result: 在提出的ChronoBench基准（包含12个子任务和17,689个QA对）上进行评估，GeoChrono模型超越了领先的商业MLLM超过20%，达到了SOTA性能。同时，其C2FComp组件在减少超过56%视觉令牌的同时，保留了模型94.6%的性能。

Insight: 论文的主要创新点包括：1) 构建了系统性的多维基准ChronoBench来分解和评估长期时空理解的不同认知能力；2) 利用地理区域空间固定但语义演变的物理先验，设计了专门的时空轨迹编码器(TempEnc)来建模每个位置的演变轨迹；3) 提出了粗到细的令牌压缩器(C2FComp)，能自适应地保留动态区域并压缩静态背景，显著提升效率。

Abstract: Remote sensing offers an unparalleled vantage point for observing the Earth’s long-term surface evolution, yet it demands that a model not only perceive land cover at isolated moments, but also track changes, memorize evolution histories, and reason across time and space. However, existing studies lack a systematic evaluation that dissects these distinct competencies. To fill this gap, we introduce ChronoBench, a multidimensional benchmark that decomposes this task into four progressive cognitive levels (i.e., Land Cover Perception, Temporal Recognition, Long-Term Memory, and Spatio-Temporal Reasoning). The ChronoBench comprises 12 sub-tasks and 17,689 rigorously validated QA (Question-Answer) pairs. Extensive evaluations reveal that mainstream MLLMs fall drastically behind human experts, with Long-Term Memory emerging as the most critical bottleneck. Motivated by this finding, we further propose GeoChrono, an MLLM with enhanced capabilities for tracing, memorizing, and reasoning about long-term geographic evolution. Leveraging the physical prior that geographic parcels remain spatially fixed while their semantics evolve, we design a Temporal Trajectory Encoder~~(TempEnc) that constructs per-location temporal trajectories for dedicated land cover evolution modeling, and we introduce a Coarse-to-Fine Token Compressor~~(C2FComp) that adaptively preserves dynamic regions while compressing the static background. To support training, we also construct ChronoInstruct, a 104K-sample instruction-tuning dataset spanning all competency levels for training. GeoChrono achieves state-of-the-art performance on ChronoBench, surpassing the leading commercial MLLMs by over 20%, while C2FComp reduces visual tokens by over 56% while retaining GeoChrono’s 94.6% performance. The code and data will be available at https://github.com/IntelliSensing/GeoChrono

[40] Modularized Dynamic-Granularity Video LLM for Multi-Event Long Video Understanding cs.CV | cs.AIPDF

Wei Feng, Xin Wang, Yu-Wei Zhan, Yuwei Zhou, Wenwu Zhu

TL;DR: 本文提出了一种名为MoD-VLLM的模块化动态粒度视频大语言模型框架，用于解决多事件长视频理解中的挑战。该框架通过正负视频片段定位模块和模块化动态粒度反思模块，形成一个闭环，逐步定位与问题相关的视频片段，并采用动态粒度强化学习策略联合学习最优定位策略和视觉表示。

Details

Motivation: 现有视频大语言模型在处理长视频时，由于视觉令牌预算有限与需要捕获多个关键事件之间的矛盾，通常采用两阶段处理（关键帧选择和详细感知），缺乏自适应能力分配和自我修正的模块化机制，导致建模不可靠。

Result: 在多个长视频理解基准测试以及作者提出的MEventBench上进行的广泛实验表明，MoD-VLLM显著优于现有的最先进基线模型。

Insight: 创新点包括：1) 将时间定位和语义理解迭代且自反思地统一起来；2) 引入模块化调度器动态选择细粒度编码（用于相关正片段）和粗粒度编码（用于负片段）；3) 提出动态粒度强化学习策略以联合优化定位策略和视觉表示；4) 构建了MEventBench这一具有挑战性的多事件长视频基准测试。

Abstract: Video Large Language Models (Video LLMs) have made significant advancements in various video understanding tasks. However, long-video scenarios remain challenging due to the tension between limited visual token budgets and the need to capture multiple key events. Existing approaches typically process long videos in two stages, i.e., i) select keyframes and ii) perform detailed perception, which exhibit limitations: they lack a modular mechanism for adaptive capacity allocation and self-correction, resulting in unreliable modeling. To tackle these challenges, we propose MoD-VLLM, a novel Modularized Dynamic-Granularity Video LLM framework for multi-event long video understanding, which unifies temporal grounding and semantic understanding iteratively and self-reflectively. Specifically, we propose a Positive-Negative Video Segments Grounding module and a Modularized Dynamic-Granularity Reflection module, which form a closed loop to progressively localize the question-related video segments. The grounding module instructs a Video LLM to distinguish relevant from irrelevant video segments based on the video question. The reflection module employs a modularized scheduler that dynamically selects fine-grained encoding for relevant positive segments to capture detailed perception and coarse-grained encoding for negative segments to maintain global context. We further propose a dynamic-granularity reinforcement learning strategy, allowing MoD-VLLM to learn optimal grounding policies and dynamic granularity visual representation jointly. Moreover, we propose MEventBench, a challenging Multi-Event Long Video Benchmark for complex long video reasoning. Extensive experiments on several long video understanding benchmarks and our MEventBench demonstrate that MoD-VLLM significantly outperforms state-of-the-art baselines.

[41] Multimodal Ambivalence and Hesitancy Recognition via Cross-Attention and Gated Fusion cs.CVPDF

Oussama Berhili, Yassine Ouzar, Larbi Boubchir

TL;DR: 本文提出了一种用于视频中矛盾/犹豫（A/H）识别的多模态框架，融合了文本、声学和视觉模态。该方法使用预训练的F2LLM-v2-0.6B、WavLM-Large和VideoMAE V2编码器提取特征，并通过双向交叉注意力和门控多模态单元进行融合。在BAW11挑战赛的BAH数据集上，该模型在验证集上取得了0.7394的宏F1分数，相比最佳单模态基线提升了11.0%。

Details

Motivation: 解决视频中矛盾/犹豫（A/H）这一复杂情感状态的识别问题，旨在通过多模态融合来捕捉单一模态无法提供的互补线索。

Result: 在BAH数据集上，最佳单模态（仅文本）基线在测试集上的宏F1为0.6659，远超零样本Video-LLaVA基线（0.2827）。提出的多模态融合模型在验证集上达到宏F1 0.7394，相比最佳单模态基线相对提升11.0%。

Insight: 创新点在于结合了跨所有三个模态的双向交叉注意力与门控多模态单元（GMU）进行融合，并通过Optuna进行架构和超参数优化。客观来看，这种显式的跨模态交互设计是有效捕捉互补信息、提升性能的关键。

Abstract: We present a multimodal framework for Ambivalence/Hesitancy (A/H) recognition in video, developed for the ABAW11 challenge at ECCV 2026. The proposed approach fuses textual, acoustic, and visual modalities extracted from the BAH dataset using three pretrained encoders: F2LLM-v2-0.6B for transcripts (1024-d), WavLM-Large for audio (1024-d), and VideoMAE V2 for facial video (768-d). We first establish comprehensive unimodal baselines using classical classifiers (MLP, Random Forest, GBDT), each optimized via Optuna, and obtain a best unimodal Macro F1 of \textbf{0.6659} on the test set using text features alone – substantially outperforming the zero-shot Video-LLaVA baseline (Macro F1: 0.2827). Building on these baselines, we propose a multimodal fusion architecture that combines bidirectional cross-attention across all three modalities with a Gated Multimodal Unit (GMU), with both architectural and optimization hyperparameters selected through a 50-trial Optuna search. This model achieves a Macro F1 of \textbf{0.7394} on the validation set, a relative improvement of 11.0% over the best unimodal baseline, confirming that explicit cross-modal interaction captures complementary cues that no single modality provides in isolation. Final predictions on the official, unlabeled private test set are generated using this model and submitted according to the challenge protocol. Code is publicly available at https://github.com/yassineouzar/IUSD_AH/

Stefano Silvestrini, Michele Ceresoli

TL;DR: 本文研究了基于事件的多模态自运动估计网络中学习表示的几何结构，通过跨模态注意力架构融合事件张量、惯性测量和距离信号，并分析了潜在空间几何与注意力动态。研究发现嵌入位于与运动变量对齐的低维流形上，注意力权重随角激励和视觉可靠性自适应调整，且融合表示恢复了经典可观测性线索。

Details

Motivation: 探索基于事件的多模态自运动估计网络中学习表示的几何结构，以桥接传统几何优化框架与现代数据驱动融合方法。

Result: 分析表明嵌入位于低维流形并与运动变量对齐，注意力权重自适应调整，融合表示恢复经典可观测性线索，在ELOPE挑战的背景下验证了方法的有效性。

Insight: 通过跨模态注意力实现多模态融合，潜在空间几何分析揭示了学习表示与运动变量的对齐关系，为数据驱动方法提供了可解释性见解。

Abstract: Classical approaches to event-based egomotion estimation, including those adopted by the top-performing teams of the ELOPE challenge, rely on geometric optimization frameworks such as contrast maximization, homography estimation, or dense optical flow combined with analytic motion inversion. This work investigates the geometric structure that emerges inside a multi-modal network for egomotion estimation. Event tensors, inertial measurements, and range signals are fused through a cross-modal attention architecture and trained in a batch setting. We analyze the latent space geometry and attention dynamics, showing that (i) embeddings lie on low-dimensional manifolds aligned with motion variables, (ii) attention weights adapt with angular excitation and visual reliability, and (iii) the fused representation recovers classical observability cues. These results bridge analytical estimation theory and modern data-driven fusion.

[43] Test-Time Noise Guided Adaptation for Realistic Autoregressive Video Generation cs.CV | cs.AIPDF

Dimitrios Karageorgiou, Symeon Papadopoulos, Ioannis Kompatsiaris, Efstratios Gavves

TL;DR: 本文提出了一种名为TANGO（Terminal points Avoidance through Noise Guided Optimization）的方法，用于解决自回归视频扩散模型在生成长视频时因误差累积而导致的序列漂移问题。该方法通过在测试时利用扩散模型自身作为评判器，预测前向噪声分布并引导优化，以避免模型陷入无法继续生成的终止点。

Details

Motivation: 自回归视频扩散模型在生成长视频时，由于去噪序列逐渐偏离训练时见过的条件分布，会产生误差累积问题。现有方法尝试将生成的每一帧锚定在真实帧的流形上，但即使单帧接近真实流形，模型仍可能因缺乏足够知识而无法继续生成，从而到达终止点。

Result: 在VBench基准测试上，TANGO方法相比最先进方法取得了3.1%的绝对提升；在15秒视频上，平均将Fréchet Video Distance降低了28.3%。

Insight: 创新点在于提出了一种测试时噪声引导优化策略，利用扩散模型自身预测的噪声分布与预期高斯分布的偏差来搜索替代轨迹，从而避免终止点。这提供了一种通过模型自监督来改善长序列生成稳定性的新思路。

Abstract: Autoregressive video diffusion models have enabled the generation of arbitrarily long videos by removing conditioning on future frames, thus greatly improving computational efficiency. Yet, they suffer from error accumulation over time, as the denoised sequence gradually drifts away from the conditioning distribution seen during training. Recent advances attempt to reduce this error by anchoring each generated frame to the learned manifold of real ones. However, even when all generated individual frames lie close to the real manifold, there are trajectories which the model lacks sufficient knowledge to continue without exiting it, thus reaching a terminal point. To prevent the model from being trapped in terminal points, we start from the hypothesis that for well-modeled future trajectories the distribution of the predicted noise should match the one of the forward noising process. To enforce such a prior at test time, we introduce Terminal points Avoidance through Noise Guided Optimization (TANGO), which uses the diffusion model as a critic of its own outputs, by predicting one step forward and requiring an isotropic Gaussian noise prediction. We use the deviation from this expected noise distribution to search for an alternative trajectory that does not lead to a terminal point. Our approach achieves a $3.1%$ absolute improvement on VBench over state-of-the-art, while reducing Fréchet Video Distance by $28.3%$ on average across $15$s videos. Our code is available on https://mever-team.github.io/tango.

[44] Von Mises-Fisher Mixture Model with Dynamic Shrinkage for Realistic Test-Time Transduction cs.CV | cs.LGPDF

Jiazhen Huang, Zhiming Liu, Changhu Wang, Wei Ju, Ziyue Qiao

TL;DR: 本文提出了一种名为MOON（Mixture of Von Mises-Fisher Models with Dynamic Shrinkage）的方法，用于增强视觉语言模型（VLMs）在测试时的性能。该方法从惩罚似然估计（PLE）的视角重新审视了转导学习范式，通过构建基于von Mises-Fisher混合模型的动态收缩机制，有效处理了现实评估中常见的类别分布高度不平衡问题，从而防止性能下降或崩溃。MOON是模型无关、无需训练且无需任务特定超参数调优的。

Details

Motivation: 现有的转导学习方法在现实评估中，面对高度不平衡的类别分布时，常出现性能下降甚至崩溃的问题。本文旨在从惩罚似然估计的理论视角出发，揭示其脆弱性源于缺乏锚定机制和对收缩强度的静态建模，从而提出一种更鲁棒的解决方案。

Result: 广泛的实验验证了MOON在性能和效率上的优势。虽然没有在摘要中明确提及具体的基准测试和SOTA比较，但实验结果表明该方法有效缓解了负迁移，提升了模型在不平衡场景下的表现。

Insight: 论文的核心创新点在于从惩罚似然估计的理论框架重新阐释转导学习，并据此提出了一个基于von Mises-Fisher混合分布、在实例和类别两个层面利用零样本先验动态调整收缩强度的模型。这提供了一种处理类别不平衡、抑制不可靠分配和有害更新的通用机制，且无需额外训练或复杂调参。

Abstract: A range of methods aim to enhance the performance of vision-language models (VLMs) at test time. Among them, transduction has emerged as a promising paradigm due to its strong compatibility and efficiency. However, realistic evaluations often involve highly imbalanced class distributions, which cause performance degradation or even collapse. In this work, we systematically revisit transduction from the perspective of penalized likelihood estimation (PLE), showing that PLE with a KL-divergence anchor term naturally yields an adaptive shrinkage behavior between prior anchors and empirical estimates. From this viewpoint, the brittleness of transductive methods can be attributed to the absence of anchoring mechanism and static modeling of the shrinkage strength. Therefore, we propose Mixture of Von Mises-Fisher Models with Dynamic Shrinkage (MOON). MOON is built upon a mixture of von Mises-Fisher distributions to model feature representations on the unit hypersphere. To handle imbalance, MOON dynamically adjusts the shrinkage strength using zero-shot priors at both instance and class levels. Thus, it suppresses unreliable assignments and prevents harmful updates from outlier classes, thereby mitigating negative transfer. MOON is model-agnostic, training-free, and requires no task-specific hyperparameter tuning. Extensive experiments further validate the advantage of MOON in both performance and efficiency. Our code is available at https://github.com/walawalagoose/MOON

Qinsong Li, Jing Meng, Haibo Wang, Shengjun Liu

TL;DR: 本文提出了一种名为MDND的新型深度功能映射（DFM）范式，用于解决三维形状对应问题。该框架通过结合可微分与不可微分组件，利用不可微分细化分支生成鲁棒对应作为监督信号，指导可微分分支进行无监督学习，从而克服了传统DFM因端到端可微分性限制而无法集成高精度非可微分技术的瓶颈。

Details

Motivation: 传统深度功能映射框架依赖端到端可微分性，这限制了其集成高精度但非可微分的细化技术，导致在处理具有非等距变形和拓扑噪声的挑战性形状时性能受限。

Result: 在广泛实验中，MDND在具有非等距变形和拓扑噪声的形状上表现出卓越的鲁棒性，并取得了新的最先进（SOTA）性能。

Insight: 创新点在于提出了一种双分支架构，其中不可微分分支采用新颖的多尺度迭代求解器生成鲁棒对应作为细化目标，并通过一致性损失指导可微分分支进行无监督学习，从而实现了可微分与不可微分组件的有效融合。

Abstract: Deep functional map frameworks (DFM) for shape correspondence are powerful, yet fundamentally limited by their reliance on end-to-end differentiability. This constraint prevents the integration of highly accurate, non-differentiable refinement techniques, capping their overall performance, especially on challenging non-isometric shapes. To overcome this, we introduce MDND, a novel DFM paradigm built on the principle of merging differentiable and non-differentiable components. Our framework facilitates unsupervised learning guided by an internal, non-differentiable refinement. Specifically, MDND employs a dual-branch architecture: a non-differentiable refinement branch leverages a novel, multiscale iterative solver to produce highly robust correspondences, acting as a refined target. Concurrently, a fully differentiable branch learns to predict correspondences from features. The entire system is trained end-to-end without supervision by enforcing a consistency loss that compels the differentiable branch to learn from the superior, refined results of the non-differentiable branch. Extensive experiments show that MDND sets a new state-of-the-art, demonstrating remarkable robustness on shapes with non-isometric deformations and topological noise.

[46] EgoExoMoCap: Distributed Ego-Exo Human Motion Capture cs.CV | cs.AI | cs.GR | cs.HC | cs.ROPDF

Jiaxi Jiang, Bharat Lal Bhatnagar, Nan Yang, Lingni Ma, Sebastian Starke

TL;DR: 本文提出了一种名为EgoExoMoCap的分布式框架，用于从头戴设备中联合利用自我中心（ego）和外部中心（exo）的多模态信号进行人体运动捕捉。该方法仅需多人各自佩戴智能眼镜，即可在3D世界中准确估计全局运动，并通过结合DINOv3的上下文感知图像特征来应对噪声和遮挡。在两个野外数据集上的实验表明，该方法在挑战性场景下也能鲁棒地重建运动。

Details

Motivation: 现有方法通常孤立地研究自我中心身体跟踪（估计佩戴者自身运动）或外部中心跟踪（捕捉周围人的运动），而本文旨在通过一个分布式框架联合利用这两种视角的信号，以更可扩展的方式获取真实世界的人体运动和交互数据，这对于具身AI和VR/AR应用至关重要。

Result: 在两个野外数据集上进行的大量实验表明，该方法在挑战性场景下能够鲁棒地重建运动。

Insight: 创新点在于提出了一个联合利用自我中心和外部中心多模态信号的分布式运动捕捉框架，仅需多人佩戴智能眼镜即可实现，避免了传统系统对笨重多相机设置或侵入式动作捕捉服的需求；方法结合了头部（及潜在手腕）跟踪信号以实现3D全局运动的准确估计，并利用DINOv3的上下文感知图像特征来增强对噪声和遮挡的鲁棒性。

Abstract: Human motion capture from head-mounted devices (HMDs) offers a scalable way to acquire real-world human motion and interaction data, which is crucial for applications in embodied AI and VR/AR. Existing approaches focus on either egocentric body tracking, estimating the motion of the subject wearing the device, or exocentric tracking, capturing the movements of people in the wearer’s surroundings. So far, these two paradigms have largely been explored in isolation. In this paper, we propose a novel distributed framework that jointly leverages ego- and exocentric multi-modal signals for human motion estimation from HMDs. Unlike traditional motion capture systems requiring bulky multi-camera setups or obtrusive mocap suits, our approach, EgoExoMoCap, is as simple as two (or more) people, each wearing a pair of smart glasses. The method leverages head (plus potentially wrist) tracking signals for accurate estimation of global motion in the 3D world and combines context-aware image features based on DINOv3 to achieve robustness in the presence of noise and occlusions. Extensive experiments on two in-the-wild datasets show that our approach can robustly reconstruct motion even in challenging scenarios.

[47] Exo2EgoPose: Leveraging Exocentric Demonstrations for Vision-Language guided Egocentric 3D Hand Pose Forecasting cs.CVPDF

Zhaofeng Shi, Heqian Qiu, Lanxiao Wang, Xiang Li, Hongliang Li

TL;DR: 本文提出Exo2EgoPose框架，用于解决视觉-语言引导的以自我为中心（Ego）的3D手部姿态预测任务。该框架创新性地利用稳定、全局的第三人称（Exo）演示视频作为指导，来补偿自我中心视角下有限的视野和高度动态的运动信息，从而更准确地预测未来的手部姿态。

Details

Motivation: 从自我中心视角感知多模态线索并预测细粒度动作对于机器人操作等应用至关重要。现有研究要么主要依赖信息不足的视觉输入来预测粗略的人体运动，要么遵循VRM/VLA范式，但面临机器人数据不足以及人与机器人形态差异的问题。本文观察到3D手部姿态可作为连接人与机器人动作的统一表征，因此探索了视觉-语言引导的自我中心3D手部姿态预测这一未充分研究的任务。

Result: 在AssemblyHands、Ego-Exo4D和新构建的EgoMe-pose基准测试上进行了广泛实验，结果表明该方法大幅优于现有最先进方法。此外，该方法在CALVIN数据集上展示了有效的人到机器人迁移能力并带来了性能提升。

Insight: 核心创新点在于利用成对的第三人称（Exo）视频作为监督，通过提出的双层次第三人称重建模块（DERM）重建视频级和片段级的表征，以建模空间上下文和时序动态，再通过全局到局部调制模块（GLMM）进行渐进式特征精炼，从而为自我中心姿态预测提供全面的第三人称指导。这为解决自我中心视角信息受限问题提供了一种新颖的跨视角知识迁移范式。

Abstract: Perceiving multimodal cues and forecasting fine-grained actions from an egocentric (Ego) perspective is vital for applications like robot manipulation. However, previous studies either rely mainly on under-informed visual inputs to predict coarse human motions or follow the VRM/VLA paradigm, which suffers from insufficient robot data and the gap between human and robot embodiments. We observe that 3D hand pose naturally serves as a unified representation to bridge human-robot actions. Hence, we investigate an under-explored Vision-Language guided Egocentric 3D Hand Pose Forecasting (VL-EHPF) task, which aims to predict future Ego 3D hand poses from visual observations, a language instruction, and pose states. To overcome the limited field-of-view and highly dynamic motions in the Ego view, we propose a framework dubbed Exo2EgoPose, which innovatively leverages holistic and stable exocentric (Exo) demonstrations as guidance to compensate for partial and dynamic Ego-view cues. Specifically, we introduce a Dual-level Exocentric Reconstruction Module (DERM), which incorporates the paired Exo videos as supervision to reconstruct their video-level and chunked frame-level representations, thereby modeling spatial contexts and temporal dynamics. Then, the Global-to-Local Modulation Module (GLMM) utilizes the reconstructed hierarchical Exo representations for progressive feature refinement via attention mechanisms and adaptive modulation, enabling comprehensive Exo guidance for accurate Ego hand pose forecasting. Extensive experiments on \textit{AssemblyHands}, \textit{Ego-Exo4D}, and our newly constructed \textit{EgoMe-pose} benchmarks show the superiority of our method, which outperforms state-of-the-art methods by a large margin. Moreover, it demonstrates an effective human-to-robot transfer capability and yields improvements on the \textit{CALVIN} dataset. Code will be released.

[48] Orbis 2: A Hierarchical World Model for Driving cs.CV | cs.AI | cs.LG | cs.ROPDF

Sudhanshu Mittal, Arian Mousakhan, Silvio Galesso, Karim Farid, Jonannes Dienert

TL;DR: 论文提出了Orbis 2，一种用于自动驾驶的分层世界模型。该模型将未来预测分解为两个层次：高层预测器负责在长时间尺度上预测粗略的场景结构，而低层生成器则基于高层输出生成详细的预测。此外，论文还引入了一种两阶段训练范式，结合了扩散强制目标的表征优势和教师强制目标的生成稳定性。

Details

Motivation: 当前的世界模型通常在单一抽象层次上运行，大多注重感知保真度，但缺乏现实世界下游任务所需的空间推理和语义理解能力。因此，需要一种能够兼顾高保真度与强空间语义表征的模型。

Result: 该方法在已建立的基准测试上，在驾驶世界模型的标准评估套件中取得了最先进（SOTA）的结果，包括长时程生成保真度、在反事实场景中评估的转向响应性以及内部表征质量。

Insight: 核心创新点在于分层预测架构（高层与低层）以及结合了扩散强制预训练和教师强制微调的两阶段训练范式。这种设计旨在同时获得丰富的内部表征和稳定的自回归生成能力。

Abstract: Current world models operate at a single level of abstraction, with most prioritizing perceptual fidelity while lacking the spatial reasoning and semantic understanding required for real-world downstream tasks. We present a hierarchical driving world model that factorizes future prediction across two levels operating at distinct temporal and abstraction scales: a high-level predictor that forecasts coarse scene structure over extended temporal horizons, and a low-level generator that produces detailed predictions conditioned on the high-level output. This decomposition yields high perceptual fidelity while also capturing strong spatial and semantic representations. We further show that pretraining with a diffusion forcing objective yields substantially richer internal representations than the standard teacher forcing objective, while teacher forcing – predicting only the next frame from clean context – produces more stable autoregressive rollouts. We therefore introduce a generic two-stage training paradigm that pretrains the model with diffusion forcing and fine-tunes with teacher forcing, combining the representational benefits of the former with the rollout stability of the latter. Our approach achieves state-of-the-art results across the standard suite of driving world model evaluations on established benchmarks, including long-horizon generation fidelity, steering responsiveness evaluated on counterfactual scenarios, and internal representation quality. Project page with code, demo, checkpoints and qualitative results: https://lmb-freiburg.github.io/orbis2.github.io/

[49] More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe cs.CV | cs.LGPDF

Stefan Maria Ailuro, Mario Markov, Mohammad Mahdi, Luc Van Gool, Danda Pani Paudel

TL;DR: 本文提出了一种简单而有效的遥感视觉语言模型训练方法，通过大规模多任务强化学习框架，使用单一语言策略处理文本回答和定位任务，在多种遥感基准测试中取得了有竞争力的性能。研究表明，对于遥感VLM而言，数据规模和多样性比架构创新更为重要。

Details

Motivation: 挑战遥感视觉语言模型领域普遍依赖特定架构设计的必要性，探索是否通过大规模、多样化的数据和任务训练，通用视觉语言模型也能在遥感任务上达到优异性能。

Result: 该方法在包括高分辨率、多时相、多模态和多视角任务在内的广泛遥感基准测试中取得了有竞争力或最先进（SOTA）的结果，并且随着训练数据规模的扩大，在分布内和分布外的大多数任务上均表现出持续的性能提升。

Insight: 核心创新点在于采用单一、通用的语言策略结合工具调用（如定位）来处理异构任务，并通过多任务强化学习框架进行统一训练。客观来看，其关键洞察是证明了对于遥感VLM，大规模、多样化的数据驱动训练比复杂的领域特定架构设计更为有效。

Abstract: Remote sensing vision-language models are increasingly expected to support open-ended reasoning over Earth Observation data and a variety of tasks. Most recent progress in this area has been driven by remote-sensing-specific architectural designs, often introducing new encoders, alignment modules, or task-specific fusion mechanisms. In this work, we challenge the necessity of such architectural specialization. We show that a generally capable vision-language model can achieve competitive or state-of-the-art performance at challenging remote sensing benchmarks, provided that it is trained at sufficient scale across diverse data and tasks. Our model uses a single language policy that can either answer directly in text or invoke a localization tool for segmentation and grounding. To train this heterogeneous behaviour, we employ a multi-task reinforcement learning framework with adaptive task rewards covering multiple-choice VQA, free-form VQA, captioning, detection, and segmentation across a large variety of input types. Our approach achieves competitive results across a broad set of benchmarks, including high-resolution, multi-temporal, multi-modal and multi-view tasks. Further, as training data scales, our experiments show consistent improvements across most tasks both in and out of distribution, which correlate with per-task data diversity. These findings suggest that, for remote sensing VLMs, data scale is more important than architectural novelty.

[50] PIXIE: A Zero-Shot texture-invariant 6D pose estimation framework for unseen objects with assembly defects cs.CV | cs.ROPDF

Leon Jungemeyer, Alejandro Magaña, Gautham Mohan, Matthias Karl, Daniel Werdehausen

TL;DR: 本文提出了PIXIE，一个零样本、纹理不变的6D姿态估计框架，用于估计具有装配缺陷的未见物体的姿态。该框架仅使用无纹理的3D模型，通过预训练的跨模态特征匹配器将渲染的合成深度和法线图与查询RGB图像匹配，然后通过PnP求解姿态。该方法对光照和纹理变化具有鲁棒性，并能处理模型与物理对象之间的几何偏差。

Details

Motivation: 解决当前数据驱动方法在工业环境中部署的局限性，包括资源密集型数据管道、对纹理3D模型的依赖以及对由损坏或装配缺陷引起的几何偏差的敏感性。

Result: 在广泛使用的公共基准测试中，报告了在无纹理物体上无需物体特定训练的最先进结果，并引入了一个包含装配缺陷、纹理变化和遮挡的新数据集，以展示其在实际应用中的有效性。

Insight: 创新点在于仅依赖几何信息实现零样本姿态估计，通过跨模态特征匹配和对应点过滤来增强对纹理变化和几何偏差的鲁棒性，为工业场景中的实际部署提供了更实用的解决方案。

Abstract: 6D pose estimation remains a key challenge in robotics and computer vision, particularly in industrial environments. The deployment of currently available data-driven methods is often limited by resource-intensive data pipelines, reliance on textured 3D models, and sensitivity to geometric deviations caused by damages or assembly defects. We present PIXIE, a zero-shot framework that estimates the 6D pose of an object from an RGB image using only an untextured 3D model. Synthetic depth and normal maps are rendered from sampled reference viewpoints and matched to the query image via a pretrained cross-modality feature matcher. Matched keypoints are back-projected to obtain 2D–3D correspondences for PnP-based pose estimation. Relying exclusively on geometry makes the method inherently robust to lighting and texture variation, while correspondence filtering handles geometric deviations between the model and physical object. We evaluate on widely-used public benchmarks, reporting state-of-the-art results on texture-less objects without object-specific training, and introduce a novel dataset with assembly defects, texture variations, and occlusion to demonstrate real-world applicability.

[51] DPNeXt: A Lightweight Multi-Scale Feature Fusion Framework for Efficient ViT-Based Multi-Task Dense Prediction cs.CV | cs.AI | cs.ROPDF

Jehun Kang, Jungha Wang, Youngjun Hwang, David Hyunchul Shim

TL;DR: 本文提出DPNeXt，一种轻量级多尺度特征融合解码器框架，旨在提升基于视觉基础模型（VFM）的多任务密集预测（如语义分割和深度估计）的效率。它通过双深度可分离倒置瓶颈结构优化特征融合，并引入多任务边界引导（MTBG）策略减少任务间负迁移，在保持高性能的同时大幅减少参数量和推理时间。

Details

Motivation: 在机器人感知系统中，多任务学习（MTL）需要集成语义分割和深度估计以实现全面的3D场景理解，但现有解码策略成为瓶颈，限制了视觉基础模型（VFMs）的有效利用。

Result: 在Cityscapes数据集上，DPNeXt-S超越先前最先进（SOTA）的MTL模型，DPNeXt-B进一步提升了整体性能并达到最佳结果；在NYUv2数据集上，DPNeXt-B也在语义分割和深度估计中取得最佳结果，且所需可训练参数远少于先前大规模MTL模型。与标准DPT相比，DPNeXt-S减少78.6%的可训练参数，并在资源受限的笔记本电脑硬件上实现最快的推理速度。

Insight: 创新点包括：采用双深度可分离倒置瓶颈结构进行以融合为中心的解码和独立任务模块化，提升冻结VFM的利用率；提出多任务边界引导（MTBG）策略，通过对称边界聚焦监督促进几何一致性，无需额外标注或推理成本，有效缓解任务间负迁移。从客观角度看，该框架在轻量化和效率方面具有显著优势，为实际部署提供了可行方案。

Abstract: Multi-Task Learning (MTL) in robotics perception systems supports comprehensive 3D spatial scene understanding by integrating semantic segmentation and depth estimation. While Vision Foundation Models (VFMs) are increasingly adopted as robust feature encoders, existing decoding strategies present a critical bottleneck. To address this, we propose DPNeXt, a streamlined multi-scale feature fusion decoder and efficient alternative to the standard Dense Prediction Transformer (DPT). DPNeXt uses dual depthwise separable inverted bottlenecks to improve frozen VFM utilization through fusion-centric decoding and independent task modularization. To further mitigate negative inductive transfer between tasks, we introduce the Multi-Task Boundary Guidance (MTBG) strategy. Unlike prior boundary-aware methods that add fusion modules or gating, MTBG applies symmetric boundary-focused supervision to encourage geometric consistency without extra annotation or inference cost. Experiments on Cityscapes show that DPNeXt-S outperforms prior state-of-the-art (SOTA) MTL models, while DPNeXt-B further improves the overall performance and achieves the best results among the compared methods. On NYUv2, DPNeXt-B also achieves the best semantic segmentation and depth estimation results among the compared methods while requiring substantially fewer trainable parameters than prior large-scale MTL models. Compared with the standard DPT, DPNeXt-S reduces trainable parameters by 78.6% and achieves the fastest inference speed among the compared models on resource-constrained laptop hardware. The source code, model checkpoints, and a demo video will be made available at https://github.com/kangjehun/DPNeXt.

Wasimul Karim, Nur Mohammad Fahad, Abdul Hasib Siddique, Md Rafiqul Islam, Hooman Mehdizadeh-Rad

TL;DR: 本文提出了一种名为HREM-Net的双分支深度学习框架，用于电解槽材料的多模态语义分割。该框架结合高光谱成像（HSI）和RGB图像，通过创新的注意力模块和自适应门控跨模态融合，旨在解决材料视觉相似性高、光谱重叠、形状不规则和类别不平衡等挑战。

Details

Motivation: 电解槽材料的精确分割对于氢技术中的自动化拆解、可持续回收和循环制造至关重要，但现有方法面临材料视觉相似性强、光谱重叠、形状不规则和严重类别不平衡等难题。

Result: 在Electrolyzers-HSI数据集上，HREM-Net实现了91.66%的平均类别准确率和0.82的mIoU，优于基线分割模型；在PCB-Vision数据集上的跨数据集验证中，获得了96.91%的准确率和0.93的mIoU，展示了强大的泛化能力。

Insight: 创新点包括结合HSI与RGB的双分支架构、采用Efficient Channel Attention和Coordinate Attention等模块来捕获光谱和空间特征，以及自适应门控跨模态融合与复合损失函数的设计，为工业应用如预测性维护提供了潜在解决方案。

Abstract: Accurate segmentation of electrolyzer materials is essential for automated disassembly, sustainable recycling, and circular manufacturing in hydrogen technologies. However, this task is challenging due to strong visual similarity between materials, spectral overlap, irregular shapes, and severe class imbalance. To address these challenges, we propose an AI-driven dual-branch framework, Hyperspectral-RGB Electrolyzer Materials Network (HREM-Net), that combines hyperspectral imaging (HSI) and RGB images for electrolyzer material segmentation. We implemented several innovative modules, including Efficient Channel Attention, Coordinate Attention, Mobile Inverted Bottleneck blocks, and Atrous Spatial Pyramid Pooling to capture spectral and spatial features from HSI, and RGB images. With an adaptive gated cross-modal fusion module and composite loss function, HREM-Net achieves a mean class accuracy of 91.66% and a mean Intersection over Union (mIoU) of 0.82 on the Electrolyzers-HSI dataset, outperforming baseline segmentation models. Cross-dataset validation on the PCB-Vision dataset demonstrates strong generalization with 96.91% accuracy and 0.93 mIoU. This work poses its potential as an industrial application to improve electrolyzer efficiency, thereby improving the predictive maintenance of hydrogen production.

[53] How Do VLMs Fail? Vision-Operation Misalignment in Compositional VQA cs.CVPDF

Navya Gupta, Bingjie Xu, Avinash Anand, Timothy Liu, Zhengchen Zhang

TL;DR: 本文分析了视觉语言模型在组合式视觉问答任务中的失败机制，提出了一个以操作为中心的机制框架，将VLM失败分解为四种不同的失败模式，并通过因果干预揭示了不同失败模式在模型内部的计算路径差异。

Details

Motivation: 尽管VLMs在组合式视觉问答任务上总体表现强劲，但其失败的机制基础仍未得到充分探索。本文旨在通过分析视觉操作错位，探究失败如何与特定推理操作以及内部计算路径相关联。

Result: 分析揭示了四种机制上不同的失败模式：基础失败、推理失败、属性提取失败和语言先验主导失败。通过跨所有Transformer层的三种互补因果干预，进一步证明了路径分离现象，即不同失败模式对应不同的内部计算路径。

Insight: 创新点在于提出了一个操作中心的机制框架来系统分解VLM失败，并通过因果干预实验揭示了不同失败模式在模型内部（如前馈网络、注意力层）的特定传播路径，这为有针对性地提升VLM在多媒体推理中的可靠性提供了原则性基础。

Abstract: Compositional visual question answering requires Vision-Language Models (VLMs) to execute multiple reasoning operations like object selection, spatial relation resolution, and attribute verification. Despite strong aggregate performance, the mechanistic basis of VLM failures on this task remains underexplored. To address this gap, we analyze vision-operation misalignment in VLMs by examining how failures relate to specific reasoning operations and the internal computational pathways through which they arise and propagate. We introduce an Operation-centric mechanistic framework that decomposes VLM failures by both the reasoning operation where they originate and the internal computational pathway through which they propagate. Our analysis reveals four mechanistically distinct failure modes: grounding failure, reasoning failure, attribute extraction failure, and language prior dominance failure. Each characterized by a unique relationship between visual grounding strength and answer correctness. Through three complementary causal interventions applied across all transformer layers, we further demonstrate a pathway dissociation: grounding failures route exclusively through the feedforward network, reasoning failures route through late-layer attention, and attribute extraction failures localize to the answer-position feedforward computation. This dissociation demonstrates that different failure types require fundamentally different corrective strategies, providing a principled foundation for targeted improvements to VLM reliability in multimedia reasoning.

[54] Adaptive Contrast Enhancement and Optimised Feature Matching for RootSIFT-Based Palm-Vein Recognition cs.CVPDF

Kaveen Perera, Fouad Khelifi, Ammar Belatreche

TL;DR: 本文提出了一种用于掌静脉识别的自适应对比度增强与优化特征匹配方法。通过改进的ILACS-BGOT技术增强低对比度掌静脉图像，并结合RootSIFT特征、KNN+RT匹配及MMD滤波器，在多个数据集上进行了全面的参数分析。

Details

Motivation: 解决近红外光散射和传感器限制导致的掌静脉图像低对比度问题，这是掌静脉识别中的一个主要挑战。

Result: 在CASIA、PolyU和PUT三个基准数据集上，使用EER和准确率进行评估，通过测试42种MMD阈值与RT参数组合，所提系统在泛化能力和性能上均优于现有方法，取得了显著提升。

Insight: 创新点包括ILACS-BGOT增强方法（结合双向高斯加权重叠分块以减轻块状伪影），以及系统研究MMD滤波器和RT参数对识别性能的影响；该方法可泛化至其他低对比度生物特征识别（如指静脉、掌纹）及计算机视觉中的图像增强任务。

Abstract: Palm-vein recognition is a highly secure biometric modality due to the uniqueness and subcutaneous nature of vein patterns. However, low contrast in palm-vein images, caused by NIR light scattering and sensor limitations, remains a significant challenge. To address this, we propose the Intensity-Limited Adaptive Contrast Stretching with Bidirectional Gaussian-weighted Overlapping Tiles (ILACS-BGOT) method, an enhancement of the previously developed ILACS with Layered Gaussian-weighted Overlapping Tiles (ILACS-LGOT) technique. ILACS enhances local contrast, while BGOT mitigates blocky artefacts. This study further integrates RootSIFT features with KNN+RT and incorporates the previously introduced Mean and Median Distance (MMD) filter to investigate the parameter variations of both MMD and RT, and their impact on recognition performance. A comprehensive analysis was conducted across three benchmark datasets (CASIA, PolyU, and PUT), using 42 combinations of MMD filter thresholds and RT values. Results were evaluated using EER and Accuracy. Findings reveal that higher template sizes improve performance, while varying MMD thresholds reflect dataset-specific rotational variations. The proposed system demonstrates superior generalisability, achieving significant improvements in both EER and Accuracy over existing methods. Furthermore, the underlying ILACS-BGOT mechanism suggests potential applicability beyond palm vein recognition to other biometric modalities such as finger vein and palmprint recognition, and more generally to low-contrast image enhancement across computer vision applications.

[55] Attention-Guided Saliency Maps for Interpreting Visualization Literacy in VLMs cs.CV | cs.HCPDF

Maeve Hutchinson, Abderrahmane Wassim Mehdaoui, Pranava Madhyastha

TL;DR: 本文提出了一种轻量级的诊断性显著图方法，专门用于解释基于Transformer的视觉语言模型（VLMs）在数据可视化图像上的文本生成过程。该方法通过聚合语言模型在所有注意力头和层上对视觉标记的关注，并将其映射回视觉编码器的图像块网格，从而生成每个答案标记与所关注图像区域之间的直接对应关系。

Details

Motivation: 随着视觉语言模型在需要可靠推理的分析任务中应用日益广泛，理解这些模型如何解释数据可视化成为一个重要且尚未解决的问题。

Result: 作者使用删除指标评估了该方法，验证了所生成显著图对模型行为的因果忠实性。

Insight: 创新点在于提出了一种无需梯度、计算快速的显著图生成方法，能够直接揭示VLM在生成答案时对视觉元素的关注分配，从而检查模型注意力是否与语义相关组件对齐，为模型可解释性提供了新工具。

Abstract: Understanding how vision-language models (VLMs) interpret data visualizations remains an open problem, and is increasingly important as these models are used for analytical tasks where reliable reasoning is essential. We introduce a lightweight, diagnostic saliency map method tailored for text generation over images using transformer models, the current state-of-the-art models in visualization interpretation. Our approach aggregates the language model’s attention over the visual tokens across all heads and layers, then maps this attention back onto the vision encoder’s patch grid to localise it over the image, producing a direct correspondence between each generated answer token and the image regions it attended to. This yields fast, gradient-free saliency maps that expose how VLMs allocate focus across visual elements during answer generation, enabling inspection of whether model attention aligns with semantically relevant components. We evaluate our approach using a deletion metric which validates the causal faithfulness of our saliency maps to the model’s behavior.

[56] Vision-Language Assistant for Emotional Reactions to Risky Driving cs.CVPDF

Harine Choi, Eun Hak Lee, Zhengzhong Tu

TL;DR: 本研究提出了一种名为’Keep Yelling Assistant (KYA)’的视觉语言管道，用于实时检测危险驾驶行为并生成带有情感表达的语言响应，以提升驾驶员的安全意识和舒适度。该系统包含视觉模块（使用YOLOv8检测车辆和危险行为）和语言模块（利用大语言模型根据用户偏好生成情感化回应）。

Details

Motivation: 现有自动驾驶中的视觉语言模型主要关注感知与推理，但很少考虑情感维度或真实世界的用户体验。本文旨在解决这一问题，通过结合情感智能来支持驾驶员的安全与情感健康。

Result: 在包含危险驾驶行为的行车记录仪视频和108名参与者的用户研究中进行了评估。所有测试的大语言模型（如ChatGPT-4o、Claude 3）都获得了积极评价，其中YOLOv8s与ChatGPT-4o的组合在5分制中获得了最高分4.29分，展现了良好的情感对齐能力。

Insight: 创新点在于将实时危险驾驶行为检测与可定制的情感化语言生成相结合，为车载人工智能引入了情感智能的新范式。这为提升传统和自动驾驶汽车的安全性、信任度和情感福祉提供了有前景的方向。

Abstract: This study introduces a vision-language pipeline that detects risky driving behaviors and generates emotionally expressive responses to support driver awareness and comfort. Although vision-language models have advanced perception and reasoning in autonomous driving, existing systems rarely consider the emotional dimension or real-world user experience. Keep Yelling Assistant (KYA) detects high-risk driving maneuvers in real time, such as sudden cut-ins. It then produces emotional responses through a large language model tailored to driver preferences. The framework comprises two core modules. The vision module uses YOLOv8 variants to detect nearby vehicles and identify risky behaviors such as sudden cut-ins. Key driving metrics, including relative distance, speed, and projected reach time, are extracted and normalized to produce a structured behavior log. The language module processes this log with user-defined emotional tone settings, such as neutral, humorous, and analytical, and generates verbal reactions using state-of-the-art large language models, including ChatGPT-4o, Claude 3, Gemini 2.5, and Copilot. We evaluated the proposed system using dashcam videos containing risky driving behaviors and a user study involving 108 participants. Participants selected preferred response styles, and the large language models were evaluated based on emotional alignment. All models received favorable ratings, although preferences varied across personas. Notably, the combination of YOLOv8s and ChatGPT-4o achieved the highest score of 4.29 out of 5.00. By integrating real-world perception with emotionally adaptive dialogue, KYA introduces a new paradigm for emotionally intelligent in-vehicle artificial intelligence. It offers promising directions for improving safety, trust, and emotional well-being in both conventional and autonomous vehicles.

[57] MotionForesight: Re-purposing Video Models for Future 3D Scene-Flow Prediction cs.CVPDF

Homanga Bharadhwaj, Yash Jangir

TL;DR: MotionForesight 提出了一种方法，将预训练的视频模型重新用于预测未来3D场景流。该方法从单目视频中学习人类与物体交互的物理运动规律，通过冻结大型视频和跟踪组件，训练一个轻量级适配器，将回顾性跟踪表示转换为前向预测器，从而在未见过的物体、环境和交互中实现泛化。

Details

Motivation: 研究如何从人类与物体交互的单目视频中学习对物体未来运动的预测能力，这对于在现实世界中采取行动至关重要。

Result: 仅使用4万个人类视频且无需语言等辅助输入，MotionForesight 在多样化的分布外物体、环境、视角和交互中表现出良好的泛化能力，并且显著优于使用超过百万训练视频的更大模型。

Insight: 关键创新在于重新利用视频预测模型中已编码的关于物体在人类交互中如何运动的丰富先验知识，将其从像素预测转向未来3D场景流预测，并通过冻结大部分组件实现高效训练和泛化。

Abstract: Humans can infer how objects are likely to move from passive observation: a cup may be lifted, a drawer may slide, and a lid may rotate shut. Such predictions expose the physical consequences of interaction needed to act in the real world. We study how to learn this anticipation from ordinary monocular videos of human-object interaction. Given a short observed video context, MotionForesight predicts future 3D trajectories for points on the manipulated object. This casts interaction prediction as object-centered 3D motion forecasting without any assumptions on the object properties. Our key insight is that video prediction models already encode rich priors about how objects move during human interactions. We redirect these priors from pixel prediction toward future 3D scene flow. We start from a dense 3D tracker built on a pretrained video model, generate pseudo-ground-truth tracks from complete clips, and train the forecaster using only the observed frames. We replace future RGB and geometry with learned mask latents and train a lightweight adapter to turn the retrospective tracking representation into a forward predictor, while freezing the large video and tracking components. Using just 40k human videos and no auxiliary inputs such as language, MotionForesight generalizes across diverse out-of-distribution objects, environments, viewpoints, and interactions. It also outperforms substantially larger models that use over a million training videos. These results show that we can efficiently re-purpose video priors into explicit geometric forecasts for embodied intelligence. https://motionforesight.github.io/

[58] Searching Videos as Trees: Self-Correcting Agents for Grounded Long Video QA cs.CVPDF

Ce Zhang, Ziyang Wang, Yulu Pan, Oluwatumininu Oguntola, Pranav Wagh

TL;DR: 本文提出VideoTreeSearch（VTS）框架，将基于长视频的问答任务（Grounded LVQA）建模为在自适应时间树上的迭代自校正搜索。VTS通过构建非均匀时间树，并训练智能体使用缩放、平移和回答等离散操作在树上导航，从而支持显式的回溯和恢复机制，以解决现有智能体方法因缺乏细粒度到粗粒度的回溯能力而容易过早收敛或无法从早期错误中恢复的问题。

Details

Motivation: 现有基于智能体的Grounded LVQA方法通常采用多轮探索框架，但仅支持从粗到细的裁剪操作，缺乏从细到粗的显式回溯原语，导致智能体容易过早收敛且无法从早期错误中恢复。

Result: 在CG-Bench、Haystack-LVBench和Haystack-Ego4D三个Grounded LVQA基准测试上，VTS分别比先前最强的智能体方法提升了+12.5 mIoU和+7.4 T-F1。在通用长视频QA任务（Video-MME、MLVU、LVBench）上，VTS也超越了所有先前的智能体基线，最高提升+7.1个准确率点。

Insight: 核心创新在于将长视频问答任务建模为在自适应时间树上的自校正分层搜索，通过显式的回溯和恢复操作（如zoom_out）作为可学习的原语，使智能体能够主动纠正错误路径。此外，通过轨迹合成管道生成包含错误分支和恢复路径的训练数据，结合监督微调和强化学习进行策略训练，有效提升了模型的鲁棒性和性能。

Abstract: Grounded long-video question answering (Grounded LVQA) requires answering a question about a long video while localizing the short evidence interval that supports the answer. Recent agentic methods frame this task as multi-turn exploration with a single crop_video(start, end) action, which supports coarse-to-fine narrowing but provides no primitive for fine-to-coarse backtracking. As a result, these agents typically converge prematurely and cannot recover from an early mistake. We propose VideoTreeSearch (VTS), a framework that casts grounded LVQA as iterative self-correcting search over an adaptive temporal tree. VTS constructs a non-uniform tree from visual scene boundaries so that each node corresponds to a semantically coherent segment, and trains an agent to navigate the tree through four discrete operations: zoom_in, zoom_out, shift, and answer. These operations expose backtracking and recovery as explicit, learnable primitives rather than implicit behaviors. To train this navigation, we introduce a trajectory synthesis pipeline that produces multi-step paths through the tree, including deliberate detours into incorrect branches followed by recovery. We use these trajectories for supervised fine-tuning, followed by reinforcement learning with grounding and answer-accuracy rewards. On three Grounded LVQA benchmarks (CG-Bench, Haystack-LVBench, Haystack-Ego4D), VTS outperforms the strongest prior agentic methods by +12.5 mIoU on CG-Bench and +7.4 T-F1 on Haystack-Ego4D. The learned policy also transfers to general long-video QA, surpassing all prior agentic baselines on Video-MME, MLVU, and LVBench by up to +7.1 accuracy points. Ablations confirm that self-correcting hierarchical search is the central mechanism behind these gains: removing either adaptive descent or explicit backtracking substantially degrades performance. Code is available at https://github.com/CeeZh/VTS.

[59] FVAttn: Adaptive Sparse Attention with Runtime Load Balancing for Video Generation cs.CVPDF

Hao Liu, Chenghuan Huang, Ye Huang, Zhiying Wen, Hao Liu

TL;DR: 本文提出了FVAttn系统，用于解决视频扩散Transformer中自适应稀疏注意力在多GPU序列并行下的负载不均衡问题。该系统通过运行时负载平衡、松弛感知稀疏增强和计算重叠等技术，在不影响视频质量的前提下显著提升了注意力机制的执行效率。

Details

Motivation: 视频扩散Transformer处理长时空序列时，自注意力成为高分辨率视频生成的主要瓶颈。现有的免训练稀疏注意力方法（如自适应Top-p路由）在多GPU序列并行下会产生不均匀的每头工作量，导致负载异构性和rank级拖尾问题，限制了分布式执行效率。

Result: 在Wan2.2 I2V的步长蒸馏模型上，FVAttn将平均负载不均衡从1.34降低到1.08，注意力速度比FlashAttention快4.41倍，DiT推理速度提升2.02-2.11倍，同时保持了有竞争力的视频生成质量。

Insight: 创新点包括：1）结合Top-p路由、Top-k安全阈值和视频感知块组织的稀疏路由前端；2）运行时负载平衡通过P2P通信迁移少量重负载头来缩短关键路径；3）松弛感知稀疏增强利用非关键rank的松弛时间填充高价值块；4）计算重叠技术隐藏调度和迁移开销。系统级优化实现了免训练稀疏注意力的高效分布式执行。

Abstract: Video Diffusion Transformers process long spatio-temporal sequences, making self-attention the main bottleneck in high-resolution video generation. Training-free sparse attention reduces this cost, but adaptive Top-$p$ routing creates uneven per-head workloads under multi-GPU sequence parallelism. The resulting workload heterogeneity turns sparse attention into a rank-level straggler problem. We present \method{}, a training-free sparse-attention system that improves the distributed execution efficiency of adaptive sparse attention under multi-GPU sequence parallelism. \method{} uses Top-$p$ routing, a Top-$k$ safety floor, and video-aware block organization as the sparse-routing frontend, then repairs the materialized mask at runtime. Runtime Load Balancing migrates a small number of heavy heads via P2P communication to shorten the current critical path. Slack-Aware Sparse Augmentation fills residual non-critical-rank slack with additional high-value blocks, while overlap hides scheduling and migration overhead behind existing computation. On step-distilled Wan2.2 I2V, \method{} reduces average load imbalance from 1.34 to 1.08 and delivers a $4.41\times$ attention speedup over FlashAttention, while achieving a $2.02$–$2.11\times$ DiT inference speedup with competitive video quality.

[60] Knowing the Self, Understanding the World: A Dual-Cognition Benchmark for UAV Spatio-temporal Reasoning with MLLMs cs.CVPDF

Like Liu, Zhengzheng Xu, Haitao He, Hongzhe Li, Shuchang Zhang

TL;DR: 本文提出了UAV-DualCog基准，用于评估多模态大语言模型在无人机场景下的双重认知能力，即同时推理无人机自身状态和外部环境状态。该基准包含图像和视频任务，要求进行时空定位，并通过自动化流程从语义点云构建大规模数据集。实验表明，现有MLLMs在该基准上表现不佳，尤其在自身状态推理、视点变换、精确空间定位和时间间隔定位方面存在瓶颈。

Details

Motivation: 现有面向无人机的基准测试主要关注场景理解、事件识别或导航完成，缺乏对无人机智能体所需双重认知能力的联合评估，即同时推理自身状态和多视角时空上下文中的外部环境状态。

Result: 在UAV-DualCog基准上的广泛评估表明，当前MLLMs在无人机双重认知方面远未达到可靠水平；与人类基线相比，该基准对人类可理解但对现有模型具有挑战性。

Insight: 创新点在于提出了基于双重认知视角的无人机多视角时空推理基准，并开发了从场景级语义点云自动构建可扩展数据集的流程；该基准不仅可用于评估，其衍生的训练集（UAV-DualCog-Train）也能为推进基于MLLM的无人机智能体提供有价值的结构化监督数据。

Abstract: Multimodal large language models have achieved strong performance across diverse vision-language tasks, yet their capabilities in UAV scenarios remain insufficiently explored. Recent UAV-oriented benchmarks have begun to evaluate MLLMs in aerial scenarios, but they typically focus on scene understanding, event recognition, or navigation completion, rather than jointly assessing the dual-cognition capability required for UAV agents: reasoning about both the UAV’s own state and the external environment in multiview spatio-temporal contexts. To address this gap, we present UAV-DualCog, a benchmark for aerial multiview spatio-temporal reasoning built on this dual-cognition perspective. UAV-DualCog includes both image and video tasks to jointly evaluate self-state and environment-state reasoning, while requiring spatial or temporal grounding beyond discrete answer prediction. We also develop an automated pipeline that constructs data from scene-level semantic point clouds, yielding a scalable benchmark with diverse scenes, hundreds of landmarks, and thousands of QA samples. Extensive evaluations show that current MLLMs remain far from reliable in UAV dual cognition. Self-state reasoning, viewpoint transformation, precise spatial grounding, and temporal interval localization are persistent bottlenecks, and additional validation with thinking/frontier models and a human baseline confirms that the benchmark is understandable to humans but challenging for existing models. We further construct UAV-DualCog-Train from disjoint scenes and show through a lightweight optimization probe that it provides useful structured supervision, suggesting its value not only as an evaluation benchmark but also as a data resource for advancing MLLM-based UAV agents. Project website and supplementary materials: https://uav-dualcog.lozumi.com

cs.LG [Back]

[61] Understanding Reasoning from Pretraining to Post-Training cs.LG | cs.AI | cs.CLPDF

Jingyan Shen, Ang Li, Salman Rahman, Yifan Sun, Micah Goldblum

TL;DR: 该论文使用国际象棋作为受控测试平台，研究了从预训练到后训练（特别是强化学习）的整个流程对大型语言模型推理能力的影响。研究发现，预训练损失和预训练数据量可以预测后训练强化学习的性能，并且强化学习在不同难度任务上对模型行为的影响不同。

Details

Motivation: 目前强化学习后训练通常与预训练阶段孤立研究，导致两个基本问题未解：预训练选择（模型大小、数据）如何影响强化学习的收益，以及强化学习实际对模型做了什么。标准LLM设置难以研究这些问题，因为预训练语料库庞大且不可控，且系统性地进行两阶段计算扫描成本过高。

Result: 在国际象棋测试中，给定强化学习计算水平下的后训练性能可由预训练损失预测，且强化学习奖励曲线的斜率随预训练词元数量近似线性提升。在数学领域文本上训练的1B模型也出现了相同的预测模式。

Insight: 论文提供了一个从预训练到强化学习接口的定量描述，并创建了一个受控测试平台来研究整个预训练到后训练流程中的推理科学。创新之处在于揭示了强化学习并非简单地锐化监督微调策略，而是在简单任务上放大已有偏好，在困难任务上发掘监督微调中几乎不存在的正确动作，且这一模式可迁移到其他领域（如数学）。

Abstract: Reinforcement learning (RL) has become central to improving large language models (LLMs) on complex reasoning tasks, yet RL post-training is largely studied in isolation from the pretraining that precedes it. As a result, two basic questions remain open: (1) how do pretraining choices (model size, data) shape the returns to RL compute, and (2) what does RL actually do to the model? These questions are difficult to study in the standard LLM setting: pretraining corpora are vast and uncontrolled, making it hard to attribute behaviors to pretraining versus RL, and systematic compute sweeps across both stages are prohibitively expensive. To address these challenges, we use chess as a controlled testbed for studying reasoning across the full pretraining-to-post-training pipeline. We follow the standard LLM training pipeline by pretraining language models from 5M to 1B parameters on human chess games, supervised fine-tuning on synthetic reasoning traces, and running RL on chess puzzles with verifiable rewards. Using this framework, we find that the post-RL performance at given RL compute level is well-predicted from the pretraining loss, and slope of the RL reward curves improves approximately linearly with the pretraining tokens. Beyond scaling, we find that RL does not simply sharpen the SFT policy: on easy puzzles it amplifies correct moves the SFT policy already preferred, while on hard puzzles it surfaces correct moves that were nearly absent under SFT. We further test whether our findings transfer beyond chess by training a 1B language model on math-domain text, where the same predictive pattern emerges: longer-pretrained checkpoints reach higher post-RL performance and improve faster under RL. In sum, we provide a quantitative account of the pretraining-to-RL interface and a controlled testbed for studying the science of reasoning across the full pretraining-to-post-training pipeline.

q-bio.NC [Back]

[62] STSBench: A Large-Scale Dataset for Modeling Neuronal Activity in the Dorsal Stream of Primate Visual Cortex q-bio.NC | cs.CVPDF

Ethan B. Trepka, Ruobing Xia, Shude Zhu, Sharif Saleki, Danielle Abreu Lopes

TL;DR: 本文介绍了STSBench，一个大规模数据集，用于建模灵长类视觉皮层背侧流中的神经元活动。该数据集包含超过2000个神经元在观看数千个独特自然视频时的单神经元记录，规模比现有背侧流数据集大近50倍。

Details

Motivation: 由于缺乏涵盖背侧流区域的大规模数据集，用于预测背侧流神经元响应的模型发展不足，本文旨在填补这一空白。

Result: 研究表明，STSBench数据集可用于对背侧流神经元响应的编码模型进行基准测试，并从神经活动中重建视觉输入。

Insight: 创新点在于构建了前所未有的、大规模的单神经元记录数据集，为研究背侧流（负责空间关系和运动编码）的神经机制提供了关键资源，弥补了与腹侧流研究在数据规模上的差距。

Abstract: The primate visual system is typically divided into two streams - the ventral stream, responsible for object recognition, and the dorsal stream, responsible for encoding spatial relations and motion. Recent studies have shown that convolutional neural networks (CNNs) pretrained on object recognition tasks are remarkably effective at predicting neuronal responses in the ventral stream, shedding light on the neural mechanisms underlying object recognition. However, similar models of the dorsal stream remain underdeveloped due to the lack of large scale datasets encompassing dorsal stream areas. To address this gap, we present STSBench, a dataset of large-scale, single neuron recordings from over 2,000 neurons in the superior temporal sulcus (STS), a nearly 50-fold increase over existing dorsal stream datasets, collected while Rhesus macaques viewed thousands of unique, natural videos. We show that our dataset can be used for benchmarking encoding models of dorsal stream neuronal responses and reconstructing visual input from neural activity.

eess.AS [Back]

[63] Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos eess.AS | cs.CVPDF

Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar, Lasha Koroshinadze, Nishit Anand

TL;DR: 本文提出了Audio-Visual Flamingo (AV-Flamingo)，一个完全开源的、最先进的视听大语言模型，用于对音频、图像和长视频进行联合理解与推理。该模型针对长且复杂的真实世界视听视频设计，通过大规模数据集、三阶段课程学习策略和一种新颖的时序视听交错思维链推理框架，显著提升了长视频理解能力。

Details

Motivation: 解决现有视听大语言模型主要关注短视频片段，而缺乏对长且复杂的真实世界视听视频进行有效理解和推理能力的问题。

Result: 在超过15个视听、全模态、音频和视觉基准测试中，AV-Flamingo明显优于同等规模的开源模型，并与甚至在某些情况下超越了规模大得多的开源权重和闭源模型，尤其是在长且复杂的真实世界视听理解与推理任务上。

Insight: 创新点包括：构建了大规模、强调时序、组合和跨模态推理的Audio-Visual-Skills数据集；设计了从短程感知到长程多事件推理的三阶段渐进式课程学习策略；提出了将中间推理步骤显式关联到长视听流时间戳的Temporal Audio-Visual Interleaved Chain-of-Thought推理框架，提升了时序对齐和可解释性。

Abstract: We present Audio-Visual Flamingo (AV-Flamingo), a fully open state-of-the-art audio-visual large language model (AV-LLM) for joint understanding and reasoning over audio, images, and long-form videos. Unlike prior AV-LLMs that primarily focus on short clips, AV-Flamingo is designed for understanding and reasoning over long and complex real-world (audio-visual) videos. To support this, we make three key contributions: (i) Audio-Visual-Skills, a large-scale collection of real-world videos with ~7M caption and question-answer training instances designed to emphasize temporal, compositional, and cross-modal audio-visual reasoning; (ii) a novel three-stage curriculum that progressively trains the model from short-range perception to long-horizon multi-event reasoning; and (iii) Temporal Audio-Visual Interleaved Chain-of-Thought, a reasoning framework that explicitly grounds intermediate reasoning steps to timestamps in long audio-visual streams, improving temporal alignment and interpretability. Extensive experiments across 15+ audio-visual, omni-modal, audio, and vision benchmarks show that AV-Flamingo outperforms similarly sized open models by clear margins and remains highly competitive with, and in some cases surpasses, much larger open-weight and closed models, particularly on long and complex real-world audio-visual understanding and reasoning tasks. Beyond benchmark performance, AV-Flamingo exhibits strong real-world utility and transfers well to unseen tasks, highlighting its robustness and generalization ability.

cs.AI [Back]

[64] DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings cs.AI | cs.CVPDF

Yoonhwa Jung, Junryu Fu, Mani Golparvar-Fard

TL;DR: 论文提出了DrawingVQA，这是首个针对真实世界建筑图纸的多模态大语言模型（MLLM）评估基准。该基准包含33张施工图和92个专家精心设计的问题-答案对，涵盖感知理解、上下文解释和领域专家推理三个推理深度。论文还引入了一个双重分类框架来联合分析模型在七个建筑工程维度和四个MLLM能力维度上的表现，评估发现当前SOTA MLLMs与专家表现存在显著差距。

Details

Motivation: 现有MLLM基准主要针对自然图像或示意图，缺乏对建筑工程实践中核心媒介——融合了抽象几何、符号标记、表格数据和领域文本的复杂施工图纸的评估。DrawingVQA旨在填补这一空白，推动AI驱动的理解与真实世界工程工作流的整合。

Result: 在DrawingVQA基准上对SOTA MLLMs的评估显示，模型性能与专家表现存在巨大差距，尤其是在更高层次的推理深度上。

Insight: 创新点在于创建了首个针对真实世界施工图纸的多模态VQA基准，并提出了一个将工程工作流映射到AI推理能力的双重分类评估框架，为领域专业化的多模态推理研究奠定了基础。

Abstract: We introduce DrawingVQA, the first benchmark designed to evaluate multimodal large language models (MLLMs) on real-world construction drawings – a core media in architecture, civil, and many other engineering practices. Unlike natural images or schematic floor plans, construction drawings fuse abstract geometry, symbolic notation, tabular data, annotations, and domain-specific text, forming a uniquely complex visual-textual domain core to engineering workflows. DrawingVQA bridges this gap with 33 “Issued for Construction” drawings and 92 expertly curated question-answer pairs, spanning three reasoning depths: perceptual understanding, contextual interpretation, and domain-expert reasoning. To evaluate model capabilities, we present a dual categorization framework to jointly analyze performance across seven construction-engineering and four MLLM capability dimensions – the first to explicitly map engineering workflows to AI reasoning competencies. Evaluations of state-of-the-art MLLMs reveal a substantial gap between model and expert performance, particularly at higher reasoning depths. This benchmark lays a foundation for domain-specialized multimodal reasoning to allow for advancement on integration of AI-driven understanding and real-world engineering workflows.

cs.RO [Back]

[65] Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories cs.RO | cs.CVPDF

Xiaomi Robotics Team, Jun Guo, Piaopiao Jin, Jason Li, Peiyan Li

TL;DR: 本文介绍了小米机器人1号（Xiaomi-Robotics-1），这是一个基础性的视觉-语言-动作（VLA）模型。它能够遵循多样化的语言指令，在未见过的环境中开箱即用地执行广泛的移动操作任务，并且能够以少量微调数据高效适应新的下游任务。模型采用包含预训练和后训练的两阶段训练方案，在超过10万小时的真实世界操作轨迹上进行训练，并通过可扩展的自动标注流程为轨迹片段添加描述场景状态转换的自然语言标签。

Details

Motivation: 旨在构建一个能够泛化到新环境、遵循自然语言指令执行移动操作任务，并能高效适应新任务的通用机器人基础策略模型。

Result: 在多个模拟基准测试中超越了最先进方法。具体而言，在RoboCasa365上以57.6%的成功率创下了新的SOTA（此前最佳为46.6%），在RoboDojo上平均得分20.07，显著优于之前的SOTA（13.07）。实验还展示了模型性能随预训练数据规模和模型大小增加而持续提升的良好缩放特性。

Insight: 核心创新在于两阶段训练方案，特别是利用大规模真实世界轨迹和可扩展的自动语言标注流程来赋予模型广泛且可泛化的动作生成能力。其缩放行为（scaling behavior）表明，更强的预训练模型能直接转化为更好的开箱即用机器人性能，这为构建通用机器人基础模型提供了重要路径。

Abstract: We present Xiaomi-Robotics-1, a foundational vision-language-action (VLA) model capable of (1) following diverse language instructions to perform a wide range of mobile manipulation tasks in unseen environments out-of-the-box, and (2) efficiently adapting to novel downstream tasks with minimal fine-tuning data. We propose a two-stage training recipe consisting of pre-training and post-training. During pre-training, we imbue the model with broad and generalizable action-generation capabilities by training on over 100k hours of real-world manipulation trajectories collected via UMI devices. Crucially, we develop a scalable auto-labeling pipeline that annotates trajectory clips with natural languages describing scene state transitions, providing rich and precise conditioning for action learning. During post-training, we aim to align these capabilities with robot embodiments and imperative instructions that humans naturally use to prompt robots. Extensive experiments demonstrate strong scaling behavior. Xiaomi-Robotics-1 consistently improves with increased data scales and model sizes during pre-training. This scaling behavior directly transfers to post-training, where a stronger pre-training model yields better out-of-the-box real-robot performance in unseen environments. Furthermore, Xiaomi-Robotics-1 serves as a strong robot foundation policy that can be efficiently fine-tuned on complex, dexterous tasks with high data efficiency. Across multiple simulation benchmarks, Xiaomi-Robotics-1 outperforms state-of-the-art methods. Notably, it establishes a new state-of-the-art with a 57.6% success rate on RoboCasa365, surpassing the previous best of 46.6%. Furthermore, it achieves an average score of 20.07 on RoboDojo, significantly outperforming the prior state-of-the-art (13.07). Code and model checkpoints will be released. Project page: https://robotics.xiaomi.com/xiaomi-robotics-1.html

Hadrien Crassous, Mohamed Yassine Kabouri, Minahil Raza, Joni Pajarinen, Riad Akrour

TL;DR: 本文研究在机器人导航时间和标注预算双重约束下，如何通过主动学习策略将计算机视觉目标检测器适应到未知环境中。该方法选择信息丰富的机器人轨迹和图像样本来重新训练检测器，并明确针对其失败案例。通过利用空间一致性识别标签不一致的图像，实验在AI2-THOR模拟器和真实世界的Boston Dynamics Spot机器人上验证了该方法的有效性。

Details

Motivation: 解决在未知环境中，同时面临机器人导航时间有限和标注预算有限的情况下，如何高效地适应目标检测器的问题。

Result: 在AI2-THOR模拟器和真实世界使用Boston Dynamics Spot机器人及YOLOv5检测器的实验中，与多个基线方法相比，该方法在相同预算下实现了最高的检测精度。

Insight: 创新点在于提出了一个具身化的批量主动学习变体，利用空间一致性来识别标签不一致的图像，从而无需外部监督即可指导机器人并选择相关图像进行标注。

Abstract: This paper studies how to adapt a computer vision object detector to an unknown environment under both a robot navigation time and annotation budget constraint. Our approach selects informative robot trajectories and image samples to retrain the detector, explicitly targeting its failure cases. Formally, the approach is an embodied variant of batch active learning, where at each round an agent has a limited navigation budget to collect candidate samples and a limited annotation budget for the most relevant images. We leverage spatial consistency to identify images with inconsistent labels, which are likely to provide the greatest improvement to the vision model. We evaluate the approach using different active learning objectives on large scenes from the AI2-THOR simulator and on a real-world setup using a Boston Dynamics Spot robot with the real-time object detector YOLOv5. Through comparison against several baselines, our experimental results show that spatial inconsistency helps guide the agent and select relevant images without external supervision, achieving the highest detection accuracy at the end of the adaptation process under the same budget. The open-source project can be found at https://mkabouri.github.io/embodied-active-learning-od

[67] VTLoc: Learning-based Tactile Contact Localization in Visual Point Clouds cs.RO | cs.CVPDF

Zhiyuan Wu, Zhuo Chen, Shan Luo

TL;DR: VTLoc是一个新颖的视觉-触觉框架，旨在解决机器人感知中触觉接触点定位的挑战。它利用3D点云作为视觉输入，通过几何多模态对齐模块和迭代定位更新器，从触觉读数中预测物体表面的接触位置。

Details

Motivation: 视觉和触觉是机器人感知和操作中互补的模态，但将两者集成以进行接触定位面临空间对齐的挑战，需要精确匹配触觉数据与视觉几何。

Result: 在包含100个真实世界物体的新基准测试中，VTLoc通过减少局部到全局的对应模糊性，改进了单点触觉接触定位的性能。

Insight: 创新点包括几何多模态对齐模块，通过重建伪点云并与视觉点云对齐来强制跨模态空间一致性，以及迭代定位更新器，利用融合的视觉-触觉特征迭代优化预测的接触位置。

Abstract: Vision and touch are complementary modalities essential for robotic perception and manipulation. While vision provides global object context, touch offers precise local information at contact points. Integrating these modalities for contact localization, i.e., predicting the location of touch on an object’s surface, poses significant challenges due to the need for accurate spatial alignment between tactile data and visual geometry. To address this challenge, we propose VTLoc, a novel visual-tactile framework that localizes contact points from tactile readings using a 3D point cloud as visual input. VTLoc introduces two key components: a geometric multi-modal alignment module, which reconstructs a pseudo-point cloud from fused visual-tactile features and aligns it with the visual point cloud to enforce spatial consistencies across modalities; and an iterative localizing updater, which iteratively refines the predicted contact location using fused visual-tactile features. Evaluated on a new benchmark of 100 real-world objects, VTLoc improves single-touch contact localization by reducing local-to-global correspondence ambiguity.

cs.MM [Back]

[68] MLLM-DataEngine: Closing the Loop of Multimodal Instruction Tuning Data Generation cs.MM | cs.CV | cs.LGPDF

Zhiyuan Zhao, Bin Wang, Linke Ouyang, Yiqi Lin, Pan Zhang

TL;DR: 本文提出了MLLM-DataEngine，一个用于多模态大语言模型（MLLM）指令微调数据生成的闭环系统。该系统通过分析模型在评估中的弱点，自动生成针对性的增量数据集，并迭代地提升模型能力，实现了数据生成、模型训练与评估的闭环。

Details

Motivation: 解决现有指令微调数据集收集方法通常与基准测试分离，缺乏针对性，无法有效提升MLLM特定能力的问题。

Result: 大量实验表明，MLLM-DataEngine能够以针对性和自动化的方式提升MLLM的能力，无需人工参与。

Insight: 创新点在于构建了一个完整的“分析弱点-生成数据-训练评估”的闭环迭代框架，并设计了自适应坏例采样模块和利用GPT-4结合上下文示例生成高质量数据的方法，为MLLM的数据管理提供了一个通用解决方案。

Abstract: In this paper, we propose MLLM-DataEngine, a novel closed-loop system that bridges data generation, model training, and evaluation. Within each loop iteration, the MLLM-DataEngine first analyzes the weakness of the model based on the evaluation results, then generates a proper incremental dataset for the next training iteration, and enhances the model capability iteratively. Compared with previous instruction fine-tuning dataset collection methods which are separate from the benchmarking, MLLM-DataEngine shows better targeting and can improve MLLMs’s capabilities more effectively. Firstly, we propose an Adaptive Bad-case Sampling module, which can effectively analyze model weakness based on the benchmarking results and adjust the generation of incremental datasets flexibly. Secondly, in order to ensure high-quality data for specific capability types, the most representative in-context examples and abundant information are provided to GPT-4, which helps GPT-4 fully comprehend the model’s weakness and further guarantees high-quality generated data. Through extensive experiments, we find MLLM-DataEngine could boost the MLLMs capability in a targeted and automatic manner without human participants. We hope MLLM-DataEngine could be a general solution for the following MLLMs data curation. Code, data, and model are available at https://github.com/opendatalab/MLLM-DataEngine.

cs.CR [Back]

[69] Hidden in Thought: Transferable Chain-of-Thought Artifacts Induce Harmful Behavior cs.CR | cs.CL | cs.LGPDF

Ali khalil, Aly M. Kassem, Mohamed Abdelrazek, Santu Rana, Negar Rostamzadeh

TL;DR: 本文研究了有害思维链（CoT）痕迹如何从被攻击的语言模型转移到其他模型，并提炼成可复用的越狱攻击。实验表明，移植有害CoT能显著提升开源模型的有害响应率，而通过LLooM概念挖掘识别出的四种有害推理模式（程序化、伦理脱钩、规避、目标-漏洞框架）可进一步提炼为有效的系统提示，在黑盒越狱中表现优异，尤其在强对齐模型上效果提升显著。

Details

Motivation: 动机是探究有害思维链痕迹是否能在语言模型间转移不安全行为，并提炼成可复用的越狱攻击，以揭示推理过程本身的安全漏洞。

Result: 在29个开源和5个闭源目标模型上，移植有害CoT使最脆弱开源模型的有害响应率超过80%；提炼的模式在强对齐模型上表现更佳，如在GPT-4.1 AdvBench上提升10倍；推理能力强的模型脆弱性加倍，且输出端防护（如Llama-Guard 3）常漏检有害生成。

Insight: 创新点在于发现有害推理在痕迹和模式层面均可转移，并识别出四种核心有害推理组件；客观分析表明，这强调了评估推理上下文而非仅最终输出的防御必要性，为模型安全提供了新视角。

Abstract: We investigate whether harmful chain-of-thought (CoT) traces from compromised language models can transfer unsafe behaviour and be distilled into reusable jailbreak attacks. Using an emergent-misalignment organism and a refusal-ablated jailbroken organism, we transplant harmful CoTs into $29$ open-source and $5$ closed-source targets. Transferred traces raise harmful-response rates above $80%$ on the most vulnerable open-source models, while semantically mismatched CoTs fail entirely. LLooM concept mining identifies four recurring components of harmful reasoning: proceduralisation, ethical decoupling, evasion, and target–vulnerability framing. Distilling these patterns into reusable system prompts produces effective black-box jailbreaks, outperforming direct CoT transplantation on strongly aligned models by up to an order of magnitude, including a $10\times$ improvement on GPT-4.1 AdvBench. Reasoning-enabled models are more than twice as vulnerable, and output-side safeguards such as Llama-Guard~3 frequently miss harmful generations. Our results show that harmful reasoning transfers at both the trace and pattern levels, motivating defences that evaluate reasoning context in addition to final outputs.

[70] Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents cs.CR | cs.CV | cs.LGPDF

Halima Bouzidi, Mboutidem Ekemini Mkpong, Mohammad Abdullah Al Faruque

TL;DR: 本文提出了一种名为Lucid的黑盒对抗攻击框架，针对依赖长期记忆的多模态AI代理。该框架通过向视觉数据添加人眼难以察觉的扰动，能够在两种场景下（有历史文本上下文和无历史文本上下文）破坏代理的记忆系统，使其产生攻击者期望的错误响应。

Details

Motivation: 多模态AI代理日益依赖持久性长期记忆来基于过去的视觉和文本片段进行生成，但无条件信任视觉数据会带来关键安全漏洞。本文旨在揭示并利用这一漏洞，在严格图像威胁模型下攻击多模态记忆管道。

Result: 在多种对话领域和五种黑盒记忆架构（包括图结构、LLM摘要和商业部署系统）上的评估表明，Lucid在记忆投毒攻击上达到61.6%的攻击成功率（ASR），在记忆注入攻击上达到58.4%的ASR，暴露了多模态记忆管道的结构性漏洞。

Insight: 论文的创新点在于提出了一个无需访问目标MLLM、检索编码器或文本通道的纯黑盒视觉攻击框架，并区分了基于历史文本上下文可用性的两种攻击模式。其核心洞察是，仅通过视觉扰动即可有效误导依赖记忆的多模态代理，这为AI系统的安全性和鲁棒性评估提供了新的攻击视角。

Abstract: Multimodal AI agents increasingly rely on persistent long-term memory to ground generation in past visual and textual episodes. We show that unconditional trust in visual data creates a critical vulnerability. We propose Lucid, a black-box adversarial framework that compromises multimodal memory pipelines under a strictly image-bounded threat model, requiring no access to the target MLLM, target retrieval encoder, or the text channel. Lucid crafts imperceptible perturbations to enable two distinct failure modes based on the availability of historical context: (1) Memory poisoning, an in-context attack where the adversarial image replaces a benign one whose content is reinforced by prior textual context, reliably corrupting visual recall and steering the agent toward attacker-chosen narratives; (2) Memory injection, an out-of-context attack where the adversarial image replaces a benign one in a conversation turn devoid of prior textual grounding, causing the agent to generate attacker-influenced responses with no corrective signal from memory. We evaluate Lucid across various conversation domains and five black-box memory architectures, including graph-structured, LLM-summarized, and commercially deployed systems. Lucid achieves 61.6% ASR on poisoning and 58.4% ASR on injection, exposing a structural vulnerability in multimodal memory pipelines.

Table of Contents

cs.CL [Back]

[1] Verbalizable Representations Form a Global Workspace in Language Models cs.CL | cs.AI | cs.LGPDF

[2] Large Language Models as Unified Multimodal Learners for Clinical Prediction cs.CL | cs.AIPDF

[3] EpiNarrate: Agentic Generation of Grounded Narratives from Epidemiological Scenario Projections cs.CLPDF

[4] Process Reward Informed Tree Rollout for Effective Multi-Turn RL cs.CL | cs.AI | cs.LGPDF

[5] Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning cs.CLPDF

[6] CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations cs.CL | cs.AIPDF

[7] DECODEM: Data Extraction from Corporate Organizational Documents via Enhanced Methods cs.CL | cs.AI | cs.CYPDF

[8] From Plausible to Actionable: A Position on LLM Self-Explanations cs.CLPDF

[9] Loop the Loopies! cs.CL | cs.AIPDF

[10] Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning cs.CL | cs.AIPDF

[11] Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D cs.CLPDF

[12] ToolSciVer: Multimodal Scientific Claim Verification with Visual Tool Augmented Reinforcement Learning cs.CL | cs.AIPDF

cs.CV [Back]

[13] An Empirical Study of Handcrafted Feature Learning and Convolutional Neural Networks for Facial Expression Recognition cs.CV | cs.LGPDF

[14] Rethinking the Readout: Unlocking Video Backbones for AI-Generated Video Detection cs.CVPDF

[15] Reasoning-Guided Part-Level Visual Grounding via Reinforcement Learning cs.CVPDF

[16] Training-Free Open-Vocabulary 3D Point-Cloud Segmentation on the Generalized Few-Shot Benchmark cs.CVPDF

[17] Explicit Over Implicit: Enhancing CNNs Via Complex Structure Tensor Representations for Periocular Recognition cs.CVPDF

[18] Unsupervised Keypoints for Real-Time Fall Detection: Comparative Analysis Under Real-world Conditions with Predictive Bandwidth Reduction cs.CV | cs.LGPDF

[19] Intentional Electromagnetic Interference Attacks on Facial Recognition cs.CV | cs.CR | cs.LGPDF

[20] Trajectory-aware Cross-view Geo-localization with Sequential Observations cs.CVPDF

[21] E3DGS: Unified Geometric-Photometric Equivariance for 3D Gaussian Splatting via Color-as-Geometry Embedding cs.CV | cs.LGPDF

[22] SLAPBench: Benchmarking Multimodal Large Language Models for Four-Finger SLAP Fingerprint Verification cs.CV | cs.AIPDF

[23] When Can Test-Time Adaptation Help Zero-Shot CT Vision-Language Models? cs.CVPDF

[24] Are All Tokens Necessary for Visual Place Recognition? An Empirical Study of Token Reduction for Efficient Inference cs.CVPDF

[25] Ask Twice, Look Twice: Prompt Echoing Resolves the Question-First Paradox in Vision-Language Models cs.CV | cs.AI | cs.LG | eess.IVPDF

[26] Region-Grounded Vision-Language Learning for Detection-Guided Mammographic Lesion Classification cs.CVPDF

[27] Benchmarking MRI Representations for Deep Learning-Based Focal Cortical Dysplasia Segmentation cs.CVPDF

[28] Model Merging for Medical LVLMs: A Benchmark and a Winner-Take-All Approach cs.CVPDF

[29] Efficient Frame Selection for Long Videos at Test Time with Attention-Based MLLM Selectors cs.CVPDF

[30] PE-Field 4D: Video Generation Models as Canvas cs.CVPDF

[31] HCIG: A Hierarchical Cross-Modal Incongruity Graph Network for Multimodal Sarcasm and Cyberbullying Detection cs.CV | cs.AI | cs.CLPDF

[32] GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Tracking cs.CVPDF

[33] Event3R: Asynchronous-to-Global 3D Reconstruction from Event Camera via Spatial-Temporal Feature Aggregation cs.CVPDF

[34] An Exam for Active Observers cs.CV | cs.AI | cs.CL | cs.LGPDF

[35] IoUPD: IoU-Aware Privileged Distillation for Visual Grounding with Multimodal Large Language Models cs.CVPDF

[36] Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Merging cs.CVPDF

[37] Debiasing Text-to-Image Evaluation via Implicit Cultural Alignment Reward Modeling cs.CV | cs.AI | cs.LG | cs.MMPDF

[38] SlotMem: Character-Addressable Internal Memory for Narrative Long Video Generation cs.CVPDF

[39] GeoChrono: Benchmarking and Rethinking Long-Term Temporal Understanding in Remote Sensing cs.CV | cs.AIPDF

[40] Modularized Dynamic-Granularity Video LLM for Multi-Event Long Video Understanding cs.CV | cs.AIPDF

[41] Multimodal Ambivalence and Hesitancy Recognition via Cross-Attention and Gated Fusion cs.CVPDF

[42] On the Geometry of Learned Representations in Event-Based Multi-Modal Egomotion Estimation cs.CV | cs.AIPDF

[43] Test-Time Noise Guided Adaptation for Realistic Autoregressive Video Generation cs.CV | cs.AIPDF

[44] Von Mises-Fisher Mixture Model with Dynamic Shrinkage for Realistic Test-Time Transduction cs.CV | cs.LGPDF

[45] MDND: Unsupervised Learning Guided by Non-Differentiable Refinement for Shape Correspondence cs.CVPDF

[46] EgoExoMoCap: Distributed Ego-Exo Human Motion Capture cs.CV | cs.AI | cs.GR | cs.HC | cs.ROPDF

[47] Exo2EgoPose: Leveraging Exocentric Demonstrations for Vision-Language guided Egocentric 3D Hand Pose Forecasting cs.CVPDF

[48] Orbis 2: A Hierarchical World Model for Driving cs.CV | cs.AI | cs.LG | cs.ROPDF

[49] More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe cs.CV | cs.LGPDF

[50] PIXIE: A Zero-Shot texture-invariant 6D pose estimation framework for unseen objects with assembly defects cs.CV | cs.ROPDF

[51] DPNeXt: A Lightweight Multi-Scale Feature Fusion Framework for Efficient ViT-Based Multi-Task Dense Prediction cs.CV | cs.AI | cs.ROPDF

[52] Multi-Modal Semantic Segmentation of Electrolyzer Components for Sustainable Hydrogen Technologies: A Dual-Branch Deep Learning Approach cs.CVPDF

[53] How Do VLMs Fail? Vision-Operation Misalignment in Compositional VQA cs.CVPDF

[54] Adaptive Contrast Enhancement and Optimised Feature Matching for RootSIFT-Based Palm-Vein Recognition cs.CVPDF

[55] Attention-Guided Saliency Maps for Interpreting Visualization Literacy in VLMs cs.CV | cs.HCPDF

[56] Vision-Language Assistant for Emotional Reactions to Risky Driving cs.CVPDF

[57] MotionForesight: Re-purposing Video Models for Future 3D Scene-Flow Prediction cs.CVPDF

[58] Searching Videos as Trees: Self-Correcting Agents for Grounded Long Video QA cs.CVPDF

[59] FVAttn: Adaptive Sparse Attention with Runtime Load Balancing for Video Generation cs.CVPDF

[60] Knowing the Self, Understanding the World: A Dual-Cognition Benchmark for UAV Spatio-temporal Reasoning with MLLMs cs.CVPDF

cs.LG [Back]

[61] Understanding Reasoning from Pretraining to Post-Training cs.LG | cs.AI | cs.CLPDF

q-bio.NC [Back]

[62] STSBench: A Large-Scale Dataset for Modeling Neuronal Activity in the Dorsal Stream of Primate Visual Cortex q-bio.NC | cs.CVPDF

eess.AS [Back]

[63] Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos eess.AS | cs.CVPDF

cs.AI [Back]

[64] DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings cs.AI | cs.CVPDF

cs.RO [Back]

[65] Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories cs.RO | cs.CVPDF

[66] Embodied Active Learning under Limited Annotation and Navigation Budget for Object Detection cs.RO | cs.CVPDF

[67] VTLoc: Learning-based Tactile Contact Localization in Visual Point Clouds cs.RO | cs.CVPDF

cs.MM [Back]

[68] MLLM-DataEngine: Closing the Loop of Multimodal Instruction Tuning Data Generation cs.MM | cs.CV | cs.LGPDF

cs.CR [Back]

[69] Hidden in Thought: Transferable Chain-of-Thought Artifacts Induce Harmful Behavior cs.CR | cs.CL | cs.LGPDF

[70] Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents cs.CR | cs.CV | cs.LGPDF