cs.CL [Total: 7]
cs.CV [Total: 50]
cs.RO [Total: 5]
cs.AI [Total: 3]
cs.LG [Total: 7]

cs.CL [Back]

[1] Ad Headline Generation using Self-Critical Masked Language Model cs.CL | cs.AI | cs.LGPDF

Yashal Shakti Kanungo, Sumit Negi, Aruna Rajan

TL;DR: 本文提出了一种基于强化学习策略梯度的自批判掩码语言模型方法，用于为电商平台生成产品广告标题。该方法通过联合考虑卖家希望推广的多个产品来生成广告标题，在重叠度指标和质量审核方面超越了现有的Transformer和LSTM+RL方法，并且在语法和创意质量上甚至优于人工提交的标题。

Details

Motivation: 解决电商平台大规模生成高质量、有创意的广告标题的难题，通过程序化方法利用零售内容自动生成广告标题，以提升广告的吸引力和通过网站创意质量审核的效率。

Result: 在重叠度指标和质量审核中，该方法超越了现有的Transformer和LSTM+RL方法；模型生成的标题在语法和创意质量上均优于人工提交的标题，达到了SOTA水平。

Insight: 创新点在于将强化学习策略梯度方法应用于基于Transformer的掩码语言模型，并联合多个产品信息进行条件生成，实现了在广告标题生成任务上的性能突破，为程序化创意生成提供了新思路。

Abstract: For any E-commerce website it is a nontrivial problem to build enduring advertisements that attract shoppers. It is hard to pass the creative quality bar of the website, especially at a large scale. We thus propose a programmatic solution to generate product advertising headlines using retail content. We propose a state of the art application of Reinforcement Learning (RL) Policy gradient methods on Transformer based Masked Language Models. Our method creates the advertising headline by jointly conditioning on multiple products that a seller wishes to advertise. We demonstrate that our method outperforms existing Transformer and LSTM + RL methods in overlap metrics and quality audits. We also show that our model-generated headlines outperform human submitted headlines in terms of both grammar and creative quality as determined by audits.

[2] MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning cs.CL | cs.LGPDF

Ruilin Tong, Dong Gong

TL;DR: MILES 是一个用于自改进大型语言模型推理的框架，它通过动态扩展模块化指令内存并应用优化正确性的内存组合，在测试时逐步积累可重用经验。该框架采用粗到细的检索机制，利用可学习的选择头对内存单元进行筛选，从而在有限监督下提升推理性能。

Details

Motivation: 现有基于内存的方法要么存储泛化能力差的完整解决方案模板，要么使用未针对最终答案正确性优化的启发式步骤级选择，且学习选择策略需要大规模训练数据和固定动作空间，不适合测试时内存逐步扩展且监督有限的情况。

Result: 在广泛的实验中，MILES 始终匹配或优于先前方法，同时实现了更优的准确性与效率权衡，展示了其有效性、鲁棒性和可迁移性。

Insight: 创新点在于提出了模块化指令内存结构，包含不对称的子目标嵌入和子指令对，并关联可学习的选择头，实现了粗到细的检索机制，能够在测试时约束下利用自信样本训练选择头并指导不确定样本的推理。

Abstract: Large language models (LLMs) increasingly improve their reasoning at test time via additional computation, yet most existing works treat each problem in isolation. When problems arrive sequentially, accumulating reusable experience across them can further improve performance. Existing memory-based methods either store whole-solution templates that generalize poorly to novel problems or use heuristic step-level selection that is not optimized for final-answer correctness. Learning selection policies requires large-scale training data and fixed action spaces, making such approaches unsuitable for test-time settings where memory expands incrementally and only limited supervision is available. We propose MILES (Modular Instruction Memory with LEarnable Selection for self-improving LLM reasoning), a framework that dynamically expands step-wise memory and applies correctness-optimized memory composition under realistic test-time constraints. MILES maintains modular memory units consisting of asymmetric pairs of sub-goal embeddings and sub-instructions, each associated with a learnable selection head. This memory structure enables a coarse-to-fine retrieval mechanism: The coarse level enables memory expansion and collects supervision for training selection heads from confident samples, while the fine stage applies learned selection heads to rerank coarse-level candidates and guide reasoning for uncertain samples. MILES consistently matches or outperforms prior methods while achieving superior accuracy-efficiency tradeoffs. Extensive experiments demonstrate its effectiveness, robustness, and transferability.

[3] Comprehensive Evaluation of Large Language Model Responses: A Multi-Factor Scoring System cs.CL | cs.AIPDF

Yiming Gai, Junde Lu, Xuefei Huang

TL;DR: 该论文提出了一种用于全面评估大语言模型响应质量的多因素评分系统，该系统整合了准确性、简洁性、事实一致性、可读性和连贯性五个维度，并配备了可视化结果的图形用户界面。在TruthfulQA数据集上的评估揭示了主流LLM在推理任务上的优势（最高综合得分0.6104）以及在处理复杂事实和模糊性方面的普遍局限。该框架超越了传统指标的狭隘视角，为模型潜力与缺陷的评估提供了透明、可扩展的新途径。

Details

Motivation: 现有的大语言模型评估方法通常局限于单一维度，无法全面捕捉模型的能力谱系，因此迫切需要一种更全面的评估框架。

Result: 在TruthfulQA基准数据集上的评估显示，主流LLM在推理任务上综合得分最高达到0.6104，但普遍在处理复杂事实和模糊性方面存在局限。

Insight: 创新点在于提出了一个整合多维度（准确性、简洁性、事实一致性、可读性、连贯性）的评分范式，并配套了可视化GUI，为模型评估提供了更透明、更全面的分析工具，超越了传统单一指标的限制。

Abstract: The remarkable performance of large language models (LLMs) in linguistic tasks underscores an urgent need for comprehensive evaluation of their response quality. Prevailing methods, often confined to singular dimensions, fall short of capturing the full spectrum of model capabilities. This study introduces a multifactor scoring paradigm, integrating accuracy, conciseness, factual consistency, readability, and coherence, complemented by a graphical user interface (GUI) for visualizing outcomes. Evaluations on the TruthfulQA dataset unveil mainstream LLMs’ strengths in reasoning tasks (peaking at a composite score of 0.6104) alongside pervasive limitations in navigating complex facts and ambiguities. Transcending the narrow lens of traditional metrics, this framework offers a transparent, adaptable avenue to illuminate model potential and deficiencies. Though presently focused on English tasks, its horizons beckon toward multilingual domains. This work carves a novel path for knowledge engineering and model refinement.

Andrei-George Durdun, Victor Constantinescu, Radu Tudor Ionescu

TL;DR: 本文提出了一种多模态情感分析方法，通过自动语音识别（ASR）生成文本转录，并利用机器翻译工具将其翻译成多种语言，创建多个文本模态。通过级联的跨模态Transformer块整合音频和多语言文本特征，并进一步将多模态模型（教师）的知识蒸馏到单模态（仅音频）模型（学生）中。实验表明，自动生成的文本信息能显著提升多模态情感极性分类性能，且蒸馏能增强纯音频模型而不增加推理开销。

Details

Motivation: 自动从语音中识别情感（积极或消极）是一项挑战性任务，需要同时分析语音语调和解说出的词语。现有方法依赖音频基础模型，但尚不清楚此类模型是否能全面考虑所有方面。

Result: 在大规模数据集上的实验表明，自动生成的文本信息能显著提升多模态情感极性分类性能。消融研究证实自动转录和自动翻译均有帮助，且通过蒸馏增强的纯音频模型在推理时无额外计算开销。

Insight: 创新点在于通过ASR和机器翻译自动生成多语言文本模态，并利用级联跨模态Transformer进行整合；同时，通过知识蒸馏将多模态知识迁移到单模态音频模型中，在保持推理效率的同时提升性能。

Abstract: Automatically recognizing the sentiment, positive or negative, from speech is a challenging task, requiring both the analysis of vocal inflections and the interpretation of uttered words. Recent solutions rely on audio foundation models to solve the task, but it remains unclear if such models can take all aspects into account. To this end, we propose a multimodal solution that integrates audio and text information via cross-modal transformers, where text transcripts are automatically generated via an automatic speech recognition (ASR) tool. Moreover, we create multiple text modalities by automatically translating the transcripts into multiple languages via machine translation tools. Audio and multilingual text features are combined via a cascaded architecture comprising cross-modal transformer blocks that integrate modalities one by one. We further distill knowledge from the multimodal model, called teacher, into a unimodal (audio only) model, called student. We conduct experiments on a large-scale dataset, demonstrating that the automatically generated textual information can bring significant performance boosts in multimodal sentiment polarity classification. Our ablation study confirms that both automatic transcripts and automatic translations are helpful. Moreover, we show that the audio-only model can be enhanced via distillation, boosting performance without any computational overhead during inference. To reproduce the reported results, we publicly release our code at https://github.com/andreidurdun/cross-modal-audio-sentiment.

[5] Evaluation of Multilingual Ability to Use Spatial Deictic Expressions in Vision-Language Models cs.CLPDF

Kaito Watanabe, Taisei Yamamoto, Tomoki Doi, Hitomi Yanaka

TL;DR: 本文开发了一个多语言基准测试，用于评估视觉语言模型（VLMs）在四种语言中使用空间指示表达（如“这个”和“那个”）的能力。实验发现，测试模型在使用指示词时与人类存在差异，特别是在根据物体距离选择合适指示词方面。

Details

Motivation: 评估视觉语言模型的空间推理能力，特别是处理依赖于情境的空间指示表达，这些表达需要模型结合语言和视觉空间进行联合推理，并理解不同语言中这些表达所编码的空间区分。

Result: 通过新开发的基准测试，实验显示测试模型在使用指示词时与人类方式不同，尤其是在基于物体距离选择合适指示词方面，但摘要未提及具体定量结果或与SOTA的比较。

Insight: 创新点在于构建了一个多语言基准来评估VLMs处理空间指示表达的能力，这有助于揭示模型在跨语言空间推理中的局限性，为改进模型的多语言接地能力提供了方向。

Abstract: One of the expected abilities of vision-language models (VLMs) is spatial reasoning ability based on a given text and image. To evaluate the spatial reasoning abilities of VLMs, we focus on the use of spatial deictic expressions, which are defined as spatial expressions whose referent is determined by their situational context, such as this'' and that’’. To handle spatial deictic expressions, VLMs must jointly reason over language and visual space, grounding context-dependent references in the image’s spatial structure. In addition, selecting appropriate spatial deictic expressions across languages requires VLMs to understand the language-specific spatial distinctions encoded by these expressions. In this paper, we develop a benchmark to evaluate the multilingual ability of VLMs to use spatial deictic expressions in four languages. Our experiments using this benchmark reveal that the tested models use demonstratives in a manner different from that of humans, particularly in selecting the appropriate demonstratives based on the distance to the object.

[6] R^3: Advertisement Compliance Rectification via Group-Relative Experience Extractor and Curriculum Reinforcement cs.CL | cs.LGPDF

Yuan Chen, Zhenyu Hu, Mengge Xue, Te Cao, Liqun Liu

TL;DR: 本文提出R^3框架，用于自动修正视频广告中的文本违规内容（包括语音字幕和屏幕文字），旨在平衡合规性与广告原始语义意图的保留。该框架结合了基于群体相对经验提取器的数据合成、分层奖励的课程强化学习策略，以及集成了文本识别、重写和重新渲染的完整视频修正流程。

Details

Motivation: 在线广告的严格内容审核导致大量广告被拒，手动修正不可行，而现有安全驱动的方法往往过度编辑，损害广告主的原始语义意图。本文旨在解决视频广告中文本违规的自动修正问题，同时保持意图完整性。

Result: 在工业数据集和在线A/B测试上的大量实验表明，R^3显著优于现有最先进基线方法，在违规修正和意图保留之间实现了最优权衡。

Insight: 创新点包括：通过群体相对合规经验提取器引导高质量监督数据合成；采用分层奖励的课程强化学习策略以强制合规并最大化语义一致性；构建了集文本识别、重写和重新渲染于一体的工业级视频修正框架，实现了端到端的自动化处理。

Abstract: Rigorous content moderation is crucial for online advertising but leads to millions of daily rejections. This scale renders manual rectification infeasible, particularly for video advertisements. However, existing safety-driven methods often suffer from aggressive over-editing, which compromises the advertiser’s original semantic intent merely to satisfy compliance. In this work, we target the rectification of textual violations in video ads, covering both speech transcripts and on-screen text. We propose R^3, a novel framework designed to harmonize compliance with original semantic intent preservation. Our approach integrates three key innovations: (1) an experience-driven data synthesis framework that bootstraps high-quality supervision via a group-Relative compliance experience extractor; (2) a curriculum Reinforcement learning strategy with hierarchical rewards designed to enforce compliance while maximizing semantic consistency; and (3) a comprehensive video Rectification framework seamlessly integrating text recognition, rewriting, and re-rendering for industrial deployment. Extensive experiments on industrial datasets and online A/B testing demonstrate that R^3 significantly outperforms state-of-the-art baselines, achieving an optimal trade-off between violation rectification and intent preservation.

[7] Accurate, Interdisciplinary and Transparent Structure-property Understanding with Deep Native Structural Reasoning cs.CL | cs.AI | cs.CE | cs.LGPDF

Chen Tang, Yizhou Wang, Jianyu Wu, Lintao Wang, Shixiang Tang

TL;DR: 本文介绍了SciReasoner，一个用于蛋白质、小分子和无机晶体的多模态科学基础模型，旨在通过原生结构推理实现准确、跨学科且透明的结构-性质关系理解。该模型将坐标、拓扑和周期性连接离散化为统一的结构感知词汇表，将结构标记视为可寻址的证据单元进行推理。

Details

Motivation: 解决在生物学、化学和材料科学中应用人工智能解释结构-性质关系时面临的表示与推理联合挑战，即模型需在保留领域原生结构信息的同时，展示特定证据如何在科学原理和物理约束下支持预测。

Result: 在86个基准测试中，SciReasoner在67个任务上达到了最先进的性能。具体而言，在同源性控制的基因本体预测中，将低同源性和孤儿样蛋白质的细胞组分注释F_max从0.42提升至0.55；在化学中，将单步逆合成准确率从0.63提升至0.72；其表示能分离元素和化合物相并解析高低带隙区域。双盲专家评估认为其推理轨迹在98%的情况下优于或至少可与前沿大语言模型相媲美。

Insight: 创新点在于将结构离散化为统一的可寻址证据单元，使结构成为科学约束下可检查的推理基底，从而将准确预测与可解释的科学推断联系起来。这为跨学科的结构-性质理解提供了一种新的、透明的推理框架。

Abstract: Structure-property relationships are foundational to biology, chemistry and materials science, where function, reactivity and physical response emerge from spatial, chemical and periodic organization. Mechanistically explaining these relationships requires interpreting structural evidence through scientific principles and physical constraints, from stereochemistry and bonding to symmetry, energetics and periodic order. However, applying artificial intelligence to this process presents a joint challenge of representation and reasoning: models must preserve domain-native structural information while showing how specific evidence supports predictions under these constraints. Here we introduce SciReasoner, a multimodal scientific foundation model for native structural reasoning across proteins, small molecules and inorganic crystals. SciReasoner discretizes coordinates, topologies and periodic connectivities into a unified structure-aware vocabulary, treating structural tokens as addressable evidence units during reasoning. In homology-controlled Gene Ontology prediction, SciReasoner improves Cellular Component annotation for low-homology and orphan-like proteins, increasing $F_{\max}$ from 0.42 to 0.55. In chemistry, it raises single-step retrosynthesis accuracy from 0.63 to 0.72 while generating fragment-level disconnection and precursor-verification traces. In materials science, its representations separate elemental and compound phases and resolve high- and low-band-gap regimes. Across 86 benchmarks, SciReasoner achieves state-of-the-art performance on 67 tasks. Double-blind expert evaluation rates its reasoning traces as preferred or at least comparable to those of a frontier large language model in 98% of cases. By making structure an inspectable substrate for reasoning under scientific constraints, SciReasoner connects accurate prediction with interpretable scientific inference.

cs.CV [Back]

[8] MiLSD: A Micro Line-Segment Detector for Resource-Constrained Devices cs.CV | cs.AI | cs.ROPDF

Parsa Hassani Shariat Panahi, Amir Hossein Jalilvand, M. Hassan Najafi

TL;DR: 本文提出了一种面向资源受限设备的微型线段检测器MiLSD，旨在在亚兆字节的内存预算下实现最优的线段检测精度。通过系统比较三种输出表示法，发现F-Clip中心-长度-角度表示在小模型尺寸下学习效果最好，并探索了量化、推理增强等策略对精度与内存权衡的影响。

Details

Motivation: 线段检测是视觉SLAM、3D重建和工业检测的关键模块，但现有深度学习模型即使最小也需数兆字节内存，超出了低成本微控制器（MCU）的容量。本文旨在研究在亚兆字节预算下可达到的最大精度。

Result: 在1MB激活内存预算下，结合亚像素解码、测试时增强和轻量级验证器等推理增强，MiLSD在ShanghaiTech Wireframe基准测试上的sAP10指标从10.6（25k参数，0.25 MB）提升至24.1。

Insight: 创新点在于为MCU级约束量身定制检测器，并系统性地映射了不同输出表示、比特宽度、模型容量和后处理策略在嵌入式视觉系统中的精度-内存权衡关系。客观来看，其提出的F-Clip表示法在小模型上更有效，且明确了8位量化可保持全精度性能，而4位量化会导致显著退化，这为资源受限设备的模型部署提供了实用指导。

Abstract: Line segment detection is a key building block in visual SLAM, 3D reconstruction, and industrial inspection. Recent deep learning methods have greatly improved accuracy, yet even the smallest models require several megabytes of memory, exceeding low-cost MCU capacity. This work investigates the maximum achievable accuracy under a sub-megabyte budget. We propose MiLSD, a detector tailored for MCU-level constraints, and systematically compare three output representations within a compact fully-convolutional backbone. Our study shows that the proposed F-Clip center-with-length-and-angle formulation learns most effectively at small model sizes. We find that 8-bit quantization preserves full-precision performance, while 4-bit quantization causes significant degradation, particularly in angle regression, with quantization-aware training recovering only part of the loss. With a one-megabyte activation budget and inference enhancements including sub-pixel decoding, test-time augmentation, and a lightweight verifier, MiLSD improves sAP10 on ShanghaiTech Wireframe from 10.6 (25k parameters, 0.25 MB) to 24.1 within 1 MB. Rather than competing with GPU-scale parsers, we map the accuracy memory trade-off across representations, bit-widths, capacities, and post-processing strategies for embedded vision systems.

[9] Overview of the NLPCC 2026 Shared Task 1: Difficulty-Aware Multilingual and Multimodal Medical Instructional Video Understanding Evaluation cs.CV | cs.AIPDF

Shenxi Liu, Kan Li, Mingyang Zhao, Yuhang Tian, Bin Li

TL;DR: 本文介绍了NLPCC 2026共享任务1：难度感知的多语言多模态医学教学视频理解评估（DA-MIVQA）。该任务在先前多语言多模态医学视频基准上扩展，通过根据回答问题所需证据的类型和复杂性对问题进行显式区分，构建了包含三个子任务（DA-TAGSV、DA-VCR、DA-TAGVC）的数据集，旨在评估系统在不同文本、视觉、时序和过程推理需求下的性能。

Details

Motivation: 延续NLPCC 2023-2025的系列挑战，旨在构建一个更精细的医学教学视频问答基准，通过显式区分问题的难度（如仅需文本线索的简单问题与需要视觉定位、过程理解和跨模态整合的复杂问题），以更全面地评估系统能力。

Result: 论文介绍了任务参与概况、竞赛结果和代表性系统，但未在摘要中提供具体的定量结果（如准确率）或与SOTA的比较。数据集覆盖急救、康复、护理等多个医学场景，并经过人工难度标注验证。

Insight: 主要创新点在于引入了“难度感知”的维度，对问题进行基于证据类型和复杂性的显式分类，这为评估医学视频问答系统提供了更细粒度和实用的基准。从客观角度看，其构建的多任务框架（单视频时序定位、视频语料检索、视频语料时序定位）能系统性地测试模型在不同推理需求下的综合能力。

Abstract: Following the CMIVQA, MMI-VQA, and M4IVQA challenges in NLPCC 2023–2025, we introduce the Difficulty-Aware Medical Instructional Video Question Answering (DA-MIVQA) shared task for NLPCC 2026. DA-MIVQA extends previous multilingual and multimodal medical video benchmarks by explicitly distinguishing questions according to the type and complexity of evidence required for answering. Specifically, simple questions can often be answered from subtitle-based textual cues, whereas complex questions require visual grounding, procedural understanding, and cross-modal evidence integration. The challenge contains three tracks: Difficulty-Aware Temporal Answer Grounding in Single Video (DA-TAGSV), Difficulty-Aware Video Corpus Retrieval (DA-VCR), and Difficulty-Aware Temporal Answer Grounding in Video Corpus (DA-TAGVC). The dataset is collected from public medical instructional channels, covers diverse scenarios such as first aid, emergency response, rehabilitation, nursing, and general medical education, and is manually verified with difficulty annotations. This paper presents the task motivation, dataset construction, evaluation protocol, participation overview, competition results, and representative systems of DA-MIVQA. DA-MIVQA provides a practical benchmark for evaluating medical instructional video question answering systems under varying textual, visual, temporal, and procedural reasoning requirements.

[10] SpaR3D-MoE: Adaptive 3D Spatial Reasoning from Sparse Views Meets Geometry-Inductive Mixture-of-Experts cs.CV | cs.AIPDF

Haida Feng, Hao Wei, Haolin Wang, Shiwei Li, Chade Li

TL;DR: 本文提出了SpaR3D-MoE框架，旨在解决多模态大语言模型在从稀疏RGB输入理解3D空间几何时的瓶颈。该框架通过自适应时空流形采样构建几何感知的时空图来提取关键帧，并利用由指令-姿态感知路由器驱动的异构几何归纳混合专家模型，自适应地路由多模态令牌以解决模态冲突。

Details

Motivation: 现有3D感知模型要么依赖昂贵的3D特定数据，要么仅使用RGB输入并采用启发式采样和单一、浅层的融合方式，这分别破坏了关键的时空连通性并导致跨不同空间任务的模态竞争。本文旨在克服这些瓶颈，使MLLM仅从稀疏RGB输入中获得几何感知能力。

Result: 在VSI-Bench、ScanQA和SQA3D上的大量实验表明，该方法达到了最先进的性能。特别是在VSI-Bench上获得了63.5的最高平均分，比最强基线高出7.8个绝对点，在路线规划和相对方向任务上分别实现了35.4%和51.4%的相对改进。

Insight: 创新点在于提出了自适应时空流形采样机制来构建几何感知时空图以保留拓扑连通性，以及引入了由指令-姿态感知路由器驱动的异构几何归纳混合专家模型，以自适应路由解决跨模态竞争，实现了仅从稀疏RGB输入进行端到端的自适应3D空间推理。

Abstract: Recent Multimodal Large Language Models (MLLMs) struggle to bridge the representational gap between 2D semantic understanding and 3D spatial geometry. Existing 3D-aware models either rely on costly 3D-specific data or utilize RGB-only inputs with heuristic sampling and monolithic, shallow fusion, which respectively disrupt essential spatiotemporal connectivity and induce modality contention across diverse spatial tasks. To overcome these bottlenecks, we introduce SpaR3D-MoE, an end-to-end framework that enables adaptive spatial reasoning by equipping MLLMs with geometry-aware capabilities from only sparse RGB inputs. First, we propose an adaptive spatiotemporal manifold sampling mechanism that constructs a geometry-aware spatiotemporal graph to extract informative keyframes, effectively mitigating sequence redundancy while preserving the scene’s topological connectivity. Second, we introduce the heterogeneous geometry-inductive Mixture-of-Experts driven by an instruction-pose aware router, which adaptively routes multimodal tokens to specialized experts, resolving the cross-modal contention inherent in monolithic fusion. Extensive experiments on VSI-Bench, ScanQA, and SQA3D demonstrate that our method achieves state-of-the-art performance. Notably, SpaR3D-MoE achieves the highest average score of 63.5 on VSI-Bench, outperforming the strongest baseline by 7.8 absolute points, alongside relative improvements of 35.4% and 51.4% in Route Plan and Relative Direction tasks, respectively.

[11] ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities cs.CV | cs.AIPDF

Aavash Chhetri, Bibek Niroula, Eduard Vazquez, Yash Raj Shrestha, Prashnna Gyawali

TL;DR: 本文提出ProMoE-FL框架，用于解决多模态联邦学习中模态缺失的问题。该框架通过构建全局客户端感知的原型库来捕获跨机构的临床先验知识，并利用基于原型和模态索引的混合专家模型进行方向感知的特征合成，以动态生成缺失模态的特征。

Details

Motivation: 现有方法通常依赖额外公共数据集或仅基于可用模态进行简单特征合成，存在局限性。本文旨在开发一种更鲁棒的多模态联邦学习方法，以应对模态缺失的挑战。

Result: 在四个公开胸部X光数据集（MIMIC-CXR、NIH Open-I、PadChest和CheXpert）上进行的广泛评估表明，ProMoE-FL在同质和更具挑战性的异质设置中均持续优于现有最先进方法。

Insight: 创新点在于构建全局客户端感知原型库来捕获跨机构模态先验，并设计基于原型和模态索引的条件混合专家模型，实现方向感知的专家路由以动态合成缺失特征，这为联邦学习中的模态缺失问题提供了新思路。

Abstract: In this paper, we address the problem of multimodal federated learning with missing modality. Existing methods utilize an additional public dataset or perform naive feature synthesis that is based solely on the available modality. To address these limitations, we propose ProMoE-FL, a Prototype-conditioned Mixture-of-Experts framework for robust missing-modality feature synthesis in multimodal federated learning. ProMoE-FL builds a global client-aware prototype bank that captures clinically meaningful modality priors across institutions. Our Mixture of Experts is conditioned on these prototypes and modality indices to enable direction-aware expert routing for dynamically synthesizing missing features. We perform extensive quantitative and qualitative evaluations on four public chest X-ray datasets (MIMIC-CXR, NIH Open-I, PadChest, and CheXpert) and demonstrate that ProMoE-FL consistently outperforms state-of-the-art methods in both homogeneous as well as the more challenging heterogeneous settings.

[12] Dynamic-in-Few-Step: Unifying Dynamic Computation and Few-Step Distillation for Efficient Video Generation cs.CV | cs.AI | cs.LGPDF

Yu Cheng, Siyue Yao, Zhongang Qi, Shanyan Guan, Wei Li

TL;DR: 本文提出了一种名为Dynamic-in-Few-Step的后训练加速框架，旨在统一动态计算与少步蒸馏，以高效生成视频。该方法通过将动态结构稀疏化直接集成到蒸馏过程中，将预训练的视频扩散模型转化为紧凑的、针对特定去噪步骤的专家混合模型，从而在保持生成质量的同时显著降低计算成本。

Details

Motivation: 视频扩散模型虽然生成质量优越，但计算成本过高。现有的少步蒸馏技术虽然能加速推理，但通常在所有去噪阶段强制使用静态模型架构，忽略了不同噪声水平固有的不同计算需求。本文旨在利用这种冗余，通过联合优化去噪步骤和结构化模型稀疏性来解决此问题。

Result: 在Wan-14B模型上，该方法在4步蒸馏的基础上进一步移除了24%的每步浮点运算，实现了1.2倍的挂钟时间增益，相比50步的教师模型达到了30倍的加速，同时保持了有竞争力的生成质量。

Insight: 主要创新点在于将动态结构稀疏化与少步蒸馏过程进行联合优化，并引入了渐进式训练策略和输出展开机制以解决训练不稳定性。该方法与现有加速技术正交，通过创建步骤特定的专家混合模型，实现了计算效率与模型性能的统一。

Abstract: Video Diffusion Models (VDMs) have demonstrated superior generation quality but suffer from prohibitive computational costs. While recent few-step distillation techniques significantly accelerate inference, they typically enforce a static model architecture across all denoising stages, ignoring the varying computational demands inherent to different noise levels. In this work, we propose a novel post-training acceleration framework that exploits this redundancy by integrating dynamic structural sparsification directly into the distillation process. Unlike conventional post-hoc compression applied to a fixed diffusion pipeline, our approach jointly optimizes the denoising steps and structured model sparsity, transforming a pre-trained VDM into a compact, step-specific Mixture-of-Models (MoM). To address the training instability arising from this joint optimization, we introduce a Progressive Training Strategy coupled with an Output Rollout Mechanism, which ensures the coherent learning of structural decisions across timesteps. Furthermore, we develop a specialized inference engine to deploy the resulting MoM efficiently. Our method is orthogonal to existing acceleration techniques and highly effective: On Wan-14B, it removes 24% of the per-step FLOPs on top of 4-step distillation, adding a 1.2x wall-clock gain and reaching a 30x speedup over the 50-step teacher while preserving competitive generation quality.

[13] CoMind: Understanding Collaborative Human Activity from Multiple Minds and Views cs.CVPDF

Alexey Gavryushin, Dingxi Zhang, Zhao Huang, Alexandros Delitzas, Jiaqi Chen

TL;DR: 该论文提出了一个名为CoMind的新型多模态数据集，用于研究真实世界中的协作人类活动，特别是烹饪场景。该数据集整合了自我中心和他者视角的视频、高质量音频、视线追踪以及3D场景和物体扫描，并标注了共享注意力、社交线索和交互行为。基于此，论文建立了联合注意力估计、社交条件化物体交互预测和协作交接预测三个基准任务，旨在推动AI系统对复杂社交交互建模和协作行为推理的研究。

Details

Motivation: 现有研究多集中于对协调和任务执行的建模，而支持协作的认知过程（如心理理论）在自然场景中难以研究。为了填补这一空白，作者旨在创建一个能够捕捉真实世界协作、并支持研究这些认知过程的多模态数据集。

Result: 论文提出了CoMind数据集及其三个基准任务（联合注意力估计、社交条件化物体交互预测、协作交接预测），为评估AI系统在协作环境中的感知和推理能力提供了标准。

Insight: 创新点在于创建了一个集成了多视角视频、音频、视线和3D场景数据的、时间对齐且标注丰富的真实协作数据集，这为研究协作中的心理理论和多模态感知提供了前所未有的资源。从客观角度看，该数据集的设计有望推动具身智能、人机协作和社交AI在理解复杂、动态的人类协作行为方面的发展。

Abstract: Human-human collaboration is a fundamental aspect of everyday life, essential to success in a wide range of goal-directed activities from household tasks to professional teamwork. While much research has focused on modeling coordination and task execution, the cognitive processes that support such collaboration, particularly Theory of Mind (the ability to infer the mental states of others), remain difficult to study in natural settings. To address this gap, we introduce a novel egocentric and exocentric video dataset capturing real-world collaboration in cooking scenarios. The dataset integrates multi-perspective video, high-quality audio, gaze tracking, and 3D scene and object scans, with annotations for shared attention to objects, social cues and interactions between agents, as well as agent-object interactions. We establish benchmarks for Joint Attention Estimation, Socially Conditioned Object Interaction Anticipation, and Collaborative Handover Prediction, enabling research on multimodal perception, proactive assistance, and collaborative planning. By providing temporally aligned, richly annotated multimodal data, CoMind facilitates the development and evaluation of AI systems capable of modeling complex social interactions and reasoning about human behaviors in collaborative environments. Our dataset and benchmarks are made available at https://comind.ethz.ch/.

[14] Hardware-aware Graph Neural Networks prunning for embedded event-based vision cs.CVPDF

Piotr Wzorek, Kamil Jeziorek, Tomasz Kryjak

TL;DR: 本文提出了一种针对嵌入式异构FPGA平台的硬件感知图卷积神经网络剪枝与量化优化策略，旨在适应事件相机的实时低功耗处理需求。该方法通过精细网格搜索和贪婪逐层迭代深化搜索探索设计空间，在CIFAR-10、MNIST-DVS和N-Caltech101数据集上实现了最高31.4%的BRAM内存减少，同时控制精度损失在可接受范围内。

Details

Motivation: 事件相机在动态环境中性能优异，但移动机器人应用需要满足低延迟、低功耗的实时数据处理需求，现有方法需通过硬件加速保持事件数据的时间稀疏性。

Result: 在CIFAR-10数据集上实现28.8%的BRAM内存减少（精度下降1.65%），MNIST-DVS数据集上减少31.4%（精度下降3.55%），N-Caltech101数据集上减少26.5%（精度下降5.18%）。

Insight: 创新点在于将硬件约束（如片上内存）直接融入神经网络架构优化过程，通过系统性的设计空间搜索方法实现模型结构与嵌入式平台的协同适配，为边缘计算场景提供了可扩展的模型压缩框架。

Abstract: Event-based cameras are gaining popularity as the sensor of choice for mobile robotics, due to their high performance in dynamic environments. However, these applications require efficient real-time data processing with low latency and power consumption. One strategy to meet these stringent requirements is hardware acceleration of efficient algorithms that preserve the temporal sparsity of event data. In this work, we propose an optimization strategy for Graph Convolutional Neural Networks models aimed at adapting their architecture to the limited resources of embedded heterogeneous FPGA platforms. Our method incorporates hardware-aware pruning and quantization, taking into account the trade-off between on-chip memory savings and inference accuracy. Strategic exploration of the design space with Fine Grid Search and Greedy layer-wise Iterative Deepening Search methods enables flexible adaptation of the model architecture to the target platform. Our approach was evaluated across various network configurations and multiple datasets, resulting in BRAM memory reductions of 28.8% for CIFAR-10 (with a 1.65% decrease in accuracy), 31.4% for MNIST-DVS (accuracy drop of 3.55%), and 26.5% for N-Caltech101 (with a 5.18% accuracy reduction).

[15] A Good Initialization is All You Need for Faithful Visual Attribution cs.CVPDF

Zihan Gu, Jiayu Wang, Hua Zhang, Yue Hu

TL;DR: 本文研究了视觉归因中的紧凑证据掩码生成问题，提出两种前向搜索方法CoPAIR和TRACE，用于高效地识别支持模型预测的关键图像区域。这些方法在ImageNet分类和MLLM归因任务上取得了SOTA性能，并展示了紧凑掩码在模型修复中的实用性。

Details

Motivation: 现有基于搜索的扰动方法通常输出所有区域的完整排序，但许多应用（如MLLM归因和修复）仅需要一个紧凑的top-k证据掩码。本文旨在解决这一掩码优先的归因问题，以更高效地提供可操作的归因输出。

Result: 在ImageNet分类任务上，使用CLIP ViT-L/14、CLIP RN101和ResNet-101模型，所提方法在包含前向调用计量的忠实完整排序归因中建立了新的SOTA前沿。在POPE和RePOPE基准上，使用Qwen2.5-VL-3B-Instruct和LLaVA-v1.5-7B模型，TRACE+Greedy取得了最强的基于搜索的MLLM归因结果；直接TRACE掩码在RePOPE修复任务中达到了94.44%和96.00%的单点修复率。

Insight: 创新点在于将归因问题从完整排序重构为紧凑掩码生成，并提出了两种高效的前向搜索策略：CoPAIR通过粗粒度候选预热完整搜索，TRACE使用交叉熵采样直接搜索固定基数细粒度掩码。其核心洞察是紧凑证据掩码本身可作为可操作的归因输出，而不仅仅是完整排序的前缀。

Abstract: Faithful visual attribution identifies which image regions support a model prediction. Search-based perturbation methods lead the insertion–deletion faithfulness frontier by masking regions and measuring score changes, but they usually output a complete ordering of all regions. Many applications, especially MLLM attribution and repair, only need a compact top-(k) evidence mask. We study this mask-first attribution problem. An exactly (k)-region mask is combinatorial: useful evidence can depend on interactions among fine regions. Coarse grouping can stabilize early search but aggregates redundant content, whereas one-step scoring can miss high-value combinations. We introduce two forward-only methods. \textsc{CoPAIR} uses a PhaseWin–Greedy gap diagnosis to construct coarse singleton/pair candidates that warm-start full-ordering search. \textsc{TRACE} directly searches fixed-cardinality fine-region masks with cross-entropy sampling, elite retention, and distribution updates, with a finite-budget recovery analysis. The resulting evidence set can be returned as a compact attribution mask or used to initialize Greedy or PhaseWin when a complete ranking is required. Across ImageNet classification with CLIP ViT-L/14, CLIP RN101, and ResNet-101, our initialized search methods establish a new state-of-the-art frontier for faithful full-ordering attribution under inclusive forward-call accounting. On POPE and RePOPE with Qwen2.5-VL-3B-Instruct and LLaVA-v1.5-7B, \textsc{TRACE}+Greedy gives the strongest search-based MLLM attribution results. Direct \textsc{TRACE} masks further achieve single-point RePOPE repair rates of (94.44%) and (96.00%), showing that compact evidence masks can be actionable attribution outputs, not merely prefixes of full rankings.

[16] WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligence cs.CVPDF

Xiangyu Han, Mengyu Yang, Jiaqi Li, Bowen Chang, Ziyu Chen

TL;DR: 该论文提出了WildCity数据集，这是一个由自动驾驶车队在复杂城市环境中采集的真实世界多模态数据集，包含18条平均83.7公里长的轨迹，旨在解决AI在城市尺度上构建空间表示的挑战。论文还建立了城市定制的重建基线，并将重建环境转换为闭环模拟器，以推动城市规模渲染和空间智能的发展。

Details

Motivation: 尽管近期基础模型在场景重建和具身智能方面取得了进展，但由于缺乏城市尺度的数据，AI在构建与人类认知相当的空间表示方面仍面临挑战。WildCity旨在填补这一空白，为城市规模的渲染、模拟和空间智能研究提供真实世界的测试平台。

Result: 论文建立了城市定制的重建基线，并将重建环境转换为闭环模拟器，但摘要中未提及具体的定量实验结果或基准测试性能。

Insight: 创新点在于引入了首个大规模、真实世界的城市尺度多模态数据集，并系统分析了实现可模拟城市数字孪生的关键挑战（可扩展性、外推性和不确定性），为AI在人类认知尺度上进行空间感知、记忆和推理的研究提供了重要资源。

Abstract: Humans can navigate an unfamiliar city and gradually form a coherent spatial mental map spanning tens of square kilometers. Can AI build spatial representations at a comparable scale? Although recent foundation models have advanced scene reconstruction and embodied intelligence, scaling to entire cities remains an open challenge, primarily due to the lack of city-scale data. To bridge the gap, we introduce WildCity, a real-world multimodal dataset collected by autonomous fleets traversing complex urban environments. Our dataset includes 18 trajectories, each averaging 83.7 kilometers in length, and preserves the core challenges of in-the-wild perception, e.g., dynamic objects, lighting variations, and imperfect camera poses. We further establish an urban-tailored reconstruction baseline and convert the reconstructed environments into a closed-loop simulator. Beyond the dataset and baseline, we systematically analyze the key challenges on the path to simulation-ready urban digital twins: scalability, extrapolation, and uncertainty. Ultimately, WildCity aims to catalyze progress not only in city-scale rendering, but more broadly in the pursuit of AI that can perceive, remember, and reason across space at a scale comparable to human cognition. Project page: https://han-xiangyu.github.io/Wild-City/

[17] URS-Stereo: Uncertainty-Guided Residual Search for Real-Time Stereo Matching cs.CVPDF

Pouya Sohrabipour, Chaitanya kumar reddy Pallerla, Dongyi Wang

TL;DR: 本文提出了URS-Stereo，一种用于实时立体匹配的由粗到细框架。该框架通过引入不确定性引导的残差搜索模块（UGRSM），在视差细化前预测传播视差的可靠性并自适应地调整局部代价体量的搜索中心，从而解决传统方法因传播视差不准确导致匹配失败的问题，在保持实时效率的同时提升了鲁棒性。

Details

Motivation: 现有的由粗到细立体匹配方法严重依赖前一阶段传播的视差估计的准确性。当传播的视差不准确时，真实对应点可能落在预定义的局部搜索范围之外，导致后续细化过程中出现无法恢复的匹配失败。本文旨在解决这一局限性。

Result: 在SceneFlow、KITTI 2012、KITTI 2015、Middlebury和ETH3D等多个基准数据集上的广泛实验表明，URS-Stereo在保持实时推理速度的同时，持续改进了视差估计精度，验证了所提不确定性引导搜索策略的有效性。

Insight: 主要创新点在于提出了不确定性引导的残差搜索模块（UGRSM），它联合预测传播视差的可靠性和残差搜索偏移量，从而在视差细化前动态自适应地重新定位局部代价体量的中心。这为提升由粗到细框架的鲁棒性提供了一种新思路，即利用不确定性来指导搜索区域的动态调整。

Abstract: Real-time stereo matching is crucial for robotics, autonomous systems, and embedded vision applications, where both computational efficiency and disparity accuracy are required. Recent coarse-to-fine stereo matching methods improve efficiency by progressively refining disparity estimates using local cost volumes at higher resolutions. However, these methods rely heavily on the accuracy of propagated disparity estimates from previous stages. When the propagated disparity is inaccurate, the ground-truth correspondence may fall outside the predefined local search range, leading to unrecoverable matching failures during subsequent refinement. In this paper, we propose URS-Stereo, a real-time coarse-to-fine stereo matching framework that addresses this limitation through uncertainty-guided search adaptation. Specifically, we introduce an Uncertainty-Guided Residual Search Module (UGRSM), which predicts the reliability of propagated disparities together with residual search offsets to adaptively relocate the centers of local cost volumes before disparity refinement. By dynamically adjusting the search region according to the confidence of the propagated disparity, the proposed method significantly improves the robustness of local correspondence estimation while preserving the computational efficiency of coarse-to-fine stereo matching. Extensive experiments on SceneFlow, KITTI 2012, KITTI 2015, Middlebury, and ETH3D demonstrate that URS-Stereo consistently improves disparity estimation while maintaining real-time inference speed, validating the effectiveness of the proposed uncertainty-guided search strategy

[18] Gen4U: Unifying Video Generation and Understanding via Diffusion cs.CV | cs.LGPDF

Michael King, Aravindh Mahendran, Matthew Koichi Grimes, Fedor Kitashov, Adham Elarabawy

TL;DR: 本文提出Gen4U框架，通过分析视频扩散模型的中间激活，发现其潜在空间具有高度结构化特征，能够同时捕获高级语义和低级细节。该框架利用冻结的、大规模视频扩散模型作为视频编码器，在单一前向传播中统一视频生成与理解任务，无需微调即可在多种视频理解任务上取得有竞争力的性能，同时保持模型原有的高质量视频生成能力。

Details

Motivation: 先前研究表明扩散模型的表征主要捕获低级几何信息，而在高级语义理解方面存在局限。本文旨在探究最先进的视频扩散模型是否能够克服这一限制，并探索如何将其强大的生成能力重新用于视频理解任务，从而统一生成与理解范式。

Result: 实验表明，Gen4U框架在广泛的视频任务上表现优异，包括视频分类、深度估计、相机姿态估计以及图像和视频描述生成等语义和非语义任务。该框架无需微调，仅使用冻结的预训练视频扩散模型作为编码器，就取得了极具竞争力的感知性能。

Insight: 核心创新点在于揭示了视频扩散模型潜在空间的结构化演化规律（随网络深度和噪声水平变化），并据此提出了一个新颖的框架，将生成模型直接用作理解任务的通用编码器。这为统一生成与理解提供了一个高效、无需任务特定适配的可行路径，突破了传统上两类模型分离的范式。

Abstract: Prior work suggests that diffusion representations capture low-level geometry but struggle with high-level semantics. We demonstrate that state-of-the-art video diffusion models overcome this limitation. By systematically probing their intermediate activations using recent mutual-kNN alignment metrics, we reveal a highly structured latent space where visual representations evolve across both network depth and noise levels. We show that while moderate noise levels yield linearly separable global semantics, fine-grained details persist at lower noise levels but become spatially scattered, requiring attention mechanisms to decode. Building on these insights, we introduce Gen4U (Generation for Understanding), a framework that repurposes these generative representations with a single forward pass. Our experiments establish that frozen, large-scale video diffusion models function as highly competitive video encoders across a wide spectrum of tasks, spanning semantic and non-semantic objectives (video classification, depth estimation, camera pose estimation, image and video captioning). Bypassing fine-tuning, Gen4U unifies the generation and understanding paradigms, achieving strong perception performance while fully preserving the model’s ability to generate high-quality video.

[19] Ensemble Deep Learning Approaches for AI-Altered Video Detection cs.CVPDF

Laiba Khan, Hung-Mao Wu, Wei Lin, Frank Bi, Yousef Abdelhadi

TL;DR: 本文提出了一种用于检测AI篡改视频的多模态集成深度学习系统，该系统结合了音频和视觉分析，通过集成多个模型来提高检测的鲁棒性。系统使用AASIST进行音频检测，并采用EfficientNet、XceptionNet和MesoNet分析视频帧的视觉特征，通过平均融合和堆叠等策略整合各模型的预测分数。

Details

Motivation: 随着AI生成视频的普及，区分真实与篡改内容变得日益困难，现有单模型检测方法难以泛化到不同类型的深度伪造视频，因此需要开发更鲁棒的检测系统。

Result: 实验表明，单个模型在训练数据集上表现良好，但在更多样化的数据集上性能下降；集成方法通过结合多个模型的预测，提高了整体鲁棒性，在不同类型深度伪造上表现更一致，但对未见过的篡改技术的泛化仍是核心挑战，平均准确率约为70%。

Insight: 创新点在于采用多模态（音频+视觉）集成学习框架，通过模型集成策略（如平均融合和堆叠）提升检测的泛化能力；客观分析认为，结合多种信息源和集成学习是应对深度伪造检测泛化挑战的有效途径。

Abstract: The increasing accessibility of artificial intelligence has led to a rapid rise in AI-generated videos, making it more difficult to distinguish between real and manipulated content. Many existing detection methods rely on a single model and often struggle to generalize across different types of deepfakes. In this work, we developed a multimodal deepfake detection system that combines both audio and visual analysis using an ensemble of models. The system includes AASIST for audio-based detection, and EfficientNet, XceptionNet, and MesoNet for analyzing visual features in video frames. The pipeline takes a video as input, separates the audio, and extracts face frames using MTCNN. Each model produces a score indicating the likelihood of the input being fake. These scores are then combined using ensemble strategies, including mean averaging and stacking. Mean fusion provides a simple and stable baseline, while stacking uses a trained meta-model to learn how to combine predictions more effectively. Results show that while individual models perform well on the datasets they were trained on, their performance drops when tested on more diverse datasets. The ensemble approach helps improve overall robustness by combining predictions from multiple models, leading to more consistent performance across different types of deepfakes. This suggests that using both audio and visual information together is a more reliable approach for deepfake detection. Our results highlight generalization to unseen manipulations as the central open challenge, with average accuracy around 70%.

[20] Geometric Collapse: When Vision Models Fail to Verify Physical Causality cs.CVPDF

Wentao Zhang, Jinhu Qi, Weiqiang Jin, Yifei Zhang, Chan-Tong Lam

TL;DR: 本文研究了大规模自监督学习模型在密集几何预测任务中是否具备物理合理性验证能力。作者提出了一种名为Scrambled Edges的受控反事实方法，通过注入违反表面连续性、光照一致性和遮挡顺序的边缘线索来测试模型。实验发现，当前模型缺乏隔离物理上不合理边缘线索的可靠机制，导致几何预测的全局性崩溃。

Details

Motivation: 旨在探究大规模自监督学习在提升密集几何预测性能的同时，是否能在推理时进行物理合理性检查，即模型能否识别并拒绝违反物理因果关系的视觉线索。

Result: 在NYU Depth v2和KITTI数据集上测试CNN、ViT及自监督学习深度预测器时，Scrambled Edges导致的预测偏差比能量匹配噪声高3.2倍；扩散模型和流匹配深度估计器虽有所衰减但仍出现显著崩溃。即使已知损坏区域，输出级修复仅恢复47%，且误差会传播到掩码外部。

Insight: 创新点在于设计了Scrambled Edges这一受控反事实测试方法，能分离边缘证据、高频能量和边缘稀疏性的影响。客观而言，该研究揭示了当前密集预测模型普遍缺乏物理合理性验证机制，为未来设计显式的合理性评分和选择性线索集成提供了动机。

Abstract: Recent progress in large-scale self-supervised learning has improved dense geometric prediction, but it remains unclear whether such scaling yields inference-time physical plausibility checks. We propose Scrambled Edges, a controlled counterfactual that injects salient edge-like cues while violating surface continuity, illumination coherence, and occlusion ordering. With energy-matched and structure-matched controls, we isolate the effect of unsupported edge evidence from high-frequency energy and edge sparsity. Across CNN/ViT/SSL depth predictors on NYU Depth v2 and KITTI, Scrambled Edges induce up to 3.2x larger deviation from clean predictions than energy-matched noise; additional diffusion and flow-matching depth estimators show attenuated but still significant collapse. The resulting Geometric Collapse propagates globally: even with oracle knowledge of the corrupted region, output-level repair recovers only 47%, with substantial error outside the mask. These findings provide controlled behavioral evidence that current dense predictors lack reliable mechanisms to quarantine physically unsupported edge cues, motivating explicit plausibility scoring and selective cue integration.

[21] Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild cs.CV | cs.LGPDF

Trang Nguyen, Sidong Zhang, Shiv Shankar, Gauri Jagatap, Deepak Chandran

TL;DR: 该论文介绍了Video2Reaction数据集，这是一个将短视频片段映射到社交媒体上观众所表达的情感分布的多模态数据集，用于预测观众反应。论文还开发了一个基于开源大语言模型的两阶段多智能体标注流程，以低成本持续收集标注数据，并建立了首个野外视频到反应分布预测的基准。实验表明，预训练的基础视频模型在零样本设置下表现不佳，但经过微调后可以成为最先进的预测器。

Details

Motivation: 理解和预测观众对视频内容的反应对于改进内容创作、推荐系统和媒体分析至关重要，但目前缺乏能够映射视频到观众情感分布的数据集和基准。

Result: 论文在Video2Reaction数据集上建立了首个基准，结果显示，预训练基础视频模型在零样本设置下失败，而微调后能成为SOTA预测器。然而，任务仍然具有挑战性，即使在主导反应预测中，最强的方法（LLaVA-Next）也仅达到77%的Top-3 F1分数。

Insight: 论文的创新点在于构建了大规模、映射视频到观众情感分布的数据集，并提出了一个仅使用开源LLM、成本效益高的两阶段多智能体标注流程来处理任务固有的噪声和主观性。从客观角度看，这项工作为建模集体观众反应这一难题提供了首个可靠的基准和训练资源。

Abstract: Understanding and forecasting audience reactions to video content are crucial for improving content creation, recommendation systems, and media analysis. To enable audience reaction prediction and other content engagement applications, we introduce $\textbf{Video2Reaction}$, a multimodal dataset that maps short movie segments to a distribution of $\textit{induced emotions}$ of viewers in the wild, as expressed through social media. $\textbf{Video2Reaction}$ spans more than 10,000 videos and serves as a reliable benchmark as well as a training resource for audience reaction prediction. To enable cost-effective continuous annotations as reactions may change over time, we develop a two-stage multi-agent pipeline using only open-source LLMs, achieving 86% correctness under blind human verification despite the inherently noisy and subjective nature of the task. We establish the first benchmark for video-to-reaction-distribution prediction in the wild and show that pretrained foundation video models fail in zero-shot settings, while finetuning transforms them into state-of-the-art predictors capable of modeling both full reaction distributions and dominant responses from video alone. However, the task remains challenging: even the strongest methods achieve only 77% Top-3 F1 in dominant reaction prediction (LLaVA-Next), highlighting a substantial gap in modeling collective audience reaction. \modification{Dataset and code are available at our project page: https://information-fusion-lab-umass.github.io/video2reaction-bench.github.io

[22] LoCA: Spatially-Aware Low-Rank Convolutional Adaptation of Vision Foundation Models cs.CV | cs.AI | cs.LGPDF

Sojung An, Junha Lee, Sujeong You, Nam Ik Cho, Donghyun Kim

TL;DR: 本文提出了一种针对视觉基础模型（VFMs）的参数高效微调方法——低秩卷积适配（LoCA），旨在解决卷积层中空间与通道信息纠缠的问题。该方法通过解耦通道和空间适配，在保留预训练空间先验的同时，实现了在细粒度分类、领域泛化语义分割和生成任务上的优异性能。

Details

Motivation: 现有低秩适配（LoRA）方法主要针对Transformer的自注意力层设计，而卷积核本质上是4D张量，将其强制转换为2D矩阵会破坏固有的空间拓扑结构，因此需要一种卷积感知的参数高效微调框架。

Result: 实验结果表明，LoCA在细粒度分类、领域泛化语义分割和生成基准测试中取得了具有竞争力或最先进的性能，有效保留了预训练的空间先验。

Insight: 创新点在于通过解耦通道和空间适配来处理卷积层的空间-通道纠缠问题，具体包括引入低秩通道适配进行密集跨通道混合，以及通过奇异值分解（SVD）从预训练卷积核中提取并优化空间基。这为卷积网络的参数高效微调提供了新思路。

Abstract: Pre-trained Vision Foundation Models (VFMs) provide strong visual representations for diverse downstream tasks. The key challenge of VFM adaptation stems from the prohibitive costs of full fine-tuning and catastrophic forgetting. To address this, Low-Rank Adaptation (LoRA) has emerged as the prevailing paradigm for Parameter-Efficient Fine-Tuning (PEFT). However, LoRA is typically designed for transformer self-attention layers parameterized by 2D matrices. Since convolutional kernels inherently couple spatial and channel information within a 4D tensor, forcing them into a monolithic 2D matrix disrupts the inherent spatial topology. In this paper, we propose Low-Rank Convolutional Adaptation (LoCA), a convolution-aware PEFT framework that addresses spatial-channel entanglement by decoupling channel and spatial adaptation. LoCA introduces a low-rank channel adaptation for dense cross-channel mixing and refines spatial bases extracted from pre-trained kernels via Singular Value Decomposition (SVD). Experimental results show that LoCA preserves pre-trained spatial priors and achieves competitive or state-of-the-art performance across fine-grained classification, domain-generalized semantic segmentation, and generative benchmarks.

[23] Seeing What Matters: Lesion-Aware High-Resolution Patch Discovery and Fusion for Chest X-ray Report Generation cs.CVPDF

Yingshu Li, Yunyi Liu, Zhenghao Chen, Tong Chen, Zailong Chen

TL;DR: 本文提出LePaX（Lesion-Aware High-Resolution Patch Discovery and Fusion）框架，用于解决胸部X光（CXR）报告生成中高分辨率感知的挑战。该框架通过可学习的空间分辨率分配（LSRA）自适应地提取诊断相关的高分辨率图像块，并利用全局-区域融合（GRF）将高分辨率区域信息写回全局特征网格，从而在不增加视觉令牌数量的前提下，实现了对原生高分辨率CXR（最高1920x1920）的高效感知。

Details

Motivation: 现有放射学报告生成（RRG）系统通常因预训练视觉编码器的固定令牌预算而依赖大幅下采样的输入（如256x256），这抑制了原生高分辨率图像中细微但临床重要的线索。然而，实现高分辨率感知面临挑战：简单的分块会导致令牌数量激增，而全局压缩则会抑制细微病变并降低诊断保真度。

Result: 在多个CXR基准测试上的实验表明，LePaX在临床和语言指标上均取得一致提升，同时实现了原生分辨率CXR感知，其使用的视觉令牌数量比简单的高分辨率分块方法少10倍以上。

Insight: 创新点在于将高分辨率感知建模为固定令牌预算下的约束空间分辨率分配问题，并引入了LSRA（学习空间效用图以自适应分配高分辨率容量）和GRF（通过空间对齐的分辨率写回实现令牌保留的区域到全局精炼）两个关键组件。从客观角度看，该工作借鉴了放射科医生的工作流程（先定位可疑区域再进行高分辨率评估），实现了高效且诊断导向的高分辨率信息融合，避免了令牌膨胀问题。

Abstract: Despite rapid advances in chest X-ray (CXR) foundation models, most radiology report generation (RRG) systems still rely on heavily downsampled inputs (e.g., 256x256) due to the fixed visual token budgets of pretrained vision encoders, suppressing subtle yet clinically important cues present in native-resolution images. However, enabling high-resolution (high-res) perception remains challenging: naive tiling causes prohibitive token inflation, while global compression suppresses subtle lesions and degrades diagnostic fidelity. Inspired by radiologists’ workflow, localizing suspicious regions before detailed high-res assessment. We propose Lesion-Aware High-Resolution Patch Discovery and Fusion for Chest X-ray Reporting (LePaX), the first RRG framework that enables efficient high-res CXR perception (up to 1920x1920) without increasing the vision-token count. LePaX formulates high-res perception as a constrained spatial resolution allocation problem under a fixed token budget and introduces two key components: Learnable Spatial Resolution Allocation (LSRA), which learns a spatial utility map that adaptively allocates limited high-res capacity to diagnostically relevant regions, enabling targeted extraction of high-res patches from native CXRs; and Global-Regional Fusion (GRF), which performs token-preserving region-to-global refinement by projecting high-resolution regional evidence back onto the global feature grid through spatially aligned resolution write-back, avoiding token inflation. Experiments on multiple CXR benchmarks demonstrate that LePaX consistently improves both clinical and linguistic metrics while enabling native-resolution CXR perception with over 10x fewer visual tokens than naive high-res tiling.

[24] Compass: Prostate Cancer Detection Needs Multi-View Context cs.CV | cs.LGPDF

Paul F. R. Wilson, Mohamed Harmanani, Zhuoxin Guo, Obed K. Dzikunu, Hannes Cash

TL;DR: 本文提出Compass方法，通过建模微超声（μUS）检查为二维图像流，整合前列腺旋转扫描视频和活检时刻的帧，利用基于探头旋转角度的Transformer进行证据聚合，预测帧级和检查级风险评分，以提升前列腺癌检测性能。

Details

Motivation: 现有AI方法多孤立分析单帧μUS图像，而临床专家通常评估完整记录的视频研究以利用三维上下文改进检测，因此受此临床工作流程启发，旨在开发能整合多视图上下文的AI方法。

Result: 在多中心临床试验数据集上评估，Compass相比文献中的基线AI方法和临床专家提供的风险评分表现出强劲性能，突显了多视图上下文在μUS前列腺癌检测中的价值。

Insight: 创新点在于将μUS研究建模为图像流，并联合旋转扫描视频与活检帧，通过条件Transformer聚合证据；客观分析认为其模拟了临床工作流程，有效利用了时空上下文信息，为基于μUS的诊断提供了互补工具。

Abstract: Artificial intelligence (AI) analysis of micro-ultrasound ($μ$US) has shown promise for prostate cancer (PCa) detection. However, most existing AI methods focus on the analysis of single $μ$US images in isolation. By contrast, expert $μ$US readers typically assess a full recorded video study, which provides three-dimensional context, to improve PCa detection compared to single-frame analysis. Inspired by this clinical workflow, we propose Compass, a novel AI methodology which models a $μ$US study as a stream of 2D images. Compass jointly integrates rotational sweep videos of the prostate with $μ$US frames acquired at the moment of biopsy, and performs evidence aggregation across the study using a transformer conditioned on the probe’s rotational angle. Finally, a decoder head predicts frame-level and study-level risk scores for the patient. The model is trained and evaluated using a multi-center clinical trial dataset of $μ$US studies, including continuous rotational scans of the prostate and videos captured during biopsy acquisition. We compare the proposed method to baseline AI methods from the literature and to risk scores provided by clinical experts. Our framework shows strong performance, highlighting the value of multi-view context for $μ$US PCa detection, and providing a potentially powerful tool to complement human expertise in $μ$US-based PCa diagnosis. Our code is available at: https://github.com/mharmanani/Compass.

[25] General Incomplete Multimodal Learning via Dynamic Quality Perception cs.CVPDF

Xiangyu Meng, Shicai Wei

TL;DR: 本文提出了一种名为GIML（通用不完整多模态学习）的统一框架，旨在同时处理模态间缺失（整个模态缺失）和模态内退化（模态存在但严重损坏）这两种在实际中常共存的复杂缺失模式。其核心是通过动态质量感知，将异构缺失模式建模为连续的模态信息退化过程，从而实现基于退化程度的自适应融合。

Details

Motivation: 现有方法主要关注模态间缺失，而忽略了模态内退化问题，导致在两种缺失模式共存的现实场景中效果不佳。本文旨在解决这一局限性，构建一个能统一处理两类不完整性的鲁棒多模态学习框架。

Result: 在包含多种模态类型的数据集上进行的广泛实验证明了GIML的有效性和通用性。

Insight: 主要创新点在于：1）将异构缺失统一建模为连续信息退化，实现退化感知的自适应融合；2）提出噪声感知质量估计器，通过可控噪声注入学习从损坏特征到噪声强度的映射；3）设计噪声-语义解耦模块，分离语义信息与噪声干扰，提升对未见损坏模式的鲁棒性和泛化能力。

Abstract: Multimodal learning robust to missing modalities is essential for real-world applications. Existing methods mainly focus on inter-modality missing, where entire modalities are absent, while overlooking intra-modality degradation, where modalities are present but severely corrupted. In practice, these two types of missing often coexist, making existing approaches ineffective. To address this limitation, we propose General Incomplete Multimodal Learning (GIML), a unified framework that simultaneously handles both inter-modality missing and intra-modality degradation through dynamic quality perception. Specifically, GIML models heterogeneous missing patterns as continuous modality information degradation, enabling degradation-aware adaptive fusion. To achieve reliable quality perception, we introduce a Noise-aware Quality Estimator that learns the mapping from corrupted features to noise intensity through controlled noise injection. Furthermore, we propose a Noise-Semantic Decoupled module that separates semantic information from noise interference. This improves robustness and generalization to unseen corruption patterns. Extensive experiments across datasets with diverse modality types demonstrate the effectiveness and generality of GIML. Code is available at: https://github.com/Yu-Five/GIML.

[26] Ego-Human Motion Prediction with 3D-Aware LLM cs.CVPDF

Yujin Bae, Jaewoo Jeong, Hyeonseong Kim, Kuk-Jin Yoon

TL;DR: 本文提出了Ego3DLM模型，用于从第一人称视角预测人体运动。该模型的核心创新在于将3D空间语义场景理解与整体跨模态预测相结合，通过单次自回归解码同时生成未来的姿态和语言描述，从而确保物理合理性和语义一致性。

Details

Motivation: 现有方法在利用语言作为先验进行第一人称运动预测时，大多忽略了决定运动如何展开的3D空间和语义上下文，并且将姿态和语言预测作为独立的推理流来处理，这导致了预测结果的不一致。

Result: 在Nymeria基准测试上的实验表明，Ego3DLM在未来的运动预测、过去的运动追踪和运动描述任务上均达到了最先进的性能水平。

Insight: 论文的创新点在于强调了3D场景感知对于准确运动预测的必要性，并提出了一个将姿态与语言描述进行整体联合预测的框架。其采用的三阶段训练方案（空间语义预训练、整体指令微调、基于GRPO的强化微调）以及跨模态一致性奖励机制，是值得借鉴的技术路径。

Abstract: Anticipating human motion from an egocentric perspective is fundamental for proactive assistance in AR/VR, human-robot collaboration, and embodied AI. While recent works incorporate language as a semantic prior to reduce the ill-posed nature of egocentric forecasting, they largely neglect the 3D spatial and semantic context that governs how motion unfolds, and treat pose and language prediction as separate inference streams. We introduce Ego3DLM, built on two core principles: accurate motion forecasting requires explicit spatial and semantic understanding of the 3D environment, and pose and language must be predicted holistically in a single pass, since motion is inherently tied to the semantic interpretation of actions being performed. Given three-point tracking, 3D scene features, and egocentric video, Ego3DLM simultaneously decodes past pose, future pose, past narration, and future narration in a single autoregressive pass, grounding predicted poses and descriptions in one another to enforce cross-modal and temporal consistency. We adopt a three-stage training scheme: (1) spatial-semantic scene awareness pretraining; (2) holistic instruction tuning over all four outputs in a single pass; and (3) GRPO-based reinforcement finetuning with intra- and inter-modal rewards that directly optimize pose-language fidelity. Experiments on the Nymeria benchmark demonstrate that Ego3DLM achieves state-of-the-art performance across future motion prediction, past motion tracking, and motion description, showing that 3D scene grounding and holistic cross-modal prediction yield physically plausible and semantically coherent motion forecasts. The project page is available at https://jaewoo97.github.io/Ego3DLM/.

[27] Self-Supervised Pretraining Improves Cross-Site and Cross-Scale Robustness of Point Cloud Leaf-Wood Segmentation cs.CV | cs.AIPDF

Heeju Mun, Tackang Yang, Yunsoo Nam, Changhyun Choi

TL;DR: 本研究探索了自监督学习（SSL）在点云叶木分割任务中的适用性，通过在ShapeNet-55及2400棵单树点云数据上预训练Point-M2AE模型，并采用递归体素细分处理输入点密度变化，使模型能在单树和林分尺度上无需架构调整即可运行。预训练模型在针叶树和阔叶树的木材IoU上分别从60.5%提升至70.0%和从69.7%提升至76.3%，在跨四个国家、三个气候带的基准测试中表现出最小的跨站点变异和最高整体性能。下游测试中，在热带森林的木材体积估计上，该方法实现了最低误差（MAE = 2.40 m³），显著优于算法基线。

Details

Motivation: 现有叶木分割方法在不同森林类型和站点间准确性差异大，且自监督学习在林业点云任务（如生物量回归和单树分割）中已提升模型泛化能力，但其在叶木分割中的适用性尚未验证。

Result: 预训练模型在针叶树和阔叶树的木材IoU上分别达到70.0%和76.3%，优于未预训练模型；在跨站点基准测试中，相比LeWos、CWLS和PointTransformer等方法，实现了最小跨站点变异和最高整体性能；林分尺度分割保持与单树尺度相当的准确性（阔叶林mIoU为84.7%，针叶林为77.7%）；下游木材体积估计误差最低（MAE = 2.40 m³），仅为算法基线的一半左右。

Insight: 创新点包括将自监督预训练应用于点云叶木分割以提升跨站点和跨尺度鲁棒性，以及采用递归体素细分处理点密度变化，使模型能无缝适应不同尺度而无需架构修改。从客观角度看，该方法通过SSL预训练增强了模型泛化能力，并在实际下游任务（如木材体积估计）中验证了性能提升的有效性。

Abstract: The accuracy of existing leaf-wood segmentation methods for tree point clouds varies across forest types and sites. Self-supervised learning (SSL) on point clouds has improved the generalization of deep learning models for forestry point cloud tasks, including biomass regression and individual tree segmentation, but its applicability to leaf-wood segmentation remains untested. In this study, we pretrained Point-M2AE, a widely used SSL architecture for point clouds, on ShapeNet-55 augmented with 2,400 individual tree point clouds. For fine-tuning and inference, we used recursive voxel subdivision to handle the wide variation in point density across inputs, allowing the same model to operate at both individual-tree and plot scales without architecture change. Compared to the model without pretraining, the pretrained model improved wood IoU from 60.5% to 70.0% for needleleaf and from 69.7% to 76.3% for broadleaf trees. On a benchmark spanning four countries across three climatic zones, the pretrained model achieved the smallest cross-site variation and highest overall performance among compared methods (LeWos, CWLS, and PointTransformer). Plot-level segmentation maintained accuracy comparable to individual-tree performance, with mIoU of 84.7% for broadleaf and 77.7% for needleleaf plots, showing that the model generalizes across scales without additional finetuning. As a downstream test in tropical forests, where dense canopies make segmentation challenging, we applied our model and a quantitative structure model to estimate wood volume for 28 trees from Guyana, Indonesia, and Peru to assess whether the segmentation improvements from SSL pretraining translate into improved downstream performance. The resulting volume estimates achieved the lowest error among all methods tested (MAE = 2.40 m$^3$), less than half that of algorithmic baselines (LeWos: 5.94 m$^3$; CWLS: 5.27 m$^3$).

[28] SHTA: Semantic Hard Token Correction and Center Alignment for Semi-Supervised Medical Image Segmentation cs.CVPDF

Zhuoru Zhang, Yiheng Zhong, Zimu Zhang, Xiaofeng Liu

TL;DR: 本文提出SHTA方法，一种轻量化的训练时语义表示分支，旨在解决半监督医学图像分割中困难区域语义表示不一致的问题。该方法通过语义分配、硬令牌精炼和语义中心对齐来优化中间语义表示，无需增加推理成本，并在多个代表性框架上验证了其有效性。

Details

Motivation: 现有半监督医学图像分割方法主要关注提升监督质量，但未能显式地强制困难区域学习到的语义表示具有一致性，导致即使预测级监督增强，语义分配不稳定的困难区域仍难以建立一致的表示，限制了分割性能的进一步提升。

Result: 在Synapse和AMOS数据集上，将SHTA集成到GA-CPS、CPS、URPC和MagicNet等框架中，实验结果表明SHTA带来了配对性能的一致提升，尤其在分割精度、弱器官恢复和语义模糊减少方面有显著增益，且仅增加训练时开销。

Insight: 创新点在于不引入额外的预测监督，而是通过优化中间语义表示来提升困难区域的语义一致性，这是一种轻量化的训练时增强策略，可无缝集成到现有框架中，为提升半监督分割的表示学习提供了新思路。

Abstract: Recent advances in semi-supervised medical image segmentation have achieved remarkable performance through prediction consistency, pseudo-label supervision, and hard-region supervision. However, these methods primarily improve supervision quality rather than explicitly enforcing semantic consistency in the learned representations of hard regions. Consequently, even under increasingly stronger prediction-level supervision, difficult regions exhibiting unstable semantic assignment often fail to establish semantically consistent representations during training, thereby limiting further segmentation improvement. To address this issue, we propose SHTA (Semantic Hard Token Correction and Center Alignment), a lightweight training-time semantic representation branch. Instead of introducing additional prediction supervision, SHTA refines intermediate semantic representations through Semantic Assignment, Hard Token Refinement, and Semantic Center Alignment, thereby improving semantic consistency in hard regions while preserving the original prediction pathway and introducing no additional inference cost. We integrate SHTA into representative semi-supervised segmentation frameworks, including GA-CPS, CPS, URPC, and MagicNet, and conduct evaluations on the Synapse and AMOS datasets. Experimental results demonstrate that SHTA delivers consistent paired improvements across frameworks, with especially clear gains in segmentation accuracy, weak-organ recovery, and semantic ambiguity reduction, while incurring only training-time overhead. The code is available at https://anonymous.4open.science/r/release_SHTA-42D5/.

[29] AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning cs.CV | cs.AIPDF

Kyuan Oh, Bumsoo Kim

TL;DR: 本文提出了AnchorPrune，一种无需训练、无需修改模型的轻量级视觉令牌剪枝框架，旨在解决大视觉语言模型中高分辨率输入导致大量冗余视觉令牌带来的高推理成本问题。该方法通过构建受保护的相关性锚点，并基于重要性加权的新颖性扩展上下文，在保持查询关键证据的同时，提高视觉覆盖范围，从而优化精度与效率的权衡。

Details

Motivation: 现有剪枝方法在激进压缩下，相关性驱动和多样性驱动的目标可能冲突：前者可能过度集中预算于局部相关证据，后者可能抑制必要令牌或保留独特但无信息的区域。本文旨在解决这一冲突，实现更高效的视觉令牌剪枝。

Result: 在图像和视频视觉语言模型及基准测试中，AnchorPrune在无需训练的基线方法上持续改善了精度-效率权衡，尤其在严重压缩下表现突出。例如，在LLaVA-NeXT-7B模型上，仅使用2880个视觉令牌中的160个（约5.6%），就能保持97.6%的全令牌性能。

Insight: 创新点在于提出了“相关性锚定的上下文扩展”原则：首先自适应地确定一个受保护的、紧凑的查询关键证据锚点，然后基于重要性加权的新颖性分配剩余预算来扩展上下文。这种有序设计防止了上下文扩展替换掉必要的查询线索，同时提高了整体视觉覆盖，是一种架构感知的轻量级解决方案。

Abstract: Large vision-language models incur substantial inference costs because high-resolution inputs introduce thousands of visual tokens, many of which are redundant for a given query. Existing pruning methods often combine query relevance and token diversity, yet these objectives can conflict under aggressive compression: relevance-driven selection may overconcentrate the budget on correlated local evidence, while diversity-driven selection may suppress indispensable tokens or retain distinct but uninformative regions. We introduce AnchorPrune, a training-free framework that first constructs a protected relevance anchor and then expands it with complementary visual context. AnchorPrune adaptively determines the anchor size from the novelty profile of relevance-ranked tokens, preserving a compact set of query-critical evidence, and allocates the remaining budget through importance-weighted novelty to recover informative, non-redundant context relative to the anchor. This ordered design prevents contextual expansion from displacing indispensable query cues while improving overall visual coverage. AnchorPrune is lightweight, architecture-aware, and requires neither retraining nor model modification. Across image and video vision-language models and benchmarks, it consistently improves the accuracy-efficiency trade-off over training-free baselines, particularly under severe compression. On LLaVA-NeXT-7B, AnchorPrune preserves 97.6% of full-token performance using only 160 of 2,880 visual tokens. These results establish relevance-anchored contextual expansion as an effective principle for efficient multimodal inference. Code is available at https://github.com/MULTI-cau/AnchorPrune.

[30] AT-Attn: Temporal-Aware Cross-Attention for Longitudinal Multimodal Alzheimer’s Disease Diagnosis cs.CV | cs.AIPDF

Xinyue Du, Yibo Liu, Zhenglei Zhou, Xuancheng Yao, Weimin Zhong

TL;DR: 本文提出了一种名为AT-Attn的时间感知多模态框架，用于整合纵向收集的阿尔茨海默病（AD）患者的MRI影像和临床信息，以支持诊断。该框架通过变化与时间编码、时间偏置的非对称交叉注意力和门控融合机制，有效处理了数据采集不规则、MRI噪声或间歇性缺失的问题，并在ADNI数据集上验证了其优于单模态和简单多模态融合基线的性能。

Details

Motivation: 在纵向AD诊断中，临床和影像信息通常在非规则时间点采集，简单的多模态融合在MRI数据存在噪声或缺失时会降低性能，因此需要一种能有效整合不规则时序多模态数据的方法。

Result: 在包含1520名患者的ADNI数据集上，使用患者级五折交叉验证，AT-Attn模型在准确率、宏F1、ROC-AUC和PR-AUC指标上均优于单模态和简单多模态融合基线，并与强大的表格数据基线保持竞争力。

Insight: 创新点在于提出了一个结合变化与时间编码、时间偏置非对称交叉注意力和门控融合的时序感知多模态融合框架；客观来看，其核心贡献是设计了一种受约束的融合策略，使结构MRI能在存在噪声或缺失时，仍能提供具有临床相关性的互补信息。

Abstract: In longitudinal Alzheimer’s disease (AD) diagnosis support, clinical and imaging information is often collected at irregular visits. Integrating these multimodal observations may improve diagnostic assessment, but naive fusion can degrade performance when MRI is noisy or intermittently unavailable. We propose AT-Attn, a temporal-aware multimodal framework that combines Change-and-Time encoding, time-biased asymmetric cross-attention, and gated fusion to integrate MRI with longitudinal clinical information. We evaluate AT-Attn on an MRI-retained ADNI cohort of 1,520 patients using structural MRI, six cognitive-scale trajectories, and seven static clinical variables under patient-level five-fold cross-validation. The main asymmetric AT-Attn model achieves accuracy 0.719+/-0.024, macro F1 0.721+/-0.023, ROC-AUC 0.873+/-0.013, and PR-AUC 0.783+/-0.018, outperforming unimodal and naive multimodal fusion baselines while remaining competitive with strong tabular baselines. These results suggest that a temporal-aware and constrained fusion strategy can help structural MRI contribute clinically relevant complementary information for patient-level AD diagnosis support.

[31] Tree-of-Thoughts Reasoning for Text-to-Image In-Context Learning cs.CV | cs.AIPDF

Stepanida Alekseeva, Jenifer Kalafatovich, Seong-Whan Lee

TL;DR: 本文提出了一种用于文本到图像上下文学习（T2I-ICL）的思维树（ToT）推理框架。该框架通过引入多阶段推理和选择层，在构建最终图像合成提示前生成、评估并选择多个候选假设，以解决现有模型在组合推理和提示构造敏感性方面的不足。

Details

Motivation: 当前最先进的多模态大语言模型在文本到图像上下文学习任务中表现不佳，主要受限于组合推理能力弱和对提示构造过于敏感。

Result: 在CoBSAT基准测试上的定性和定量结果表明，与基线及思维链提示策略相比，该方法无需额外训练或微调，即可实现更一致、语义更对齐的图像生成。

Insight: 核心创新在于将思维树推理范式引入多模态上下文学习，通过探索替代推理分支和选择连贯解释来缓解提示歧义和组合错误，这是一种无需模型参数更新的结构化推理增强方法。

Abstract: In text-to-image in-context learning (T2I-ICL), a model has to infer a latent compositional pattern from fewshot demonstrations for generating a query image. Recent studies show that state-of-the-art multimodal large language models struggle with this setting, particularly due to limited compositional reasoning and sensitivity to prompt construction. In this work, we propose a Tree-of-Thoughts (ToT) reasoning framework for T2I-ICL that introduces a multi-stage reasoning and selection layer that generates, evaluates, and selects among multiple candidate hypotheses before constructing the final prompt for image synthesis. By exploring alternative reasoning branches and selecting a coherent interpretation, the proposed approach mitigates prompt ambiguity and compositional errors. We implement the proposed approach in a complete ToT-T2IICL inference pipeline and evaluate it on the CoBSAT benchmark. Both qualitative and quantitative results show that structured multi-branch reasoning leads to more consistent and semantically aligned image generation compared to baseline and Chain-of-Thought prompting strategies, without any additional training or fine-tuning.

[32] Video-Based Detection of squint and cataract for accessibility-aware adaptive web interface rendering cs.CV | cs.HCPDF

Amar Ranjan Dash, Manas Ranjan Patra

TL;DR: 本文提出了一种基于实时视频的自动化检测系统，用于斜视和白内障的识别。该系统利用计算机视觉和图像处理方法，通过MediaPipe Face Mesh提取眼部几何特征进行斜视多分类，并通过灰度强度和直方图分析评估白内障的存在与严重程度。实验结果表明，该系统在斜视检测和白内障分类上分别达到了98.39%和96.90%的高准确率，并计划与自适应用户界面和网络无障碍系统集成，以辅助视觉障碍人群。

Details

Motivation: 斜视和白内障是严重影响视觉感知和交互能力的主要眼部疾病，当前缺乏低成本、可大规模部署的实时自动化检测方案，因此本文旨在开发一种基于视频的检测系统，以提升可访问性。

Result: 在斜视检测任务上准确率达到98.39%，白内障分类准确率为96.90%，表明系统具有高精度，但未提及具体基准测试或与SOTA的比较。

Insight: 创新点在于结合MediaPipe Face Mesh进行实时眼部特征提取，以及利用灰度强度和直方图分析白内障严重程度，为低成本、大规模部署的视觉障碍辅助系统提供了可行方案。

Abstract: Squint and cataract are major ocular disorders that majorly affect visual perception and interaction capability. This paper proposes a real-time video-based automated detection system for squint and cataract detection based on computer vision and image processing methods. The proposed system uses a media-pipe face-mesh (a 478-point facial landmark detection model) to extract geometric ocular features for multi-class squint classification. Simultaneously, The presence and severity cataract is estimated through grayscale intensity and histogram-based lens opacity analysis. The system records short video sequences with standard laptop or mobile cameras, which can be deployed at low costs and on a large scale. The experimental performance has shown great accuracy in the detection of squint (98.39%) and classification of cataract (96.90%). Besides automatic ocular analysis, the proposed framework is also made accessible for visual impairment inference which will be integrated with future adaptive user interface and Web accessibility systems for people with visual impairment.

[33] ColorFM: An Optimization-to-Learning Framework for Color Transfer via Flow Matching cs.CVPDF

Yuhang He, Kai Zhang, Xiaoming Li, Du Chen, Jian Yang

TL;DR: 本文提出ColorFM，一种从优化到学习的颜色迁移框架，通过流匹配将颜色迁移重新表述为沿速度场的像素分布传输。框架包含ColorFM-O（实例特定优化方案）和ColorFM-L（高效前馈模型），前者生成精确的伪监督数据，后者实现快速推理，结合了优化的准确性和前馈推理的速度。

Details

Motivation: 现有颜色迁移方法常存在全局映射不准确、语义错位和视觉伪影等问题，本文旨在通过优化到学习的框架解决这些挑战，提升颜色迁移的精确性和语义一致性。

Result: 大量实验表明，ColorFM-L在视觉质量、结构保真度和语义一致性方面优于现有最先进方法，在相关基准测试中达到SOTA水平。

Insight: 创新点包括将颜色迁移重新表述为流匹配问题，通过分层颜色耦合和语义先验指导优化速度场，并利用生成的伪监督数据训练前馈模型，实现优化准确性与推理效率的结合。

Abstract: Color transfer aims to align the color distribution of a source image with that of a reference image while preserving structural and semantic consistency. However, existing methods often suffer from inaccurate global mapping, semantic misalignment, and visual artifacts. To address these issues, we propose ColorFM, an optimization-to-learning framework. ColorFM connects online optimization to offline inference by reformulating color transfer as the transport of pixel distributions along velocity fields via Flow Matching. Specifically, we introduce ColorFM-O, an instance-specific optimization scheme that fits the velocity field through hierarchical color coupling guided by semantic priors. By numerically integrating the induced flow trajectories, ColorFM-O produces precise and semantically consistent color transfer results, while generating high-quality paired data as pseudo-supervision. Building upon this, we design ColorFM-L, an efficient feed-forward model trained on the generated pairs. Through implicit state modeling, ColorFM-L extracts deep semantic features to predict flow parameters for bidirectional linearized transport, ensuring accurate color transfer. Extensive experiments demonstrate that ColorFM-L outperforms state-of-the-art methods in visual quality, structural fidelity, and semantic consistency, successfully combining the accuracy of optimization with the speed of feed-forward inference.

[34] Widest-Path Reachability Fields for Connectivity-Preserving Slender Structure Segmentation cs.CV | eess.SYPDF

Youcheng Zong, Runda Jia, Minxuan Hu, Weilan Su, Dakuo He

TL;DR: 本文提出了一种名为Widest-Path Reachability Fields (WPRF)的模块，用于解决细长曲线结构（如视网膜血管、裂缝和道路）分割中的拓扑断裂问题。该模块通过可微的最大-最小可达性目标，将梯度流重新导向连接瓶颈像素，从而在保持拓扑连通性的同时提升分割性能。

Details

Motivation: 现有基于像素级重叠优化的模型在分割细长结构时，常因梯度均匀分布而忽略稀疏的瓶颈像素，导致拓扑断裂，这被称为拓扑梯度饥饿（TGS）问题。本文旨在直接优化端到端的可达性，以保持分割结果的拓扑正确性。

Result: 在九个架构和六个数据集（包括新提出的口腔微血管数据集OMVIS）上的实验表明，WPRF在87%的实验中提升了性能，在结构脆弱的数据集上clDice指标提高了7.2个百分点，且无需推理开销。

Insight: 创新点在于提出了一种可插拔、骨干无关的WPRF模块，通过可微的最大-最小代数直接优化可达性，避免了后处理骨架化或同调计算，使梯度能聚焦于连接瓶颈。这为拓扑敏感的分割任务提供了一种高效的梯度路由机制。

Abstract: Segmenting slender curvilinear structures such as retinal vessels, cracks, and roads demands topological correctness, as even a single-pixel discontinuity can fragment a continuous network and invalidate downstream analysis. Under standard binary-mask supervision, models optimized for pixel-level overlap frequently produce topologically broken predictions. We trace this to a fundamental mismatch: pixel-wise losses distribute gradients uniformly, yet connectivity hinges on a sparse set of bottleneck pixels. These pixels are vastly outnumbered by thick structures and background, rendering their aggregate gradient contribution negligible. We term this phenomenon topological gradient starvation (TGS). To address it, we propose Widest-Path Reachability Fields (WPRF), a differentiable Max-Min reachability objective that redirects gradient flow to connectivity bottlenecks. The module is plug-and-play, backbone-agnostic, and incurs no inference overhead. WPRF implements a differentiable Max-Min objective via dynamic programming on a domain-restricted graph, coupled with a bottleneck-aware observation term that balances gradient contributions across varying structures. Compared to prior topology-aware losses that rely on post-hoc skeletonization or homology computation, WPRF directly optimizes end-to-end reachability via differentiable Max-Min algebra, enabling gradient flow to concentrate on connectivity bottlenecks without auxiliary structures. We introduce OMVIS, a new oral microvessel segmentation dataset. Experiments across nine architectures and six datasets validate the bottleneck-focused gradient routing mechanism. WPRF improves 87% of experiments with fixed hyperparameters and achieves clDice gains of 7.2 percentage points on structurally fragile datasets.

[35] Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering cs.CV | cs.LGPDF

Miguel Lopez-Duran, Elena Marrero, Julian Fierrez, Marta Robledo-Moreno, Ruben Vera-Rodriguez

TL;DR: 本文对8个开源预训练视觉语言模型在文档视觉问答任务上进行了全面的跨领域评估，涵盖工业文档、信息图和演示幻灯片三种文档类型。研究发现，大型VLM在结构化布局上具有强大的零样本能力，但在视觉复杂布局上性能显著下降；监督微调对小模型提升更明显，且视觉理解是DocVQA的主要瓶颈。

Details

Motivation: 尽管视觉语言模型在文本-视觉任务中表现优异，但其在不同文档领域的鲁棒性和可迁移性尚未得到充分探索，本文旨在系统评估VLM在跨文档领域的适应能力。

Result: 在DocVQA任务上，大型VLM在零样本评估中对结构化布局表现良好，但在信息图和幻灯片等复杂布局上性能下降；监督微调使小模型获得更高相对增益；仅用50个目标领域样本进行微调，模型就能快速适应新领域，在某些情况下甚至超过全监督模型。

Insight: 研究揭示了视觉理解（而非知识缺乏）是DocVQA的关键瓶颈；跨领域少样本学习实验表明，模型通过少量目标领域数据即可快速适应，这为领域自适应提供了高效策略；参数缩放虽重要，但监督微调对小架构模型效益更显著。

Abstract: Document Visual Question Answering (DocVQA) presents a complex multimodal challenge, requiring models to exploit visual, textual, and layout information from documents. Although Vision-Language Models (VLMs) have shown remarkable performance in text-vision tasks, their robustness and transferability to different document domains remains underexplored. In this study, we present a comprehensive evaluation of 8 open-source pretrained VLMs on DocVQA in three different document domains: industrial documents of varying type, infographics, and presentation slides. We systematically assess model performance under zero-shot evaluations, fully supervised finetuning with inter- and intra-dataset evaluations, and few-shot learning evaluations of knowledge transfer between domains. Our findings demonstrate that while large pretrained VLMs possess strong zero-shot baselines for structured layouts, their performance strongly decreases on visually complex layouts of infographics and slides. Although parameter scaling is a dominant factor on performance, supervised finetuning yields higher relative gains in smaller architectures. Furthermore, our cross-domain and few-shot experiments show that visual understanding is the main bottleneck for DocVQA, not a lack of knowledge from the VLMs. Using 50 target domain samples, the models finetuned in DocVQA with datasets of different domains rapidly adapt to the target domain documents, even surpassing their fully supervised counterparts in some cases.

[36] NoDrift3R: Raymap-Guided Coupling for Drift-Robust Unposed Feed-Forward 3D Reconstruction cs.CVPDF

Xiangyu Sun, Liu Liu, Seungkwon Yang, Jingbing Han, Seungtae Nam

TL;DR: 本文提出NoDrift3R，一种用于无姿态前馈3D高斯溅射（3DGS）重建的鲁棒框架。该方法通过一个射线图引导耦合模块（RGC），显式地将几何与外观耦合，并引入双频视角调度策略来稳定长序列学习，旨在解决长图像序列中相机姿态估计漂移导致的性能下降问题。

Details

Motivation: 现有无姿态前馈3DGS方法在长图像序列中性能显著下降，主要原因是累积的相机姿态估计漂移，该误差会传播到几何建模中并严重限制渲染保真度。同时，基于SfM的伪真值姿态会引入传感器噪声，而纯基于渲染的监督则常因几何与姿态的纠缠优化导致不稳定和局部最优。

Result: 在领域内和跨领域数据集上的大量实验表明，该方法在渲染和姿态估计方面均取得了一致的性能提升，在长序列上显著提高了鲁棒性。消融研究验证了其核心见解的有效性。

Insight: 核心创新在于显式设计的几何-外观协同机制（通过RGC模块实现双向反馈）以及双频视角调度策略，这被认为是实现可扩展且抗漂移的无姿态前馈3D重建的关键。该方法为解决长序列重建中的漂移问题提供了一个新的协同优化思路。

Abstract: Pose-Free Feed-forward 3D Gaussian Splatting (3DGS) has recently emerged as a powerful paradigm for fast scene reconstruction. However, its performance degrades significantly in long image sequences due to cumulative camera pose estimation drift, which propagates errors into geometric modeling and severely limits rendering fidelity. In this work, we revisit the long-sequence bottleneck and identify pose drift as the primary factor restricting reconstruction quality. Furthermore, while SfM-based pseudo ground-truth poses introduce sensor noise, purely rendering-based supervision often leads to optimization instability and local minima due to the entangled optimization of geometry and pose. To address the challenges, we propose a synergistic pose-free framework that explicitly couples geometry and appearance via a Raymap-Guided Coupling Module (RGC). Concretely, we anchor Gaussian centers to raymap-induced geometry and jointly optimize RGB reconstruction, raymap consistency, and camera regularization under a unified objective, yielding a bidirectional feedback loop: stronger geometry improves rendering, and appearance supervision in turn refines geometry and pose. To further stabilize learning across wide temporal ranges, we introduce a Dual-Frequency Viewpoint Scheduling strategy that combines easy-to-hard interval expansion with replay of short-interval pairs. Extensive experiments across in-domain and cross-domain datasets show consistent gains in both rendering and pose estimation, with notably improved robustness on long sequences. Ablation studies validate our central insight: explicitly designed geometry-appearance synergy is the key to scalable and drift-robust pose-free feed-forward 3D reconstruction.

[37] Prototype-Anchored Generalized Manifold Regression for Unknown-Domain Object Detection cs.CVPDF

Zihao Zhang, Aming Wu, Yang Li, Yahong Han

TL;DR: 本文研究单域广义目标检测问题，提出了一种基于流形回归的方法MR-DCoT。该方法通过视觉-文本双链思维模块生成结构化离群样本，并利用类特定原型锚定学习校正算子，将偏离特征投影回源语义流形，从而缩小分布差距并提升模型在未知域上的鲁棒性。

Details

Motivation: 现有单域广义目标检测方法主要依赖数据增强或文本提示等模拟驱动策略，难以覆盖真实场景的动态变化，容易对合成风格过拟合且对复杂结构退化鲁棒性有限。本文基于流形假设，认为语义特征应位于紧凑稳定的低维流形上，因此需要将偏离样本校正回语义流形而非穷举模拟外部扰动。

Result: 在三个互补基准测试（恶劣天气检测、真实到艺术风格泛化、零样本语义分割）上的大量实验表明，该方法具有有效性和通用性。

Insight: 创新点在于将未知域泛化问题形式化为流形回归问题，并设计了视觉-文本双链思维模块与类特定原型锚定相结合的闭环框架，通过生成结构化离群样本和语义校正的集成来提升模型泛化能力。

Abstract: In this paper, we study Single-Domain Generalized Object Detection (Single-DGOD), which aims to transfer a detector trained on a single source domain to multiple unseen domains. Existing methods mainly rely on simulation-driven strategies, such as data augmentation or textual prompts, to enlarge the training distribution. However, finite simulations can hardly cover the dynamic variations of real-world scenarios, often causing overfitting to synthetic styles and limited robustness to complex structural degradations. Inspired by the manifold hypothesis, we argue that semantic features, despite diverse visual changes, should lie on a compact and stable low-dimensional manifold. Therefore, robust generalization requires rectifying deviant samples back to this semantic manifold, rather than exhaustively simulating external perturbations. To this end, we propose Manifold Regression with Visual-Text Dual Chain-of-Thought (MR-DCoT), which formulates unknown-domain generalization as a manifold regression problem. MR-DCoT first uses a Visual-Text Dual Chain-of-Thought module to combine VLM-guided semantic evolution with diffusion-based structural perturbation, generating structured off-manifold hard examples. It then introduces Class-Specific Prototype Anchoring to learn a rectification operator that projects deviant features toward the source semantic manifold. By integrating outlier generation and semantic correction into a closed loop, MR-DCoT effectively narrows the distribution gap and improves robustness under unseen shifts. Extensive experiments on three complementary benchmarks, including adverse-weather detection, real-to-art generalization, and zero-shot semantic segmentation, demonstrate the effectiveness and versatility of our method.

[38] [`Attention-Guided Cross-Temporal Clustering for Self-Supervised Video Object Segmentation](https://arxiv.org/abs/2607.07230) cs.CVPDF

Waqas Arshid, Mohammad Awrangjeb, Alan Wee-Chung Liew, Yongsheng Gao

TL;DR: 本文提出了一种名为’注意力引导的跨时间聚类’的自监督视频对象分割框架，旨在通过结合注意力引导的令牌选择和轻量级时间聚类来学习中层、部分感知的表征，以在无需人工标注的情况下实现准确的空间分割和一致的时间跟踪。

Details

Motivation: 解决监督方法依赖昂贵密集标注数据、现有自监督方法在无约束多对象场景中难以同时保持空间准确性和时间一致性的问题，特别是避免使用光流、合成运动线索或任务特定预训练来提升可扩展性和泛化能力。

Result: 未在摘要中提及具体的定量结果或基准测试，但宣称该方法能通过轻量级模块实现跨分辨率和运动模式的高效扩展，提升自监督视频对象分割的性能。

Insight: 创新点在于使用注意力引导的令牌选择和软部分分配的时间对齐，通过显著性加权的对称一致性目标，在像素或整个对象级别之外操作，以学习更具鲁棒性的中层表征，从而增强自监督学习在复杂视频场景中的适用性。

Abstract: Video object segmentation (VOS) is a fundamental task in video understanding, requiring accurate delineation and consistent tracking of objects across frames. While supervised methods achieve strong performance, they rely on densely annotated datasets that are costly to obtain and have limited domain coverage. Self-supervised learning offers a promising alternative by removing the need for manual labels; however, existing approaches often struggle to jointly maintain spatial accuracy and temporal coherence, particularly in unconstrained multi-object scenarios. Many rely on optical flow, synthetic motion cues, or task-specific pretraining, limiting scalability and generalisation. We propose a self-supervised framework, Cross-Temporal Consistency and Clustering, that learns mid-level, part-aware representations by combining attention-guided token selection with lightweight temporal clustering. Instead of operating at the pixel or whole-object level, the method aligns soft part assignments across time using a saliency-weighted symmetric consistency objective. The framework leverages a frozen transformer backbone with lightweight modules for adaptive token selection and multi-offset temporal alignment, enabling efficient scaling across resolutions and motion patterns.

[39] DiffCVE: Diffusion-based Compressed Video Enhancement cs.CVPDF

Wenqiang Xiao, Wenzhuo Ma, Junxi Zhang, Zhenzhong Chen

TL;DR: 本文提出DiffCVE，一种基于扩散模型的压缩视频增强方法，旨在解决严重压缩视频中复杂的伪影模式和严重信息丢失导致的感知质量下降问题。该方法通过编码先验增强的双重条件分支、压缩退化语义提示机制和编码先验引导的加权融合模块，有效利用压缩视频的编码先验信息来指导扩散去噪过程，从而提升视觉质量。

Details

Motivation: 现有扩散模型在视觉修复方面展现出强大生成能力，但直接应用于压缩视频增强时，往往忽略了压缩退化特性，并可能引入结构不一致的幻觉伪影。因此，需要一种能够有效建模压缩视频特性和编码先验信息的方法。

Result: 大量实验表明，该方法在提升感知质量方面是有效的，尤其是在严重压缩场景下。

Insight: 创新点在于设计了编码先验增强的双重条件分支来联合建模压缩视频和编码先验条件，引入了压缩退化语义提示机制使扩散过程感知压缩严重程度，并提出了编码先验引导的加权融合模块来融合特征。这为利用扩散模型进行视频增强时如何有效结合领域特定先验知识提供了新思路。

Abstract: Perceptual quality enhancement of severely compressed videos remains challenging due to complex artifact patterns and substantial information loss. Recent diffusion models have demonstrated strong generative capability for visual restoration, but directly applying them to compressed video often ignores compression degradation characteristics and may introduce structure-inconsistent hallucinations. To address this issue, this paper presents a diffusion-based compressed video enhancement method, named DiffCVE. Coding Prior-enhanced Dual Conditioning (CPDC) branches are designed to jointly model compressed video and coding prior conditions, where coding priors including residuals and motion vectors provide complementary structural and motion guidance during the diffusion denoising process. To make the diffusion process aware of compression severity, a Compression Degradation Semantic Prompting (CDSP) mechanism is introduced to leverage QP-conditioned textual prompts together with LoRA fine-tuning. In addition, a Coding Prior-guided Weighted Fusion (CPWF) module is incorporated into the VAE decoder to fuse VAE encoder and coding prior encoder features with QP-predicted weights. Extensive experiments demonstrate the effectiveness of the proposed method in improving perceptual quality, especially under severe compression settings. The project page with enhanced video demonstrations is available at https://wqmaker.github.io/projects/DiffCVE/.

[40] Vision Foundation Models in Radiology: A Scoping Review of Data, Methodology, Evaluation and Clinical Translation cs.CV | cs.AI | cs.LGPDF

Alejandro Vergara-Richart, Xavier Rafael-Palou, Almudena Fuster-Matanzo, Ignacio Iborra Roncales, Ángel Alberich-Bayarri

TL;DR: 本文对2017年至2026年间发表的、专门在放射影像数据上训练的视觉基础模型（VFMs）研究进行了范围综述，共纳入67项研究。综述从数据规模与异质性、架构与预训练可扩展性、下游任务可迁移性与泛化性三大支柱进行了梳理。研究发现，放射学专用VFMs展现出良好的迁移能力，但其临床转化仍受限于数据代表性不足、评估基准不统一、报告不完整以及缺乏面向部署的评估。

Details

Motivation: 放射影像领域视觉基础模型（VFMs）的开发日益增多，但其定义、开发和评估方法仍存在很大异质性。本研究旨在通过系统性综述，厘清该领域的研究现状、关键方法、评估实践及临床转化面临的挑战。

Result: 综述发现，数据集主要集中在脑部MRI、胸腹部CT和胸部X光，规模从不足10万到数百万图像不等。模型架构以Transformer为主，预训练方法以掩码图像建模、对比学习和多阶段方法为主导。评估主要集中在分割和分类任务上，而对跨中心、跨扫描仪、跨解剖部位和跨模态的泛化性验证报告不一致。

Insight: 论文的创新点在于首次对放射学专用视觉基础模型领域进行了系统性的范围综述，并构建了涵盖数据、方法、评估三大支柱的分析框架。从客观角度看，其核心洞察在于明确指出了当前研究在数据代表性、评估基准标准化、报告完整性以及面向临床部署的评估方面的关键不足，为未来研究指明了方向。

Abstract: Vision foundation models (VFMs) are increasingly being developed for radiological imaging, yet their definition, development and evaluation remain heterogeneous. We conducted a PRISMAScR scoping review of peer-reviewed studies published between January 2017 and March 2026 describing foundation models trained exclusively on radiological imaging data. Sixty-seven studies were included and mapped across three pillars: data scale and heterogeneity, architectural and pretraining scalability, and downstream transferability and generalization. Datasets primarily covered brain MRI, thoracoabdominal CT, and chest X-ray, ranging from fewer than 100,000 samples to multi-million-image cohorts. Transformer-based architectures and self-supervised pretraining predominated, particularly masked image modeling, contrastive learning and multi-stage approaches. Evaluation focused mainly on segmentation and classification, whereas cross-center, cross-scanner, anatomical and modality-shift validation was inconsistently reported. Alignment with FUTURE-AI principles was uneven. Overall, radiology-specific VFMs show promising transferability, but clinical translation remains constrained by limited data representativeness, heterogeneous benchmarks, incomplete reporting and insufficient deployment-oriented evaluation.

[41] An Edge-aware Prompt-enhanced SAM for Ultrasound Image Segmentation cs.CVPDF

Wenhao Li, Fangyi Liu, Bo Du

TL;DR: 本文提出了一种名为EP-SAM的边缘感知提示增强SAM模型，用于解决SAM在超声图像分割中边界描绘不佳的问题。该方法通过从图像编码器中提取多块特征来丰富从粗到细的语义表示，并引入边缘感知监督以提高对轮廓模糊和斑点噪声的鲁棒性，从而生成高质量提示以有效引导模型关注目标区域。

Details

Motivation: SAM在自然图像上表现出色，但在超声图像分割中，其性能常因边界描绘不佳而受限，本研究旨在解决这一局限性。

Result: 在多个基准测试上的实验结果表明，EP-SAM持续优于现有的基于SAM的方法。

Insight: 创新点在于将多块特征提取与边缘感知监督相结合，以增强SAM在超声图像中的语义表示和边界处理能力，从而生成更有效的提示来引导分割。

Abstract: Ultrasound image segmentation is essential for delineating anatomical structures and lesions, providing the foundation for accurate diagnosis. While the Segment Anything Model (SAM) has demonstrated remarkable success on natural images, its performance on ultrasound data is often hindered by poor boundary delineation. To address this limitation, we propose EP-SAM, an edge-aware and prompt-enhanced adaptation of SAM. Specifically, we leverage multi-block feature extraction from the image encoder to enrich coarse-to-fine semantic representations, while edge-aware supervision of the image encoder improves robustness to contour ambiguity and speckle noise. By integrating these complementary cues, EP-SAM generates high-quality prompts that effectively guide the model toward target regions of interest. Experimental results on multiple benchmarks demonstrate that EP-SAM consistently outperforms existing SAM-based methods.

[42] Unraveling Machine Behavior by Multi-Level Bias Analysis and Detection: Methodology and Application to Computer Vision cs.CVPDF

Ignacio Serna, Aythami Morales, Julian Fierrez

TL;DR: 该研究通过多层级分析（潜在空间、层激活和网络参数）来调查神经网络中偏见的存留与传播，并提出了三种偏见检测方法：SpaceBias、ActivationBias和WeightBias。这些方法旨在揭示偏见在网络架构内部不同层级的具体表现，而非仅将模型视为黑箱评估其输出。实验在DiveFace数据集（性别分类）和彩色MNIST基准（数字分类）上进行，训练并评估了超过127,000个具有不同偏见程度和类型的模型。

Details

Motivation: 传统方法通常将神经网络视为黑箱，仅评估其输出结果，而无法深入理解偏见如何在网络架构内部产生和传播。本研究旨在通过多层级分析，揭示偏见在模型内部（如潜在空间、激活和参数）的具体表现形式，从而提供更细致、深入的AI模型行为洞察。

Result: 在DiveFace数据集（72,000张人脸图像）的性别分类任务和具有可控偏见严重性的彩色MNIST基准数字分类任务上进行了实验。结果表明，当训练分布接近平衡时，内部差异（以及检测性能）会平滑下降。研究训练和评估了超过127,000个模型，验证了所提方法在量化偏见严重性方面的有效性。

Insight: 创新点在于提出了一个多层级偏见分析框架（潜在空间、激活、参数），并引入了SpaceBias这一新方法（利用邻域概率分布和KS检验量化潜在空间偏见）。客观来看，该研究将偏见检测从模型输出“黑箱”评估转向内部架构分析，提供了更细粒度的诊断工具，有助于理解偏见在神经网络中的传播机制。

Abstract: This study investigates the presence and propagation of bias within Neural Networks through a comprehensive multi-level analysis spanning the learned latent space, layer activations, and the network’s parameters. Based on this taxonomy, we propose three bias detection approaches: 1) SpaceBias (new method), which characterizes the latent space prior to the final classification layer using neighbor-probability distributions and quantifies bias with the two-sample Kolmogorov-Smirnov test on the per-group distributions. 2) ActivationBias (extension of the existing method InsideBias), which analyzes the activations of neural network filters and quantifies bias via a Mann-Whitney U test, based on the observed fact that underrepresented groups exhibit lower activation levels in the final convolutional layers. 3) WeightBias (extension of the existing method IFBiD), which uses a secondary neural network trained to identify biased patterns directly in the parameters of task-specific models. Unlike conventional methods, which assess neural network outcomes and treat the model as a black box, our proposed techniques provide insight into how biases manifest within the network architecture itself at different levels, offering a more nuanced and detailed understanding. Experiments are conducted on two complementary applications: gender classification in the DiveFace dataset (72,000 face images) and digit classification on a colored-MNIST benchmark with controlled bias severity. In total, more than 127,000 models with varying degrees and types of bias were trained and evaluated. The severity sweep shows that the internal disparity, and with it the detection performance, decreases smoothly as the training distribution approaches balance. The results highlight the importance of methods that provide deeper insight into the behavior of AI models.

[43] InfraQR: Edge-Placed QR-Inspired Structured Patch Attacks on Infrared Vision-Language Models cs.CVPDF

Xin Li, Jiaju Han, Ma Yaqi, Chengyin Hu, Yingying Zhao

TL;DR: 本文提出InfraQR，一种受QR码启发的结构化补丁攻击方法，针对红外视觉语言模型。该方法在图像边界放置紧凑的结构化补丁，通过代理CLIP风格编码器优化可学习网格单元，生成的补丁具有近二值化结构外观，但无需是有效的机器可读QR码。在红外分类、描述迁移和视觉问答任务中评估表明，该方法能显著降低模型性能，并揭示红外视觉语言模型对结构化边缘扰动的脆弱性。

Details

Motivation: 红外视觉语言模型在低光和恶劣视觉条件下应用广泛，但其对局部结构化扰动的鲁棒性尚未充分探索。现有红外对抗研究主要关注目标检测器，而红外视觉语言模型的安全性缺乏系统性检验。

Result: 在包含300张图像的红外基准测试中，InfraQR显著降低了多个CLIP风格分类器的准确率，例如将OpenAI CLIP的准确率从98.67%降至0.70%。生成的对抗图像还能迁移到黑盒描述生成和视觉问答模型，导致描述语义退化，并在基于GPT-5.4的评估中引发更多错误答案。

Insight: 创新点在于提出一种边界放置的结构化补丁攻击，无需直接遮挡目标物体，且补丁无需是有效QR码；这揭示了跨任务鲁棒性研究的重要性，即对抗扰动可影响分类、描述和问答等多种视觉语言任务。

Abstract: Infrared vision-language models are increasingly used for perception under low-light and adverse visual conditions, yet their robustness to localized structured perturbations remains underexplored. Existing infrared adversarial studies mainly focus on object detectors, leaving the security of infrared vision-language models less systematically examined. We present InfraQR, a QR-inspired structured patch attack for infrared vision-language models. Unlike localized attacks that attach perturbations to the target object, InfraQR places a compact structured patch along image boundaries and optimizes learnable grid cells through surrogate CLIP-style encoders. The resulting patch has a near-binary structured appearance, but is not required to be a valid or machine-readable QR code. We evaluate InfraQR on infrared classification, caption transfer, and question-answer-aware visual question answering (VQA) tasks. On a 300-image infrared benchmark, InfraQR sharply reduces the accuracy of multiple CLIP-style classifiers, including reducing OpenAI CLIP accuracy from 98.67% to 0.70%. The generated adversarial images also transfer to black-box captioning and VQA models, causing semantic degradation in captions and more error-prone answers under GPT-5.4-based evaluation. These results show that infrared vision-language models remain vulnerable to structured edge-placed perturbations, motivating further study of cross-task robustness beyond direct object occlusion.

[44] CarbonCLIP: Enhance Carbon Prediction from Satellite Imagery via Integrated Street-View Semantics and Temporal Context Training cs.CV | cs.AIPDF

Zeru Yang, Fang-Ying Gong, Steve H. L. Yim, Chau Yuen

TL;DR: CarbonCLIP是一个面向任务的多模态蒸馏框架，旨在通过双分支对比学习将上下文知识整合到统一的卫星表征中，以提升基于卫星图像的碳排放预测精度。该方法在预训练阶段利用街景图像生成的细粒度文本描述和月份编码器分别提供空间语义先验和时间先验，而在推理阶段仅需卫星图像，支持可扩展部署。在北京和新加坡的实验表明，其性能优于基线方法。

Details

Motivation: 现有城市碳排放估算方法因数据源异质性和遥感数据缺乏细粒度语义-时间上下文，难以在不同城市间一致应用。本文旨在解决卫星数据在反映地面人类活动方面的语义-时间上下文缺失问题，以提升跨城市碳排放预测的鲁棒性和准确性。

Result: 在北京和新加坡两个城市上的实验表明，CarbonCLIP的性能超越了基线方法，验证了其能够有效地将多模态知识迁移到卫星表征中，为基于卫星的城市碳建模提供了鲁棒的解决方案。

Insight: 创新点在于通过双分支对比学习框架，显式地弥合了自上而下的卫星视图与地面人类活动之间的语义鸿沟，并引入了由大模型自动生成的街景文本描述作为语义先验以及月份编码器作为时间先验。从客观角度看，其“仅需卫星图像进行推理”的设计巧妙地解决了多模态数据在部署时的可获得性问题，增强了方法的实用性。

Abstract: Accurately estimating urban carbon emissions is critical for sustainable urban planning, yet many existing approaches remain difficult to apply consistently across cities due to data-source heterogeneity and the lack of fine-grained semantic-temporal context in remote sensing data. We propose CarbonCLIP, a task-oriented multimodal distillation framework that improves satellite-based carbon emission prediction by transferring contextual knowledge into a unified satellite representation through dual-branch contrastive learning. Unlike conventional methods that rely on static visual features, CarbonCLIP explicitly bridges the gap between top-down satellite views and ground-level human activities. Specifically, the spatial branch uses fine-grained textual descriptions automatically generated from street-view images by Large Multimodal Models (LMMs) to provide semantic priors reflecting building functions, infrastructure, and urban activities, while the temporal branch employs a month encoder to encode temporal priors associated with monthly emission variation. CarbonCLIP requires multimodal data only during the pretraining phase; during inference, it relies solely on satellite imagery, thereby supporting scalable deployment when ground-level data are unavailable at inference. Experiments on Beijing and Singapore demonstrate that CarbonCLIP outperforms baselines in both study cities. The results validate that our method effectively transfers multimodal knowledge into satellite representations, offering a robust solution for satellite-based urban carbon modeling.

[45] SoccerNet 2026 Challenges Results cs.CVPDF

Anthony Cioppa, Silvio Giancola, Håkan Ardö, Mohamad Dalal, Jan Held

TL;DR: SoccerNet 2026挑战赛是第六届年度开放基准测试活动，旨在推动体育视频理解领域的计算机视觉研究。本次挑战赛包含五个基于视觉的任务：球动作预测、球员中心球动作检测、新视角合成、Spiideo SoccerNet Synloc运动员定位和视觉问答。该论文描述了任务设置、评估协议、排行榜及领先提交结果，以记录各任务在保留测试数据上的当前技术水平。

Details

Motivation: 通过组织年度基准挑战赛，为体育视频理解领域提供标准化数据集和评估框架，以促进计算机视觉技术在足球视频分析中的研究进展，解决如动作预测、多模态理解等实际问题。

Result: 挑战赛吸引了广泛参与，共有427个团队提交了1,129份结果，覆盖五个任务，其中28个团队贡献了经过评审的技术报告；论文基于保留的挑战数据呈现了各任务的排行榜和领先方法性能，反映了当前SOTA水平。

Insight: 创新点在于构建了涵盖时序预测、多目标关联、多视角合成、单图像定位和多模态问答的综合性基准套件，为体育视频分析提供了统一的评估平台；客观来看，其任务设计强调了从低级视觉到高级语义理解的完整研究链条，有助于推动领域向更细粒度和实用化方向发展。

Abstract: The SoccerNet 2026 Challenges constitute the sixth annual edition of the SoccerNet open benchmarking effort, dedicated to advancing computer vision research in sports video understanding. This year’s challenges span five vision-based tasks: (1) Ball Action Anticipation, predicting the timing and class of ball-related actions within a short future window from a preceding observation window; (2) Player-Centric Ball Action Spotting, temporally localizing and classifying ball-related actions while assigning each action to the acting player through team affiliation and jersey number; (3) Novel View Synthesis, rendering images from unobserved camera poses in multi-view football scenes; (4) Spiideo SoccerNet Synloc, localizing athletes in real-world pitch coordinates from a single calibrated static-camera image; and (5) Visual Question Answering, answering multiple-choice questions about football broadcasts across text, image, and video inputs. For each task, participants were provided with annotated data, a unified evaluation protocol, and a public baseline. This edition saw broad participation, with 427 teams submitting 1,129 entries across the five tasks and 28 teams contributing reviewed technical reports. This paper describes each task and its evaluation protocol, presents the challenge leaderboards, and summarizes the leading submissions, with the aim of documenting the current state of each task as measured on held-out challenge data.

[46] HAJJv2-CrowdCount: Zero-Shot Benchmark for Dense Crowd Counting cs.CV | cs.AIPDF

Reem AlYabis, Fares AlTuwaim, AlJawharh AlOtaibi, Mohamed Eltahir

TL;DR: 本文介绍了HAJJv2-CrowdCount基准数据集，该数据集为HAJJv2视频的测试片段提供了每秒的人工标注人群计数。作者利用该数据集评估了三种最新的零样本人群计数范式（YOLO-World、APGCC和SAM3Count）在麦加朝觐视频这一具有挑战性场景下的性能。

Details

Motivation: 解决在麦加朝觐视频中进行自动化人群计数的难题，该场景因摄像机俯拍角度陡峭、个体遮挡严重、单帧人数极多而违反现有模型的假设，且缺乏公开、细粒度的基准测试。

Result: 在整体性能上，基于提示的分割模型SAM3Count取得了最低的平均绝对误差（MAE 70.4），优于YOLO-World（MAE 92.0）和APGCC（MAE 152.9）。然而，在最密集的帧（部署中最相关的场景）上，基于检测和分割的方法性能急剧下降（MAE > 300），而基于点的方法APGCC则稳健得多（MAE 114.9）。

Insight: 论文的主要创新在于发布了细粒度的HAJJv2-CrowdCount基准数据集。一个关键的客观发现是，在极端密集和遮挡的场景下，基于点估计的计数方法比基于检测或分割的先进零样本方法更具鲁棒性，这对实际部署（如朝觐人群管理）具有决策参考价值。

Abstract: Automated crowd counting in Hajj video is difficult not because current models lack capacity, but because the footage violates the assumptions those models were built on: cameras observe the crowd from steep, near-vertical angles, individuals occlude one another extensively, and a single frame can contain well over a thousand people. Benchmarks that test crowd counting in such an environment are either private or not detailed per second. We revisit the HAJJv2 dataset and contribute HAJJv2-CrowdCount: per-second human-annotated crowd counts for its testing videos. Using these annotations, we benchmark three recent zero-shot counting paradigms: an open-vocabulary detector (YOLO-World), a point-based counter (APGCC), and a promptable segmentation-based counter (SAM3Count). SAM3Count attains the lowest overall mean absolute error (MAE 70.4, 95% CI 56.0-86.1), ahead of YOLO-World (92.0) and APGCC (152.9). This ordering reverses, however, in the regime most relevant to deployment: on the densest frames, the detection- and segmentation-based counters both degrade sharply (MAE exceeding 300), while the point-based counter degrades far more gracefully (MAE 114.9). This inversion is decision-relevant for Hajj crowd management, where reliable counts are needed most precisely in the densest and most occluded scenes. The annotations are released to support reproduction and extension of these results.

[47] BUS: Brain-Inspired Unsupervised Self-Reflection for Advanced Multimodal Reasoning cs.CVPDF

Jiacheng Yang, Tongying Xiao, Yunkai Dang, Cong Wang, Yuekun Yang

TL;DR: 本文提出了一种受神经科学启发的无监督自反思框架BUS，旨在提升视觉语言模型在复杂视觉任务中的推理能力。该框架通过模拟人脑的后向预测机制，利用无标签数据训练模型进行自反思，从而减少对标注数据的依赖，并与主流微调方法兼容。

Details

Motivation: 当前视觉语言模型在处理需要一致性和细粒度推理的复杂视觉任务时表现不佳，且现有自反思方法依赖大量标注数据且缺乏显式的反思行为。本文受神经科学中人脑高效后向预测机制的启发，旨在弥补这一差距。

Result: 在8个基准测试上的广泛实验表明，BUS在多种复杂视觉任务中有效，仅使用无标签训练数据即可显著超越基础模型，验证了后向预测能力对VLM推理的关键作用。

Insight: 创新点在于将神经科学中的后向预测机制引入VLM的自反思训练，提出了一种无需标注的无监督学习框架，这为增强模型推理能力提供了新的生物启发式途径，并展示了与监督微调和强化学习等方法的兼容性。

Abstract: Current Vision-Language Models (VLMs) often struggle to handle complex visual tasks that require consistent and fine-grained reasoning. Recent methods aim to train models to facilitate self-reflective reasoning, i.e., reviewing and improving the generated reasoning. However, they require large volumes of annotated data and lack explicit reflective behavior during test time. This work aims to bridge this gap through inspiration from neuroscience. The human brain exhibits efficient backward prediction, i.e., predicting which current states are likely to precede a given future state. In this work, we first verify that mainstream VLMs can perform backward prediction, similar to the human brain. Then, we propose Brain-inspired Unsupervised Self-reflection (BUS), a label-free training framework to enhance reflective reasoning capability in challenging image analysis. BUS enables VLMs to perform backward prediction and provide explicit learning signals on data without ground-truth labels. In this way, BUS eliminates reliance on annotated data while improving reasoning performance. Notably, BUS is compatible with popular fine-tuning methods, such as Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). Finally, extensive experiments on 8 benchmarks demonstrate the effectiveness of BUS across a wide range of complex visual tasks. It achieves notable improvements over the base model while using only unlabeled training data. Our experimental findings validate that backward prediction capability is critical for VLM reasoning.

[48] MMAgent-R$^2$: Learning to Rerank and Reject for Agentic mRAG cs.CVPDF

Tao Zhang, Ziqi Zhang, Zongyang Ma, Yuxin Yang, Bing Li

TL;DR: 本文提出了MMAgent-R²，一种基于智能体（agentic）的多模态检索增强生成（mRAG）框架，用于解决知识库视觉问答（KB-VQA）任务中，当知识库存在大量视觉相似实体时，传统检索器难以区分、导致候选集被干扰项污染的问题。该框架通过集成视觉重排序和主动拒绝机制作为内部验证，突破了固定候选集的限制，并利用GRPO训练联合优化检索、验证和答案生成。

Details

Motivation: 现有mRAG方法依赖全局视觉特征匹配候选实体，当知识库包含大量视觉相似实体时，检索器难以区分，导致候选集中充斥视觉相似但事实不匹配的干扰项，且后续处理步骤受限于此固定候选集，检索失败的错误会传播到最终答案。

Result: 在InfoSeek、E-VQA和MMhops基准测试上的实验表明，MMAgent-R²取得了最先进的性能，在具有挑战性的检索场景和复杂的多图像多跳推理任务中优势尤为显著。

Insight: 主要创新点在于将视觉重排序（直接比较查询和候选图像以捕捉超越文本描述的判别性细节）和主动拒绝（丢弃不可靠结果并在未找到可靠匹配时检索额外候选）作为内部验证机制集成到智能体框架中，并通过复合奖励函数和GRPO训练实现检索、验证与生成的联合优化，突破了传统固定候选集的限制。

Abstract: Knowledge-based Visual Question Answering (KB-VQA) requires models to retrieve visual entities matching the query image from large-scale encyclopedic knowledge bases and answer related questions. Existing multimodal Retrieval Augmented Generation (mRAG) methods rely on global visual features to match candidate entities, yet when the knowledge base contains numerous visually similar entities, the retriever struggles to distinguish them, populating the candidate set with visually similar but factually mismatched distractors. Since subsequent processing steps such as noise filtering are also confined to this fixed candidate set, errors from failed retrieval inevitably propagate to the final answer. To address these challenges, we propose MMAgent-R$^2$, an agentic mRAG framework that integrates visual reranking and active rejection as its internal verification mechanism. Visual reranking directly compares query and candidate images, capturing discriminative details beyond textual descriptions to precisely identify the target entity among similar candidates; active rejection discards unreliable results and retrieves additional candidates when no confident match is found, moving beyond the fixed candidate pool. We design a composite reward function with step-level verification rewards and achieve joint optimization of external retrieval, internal verification, and answer generation via GRPO training. Experiments on InfoSeek, E-VQA, and MMhops demonstrate that \ours{} achieves state-of-the-art performance, with particularly notable advantages in challenging retrieval scenarios and complex multi-image multi-hop reasoning tasks.

[49] When Prompts Ignore Structure: Graph-Based Attribute Reasoning for Calibrated VLMs cs.CV | cs.AI | cs.LGPDF

Tanay Sodha, Aditya Sharma, Ramya Hebbalaguppe, Vinti Agarwal, Pranav Murthy Yeluripaty

TL;DR: 本文提出了一种名为ARGTCA的方法，用于改进视觉语言模型（VLMs）在测试时适应中的置信度校准问题。该方法通过构建符号属性图来建模类别与属性之间的关系，并利用图注意力网络（GAT）学习捕获属性间依赖关系的嵌入表示，从而缓解了传统提示调优导致的过自信和校准退化。

Details

Motivation: 现有方法在提升视觉语言模型零样本准确率时，常因基于熵的过自信而导致置信度校准不可靠，且忽略了类别属性之间的结构关系。本文旨在通过显式建模属性间的依赖关系，实现更可靠的测试时适应。

Result: 在九个基准测试上的实验表明，所提出的ARGTCA-DIV变体将平均期望校准误差（ECE）较基线降低了约37%，达到最佳性能；ARGTCA-DISC变体则始终表现第二好，将平均ECE降低了约17%。

Insight: 创新点在于将（类别，属性）对表示为符号属性图中的节点，并利用图注意力网络进行对比学习，以捕获属性间的结构化依赖。这为视觉语言模型的测试时适应提供了一种基于结构化属性推理的原则性方法。

Abstract: Reliable confidence estimation remains a key limitation of test-time adaptation in vision-language models (VLMs), where prompt tuning improves zero-shot accuracy but often degrades calibration due to entropy-driven overconfidence. Prior approaches mitigate this using LLM-derived class attributes and contrastive regularization, yet treat attributes independently, ignoring their relational structure. We propose ARGTCA, which represents (class, attribute) pairs as nodes in a Symbolic Attribute Graph and trains a Graph Attention Network (GAT) using contrastive objectives to produce structurally informed embeddings that capture inter-attribute dependencies. We introduce two attribute selection strategies: ARGTCA-DIV for intra-class diversity and ARGTCA-DISC for inter-class discrimination. Experiments across nine benchmarks show that ARGTCA-DIV reduces average Expected Calibration Error (ECE) by approximately ~37% over baselines, while ARGTCA-DISC consistently performs as the second-best variant, reducing average ECE by approximately ~17% over baselines. These results suggest that modeling symbolic attribute interactions provides a principled approach for reliable test-time adaptation in VLMs.

[50] Heterogeneity-Adaptive Diffusion Schrodinger Bridge for PET-Guided Whole-Body MRI Translation cs.CV | cs.AI | cs.LGPDF

Chengbo Wang, Jiacheng Yu, Linjie Bian, Ming Qi, Xiaosheng Liu

TL;DR: 本文提出了一种新颖的异质性自适应扩散薛定谔桥（HA-DSB）框架，用于解决全身MRI翻译中因解剖区域和病变组织导致的异质性分布挑战。该方法通过整合视觉语言模型生成的区域上下文嵌入进行区域特定建模，并利用PET图像的病变感知代谢先验，通过双阶段引导机制（前向过程的噪声调制和后向过程的注意力机制）直接增强病变区域的保真度。

Details

Motivation: 全身多模态医学成像（如PET-MR）的采集时间过长是临床实践中的主要障碍，而现有的基于深度学习的MRI翻译模型通常针对特定解剖区域，难以处理全身扫描中高度异质的特征分布（包括不同解剖区域和病变组织）。

Result: 实验表明，该方法在全身MRI翻译任务中，在不同身体区域均表现出优越性，并且在PET引导下，病变区域的翻译质量得到了提升。

Insight: 创新点在于将翻译问题明确建模为源与目标分布之间的随机传输，并整合了区域上下文嵌入和病变感知的PET先验。客观来看，其利用VLM进行区域自适应建模，以及将PET信息通过双阶段（噪声调制和注意力）机制直接融入扩散桥动力学的设计，是针对医学图像翻译中异质性和病变保真度问题的有效且新颖的解决方案。

Abstract: While whole-body multimodal medical imaging scanners have been increasingly recognized for more effective medical applications, the excessive long acquisition time in PET-MR scanning is a major obstacle in more efficient clinical practice. Deep learning-based MRI translation provides a potential solution to reduce scan duration. However, current models often focus on specific anatomical regions and face challenges for whole-body scans that consists of highly heterogeneous feature distributions mainly due to (1) different anatomical regions across whole-body, and (2) lesions or pathological tissues. This paper tackles the challenges through a novel Heterogeneity-Adaptive Diffusion Schrodinger Bridge (HA-DSB) framework. By explicitly modeling translation as stochastic transport between source and target distributions, HA-DSB incorporates region context embeddings derived from a vision-language model (VLM) to enable region-specific modeling. To enhance fidelity of the pathological tissue, lesion-aware metabolic prior from PET is integrated directly into the bridge dynamics through a dual-stage guidance mechanism. Specifically, a PET-guided noise modulation module adaptively scales spatial diffusion perturbations during the forward process, while PET features are leveraged during the reverse process to selectively amplify lesion-relevant structures via an attention mechanism. Experiments demonstrate the superiority of our method across different body regions in whole-body MRI translation and show improved translation quality in lesion areas under PET guidance. Our code is available at Github.

Kanglei Zhou, Ruizhi Cai, Xinning Wang, Yijian Zheng, Liyuan Wang

TL;DR: 本文提出了一种名为DualAlign的两阶段多模态融合框架，用于动作质量评估。该框架首先通过最大化RGB视频、光流和骨架模态之间的共享结构信息来构建连贯的视觉表示，然后在视觉稳定后融入文本语义，使高级描述补充而非扭曲底层视觉流形。为了在真实多模态条件下评估框架，作者还引入了包含多种模态和结构化文本的MM-JDM数据集。

Details

Motivation: 解决动作质量评估中现有多模态方法面临的两大挑战：异构模态导致的跨模态错位与不稳定融合，以及可靠多模态标注成本高昂导致的数据集多样性有限。

Result: 在提出的MM-JDM数据集上，DualAlign比现有最优方法的平均相关性提高了21.16%；在RG和Fis-V基准测试上分别取得了3.53%和5.95%的性能提升，并在模态缺失和标签稀缺条件下保持鲁棒性。

Insight: 创新点在于两阶段自适应对齐策略：先对齐视觉模态以构建稳定表示，再融入文本语义进行互补。从客观角度看，其提出的MM-JDM数据集模拟了真实世界的模态噪声、类别不平衡和标签稀缺，为多模态融合与对齐研究提供了具有挑战性的基准。

Abstract: Action Quality Assessment (AQA) aims to evaluate how well a person performs a movement, which is essential in applications such as sports scoring, skill assessment, and healthcare. However, unimodal approaches often struggle to capture subtle cues of movement quality in real-world settings. Although multi-modal inputs provide complementary information, existing methods still face two major challenges: heterogeneous modalities often lead to cross-modal misalignment and unstable fusion, and reliable multi-modal annotation is costly, resulting in limited dataset diversity. To address these challenges, we propose DualAlign, a two-stage multi-modal fusion framework with adaptive alignment. The framework first constructs a coherent visual representation by maximizing shared structural information across RGB video, optical flow, and skeleton modalities. Textual semantics are then incorporated after visual stabilization, allowing high-level descriptions to complement rather than distort the underlying visual manifold. To evaluate the framework under realistic multi-modal conditions, we introduce MM–JDM, a movement-quality assessment dataset integrating RGB videos, optical flow, skeleton sequences, and structured text. MM–JDM naturally exhibits modality noise, class imbalance, and label scarcity, making it a challenging benchmark for studying multi-modal fusion and alignment. Extensive experiments show that DualAlign improves average correlation on MM–JDM by 21.16% over the state-of-the-art methods and achieves gains of 3.53% and 5.95% on the RG and Fis-V benchmarks, respectively. DualAlign also remains robust under missing-modality and label-scarce conditions.

[52] HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models cs.CV | cs.AIPDF

Feng He, Zhenting Wang, Qifan Wang, Qiang Guan, Dongfang Liu

TL;DR: 该论文研究了视觉语言模型（VLMs）中的后幻觉推理（PHR）阶段，即幻觉语义进入模型推理上下文并影响下游预测的过程。作者提出了HIVE评估框架，通过对比忠实描述和幻觉描述，在九个任务和九个模型上进行了系统性分析。研究发现幻觉描述在视觉语言任务中常能提高准确性，而在纯文本任务中效果有限或不稳定，揭示了幻觉语义对多模态推理的动态影响。

Details

Motivation: 现有研究主要关注在生成时检测或抑制幻觉，但幻觉常源于部分或模糊的视觉证据，且其进入推理上下文后的影响尚未被充分探索。论文旨在系统研究后幻觉推理阶段，以理解幻觉语义如何影响下游预测。

Result: 在九个多模态任务和九个模型上的实验表明，幻觉描述在视觉语言任务（如VQA、图像检索）中常能提高准确性，而在纯文本任务（如文本蕴含、问答）中效果有限或不稳定，揭示了结构化的模态依赖模式。

Insight: 创新点在于首次系统研究后幻觉推理阶段，并构建了HIVE可控评估框架。客观分析表明，幻觉线索能拓宽语义覆盖并重塑推理动态，同时保持推理稳定性，这挑战了将幻觉仅视为错误的传统观点，为提升多模态系统可靠性和可解释性提供了新视角。

Abstract: Hallucinations in vision language models (VLMs) are commonly treated as semantic errors, yet they often arise from partial or ambiguous visual evidence. Prior work mainly focuses on detecting or suppressing hallucinations at generation time, leaving the subsequent reasoning stage largely unexplored. In this work, we study Post Hallucination Reasoning (PHR), the stage in which hallucinated semantics enter the model’s inference context and influence downstream predictions. To systematically investigate PHR, we introduce HIVE, Hallucination Inference and Verification Engine, an evaluation infrastructure that enables controlled comparisons between faithful and hallucinated captions. Across nine tasks and nine models, we observe structured modality dependent patterns: hallucinated captions often improve accuracy on vision language tasks, while text only tasks exhibit limited or unstable effects. Further analyses show that hallucinated cues broaden semantic coverage and reshape reasoning dynamics while preserving stable inference. These findings highlight that hallucinated semantics may influence downstream reasoning once they enter the model’s inference context. Understanding this post hallucination stage is important for improving the reliability and interpretability of multimodal reasoning systems. Code is publicly available at https://github.com/hefengcs/HIVE.

[53] Learning to Unify Deformable Shape and Texture Representations for Cardiac Video Classification cs.CVPDF

Tonmoy Hossain, Miaomiao Zhang

TL;DR: 本文提出了一种新颖的心脏视频分类模型，首次在可变形形状和图像纹理表示的集成空间中学习时序特征。该方法通过设计潜在空间中的双向交叉注意力机制来融合潜在的可变形形状和图像特征，使每种模态能根据时空对应关系自适应地加权另一种模态。与现有方法对所有心脏时相进行均匀加权不同，该方法能动态调整源自图像的形状和纹理表征随时间变化的贡献度。

Details

Motivation: 现有深度网络通过简单拼接来结合可变形形状和纹理特征，这既未充分利用它们的互补性，也未学习跨模态特征依赖关系，且导致对所有时间点施加均匀注意力，忽略了不同心脏时相在诊断重要性上的差异。

Result: 该方法在心脏电影磁共振（CMR）视频数据集上实现了最先进的分类性能，并通过注意力机制识别出诊断关键的心脏时相和模态贡献，从而提高了可解释性。

Insight: 创新点在于首次在可变形形状和图像纹理的集成潜在空间中学习时序特征，并引入双向交叉注意力机制进行自适应融合，以及动态调整不同时相中形状和纹理表征的贡献，这提供了更精细的跨模态交互和时序建模方法。

Abstract: Deformable shape representations have proven to be robust complements to texture features in cardiac image classification, offering geometric priors that are invariant to imaging artifacts and intensity variations. However, existing deep networks perform simple concatenation to combine these distinct feature representations, which neither fully exploits their complementary nature nor learns cross-modal feature dependencies. Furthermore, this results in uniform attention across all timepoints; hence ignoring the varying diagnostic importance across the cardiac phases. In this paper, we propose a novel cardiac video classification model that, for the first time, learns temporal features in an integrated space of deformable shape and image texture representations. In particular, we design a bi-directional cross-attention in the latent space to fuse latent deformable shape and image features, allowing each modality to adaptively weight the other based on spatio-temporal correspondence. In contrast to current methods that apply uniform weighting across all the cardiac phases, our approach learns to dynamically adjust the contributions of shape and texture representations, derived from images, over time. We demonstrate state-of-the-art classification performance on a cine cardiac magnetic resonance (CMR) video dataset, achieving improved interpretability from attention mechanisms that identify diagnostically critical cardiac phases and modality contributions.

[54] Infinite Worlds with Versatile Interactions cs.CVPDF

Zelin Gao, Qiuyu Wang, Jiapeng Zhu, Jingye Chen, Zichen Liu

TL;DR: 本文介绍了LingBot-World 2.0（又称LingBot-World-Infinity），这是LingBot-World的升级版本，包含四项主要改进：实现了无限交互时长并保持输出质量；通过蒸馏获得实时变体模型，支持60fps的720p视频流驱动；引入了更多样化的交互元素和文本驱动事件；首创在世界建模中整合智能体框架，由规划角色行为的驾驶智能体和合成环境元素的导演智能体协作。系统还支持多玩家同时体验，并提供了易于部署的轻量级模型。

Details

Motivation: 旨在提升交互式世界模拟器的能力，解决现有模型在交互时长、响应速度、交互多样性以及环境动态生成方面的局限性。

Result: 模型实现了无限交互时长，实时变体模型能驱动60fps的720p视频流，交互元素多样性显著提升。

Insight: 创新点在于采用了因果预训练范式以实现无限交互，通过蒸馏获得实时模型，以及在世界建模中引入由驾驶智能体和导演智能体组成的智能体框架来协同规划行为与合成环境，这为动态、开放式的虚拟世界构建提供了新思路。

Abstract: We present LingBot-World 2.0 (also known as LingBot-World-Infinity), an advanced iteration of LingBot-World featuring four distinct upgrades. (1) Our model achieves an unbounded interaction horizon while maintaining consistent output quality, benefiting from a carefully crafted causal pretraining paradigm. (2) Through distilling a real-time variant from the base model, our system guarantees rapid response time, sufficient to drive 720p video streams at 60 fps. (3) Compared to the previous version, this update introduces highly diverse interactive elements, comprising a broader spectrum of actions (e.g., attacking, archery, spell-casting, and shooting) alongside a richer variety of text-driven events. (4) We pioneer the integration of an agentic harness within the domain of world modeling, wherein a pilot agent is tasked with planning and executing character behaviors, while a director agent is responsible for synthesizing novel environmental elements as the scene progresses. Additionally, to facilitate a shared experience, we develop an interface that permits multiple players to simultaneously immerse themselves in this vivid world simulator. We pair our primary 14B model with a lightweight 1.3B counterpart, which supports effortless deployment on a single GPU.

[55] AA-ViT: Anatomically Aware Vision Transformer with Structural and Frequency Guidance for Contrast Enhanced Brain MRI Synthesis cs.CVPDF

Talha Meraj, Tom Flannery, Charlie Cummins, Matt Townend, Thomas C Booth

TL;DR: 本文提出了一种名为AA-ViT的解剖学感知视觉Transformer模型，用于从非对比增强的MRI模态（T1、T2、FLAIR）合成对比增强脑部MRI（CEMRI）。该模型通过结构和频率引导，旨在更好地保留解剖边界和肿瘤特异性精细结构。在BraTS 2021数据集上的实验表明，该方法在PSNR和SSIM指标上超越了现有最佳方法，并获得了临床专家的初步验证。

Details

Motivation: 标准MRI对比度有限且存在伪影，常需使用造影剂来增强病灶可见性，但造影剂的使用对肾功能不全等患者存在禁忌且不总是可行。现有基于生成式AI的CEMRI合成方法依赖隐式特征学习，难以有效保留解剖边界和肿瘤精细结构。

Result: 在BraTS 2021数据集上，所提方法在PSNR和SSIM指标上达到了最先进水平（SOTA）。对19个不同胶质瘤病例的临床评估由三位神经放射科医生和一位神经外科医生进行，平均评分为3.94/5，提供了先前研究中罕见的初步临床验证。

Insight: 创新点在于将解剖学先验知识（结构和频率引导）明确整合到视觉Transformer架构中，以增强对解剖和病灶边界的保留能力。这为医学图像合成任务提供了一种结合领域知识与深度学习模型的新思路，其临床评估流程也为AI医疗研究提供了可借鉴的验证方法。

Abstract: Accurate tumour localization and diagnosis is a critical component of clinical care for brain cancers. Magnetic Resonance Imaging (MRI) is the most commonly used imaging modality due to its superior soft-tissue contrast. However, standard MRI often exhibits limited contrast and imaging artifacts, which necessitates the use of contrast agents to enhance lesion visibility. The administration of chemical contrast agents is not always feasible and may be contraindicated in patients with renal impairment or other health conditions. As a result, developing accurate and non-invasive contrast enhanced MRI (CEMRI) synthesis methods has clinical importance. In recent years, numerous approaches for CEMRI synthesis have been proposed, predominantly relying on generative artificial intelligence models. While these methods demonstrate promising performance, their dependence on implicit feature learning often limits their ability to preserve anatomical boundaries and tumour-specific fine structures. To address these challenges, we propose an anatomically aware frequency-and-structure-guided vision transformer (AA-ViT), for CEMRI synthesis using pre-contrast MRI modalities (T1, T2, and FLAIR). Experiments on the BraTS 2021 dataset demonstrate that the proposed method preserves anatomical and lesion boundaries, achieving higher PSNR and SSIM than state-of-the-art approaches. Clinical evaluation by three neuroradiologists and a neurosurgeon on 19 randomly selected cases across diverse gliomas yielded a mean score of 3.94/5, providing preliminary clinical validation rarely seen in prior studies. Synthetic post-contrast scans from our model could lower scanning costs, shorten imaging time, and avoid the potential risks of using gadolinium-based contrast agents.

[56] Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence cs.CVPDF

Shuailei Ma, Jiaqi Liao, Xinyang Wang, Jingjing Wang, Chaoran Feng

TL;DR: 本文提出了LingBot-Video，一种基于扩散Transformer（DiT）的视频预训练范式，专门为具身智能设计。通过采用混合专家（MoE）架构来平衡建模能力与推理效率，构建包含机器人导向视频的数据集，并引入多维奖励系统来增强物理合理性和任务完成的对齐，旨在弥合视频生成模型与机器人控制之间的领域鸿沟。

Details

Motivation: 当前视频生成模型主要关注内容创作，存在视觉保真度和创造性优先于计算效率和物理真实性的设计倾向，导致与机器人控制等具身智能任务存在领域不匹配问题。

Result: 综合评估验证了其作为视频基础模型的性能和效率，该模型是首个开源的大规模MoE视频基础模型。

Insight: 创新点在于从架构（采用可扩展的MoE框架）、数据（构建机器人导向的视频数据集增强对动作和世界动态的理解）和训练（引入超越美学和运动一致性的多维奖励系统以强化物理合理性与任务对齐）三个维度系统性地定制视频预训练范式，以桥接数字创造与物理执行。

Abstract: Despite the recent promise in robot control, video generative models suffer from a domain mismatch due to their primary focus on content creation. For example, their design inherently prioritizes visual fidelity and creativity over computational efficiency and physical realism. In this work, we present LingBot-Video, a DiT-based video pretraining paradigm specifically tailored for embodied intelligence. From the architecture perspective, we adopt the Mixture-of-Experts (MoE), instead of dense, framework to achieve a better trade-off between modeling capacity and inference efficiency, and manage to scale it up from scratch. From the data perspective, we construct a data profiling engine that augments standard internet videos with extensive robot-oriented footage, encompassing manipulation, navigation, and egocentric perspectives, to equip the base model with an intrinsic understanding of actions and world dynamics. From the training perspective, we develop a multi-dimensional reward system to enforce the alignment regarding physical rationality and task completion, going beyond standard criteria such as aesthetics, prompt-following, and motion consistency. Comprehensive evaluations validate its performance and efficiency as a video foundation model. We contribute LingBot-Video as the inaugural large-scale, open-source MoE video foundation model to the community, in a pioneering effort to bridge digital creativity and physical actuation.

[57] MedPMC: A Systematic Framework for Scaling High-Fidelity Medical Multimodal Data for Foundation Models cs.CV | cs.LGPDF

Hyunjae Kim, Dain Kim, Pan Xiao, Serina S. Applebaum, Younjoon Chung

TL;DR: 本文提出了MedPMC，一个自动化、可持续更新的框架，用于从PubMed Central (PMC)的许可文献中构建高质量、大规模的医学多模态数据集。该框架处理了610万篇PMC文章，生成了1100万个医学图像-文本对，并通过组件评估和基准测试证明了其在提升医学多模态基础模型性能方面的有效性。

Details

Motivation: 医学本质上是多模态的，但多模态基础模型的发展受限于大规模、高质量临床数据的获取。现有的PMC衍生资源在保真度、可重复性和临床验证方面存在不足，因此需要一种系统方法来构建高保真的医学多模态数据基础设施。

Result: 在26个涵盖11个专业的基准测试中，使用MedPMC数据训练的CLIP风格模型在零样本AUC上比最强的架构匹配生物医学CLIP基线平均提高了7.1个百分点，尽管使用的图像-文本对数量不到一半。作为多模态大语言模型的视觉编码器，它在两个医学视觉问答基准上分别提高了1.9和16.9个百分点，并在10,524张耶鲁纽黑文健康系统的皮肤病照片中，将形态到图像检索的Recall@5提高了11.7个百分点。

Insight: 创新点在于提出了一个自动化、可扩展的框架，能够从科学文献中高效提取高保真医学多模态数据，并通过严格的组件评估（如图像筛选、多面板检测、分类）确保数据质量。客观来看，该方法通过利用许可文献解决了医学数据稀缺问题，其系统化流程和临床验证显著提升了数据集的可靠性和下游模型性能，为医学AI提供了可重复的基础设施。

Abstract: Medicine is inherently multimodal, requiring clinicians to synthesize information across diverse data streams. Yet the development of multimodal foundation models is constrained by limited access to large-scale, high-quality clinical data. Although PubMed Central (PMC) offers a complementary source of expert-authored image-text data, existing PMC-derived resources remain limited in fidelity, reproducibility, and clinical validation. We introduce MedPMC, an automated, continuously updatable framework that transforms permissively licensed literature into high-fidelity infrastructure for medical multimodal models. Applied to 6.1 million PMC articles, MedPMC curated 11 million medical image-text pairs. Component evaluations showed strong performance for initial screening (F1 = 93.2), multi-panel figure detection (F1 = 96.5), figure separation (mAP = 89.8), caption separation and alignment (F1 = 81.4; ROUGE-L = 85.3), and medical figure classification (F1 = 96.5). Manual review by five annotators, three with medical training, found 95.3% of MedPMC images medically relevant, versus 19.7% in a prior PMC-derived dataset. Across 26 benchmarks spanning 11 specialties, a MedPMC-trained CLIP-style model improved average zero-shot AUC by 7.1 percentage points over the strongest architecture-matched biomedical CLIP baseline despite using fewer than half as many image-text pairs. As the vision encoder in a multimodal large language model, it improved medical visual question-answering by 1.9 and 16.9 percentage points across two benchmarks. In 10,524 Yale New Haven Health System dermatology photographs, it improved morphology-to-image retrieval Recall@5 by 11.7 percentage points. These findings show that high-fidelity literature curation strengthens medical multimodal foundation models across benchmark and clinical settings. We publicly release the framework, corpus, benchmarks, and pretrained models.

cs.RO [Back]

Antonio Cano, Guillermo Pérez, Luis Merino, Randy Gomez

TL;DR: 本文提出了一种多模态语音活动预测（MM-VAP）框架，用于社交机器人中的轮转预测。该框架将原有的仅音频VAP扩展至同步的视听输入，并利用预训练的视听骨干网络，通过低秩适应（LoRA）进行调整，结合说话人注意力机制和语义一致性损失来提升预测性能。

Details

Motivation: 轮转预测是社交机器人参与人机交互（尤其是作为调解者时）的关键需求，机器人需要预测对话动态而非仅对停顿做出反应。

Result: 在NoXi和NoXi+J数据集上的实验显示，该方法超越了当前基线，特别是在某些轮转事件上表现更优；在Haru EDR语料库上的进一步评估也支持了该方法在面向调解的人机交互中的适用性。

Insight: 创新点包括将VAP扩展为多模态框架，利用预训练骨干网络进行低秩适应，以及引入语义一致性损失来正则化输出空间；从客观角度看，该方法有效整合了视听信息并建模说话人间关系动态，提升了轮转预测的准确性。

Abstract: Turn-taking prediction is a key requirement for social robots involved in human-human interaction, particularly in mediator settings, where the robot must anticipate conversational dynamics rather than merely react to pauses. This work presents a Multimodal Voice Activity Projection (MM-VAP) framework that extends the original audio-only VAP formulation to synchronized audio-visual inputs while preserving its self-supervised future-projection objective. The proposed approach builds on pretrained audio-visual backbones originally optimized for speech-related tasks and adapts them through Low-Rank Adaptation to the multimodal turn-taking problem. After independent speaker encoding, an inter-speaker attention stage models the relational dynamics required to project future voice activity. In addition, a semantic consistency loss is introduced to regularize the 256-state output space according to higher-level dialogue activity patterns. Experiments on NoXi and NoXi+J showed improvements over the current baselines, particularly for some turn-taking events. Additional evaluation on the Haru EDR corpus further supported the suitability of this direction for mediation-oriented human-robot interaction.

[59] Dynamic Object Detection and Tracking in Construction: A Fisheye Camera and LiDAR Sensor Fusion Model cs.RO | cs.CVPDF

Yilong Chen, Huili Huang, Yong K. Cho

TL;DR: 本文提出了一种用于施工现场动态物体检测与跟踪的传感器融合框架，结合了激光雷达的精确性和鱼眼相机的语义丰富性。该方法通过将注册点云中的移动物体投影到2D圆柱全景图上进行语义标注，并与实时图像检测对齐以更新卡尔曼滤波器观测，从而实现对动态与静态状态转换物体的鲁棒处理。

Details

Motivation: 为了解决机器人在复杂施工现场与人类安全协同作业时对动态物体进行鲁棒检测与跟踪的需求，现有基于激光雷达的SLAM和占据栅格方法虽可行，但许多先进的3D视觉方法严重依赖预训练神经网络且需额外后处理来识别移动物体，因此探索结合激光雷达精度与RGB图像语义的传感器融合技术成为有前景的替代方案。

Result: 所提出的系统在真实世界施工环境中表现出高精度、简单性和鲁棒性，特别是在处理物体在动态与静态状态间转换时表现优异，适合实际部署。

Insight: 创新点在于将激光雷达点云中的移动物体检测与鱼眼相机实时图像语义标注通过2D圆柱投影对齐，实现传感器融合以增强卡尔曼滤波器更新，从而提升动态物体跟踪的鲁棒性和适应性，为复杂环境下的机器人感知提供了可借鉴的多模态融合方法。

Abstract: Robust dynamic object detection and tracking are essential for enabling robots to operate safely and effectively alongside humans in complex environments such as construction sites. While LiDAR-based SLAM and occupancy grid methods offer viable solutions for detecting and tracking motion, many state-of-the-art 3D vision approaches rely heavily on pre-trained neural networks and require additional post-processing to identify moving objects. Sensor fusion techniques, combining the precision of LiDAR with the semantic richness of RGB imagery, offer a promising alternative. In this work, we present a novel framework that enhances a quadruped robot equipped with a LiDAR sensor and an upward-facing fisheye camera for real-time dynamic object detection and tracking. After identifying moving objects within a registered point cloud, our method assigns semantic labels by projecting 3D coordinates onto a 2D cylindrical panorama, aligning with real-time image-based detections for observation update of the Kalman filter. The proposed system demonstrates high precision, simplicity, and robustness, particularly in handling objects transitioning between dynamic and static states, thus it is well-suited for deployment in real-world construction environments.

[60] SonoRank: Towards Calibration-Free Real-Time Finger Flexion Detection from Forearm Ultrasound Sequences cs.RO | cs.CVPDF

Dean Zadok, Alon Wolf, Alex M. Bronstein, Oren Salzman

TL;DR: 本文提出SonoRank方法，旨在实现无需用户特定校准的实时手指弯曲检测。该方法通过从超声序列中学习手指相对运动幅度的排序表示，并利用操作开始时采集的静止参考帧进行微调，以分类各手指是否主动弯曲。

Details

Motivation: 当前基于表面肌电图的假手功能有限且易被弃用，超声成像虽能实时观测肌肉活动并控制更多自由度，但现有方法需针对每位用户进行微调，阻碍了其商业化应用。

Result: 在包含12名受试者的数据集上进行留一受试者交叉验证，SonoRank的F1分数比跳过排序阶段的直接分类基线提高了28%，表明该方法在实现用户无关控制方面具有有效性。

Insight: 创新点在于引入成对排序作为预训练信号，以学习与用户无关的手指运动表示，这为基于超声的假肢实现无需校准的实用部署提供了重要进展。

Abstract: Powered prosthetic hands are frequently abandoned, largely due to the limited functionality of current devices that rely on surface electromyography (sEMG). Sonomyography (ultrasound) has emerged as a promising alternative, owing to its ability to observe muscle activity in real time and control a greater number of degrees of freedom. Yet, existing ultrasound-based methods require per-user fine-tuning, limiting their commercialization. We propose SonoRank, an important step towards calibration-free finger flexion detection from forearm ultrasound video. SonoRank first learns to rank pairs of ultrasound sequences by their relative motion magnitude for each of the five fingers. The learned representations are then fine-tuned to classify whether each finger is actively flexing, using a rest reference that is captured at the beginning of the operation. Under 12-fold leave-one-subject-out cross-validation on a dataset of twelve subjects with synchronized kinematics, SonoRank achieves a 28% improvement in F1 score over direct classification baselines that skip the ranking stage. These results establish pairwise ranking as an effective pretraining signal for subject-independent control, bringing ultrasound-based prosthetics closer to practical, calibration-free deployment.

[61] EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI cs.RO | cs.CVPDF

Xinjie Wang, Liu Liu, Taojun Ding, Andrew Choi, Chaodong Huang

TL;DR: 本文提出了EmbodiedGen V2，一个用于具身智能的生成式3D世界引擎，旨在自动化构建可直接用于策略训练的可执行仿真环境。它通过一个统一的仿真就绪表示，将跨模拟器的资产、交互功能、任务驱动的世界、大规模多房间场景和状态化Vibe Coding集成到一个可生成、可编辑和可重用的仿真流程中。

Details

Motivation: 当前仿真就绪3D资产生成技术发展迅速，但将这些资产组装成可用于策略训练的任务环境仍然主要依赖手动操作，这限制了可扩展的闭环学习。EmbodiedGen V2旨在填补这一空白。

Result: 评估显示，其资产流水线达到了96.5%的人工接受率和98.6%的碰撞成功率，83.3%的任务驱动世界无需手动修改即可直接用于下游仿真。使用生成环境进行在线强化学习将仿真成功率从9.7%提升至79.8%，并迁移到真实机器人上，任务成功率从21.7%提升至75.0%。

Insight: 主要创新点在于提出了一个统一的、可生成的仿真就绪表示和流水线，将资产、交互、任务和场景生成自动化集成，从而为具身策略的训练、评估和部署提供了可扩展的仿真基础设施。其将Vibe Coding等概念整合以实现状态化、可编辑的世界构建，是一个值得关注的系统级创新。

Abstract: We present EmbodiedGen V2, a generative 3D world engine for building executable sim-ready environments for embodied intelligence. Sim-ready 3D asset generation has advanced rapidly, yet assembling such assets into policy-ready task environments remains largely manual, limiting scalable closed-loop learning. EmbodiedGen V2 addresses this gap through a unified sim-ready representation that connects cross-simulator assets, interaction affordances, task-driven worlds, large-scale multi-room scenes, and stateful Vibe Coding into a generative, editable, and reusable simulation pipeline. The generated environments support manipulation, navigation, mobile manipulation, cross-simulator deployment, and embodied policy training. In evaluation, the asset pipeline achieves 96.5% human acceptance and 98.6% collision success, and 83.3% of task-driven worlds are directly usable for downstream simulation without manual modification. Online reinforcement learning with generated environments further improves simulation success from 9.7% to 79.8%, and transfers to real robots with task success increasing from 21.7% to 75.0%. These results establish EmbodiedGen V2 as scalable simulation infrastructure for training, evaluating, and deploying embodied policies.

[62] Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation cs.RO | cs.CVPDF

Hongyu Qu, Jianzhe Gao, Xiaobin Hu, Shaohuan Yang, Xinlei Yu

TL;DR: 本文提出LaMem-VLA，一种用于机器人操作的视觉-语言-动作（VLA）模型新框架。它通过将历史经验重构为潜在记忆令牌，并直接将其与VLA推理交织，解决了现有模型在长视野、时间依赖性任务上的困难。

Details

Motivation: 主流VLA模型在马尔可夫假设下主要根据当前观察预测动作，难以处理长视野、时间依赖的任务。现有基于记忆的VLA方法将记忆置于VLA推理的原生潜在嵌入空间之外，阻碍了历史经验与多模态推理和动作形成的流畅交织。

Result: 在SimplerEnv和LIBERO基准测试上的大量实验证明了LaMem-VLA的优越性。

Insight: 核心创新在于提出了一个完全在连续潜在空间内表示、检索和利用历史经验的框架，其四个协调组件（策展人、搜寻者、压缩器、编织者）实现了记忆与VLA推理的直接、原生融合，从而更有效地指导动作生成。

Abstract: Mainstream Vision-Language-Action (VLA) models predict actions primarily from the current observation under a Markovian assumption, thus struggling with long-horizon, temporally dependent tasks. Existing memory-augmented VLAs either expand the observation window or retrieve history from the memory bank as auxiliary policy-side context. However, they leave memory outside the native latent embedding space of VLA reasoning, preventing historical experience from being fluidly interleaved with multimodal reasoning and action formation. To this end, we introduce LaMem-VLA, a latent-memory-native framework that reconstructs historical experience into latent memory tokens and directly interweaves them with VLA reasoning. At its core, LaMem-VLA introduces four coordinated components: (i) a curator that organizes historical experience into two complementary short-term and long-term memory vaults; (ii) a seeker that queries both vaults using the multimodal cognition to retrieve context-relevant evidence; (iii) a condenser that reconstructs the retrieved evidence into compact short-term and long-term latent memory tokens; and (iv) a weaver that injects these memory tokens with the current observation and instruction into one continuous embedding sequence. By representing, retrieving, and consuming historical experience entirely in the same continuous latent space, LaMem-VLA enables memory to directly participate in VLA reasoning and guide action generation under a bounded context. Extensive experiments on SimplerEnv and LIBERO demonstrate the superiority of our LaMem-VLA.

cs.AI [Back]

[63] When Does In-Context Search Help? A Sampling-Complexity Theory of Reflection-Driven Reasoning cs.AI | cs.CLPDF

Yotam Wolf, Noam Wies, Amnon Shashua

TL;DR: 本文对大型语言模型中的上下文搜索进行了理论分析，将其建模为推理轨迹的近似推断过程，其中基础模型定义先验分布，自我反思提供后验更新反馈。研究表明，当反思能可靠定位早期错误时，上下文搜索能带来指数级改进，仅需多项式次顺序尝试即可解决零样本通过率极低的问题；反之，若该特性失效，则相比并行采样无渐进优势。

Details

Motivation: 研究动机在于理解上下文搜索（模型迭代生成、批判和修订解决方案）何时有效，通过建立采样复杂度理论来分析其底层机制与性能边界。

Result: 理论分析表明，在反思可靠定位错误时，上下文搜索能实现指数级改进，且这种收益具有鲁棒性和可学习性——近似后验更新足够有效，通过搜索轨迹的交叉熵训练能以多项式样本复杂度恢复所需行为。研究在真实大型推理模型上验证了关键定性预测。

Insight: 创新点在于将上下文搜索形式化为近似推断问题，建立了采样复杂度理论框架，揭示了反思定位错误能力对搜索效率的决定性作用，并证明了该机制与强化学习中可验证奖励下的最优策略扩展具有一致性。

Abstract: Training large language models (LLMs) with extended reasoning has enabled in-context search, in which models iteratively generate, critique, and revise solution attempts. We provide a theoretical analysis of in-context search by modeling it as approximate inference over reasoning traces, where the base model defines a prior and self-reflection provides feedback for posterior updates, and study the resulting inference-time sampling complexity - the number of sequential attempts needed to achieve high success probability. We show that when reflections reliably localize early mistakes, in-context search can yield exponential improvements over the base model, solving problems with exponentially small zero-shot pass rates using only a polynomial number of sequential attempts, whereas when this property fails, conditioning on past attempts offers no asymptotic benefit over parallel sampling. We further show that these gains are robust and learnable: approximate posterior updates suffice, and cross-entropy training on search rollouts recovers the required behavior with polynomial sample complexity. Finally, we show that under a stagewise abstraction of reinforcement learning with verifiable rewards, the optimal policy extension implements the same posterior reweighting rule. We validate key qualitative predictions of the theory on real large reasoning models.

[64] From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents cs.AI | cs.CL | cs.MAPDF

Haipeng Ding, Yuexiang Xie, Zhewei Wei, Yaliang Li, Bolin Ding

TL;DR: 该论文提出了一种名为EvoSOP的框架，旨在解决现有LLM代理框架依赖静态原子工具集导致的推理开销大和失败率高的问题。通过将原子动作合成为可复用的标准操作程序（SOPs），并采用构建、合并、评估和修剪的生命周期迭代优化工具集，使代理能够自我进化。实验表明，EvoSOP显著提高了任务成功率并减少了交互轮次。

Details

Motivation: 现有LLM代理框架主要依赖由细粒度原子动作（如基本文件I/O或单轮搜索）组成的静态工具集，这迫使代理为每个重复工作流重新发明低级逻辑，导致推理开销增加和失败率上升。

Result: 大量实验表明，与基线相比，EvoSOP显著提高了任务成功率，同时大幅减少了交互轮次。分析还揭示迭代工具优化促进了可靠高效的工具使用模式。

Insight: 创新点在于提出代理可以通过合成原子动作为可复用的SOPs来实现自我进化，并引入EvoSOP框架进行迭代工具优化，为自进化代理的发展提供了可扩展路径。从客观角度看，该方法将低级逻辑封装为高阶工具，减少了重复工作，提升了效率。

Abstract: Tool utilization enables Large Language Model (LLM) agents to interact with the real world and resolve complex tasks. However, existing agent frameworks predominantly rely on static toolsets composed of granular atomic actions (e.g., basic file I/O or single-turn search), which forces agents to reinvent low-level logic for every recurring workflow, leading to increased reasoning overhead and failure rates. In this study, we propose that agents can achieve self-evolution by synthesizing these atomic actions into reusable Standard Operating Procedures (SOPs), which function as callable higher-order tools that encapsulate multi-step logic. We further introduce EvoSOP, a framework that empowers agents to extract SOPs from execution trajectories and iteratively optimize the toolset through a systematic lifecycle of construction, merging, evaluation, and pruning. Extensive experiments demonstrate that EvoSOP significantly boosts task success rates while substantially reducing the number of interaction rounds compared to baselines. Our analysis also reveals that iterative tool optimization fosters reliable and efficient tool-use patterns, providing a scalable pathway for the development of self-evolving agents.

[65] RL Post-Training Builds Compositional Reasoning Strategies cs.AI | cs.CLPDF

Azwar Abdulsalam, Nishil Patel, Andrew Saxe

TL;DR: 本文研究强化学习（RL）后训练在Transformer模型中的作用，探讨其是仅放大基础模型的原始技能，还是能将原始技能组合成新的高层策略。通过在完全可观测的改写语法环境中进行实验，发现RL后训练能解决预训练模型难以解决的推理问题，并通过分阶段组合机制重组原始能力，形成可重用的稳定策略库。

Details

Motivation: 探究RL后训练的本质，即其是单纯增强基础模型中已有的原始技能，还是能够将这些技能组合成新的、更高层次的推理策略。

Result: 在仅提供二元最终答案奖励的基于轨迹的推理任务上，RL后训练解决了预训练模型即使在更大采样预算下也极少解决的保留问题，而拒绝微调方法则早期改善后陷入平台期。

Insight: RL通过分阶段组合机制（先强化原始归约，再发现有效的组合程序）重组原始能力，形成可重用且稳定的顺序组合与并行组合策略；其关键优势在于选择性探索，能集中探索有效的可重用结构，而非产生大量无效的捷径式改写；组合策略的出现不仅依赖于预训练对原始技能的暴露，更取决于预训练是否将原始能力组织成可供RL后续压缩的归约程序。

Abstract: Does RL post-training merely amplify primitive skills already latent in a base model, or can it compose primitive skills into new higher-level strategies? We study this question in a fully observable rewrite-grammar environment where the pretraining distribution is known and every generated rewrite can be audited. A Transformer is pretrained on primitive symbol-rewrite chains and post-trained on a Trace-based reasoning task with only a binary final-answer reward. RL solves held-out problems that remain rarely solved by the pretrained model even under much larger sampling budgets, while rejection fine-tuning improves early but plateaus. Trace analysis shows that RL reorganizes primitive competence through a phased compositional mechanism: it first strengthens primitive reductions, then discovers valid composed procedures. These include sequential compositions, which collapse ordered chains of primitive contractions, and parallel compositions, which combine independent primitive contractions in a single step. The composed procedures are not isolated samples; they are reused and consolidated into a stable repertoire. Comparing RL with rejection fine-tuning shows that the key difference is not exploration volume but selectivity: RFT produces many shortcut-like rewrites, much of them invalid, whereas RL concentrates exploration into valid reusable structure. Pretraining ablations show that the emergence of compositional strategies is gated not by primitive exposure alone, but by whether pretraining organizes primitive competence into reduction procedures that RL can later compress. The base model provides weak procedural ingredients; RL builds them into reliable higher-level strategies.

cs.LG [Back]

[66] Final Checkpoints Are Not Enough: Analyzing Latent Reasoning Faithfulness Along Training Trajectories cs.LG | cs.CLPDF

Hengyu Jin, Shu Yang, Di Wang

TL;DR: 本文研究了隐式推理方法在训练过程中的忠实性问题，发现其推理步骤对最终答案的因果贡献会随着训练而衰减，且这种衰减模式因答案格式（二元选择 vs. 开放式生成）而异。

Details

Motivation: 隐式推理方法在模型的连续隐状态中进行多步推理，但其不透明的隐状态引发了忠实性问题：这些隐式推理步骤是否真正因果驱动了最终答案。先前研究仅在收敛的检查点上考察了该问题，而本文旨在探究这些行为在训练过程中是如何形成的。

Result: 研究通过应用可验证的反事实编辑和噪声消融激活补丁发现，在输出层面，收敛的隐式推理方法在反事实编辑下可能表现出相似的虚假忠实性，但遵循不同的定性轨迹；在激活层面，隐式推理步骤对最终答案的因果贡献在训练过程中衰减，且输出翻转的示例正是贡献衰减的示例；此外，激活层面的轨迹因答案格式而异，在二元选择任务上衰减，在开放式解码任务上增强。

Insight: 创新点在于首次系统分析了隐式推理忠实性在训练轨迹上的动态演变，揭示了忠实性依赖于训练阶段和答案格式，挑战了仅关注最终检查点的评估范式。从客观角度看，该研究为理解和诊断隐式推理模型的内部工作机制提供了新的分析框架和工具。

Abstract: Latent reasoning methods perform multi-step inference entirely in the model’s continuous hidden states, promising more compact and efficient reasoning. However, these opaque hidden states raise a question of faithfulness: whether these latent reasoning steps causally drive the final answer. Prior work investigates this question at converged checkpoints and reports several unfaithful behaviors, such as latent reasoning steps that can be replaced without changing the answer, but leaves how these behaviors form during training unexamined. We instead track how faithfulness evolves across saved checkpoints for different latent reasoning paradigms, applying a verifiable counterfactual edit on the input and a noise-ablation activation patch on the latent reasoning steps. We find that (i) at the output level, latent reasoning methods can look similarly unfaithful at convergence under counterfactual edits while following qualitatively divergent trajectories; (ii) at the activation level, the causal contribution of latent reasoning steps to the final answer decays across training for both paradigms, with the examples that flip on the output side in (i) also being the examples on which this contribution decays; and (iii) the activation-level trajectory diverges by answer format, decaying on binary choice and rising on open-ended decoding. These findings highlight that latent reasoning faithfulness depends on training stage and answer format.

[67] Geometric Self-Distillation for Reasoning Generalization cs.LG | cs.CLPDF

Josip Jukić, Ivan Titov

TL;DR: 本文提出了一种名为GeoSD的几何自蒸馏方法，旨在解决特权上下文自蒸馏中因教师模型和学生模型预测分歧累积导致的分布外推理性能下降问题。该方法通过Hellinger损失和近端项来抑制学生模型预测的漂移，并在多个数学推理基准测试中验证了其有效性。

Details

Motivation: 特权上下文自蒸馏虽然能提供密集的教师监督，但教师模型基于特权信息（如提示或完整解题步骤）的自信预测可能与学生模型当前的理解不一致，这种分歧的累积会导致学生模型在分布外推理时性能下降。

Result: 在多个数学推理基准测试和三种模型系列（规模从1.7B到32B）上，GeoSD在保持分布内性能提升的同时，将分布外平均准确率比基础模型提高了5.7-8.6个百分点。

Insight: 创新点在于从预测分布的几何视角出发，设计了Hellinger损失（根据学生与教师的重叠程度缩放监督信号）和基于Fisher-Rao距离的近端项来共同约束预测漂移，并使用自然梯度更新，从而避免了标准匹配方法在分布外场景下因过度自信而错误收敛的问题。

Abstract: On-policy distillation is a practical post-training recipe for large language models, supplying dense teacher supervision on the student’s own trajectories. In privileged-context self-distillation, teacher and student are the same model conditioned on the same prefix, but the teacher also sees a hint or the full solution trace. This makes supervision abundant but harder to trust: the teacher can be confident about continuations its privileged view makes obvious but the student cannot yet justify. The distillation pull is strongest where teacher and student disagree most, and over many updates it accumulates into drift that degrades out-of-distribution (OOD) reasoning. We introduce GeoSD, a geometric self-distillation objective that treats this drift as movement in the student’s predictive behavior and counters it in two complementary ways. A Hellinger loss scales each teacher preference by the overlap the student already shares with it, attenuating the pull on tokens the student cannot yet support. Since these pulls still compound over training, a proximal term penalizes how far the student’s predictions drift from a recent checkpoint, measured as a Fisher-Rao distance. Both are distances in the same geometry of next-token distributions, and a natural-gradient update takes its steps in that geometry rather than in parameter space. Across mathematical reasoning benchmarks and three model families, GeoSD preserves the in-distribution gains of self-distillation while improving average OOD accuracy by 5.7-8.6 points over the base model, with gains holding across model scales from 1.7B to 32B. Analyzing why standard matching fails out of distribution, we find it wins agreement with the teacher by draining mass from alternatives at high-entropy states, resulting in confident agreement on wrong answers, whereas GeoSD keeps those alternatives in reach.

[68] Max Out GRPO Signal: Adaptive Trace Prefix Control for Hard Reasoning Problems cs.LG | cs.CLPDF

Vladislav Beliaev

TL;DR: 这篇论文提出了AdaPrefix-GRPO方法，用于解决Group Relative Policy Optimization (GRPO)在模型最难问题上梯度信号消失的问题。该方法通过在训练过程中动态调整参考解决方案前缀的长度，将问题难度控制在成功率约50%的水平，以最大化GRPO的梯度信号，并在训练后期完全撤除辅助，使部署模型能够独立解决问题。

Details

Motivation: GRPO在模型最难的问题上会停滞：当一组（group）中没有任何一次rollout成功时，组相对优势（group-relative advantages）会消失，导致该问题不产生梯度，从而浪费了最需要学习的前沿示例。

Result: 在困难的数学问题上，在匹配的训练FLOPs下，该方法将0.6B模型在训练分布外问题上的准确率提升至GRPO的2.1倍，在Qwen3-1.7B模型上为1.6倍，在AIME基准上为1.7倍，同时将轨迹长度大致减半。

Insight: 核心创新点是将解决方案前缀长度从一个静态设置转变为反馈控制器，动态调整以维持最优训练难度（成功率~50%），从而最大化GRPO的梯度信号。该方法仅需在数据准备阶段和损失掩码上实现，无需修改训练器本身，且模型越小，收益越大。

Abstract: Group Relative Policy Optimization (GRPO) stalls on a model’s hardest problems: when no rollout in a group succeeds, the group-relative advantages vanish and the problem contributes no gradient, wasting the frontier examples we most want to learn from. Prepending a correct prefix of a reference solution raises the success rate, making prefix length a continuous knob on difficulty. Concurrent methods set the knob once; AdaPrefix-GRPO turns it into a feedback controller: throughout training it adjusts how much of the solution each problem gets, holding its success rate near 50%, where GRPO’s gradient signal is largest, then withdraws the assistance entirely, so the deployed model solves problems unaided. On hard math, at matched training FLOPs, it more than doubles GRPO’s accuracy on held-out problems from the training distribution for a 0.6B model (2.1x), with 1.6x on Qwen3-1.7B and 1.7x on AIME, while roughly halving trace length. The method is implemented in data preparation plus a loss mask on prefix tokens; the trainer is otherwise stock. The smaller the model, the larger the gain.

[69] Agon: Competitive Cross-Model RL with Implicit Rival Grading of Reasoning cs.LG | cs.AI | cs.CLPDF

Vladislav Beliaev

TL;DR: 本文提出Agon方法，通过让两个竞争模型互为评分者来提升推理能力。两个模型交替扮演解题者和审阅者角色，在解决同一问题时相互竞争，奖励基于超越对方的表现，从而隐式评估推理过程。该方法在DeepMath难题集上使Qwen3模型的pass@1指标翻倍，并在代码生成任务和不同模型家族中验证了有效性。

Details

Motivation: 现有基于可验证奖励的强化学习（如GRPO）仅对最终答案评分，导致模型倾向于生成冗长内容而非提升推理质量，因为推理过程本身未被评估且缺乏优质思维标签。

Result: 在DeepMath难题集上使用Qwen3模型时，Agon方法使GRPO的pass@1指标翻倍，增益约为未经训练的Mixture-of-Agents方法的八倍；该排序结果在竞争性编程代码生成任务和不同模型家族（如Qwen3.5、Gemma 4）中均得到复现。

Insight: 创新点在于引入竞争性交叉模型强化学习框架，通过模型间隐式对抗评分替代显式过程标签或奖励模型，使推理能力在训练中被动态评估；双模型优化机制提供渐进强化的对手，克服了单模型RL的局限性，且仅需模型实力相当和行为差异即可生效。

Abstract: Reinforcement learning from verifiable rewards (e.g. GRPO) is the engine behind today’s reasoning models, yet it grades only the final answer. On hard problems this trains models to write more rather than to think better, since the trace itself is never graded and no label for good thinking exists. We introduce Agon, which makes two competing models each other’s graders. Both attempt the same problem; in alternating roles, one drafts a solution and the other reads it while solving, and each is rewarded for out-solving the other. To win, a model must out-reason a rival that has seen its work, so reasoning is judged implicitly during training, with no process labels and no reward model. Because both models are optimized, each faces a progressively stronger rival, which single-model RL cannot provide. The two need only be comparably strong and behaviorally different. At inference the pair deploys as it trains, a two-stage cascade in which one model drafts and the other answers after reading the draft. On the hard split of DeepMath with Qwen3, this doubles GRPO’s pass@1, roughly eight times the gain of an untrained Mixture-of-Agents pass over the same base. The ordering replicates on competitive-programming code and across model families (Qwen3.5, Gemma 4). For now the models talk in text; the next step is to let them reason together in latent space.

[70] LEMUR 2: Unlocking Neural Network Diversity for AI cs.LG | cs.CVPDF

Tolgay Atinc Uzun, Waleed Khalid, Saif U Din, Sai Revanth Mulukuledu, Akashdeep Singh

TL;DR: LEMUR 2 是一个大规模、可扩展的框架，旨在通过统一生成、评估和部署流程来解锁神经网络的多样性。它包含超过14,000种不同的架构和750,000条结构化训练记录，覆盖多模态任务和跨平台部署评估，为数据驱动的AI设计和LLM驱动的AutoML提供了新的基础。

Details

Motivation: 解决现有NAS基准（如NAS-Bench、NATS-Bench）覆盖范围狭窄、缺乏跨领域或部署感知评估的问题，旨在探索更广泛的架构设计空间。

Result: 该框架生成了大量架构和性能记录，并利用NN-VR和NN-Lite管道在异构移动和VR平台上进行自动化部署和延迟基准测试，提供了真实设备的性能元数据。

Insight: 创新点在于整合了基于AST的代码突变、遗传与强化学习进化、分形架构生成以及LLM引导的合成等多种生成方法，特别是引入了检索增强系统NN-RAG来利用公开代码库中的架构模式，并支持跨领域架构可转移性分析和多平台部署验证。

Abstract: Existing NAS benchmarks (e.g., NAS-Bench, NATS-Bench) cover only narrow, task-specific regions of the architectural design space and lack cross-domain or deployment-aware evaluation. LEMUR 2 introduces a large-scale, extensible framework unifying generative, evaluative, and deployment pipelines to unlock neural-network diversity. It comprises over 14,000 distinct architectures and more than 750,000 structured training records documenting model performance, hyperparameters, and task outcomes. These models were produced through AST-based code mutation, genetic and reinforcement-learning evolution, generation of fractal architectures, and synthesis guided by a Large Language Model (LLM). This includes deep models generated with the retrieval-augmented system NN-RAG, which derived and used architectural motifs from over 900 PyTorch modules extracted from public repositories. LEMUR 2 further employs NN-VR and NN-Lite pipelines for automated deployment and latency benchmarking on heterogeneous mobile and Unity-based VR platforms, providing real-device performance metadata. It spans multimodal tasks, image captioning, text-to-image synthesis, and language modeling, supporting cross-domain analysis of architectural transferability. By linking diverse architectures, tasks, and deployment data, LEMUR 2 provides the data foundation for LLM fine-tuning and coupling diverse architectural origins with large-scale, cross-platform empirical validation. This dataset defines a new basis for reproducible and data-driven AI design, advancing the emerging paradigm of LLM-driven AutoML and architectural generalization across modalities and hardware.

[71] Latent graph encoding of multimodal neuroimaging features with generative AI architectures cs.LG | cs.AI | cs.CVPDF

Ishaan Batta, Meenu Ajith, Vince Calhoun

TL;DR: 本文提出了一种多模态生成框架，用于处理结构性和功能性磁共振成像（MRI）特征，通过系统评估编码策略、潜在多模态融合和生成模型选择，旨在优化神经影像数据的编码与生成。研究比较了变分自编码器（VAE）、Transformer、生成对抗网络（GAN）和扩散模型等架构，发现采用模态感知图编码将功能连接性映射到低维潜在空间的方法优于向量化编码器或直接数据空间方法。最终提出的多模态图VAE（gMMVAE）在生成保真度、重建质量、效率和潜在空间可区分性等多个指标上超越其他生成变体，展示了其在稳健多模态神经影像分析中的潜力。

Details

Motivation: 解决神经影像数据（如结构性和功能性MRI）的复杂编码问题，通过开发最优的架构框架，以研究大脑的结构和功能特性，并优化潜在空间处理。

Result: 在大型神经影像数据集上，使用结构灰质体积（GMV）和静态功能网络连接性（sFNC）特征进行评估，提出的gMMVAE在生成保真度、重建质量、效率和潜在空间可区分性等多个指标上超越其他生成模型变体（如VAE、Transformer、GAN和扩散模型），达到先进水平。

Insight: 创新点在于引入模态感知图编码，将功能连接性有效压缩到低维潜在空间，这优于传统向量化方法；从客观角度看，该框架通过系统整合多模态数据和生成模型，为神经影像分析提供了更稳健和高效的解决方案。

Abstract: While generative models enable encoding of complex neuroimaging data for feature generation and reconstruction, developing optimal architectural frameworks with appropriate encoding and latent space processes is crucial for studying structural and functional properties of the brain. We design a multimodal generative framework for structural and functional magnetic resonance imaging (MRI) features through systematic evaluation of encoding strategies, latent multimodal fusion, and generative model selection. Using structural gray matter volume (GMV) and static functional network connectivity (sFNC) features from a large neuroimaging dataset, we analyze generative frameworks involving variational autoencoders (VAEs), transformers, generative adversarial networks (GANs), and diffusion models. Architectures that employ modality-aware graph encoding of functional connectivity into a lower-dimensional latent space outperform vectorized encoders or direct data space approaches. The proposed multimodal graph VAE (gMMVAE) surpasses alternative generative variants across multiple metrics for generation fidelity, reconstruction quality, efficiency, and latent space discriminability, highlighting its potential for robust multimodal neuroimaging analysis.

[72] Selective Timestep Weighting and Advantage-Based Replay for Sample-Efficient Diffusion RLHF cs.LG | cs.AI | cs.CVPDF

Eric Zhu, Abhinav Shrivastava, Soumik Mukhopadhyay

TL;DR: 本文针对扩散模型强化学习人类反馈（Diffusion RLHF）中反馈效率低的问题，提出了两种互补策略：选择性时间步加权和基于优势的回放机制。通过强调去噪过程中信息丰富的时间步和轨迹，该方法在保持对未见提示泛化能力的同时，显著提升了反馈效率。

Details

Motivation: 现有扩散模型RLHF方法需要大量人类或奖励模型评估，反馈效率低下，这在实际应用中成为主要瓶颈。论文旨在解决扩散RLHF中反馈效率低的问题，使其在反馈是主要限制的现实场景中更具实用性。

Result: 在相同超参数设置下，该方法相比广泛使用的扩散RLHF基线，实现了高达6倍的样本效率提升。

Insight: 核心创新点在于认识到扩散轨迹中的奖励信息分布不均，并非所有去噪时间步或轨迹对从奖励信号中学习的贡献相等。因此，提出了选择性时间步加权（从理论上与PPO的最优收敛特性关联）和基于优势的轨迹回放机制，以重用过去样本而非重复查询新奖励，从而更有效地利用梯度更新。

Abstract: Reinforcement learning from human feedback (RLHF) has emerged as a powerful paradigm for aligning generative models with human preferences. However, applying RLHF to diffusion models remains highly feedback inefficient, as existing approaches typically require large amounts of human or reward model evaluations. This limitation reduces the practicality of diffusion RLHF in realworld settings where feedback is the primary bottleneck. In this paper, we propose two complementary strategies that substantially improve the feedback efficiency of diffusion RLHF while preserving generalization to unseen prompts. Our key observation is that reward information in diffusion trajectories is unevenly distributed: not all denoising timesteps or trajectories contribute equally to learning from a reward signal. By emphasizing informative timesteps and trajectories during optimization, we obtain more effective gradient updates. First, we introduce a per-timestep weighting scheme that reweights denoising steps during policy optimization. We theoretically connect this weighting to the optimal convergence properties of proximal policy optimization (PPO) and approximate the resulting weighting trend empirically. Second, we introduce a replay mechanism that prioritizes informative trajectories, enabling the model to reuse past samples instead of repeatedly querying new rewards. Together, these strategies significantly improve the feedback efficiency of diffusion RLHF. Under identical hyperparameter settings, our approach achieves up to a 6$\times$ improvement in sample efficiency compared to widely used diffusion RLHF baselines.

Table of Contents

cs.CL [Back]

[1] Ad Headline Generation using Self-Critical Masked Language Model cs.CL | cs.AI | cs.LGPDF

[2] MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning cs.CL | cs.LGPDF

[3] Comprehensive Evaluation of Large Language Model Responses: A Multi-Factor Scoring System cs.CL | cs.AIPDF

[4] Audio Sentiment Analysis via Distillation and Cross-Modal Integration of Generated Multilingual Transcripts cs.CL | cs.AI | cs.LG | cs.SDPDF

[5] Evaluation of Multilingual Ability to Use Spatial Deictic Expressions in Vision-Language Models cs.CLPDF

[6] R^3: Advertisement Compliance Rectification via Group-Relative Experience Extractor and Curriculum Reinforcement cs.CL | cs.LGPDF

[7] Accurate, Interdisciplinary and Transparent Structure-property Understanding with Deep Native Structural Reasoning cs.CL | cs.AI | cs.CE | cs.LGPDF

cs.CV [Back]

[8] MiLSD: A Micro Line-Segment Detector for Resource-Constrained Devices cs.CV | cs.AI | cs.ROPDF

[9] Overview of the NLPCC 2026 Shared Task 1: Difficulty-Aware Multilingual and Multimodal Medical Instructional Video Understanding Evaluation cs.CV | cs.AIPDF

[10] SpaR3D-MoE: Adaptive 3D Spatial Reasoning from Sparse Views Meets Geometry-Inductive Mixture-of-Experts cs.CV | cs.AIPDF

[11] ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities cs.CV | cs.AIPDF

[12] Dynamic-in-Few-Step: Unifying Dynamic Computation and Few-Step Distillation for Efficient Video Generation cs.CV | cs.AI | cs.LGPDF

[13] CoMind: Understanding Collaborative Human Activity from Multiple Minds and Views cs.CVPDF

[14] Hardware-aware Graph Neural Networks prunning for embedded event-based vision cs.CVPDF

[15] A Good Initialization is All You Need for Faithful Visual Attribution cs.CVPDF

[16] WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligence cs.CVPDF

[17] URS-Stereo: Uncertainty-Guided Residual Search for Real-Time Stereo Matching cs.CVPDF

[18] Gen4U: Unifying Video Generation and Understanding via Diffusion cs.CV | cs.LGPDF

[19] Ensemble Deep Learning Approaches for AI-Altered Video Detection cs.CVPDF

[20] Geometric Collapse: When Vision Models Fail to Verify Physical Causality cs.CVPDF

[21] Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild cs.CV | cs.LGPDF

[22] LoCA: Spatially-Aware Low-Rank Convolutional Adaptation of Vision Foundation Models cs.CV | cs.AI | cs.LGPDF

[23] Seeing What Matters: Lesion-Aware High-Resolution Patch Discovery and Fusion for Chest X-ray Report Generation cs.CVPDF

[24] Compass: Prostate Cancer Detection Needs Multi-View Context cs.CV | cs.LGPDF

[25] General Incomplete Multimodal Learning via Dynamic Quality Perception cs.CVPDF

[26] Ego-Human Motion Prediction with 3D-Aware LLM cs.CVPDF

[27] Self-Supervised Pretraining Improves Cross-Site and Cross-Scale Robustness of Point Cloud Leaf-Wood Segmentation cs.CV | cs.AIPDF

[28] SHTA: Semantic Hard Token Correction and Center Alignment for Semi-Supervised Medical Image Segmentation cs.CVPDF

[29] AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning cs.CV | cs.AIPDF

[30] AT-Attn: Temporal-Aware Cross-Attention for Longitudinal Multimodal Alzheimer’s Disease Diagnosis cs.CV | cs.AIPDF

[31] Tree-of-Thoughts Reasoning for Text-to-Image In-Context Learning cs.CV | cs.AIPDF

[32] Video-Based Detection of squint and cataract for accessibility-aware adaptive web interface rendering cs.CV | cs.HCPDF

[33] ColorFM: An Optimization-to-Learning Framework for Color Transfer via Flow Matching cs.CVPDF

[34] Widest-Path Reachability Fields for Connectivity-Preserving Slender Structure Segmentation cs.CV | eess.SYPDF

[35] Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering cs.CV | cs.LGPDF

[36] NoDrift3R: Raymap-Guided Coupling for Drift-Robust Unposed Feed-Forward 3D Reconstruction cs.CVPDF

[37] Prototype-Anchored Generalized Manifold Regression for Unknown-Domain Object Detection cs.CVPDF

[38] [`Attention-Guided Cross-Temporal Clustering for Self-Supervised Video Object Segmentation](https://arxiv.org/abs/2607.07230) cs.CVPDF

[39] DiffCVE: Diffusion-based Compressed Video Enhancement cs.CVPDF

[40] Vision Foundation Models in Radiology: A Scoping Review of Data, Methodology, Evaluation and Clinical Translation cs.CV | cs.AI | cs.LGPDF

[41] An Edge-aware Prompt-enhanced SAM for Ultrasound Image Segmentation cs.CVPDF

[42] Unraveling Machine Behavior by Multi-Level Bias Analysis and Detection: Methodology and Application to Computer Vision cs.CVPDF

[43] InfraQR: Edge-Placed QR-Inspired Structured Patch Attacks on Infrared Vision-Language Models cs.CVPDF

[44] CarbonCLIP: Enhance Carbon Prediction from Satellite Imagery via Integrated Street-View Semantics and Temporal Context Training cs.CV | cs.AIPDF

[45] SoccerNet 2026 Challenges Results cs.CVPDF

[46] HAJJv2-CrowdCount: Zero-Shot Benchmark for Dense Crowd Counting cs.CV | cs.AIPDF

[47] BUS: Brain-Inspired Unsupervised Self-Reflection for Advanced Multimodal Reasoning cs.CVPDF

[48] MMAgent-R$^2$: Learning to Rerank and Reject for Agentic mRAG cs.CVPDF

[49] When Prompts Ignore Structure: Graph-Based Attribute Reasoning for Calibrated VLMs cs.CV | cs.AI | cs.LGPDF

[50] Heterogeneity-Adaptive Diffusion Schrodinger Bridge for PET-Guided Whole-Body MRI Translation cs.CV | cs.AI | cs.LGPDF

[51] Two-Stage Multi-Modal Fusion with Adaptive Alignment for Action Quality Assessment cs.CVPDF

[52] HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models cs.CV | cs.AIPDF

[53] Learning to Unify Deformable Shape and Texture Representations for Cardiac Video Classification cs.CVPDF

[54] Infinite Worlds with Versatile Interactions cs.CVPDF

[55] AA-ViT: Anatomically Aware Vision Transformer with Structural and Frequency Guidance for Contrast Enhanced Brain MRI Synthesis cs.CVPDF

[56] Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence cs.CVPDF

[57] MedPMC: A Systematic Framework for Scaling High-Fidelity Medical Multimodal Data for Foundation Models cs.CV | cs.LGPDF

cs.RO [Back]

[58] Multimodal Voice Activity Projection for Turn-Taking in Social Robots with Voice-Activity-Related Pretrained Encoders cs.RO | cs.AI | cs.CLPDF

[59] Dynamic Object Detection and Tracking in Construction: A Fisheye Camera and LiDAR Sensor Fusion Model cs.RO | cs.CVPDF

[60] SonoRank: Towards Calibration-Free Real-Time Finger Flexion Detection from Forearm Ultrasound Sequences cs.RO | cs.CVPDF

[61] EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI cs.RO | cs.CVPDF

[62] Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation cs.RO | cs.CVPDF

cs.AI [Back]

[63] When Does In-Context Search Help? A Sampling-Complexity Theory of Reflection-Driven Reasoning cs.AI | cs.CLPDF

[64] From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents cs.AI | cs.CL | cs.MAPDF

[65] RL Post-Training Builds Compositional Reasoning Strategies cs.AI | cs.CLPDF

cs.LG [Back]

[66] Final Checkpoints Are Not Enough: Analyzing Latent Reasoning Faithfulness Along Training Trajectories cs.LG | cs.CLPDF

[67] Geometric Self-Distillation for Reasoning Generalization cs.LG | cs.CLPDF

[68] Max Out GRPO Signal: Adaptive Trace Prefix Control for Hard Reasoning Problems cs.LG | cs.CLPDF

[69] Agon: Competitive Cross-Model RL with Implicit Rival Grading of Reasoning cs.LG | cs.AI | cs.CLPDF

[70] LEMUR 2: Unlocking Neural Network Diversity for AI cs.LG | cs.CVPDF

[71] Latent graph encoding of multimodal neuroimaging features with generative AI architectures cs.LG | cs.AI | cs.CVPDF

[72] Selective Timestep Weighting and Advantage-Based Replay for Sample-Efficient Diffusion RLHF cs.LG | cs.AI | cs.CVPDF