cs.CV [Total: 46]
cs.CL [Total: 19]
cs.AI [Total: 2]
cs.LG [Total: 1]
cs.RO [Total: 5]
cs.SE [Total: 1]
eess.AS [Total: 1]
eess.IV [Total: 3]

cs.CV [Back]

[1] MemeTAG: Keyword-Driven Meme Classification through Tag Embedding Reconstruction cs.CV | cs.MMPDF

Akshit Sharma, Prashant W. Patil

TL;DR: MemeTAG 先用预训练 VLM 生成梗图关键词，再由注意力标签推断网络聚合成语义嵌入并以重建损失对齐视觉和文本特征。

Details

Motivation: 有害网络梗图具有细腻多模态语义，自动分类困难；若能显式使用图文描述性关键词，可为模型提供高层语义线索。

Result: MemeTAG 包含双目标机制：预训练视觉语言模型生成描述关键词，聚合标签推断网络 ATIN 再以注意力将其蒸馏为单一丰富语义嵌入，并通过辅助重建损失迫使视觉和文本表征深度对齐。配合三阶段训练策略，该方法在 HarMeme、Hateful Memes Challenge 和 PrideMM 上达到新的先进结果，超过既有方法。

Insight: 关键词能够把梗图中的隐含高层语义压缩成可监督的辅助目标；将其用于表示重建而非只拼接输入，有助于形成图文对齐特征。

Abstract: The proliferation of harmful internet memes poses a significant societal threat, yet their automated classification remains a formidable algorithmic challenge due to the nuanced, multimodal nature of their content. To address this, we introduce MemeTAG, a novel dual-objective framework that pioneers a keyword-aware approach to meme classification. Our core innovation is a two-part semantic guidance mechanism: first, we leverage a pretrained Vision-Language Model to generate a set of descriptive keywords, that capture the high-level semantics. Second, we introduce the Aggregated Tag Inference Network (ATIN), an attention-based module that distills these keywords into a single, rich semantic embedding. This embedding serves as a target for a novel auxiliary reconstruction loss, which compels the model to learn deeply aligned visual and textual features. This approach, combined with an efficient three-stage training strategy, establishes a new state-of-the-art on the HarMeme, Hateful Memes Challenge (HMC), and PrideMM datasets, decisively outperforming existing state-of-the-art methods.

[2] TAPe+ML: A Compact Structured Representation for Multi-Task Computer Vision cs.CV | eess.IVPDF

Sergey Kurinov, Alexey Upatov

TL;DR: TAPe+ML v3 先用结构化主动感知表征编码感知元素关系，再以紧凑模块完成分类、检测和实例分割，报告参数量低于十万。

Details

Motivation: 多数视觉系统直接处理像素张量，导致模型参数、数据和计算负担较大。作者探索将部分建模负担转移到识别前显式结构化的输入表示，以构造紧凑多任务系统。

Result: TAPe+ML v3 基于主动感知理论 TAPe，在识别前编码感知元素间关系，并以共享表征和模块化架构支持分类、目标检测及实例分割。系统组合背景与轮廓处理、局部定位、原型分类和专用子模型协调器，参数少于 10 万。在 COCO 检测上 mAP50 为 84.7、mAP50-95 为 65.3；实例分割 mask mAP50 为 80.7、mask mAP50-95 为 58.4。Imagenette 同训练条件下分类验证准确率 92%，ImageNet-Real top-1 为 89.9%；另测试视频场景检测和工业分布变化适配。

Insight: 紧凑模型可通过在输入表示层先组织感知关系来减少对参数规模的依赖；论文同时报告多任务结果和分布变化测试，探索结构化表征的效率收益。

Abstract: We present TAPe+ML v3, a compact computer vision system based on TAPe (Theory of Active Perception), a structured representation that encodes relations among perceptual elements before recognition. Instead of operating directly on pixel tensors, the system uses a shared TAPe representation and a modular recognition architecture for image classification, object detection, and instance segmentation. TAPe+ML v3 combines background and contour processing, local object localization, prototype-based classification, and a coordinator for specialized submodels. Across the reported experiments, it uses fewer than 100,000 parameters. On COCO object detection, it obtains 84.7 mAP50 and 65.3 mAP50-95. On COCO instance segmentation, it obtains 80.7 mask mAP50 and 58.4 mask mAP50-95. In classification experiments, it reaches 92 percent validation accuracy on Imagenette under an identical-training comparison with a raw-pixel baseline, and 89.9 percent Top-1 accuracy on ImageNet-Real. We also evaluate compactness in video scene detection and adaptation under distribution shift in an industrial pilot. The results suggest that shifting part of the modeling burden from network parameters to a structured input representation can support compact multi-task vision systems with reduced data, memory, and compute requirements.

Haitao Huang, Shenghao Zhao, Boyuan Tian, Shin-Fang Chng, Songlin Yang

TL;DR: 4DGS-Fixer 用多视角深度构建更完整的动态高斯初始化，再以视频扩散修复新视角序列作为伪监督，迭代改善稀疏视角动态场景重建。

Details

Motivation: 稀疏视角视频中的 4D Gaussian Splatting 存在观测不足导致的欠定问题；COLMAP 从少量输入视角得到的点云稀疏不完整，使场景大片区域缺乏高斯支持，后续优化难以恢复。

Result: 方法先估计多视角深度并融合成稠密点云，为动态 4DGS 提供更完整几何初始化；随后用预训练视频修复模型处理沿新相机轨迹、不同时间渲染的视频序列，将修复结果作为伪监督正则化并迭代精炼表示。在常用基准上，方法超过既有基线，较此前最佳结果 PSNR 提升近 2 dB。

Insight: 视频扩散先验可补足稀疏观察遗漏的场景内容，并通过新视角渲染序列对动态表示提供跨帧一致性伪监督。

Abstract: This paper addresses the challenges of dynamic scene synthesis from sparse-view videos. Existing methods employ geometric priors, adaptive optimization, or density-control strategies to improve 4D Gaussian modeling under sparse observations. However, they cannot fundamentally resolve the ill-posed problem caused by insufficient observations and missing scene information. Moreover, sparse-view 4D Gaussian Splatting (4DGS) often suffers from poor geometric initialization: with only a few input views, COLMAP typically reconstructs sparse and incomplete point clouds, leaving large scene regions without sufficient Gaussian support and making them difficult to recover through subsequent optimization. To address these limitations, we propose a novel iterative refinement framework based on a video diffusion model to improve the completeness and consistency of dynamic 4D scenes. Specifically, we first estimate multi-view depth maps and fuse them into dense point clouds to provide more complete geometric initialization for a dynamic 4DGS representation. We then employ a pretrained video restoration model to refine sequences rendered along novel camera trajectories at different time steps. The restored sequences serve as pseudo-supervision to regularize and iteratively refine the 4DGS representation. Experiments on a widely used benchmark dataset demonstrate that our method substantially outperforms existing baselines, achieving nearly a 2 dB PSNR improvement over the previous best-performing method.

[4] MarsFM: Shading-Regularized Flow Matching for Martian Relief Estimation cs.CVPDF

Marius F. R. Juston

TL;DR: MarsFM 将生成式潜在流匹配先验、立体几何监督和可微 Lunar–Lambert 阴影约束结合，从单波段 HiRISE 图像估计局部火星地形起伏。

Details

Motivation: 仅凭单幅火星轨道影像估计地形起伏存在歧义。作者希望结合生成式先验和图像成像约束改善地形估计，同时评估低步数推理在当前验证协议下的稳定性。

Result: MarsFM 是图像条件潜在流匹配模型，使用立体数据导出的几何监督和可微阴影目标；起伏、法线、梯度、曲率及序关系项约束地形结构，正仿射不变图像比较约束渲染外观。按每种积分步数收集 2024 条 patch 记录评估，1–20 个 Euler 步的归一化符号对数起伏空间配准 RMSE 为 0.0935–0.0957，衡量与 VAE 重建参考在正值参考支持域上的一致性。结果范围较窄支持该协议下少步推理，但空间、微分和频谱诊断仍显示平滑、幅度压缩及边界不匹配。

Insight: 生成式地形先验可与阴影和立体几何约束结合，但现有结果只证明对重建参考的协议内一致性；要证明物理地形分辨率提升，还需匹配基线和独立高分辨率真值。

Abstract: We present MarsFM, an image-conditioned latent flow-matching model for local Martian relief estimation from single-band HiRISE RED orthoimagery. The method combines a pretrained generative prior with stereo-derived geometric supervision and a differentiable Lunar–Lambert shading objective. Relief, normal, gradient, curvature, and ordinal terms constrain complementary aspects of terrain structure, while a positive-affine-invariant image comparison constrains rendered appearance. An evaluation comprising 2024 gathered patch records per integration-step count yields mean affine-aligned RMSE between 0.0935 and 0.0957 in normalized signed-log relief space for one to twenty Euler steps. These scores measure agreement with VAE-reconstructed references on positive-reference support. Their narrow range supports low-step inference under this protocol. Spatial, differential, and spectral diagnostics show broad terrain correspondence alongside smoothing, amplitude compression, and boundary mismatch. MarsFM provides a framework for combining learned terrain priors with image-based constraints; establishing improved physical terrain resolution requires matched baselines and independent high-resolution reference data. Data: https://huggingface.co/datasets/SuperComputer/mars_hirise_dtm_processed-6aa9b66ba461e07f; code: https://github.com/Marius-Juston/MarsRecon.

[5] Fragment-Aware Vision Transformers for Fresco-Fragment Style Classification cs.CV | cs.LGPDF

Sara Miketek, Biagio Barchielli, Nadeem Iqbal Kajla, Sinem Aslan

TL;DR: 该研究为壁画碎片风格分类逐步加入前景引导遮罩、几何正则化和分布级监督式对比目标，以适应考古图像不完整、形状不规则的视觉证据。

Details

Motivation: 艺术风格分类常基于完整作品，模型可使用全局构图和图像结构；考古发现常只有残片，视觉证据不完整、形状不规则且上下文有限，需要研究碎片感知方法。

Result: 作者从 ViT-B/16 基线出发，加入抑制背景 token 的前景引导遮罩、通过修复对齐不规则碎片边界与 ViT patch 网格的几何正则，以及基于预测分布 KL 相似性的监督式对比目标。各分支经简单可学习 logits 集成。在 CLEOPATRA 上，集成将准确率从 0.604 提升至 0.656、宏 F1 从 0.596 提至 0.648；在 POMPAAF 六种碎片条件中的四种超过最佳单分支。更复杂图融合在 POMPAAF 与简单集成持平，在 CLEOPATRA 仅有小幅数据集特定提升，未抵消复杂度。可解释性分析确认模型利用真实绘画证据，同时量化修复分支会从合成周边区域取得部分归因。

Insight: 碎片化输入需要同时处理可见前景、规则 patch 对齐和跨样本风格分布；研究还提醒，修复生成的周边像素可能参与模型决策，解释分析应区分真实与合成证据。

Abstract: Artistic style classification is usually studied on complete artworks, where models can exploit global composition, spatial organisation, and iconographic structure. In archaeological settings, however, artworks often survive only as fragmented remains, forcing recognition from incomplete, irregular, and context-limited visual evidence. We study fresco-fragment style classification using a progressive transformer-based framework. Starting from a ViT-B/16 baseline, we introduce foreground-guided masking to suppress background-only tokens, inpainting-based geometric regularisation to align irregular fragment supports with the ViT patch grid, and a supervised contrastive objective that operates on predictive distributions through a Kullback-Leibler similarity and consistently improves every branch. We combine the branches with a deliberately simple learnable logit ensemble. Experiments on CLEOPATRA and POMPAAF show that fragment-aware modelling improves over the standard ViT baseline, with the ensemble increasing accuracy from 0.604 to 0.656 and macro-F1 from 0.596 to 0.648 on CLEOPATRA, and outperforming the best single branch in four of six fragmentation settings on POMPAAF. We additionally evaluate a more complex graph-fusion variant and find that it matches the simple ensemble on POMPAAF while offering only a small, dataset-specific gain on CLEOPATRA, which does not justify its added complexity. Beyond these empirical gains, our contribution is twofold: a distribution-level contrastive objective that consistently sharpens single-branch recognition, and an interpretability analysis that verifies the models exploit genuine painted evidence, while quantifying that the inpainting-based branch draws part of its attribution from the synthesised surround.

[6] VGGT-CAD: Reconstructing Parametric CAD 3D Model with Geometric Grounding cs.CVPDF

Chunan Yu, Tianrun Chen, Fu Shen, Cheng Chen, Lanyun Zhu

TL;DR: VGGT-CAD 将预训练三维几何先验迁移到参数化 CAD 重建，以相机条件 token、可变视角聚合和免训练选帧预测可编辑建模命令。

Details

Motivation: 参数化 CAD 重建既需恢复精确几何，又要还原可编辑建模操作；视角有限且有歧义时尤为困难。现有方法多依赖二维外观线索，缺少强多视角几何先验。

Result: VGGT-CAD 从单视图或多视图观察重建参数化 CAD。相机参数被编码为条件 token，与图像 token 联合建模；可变视角跨视图上下文聚合模块适应不同视图数量；推理时的免训练几何感知选视图策略选择互补且可靠帧。表示最终由非自回归解码器输出 CAD 命令序列。作者还基于已有 CAD 数据多视角重渲染构建 VideoCAD，并在不同观察配置下的实验验证方法有效性。

Insight: 参数化建模除了外观相似，还需要显式几何约束和可执行的操作序列；视角筛选能从有限观察中尽量保留互补几何证据。

Abstract: Parametric CAD reconstruction requires recovering both precise geometry and editable modeling operations from visual observations, making it challenging under limited and ambiguous views. Existing methods mainly rely on 2D appearance cues and lack strong multi-view geometric priors. In this work, we present VGGT-CAD, a geometry-aware framework for parametric CAD reconstruction from single- and multi-view observations. We transfer pretrained 3D geometric priors into CAD reconstruction by encoding camera parameters as condition tokens and jointly modeling them with image tokens. To handle varying numbers of viewpoints, we introduce a variable-view cross-view context aggregation module that adaptively fuses multi-view features. We further develop a training-free geometry-aware view selection strategy to select complementary and reliable frames during inference. The resulting representation is decoded into CAD command sequences using a non-autoregressive decoder. We also develop VideoCAD, a large-scale multi-view video benchmark derived from existing CAD data through multi-view re-rendering. Extensive experiments demonstrate the effectiveness of VGGT-CAD for visual CAD reconstruction under different observation configurations.

[7] Multi-viewpoint Geo-localization with Event Cameras cs.CV | cs.ROPDF

Adam D. Hines, Michael Milford, Tobias Fischer

TL;DR: MegaEvent 将五个地理定位图像集转成合成事件流并微调事件视觉 Transformer，提升事件相机地点识别对视角变化的鲁棒性。

Details

Motivation: 机器人定位需要在视点变化时仍可靠，而事件相机因其时间分辨率受到关注，但现有事件定位器对视角变化研究不足，相关困难数据集也稀缺。

Result: 作者把五个用于帧式定位的带地理标签大规模数据集通过 Image-to-Event 转换生成合成事件流，并用多损失微调预训练事件视觉 Transformer，形成 MegaEvent。在三个已有事件定位数据集上的平均 Recall@1 为 82%，比次优事件方法高 20 个召回点，比直接用事件帧的帧式 VPR 高 8–26 点。新建 Springfield-Event-VPR 包含三种相机朝向的步行路线，总计 11.1 公里，MegaEvent 比最强基线高 9 个召回点。

Insight: 以合成事件数据扩展训练并显式覆盖相机视角变化，可以提升事件相机的地点表征；新基准也把跨视角鲁棒性作为定位核心条件。

Abstract: Robot localization is an ongoing challenge that demands mapping and positioning systems that are tolerant to viewpoint change. Event cameras are attracting increasing interest and adoption in robotics; however, dealing with viewpoint variance is an under-investigated problem in existing event-based localizers. In addition, event-based datasets that emphasize viewpoint variance for challenging localization situations are scarce. Here, we introduce an event-based visual place recognition (VPR) system that performs robustly under viewpoint changes. We converted five large-scale geo-tagged datasets, conventionally used to train frame-based localization systems, into synthetic event streams using Image-to-Event (I2E) conversion, and used them to fine-tune a pre-trained event-based vision transformer backbone with a multi-loss function, yielding a system we call MegaEvent that learns viewpoint-robust features for place recognition. We achieved an average Recall@1 of 82% across three existing event-based localization datasets, leading the next best event-based method by 20 recall points, and frame-based VPR models applied directly to event frames by 8 to 26 recall points. We introduce a new, challenging dataset - Springfield-Event-VPR - which features a 3.7km walking route recorded in three camera orientations for a total of 11.1km, which MegaEvent outperforms the strongest baseline by 9 recall points. The code for MegaEvent is available at https://github.com/AdamDHines/megaevent.

[8] Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing cs.CVPDF

Chongbo Zhao, Jiangming Wang, Xilai Wang, Xinyu Wang, Jingyi Tang

TL;DR: Edit-VAR 是免训练、免反演的自回归视频编辑方法，直接编码多尺度离散 token 并按概率替换编辑相关 token，同时通过晚尺度约束释放与剪枝提高质量和效率。

Details

Motivation: 文本视频编辑既需改变目标内容又要保留其他区域外观和时间连贯性。反演免方法容易约束过强、语义变化不完整；反演式方法恢复潜轨迹时的误差又可能造成源内容漂移和时序不一致。

Result: Edit-VAR 使用预训练视觉自回归视频模型，直接将源视频编码为多尺度离散 token，并以概率引导条件替换保持源内容。注意力引导、逐 token 和尺度感知调制，在编辑相关位置和生成阶段有选择地放松源约束；尺度解耦生成通过晚尺度释放约束重建运动一致细节并减少纹理碎片。残差引导 token 剪枝进一步利用最后两个高分辨率尺度冗余。实验和盲法用户研究显示，整体编辑保真、源内容保持、时间连贯和推理效率均超过既有免训练视频编辑方法。

Insight: 离散多尺度生成可把编辑控制落到 token 位置和生成阶段；只在后期尺度释放编辑约束，可在保留源视频结构的同时生成连贯细节。

Abstract: Text-guided video editing modifies target content while preserving the appearance and temporal coherence of unedited regions. Training-based approaches provide strong control but demand substantial data and computation. Training-free methods fall into inversion-free and inversion-based paradigms. Inversion-free approaches avoid trajectory recovery, but their source-preserving guidance can limit editing strength and leave semantic changes incomplete. Inversion-based approaches recover a latent trajectory before regeneration, where approximation errors can accumulate and cause source-content drift and temporal inconsistency. We introduce Edit-VAR, the first training-free and inversion-free framework for text-guided video editing with a pretrained visual autoregressive video model. Edit-VAR directly encodes the source video into multi-scale discrete tokens and performs probability-guided conditional token replacement for source preservation. Attention-guided token-wise and scale-aware modulation selectively relaxes source constraints over edit-relevant positions and generation stages. Scale-Decoupled Generation, implemented as late-scale constraint release, regenerates motion-consistent details and reduces texture fragmentation. Residual-guided token pruning further exploits redundancy at the final two high-resolution scales to reduce inference cost. Extensive experiments and a blind user study demonstrate that Edit-VAR outperforms existing training-free video editing methods overall in editing fidelity, source preservation, temporal coherence, and inference efficiency.

[9] Beyond Exact Match: Task-Aware GRPO for Cross-Domain PCBA Visual Question Answering cs.CV | cs.AIPDF

Jia Li, Li Dai, Peng Jia, Zhenzhen Hu, Chee Seng Chan

TL;DR: 该 PCBA 视觉问答系统用任务感知 GRPO 为选择题、计数题和格式分别设计奖励，并结合语义校正、自一致投票和多模型仲裁应对跨域检查。

Details

Motivation: 电路板装配检查需联合理解细粒度视觉线索、元件语义和制造标准知识。VLM 面临标准样本与真实产线图像的域差异，且问题输出形式混合选择和数值计数，单一精确匹配目标不够。

Result: 框架将标准、真实和辅助 PCB 数据统一为指令格式，并构建与视觉证据、问题语义、候选选项和真值对齐的验证推理轨迹。任务感知 GRPO 对选择题使用多成分语义奖励，对计数题使用距离感知奖励，并加入格式奖励。推理时结合答案选项语义一致性校正、自一致投票和多模型仲裁。在官方 PCBA Standard-to-Real Grand Challenge 榜单上总分为 83.24。

Insight: 不同题型需要不同误差结构的奖励：选择题需考虑语义匹配，计数则应按差距连续惩罚；再用多种推理校验提高跨域部署稳健性。

Abstract: In automated Printed Circuit Board Assembly (PCBA) inspection, standards-guided decisions require systems to jointly reason over fine-grained visual cues, component semantics, and manufacturing knowledge. Although large vision-language models (VLMs) provide a promising foundation, their deployment is hindered by the domain shift between standards-derived samples and real-world production-line imagery, together with heterogeneous output spaces spanning choice-based and numerical counting tasks. To address these challenges, we propose a multimodal reasoning framework for cross-domain PCBA visual question answering. The framework converts standards-derived, real-world, and auxiliary PCB-domain data into a unified instruction format and constructs verified reasoning traces aligned with visual evidence, question semantics, candidate options, and ground-truth answers. We further introduce Task-Aware Group Relative Policy Optimization (GRPO), which moves beyond exact-match supervision by integrating multi-component semantic rewards for choice-based questions, distance-aware rewards for counting questions, and an auxiliary format reward for valid outputs. During inference, answer-option semantic consistency correction, self-consistency voting, and multi-model arbitration are combined to improve prediction robustness. The proposed system achieves an Overall Score of 83.24 on the official PCBA Standard-to-Real Grand Challenge leaderboard, demonstrating the effectiveness of task-aware reward design and robust inference for cross-domain PCBA visual question answering.

[10] Combining Object Detection with Geometry-Aware Clustering to Distinguish Overlapping Plants in UAV Imagery cs.CVPDF

Ik Jae Lee, Hieu D. Nguyen, Mahbubur Meenar, Carlos Morrison Martinez, Cameron Connelly

TL;DR: 该方法在无人机 RGB 植株检测区域内用叶片/枝条几何聚类拆分重叠植株，无需深度传感器、像素分割或重新训练主检测器。

Details

Motivation: 密集田间冠层中的相邻植株常重叠并被检测为一个实例，降低按单株统计的可靠性；额外深度或像素级分割则增加系统需求。

Result: 方法先检测灌木级区域，再以其中叶片或枝条组件质心和径向交点构造两类几何特征，通过 K-means 或高斯混合模型判断区域含一株还是两株。密度过滤去除虚假径向交点，最后融合空间和方向几何信息。在茄子和番茄田间 UAV 图像上，茄子质心聚类 F1 为 0.89；番茄用质心-RIP 组合配 K-means 最好，准确率 0.80、精确率 1.00、F1 0.75，密度过滤明显改善 RIP 聚类。

Insight: 现有检测器的框和组件输出仍可通过几何关系二次解析，改善重叠实例计数；轻量后处理能补足深度学习检测而不增加新传感器或重训成本。

Abstract: Reliable plant-level information from unmanned aerial vehicle (UAV) imagery is important for automated crop monitoring. However, in dense crop canopies, adjacent plants frequently overlap and are detected as a single object, reducing the reliability of plant-level measurements. This study presents a geometry-aware post-detection framework for resolving overlapping plant instances using standard RGB UAV imagery. The framework combines object detection with geometric clustering of plant components. Leaves or branches detected within each bush-level region are represented using two complementary geometric features: component centroids and radial intersection points (RIPs) derived from detected plant structures. K-means and Gaussian mixture models determine whether a detected region contains a single plant or two overlapping plants. Density filtering suppresses spurious radial intersections, and a post-pipeline ensemble combines spatial and directional geometric information. The framework was evaluated using UAV imagery of eggplant and tomato crops under field conditions. Centroid-based clustering achieved an F1-score of 0.89 for eggplant, while the combined centroid-RIP approach achieved the best tomato performance, with an accuracy of 0.80, precision of 1.00, and F1-score of 0.75 using K-means. Density filtering substantially improved RIP-based clustering for tomato. The proposed approach provides a lightweight, modular engineering solution that can be integrated with existing RGB UAV monitoring pipelines without additional depth sensors, pixel-level segmentation, three-dimensional reconstruction, or retraining of the primary bush detector. The results demonstrate that geometric reasoning applied to existing detector outputs can complement deep-learning-based object detection and improve plant-level interpretation in dense agricultural canopies.

[11] S3VD: Semantic-Guidance Spatio-Temporal Scanning for Video Deraining cs.CVPDF

Kui Jiang, Yiang Chen, Yan Luo, Zhaocheng Yu, Junjun Jiang

TL;DR: S3VD 将 DINOv2 时间语义先验与时空扫描结合，通过双门控 Mamba 建模视频帧内及帧间关系，用于强降雨视频恢复。

Details

Motivation: 暴雨会破坏户外视频高频细节并引入运动模糊，影响后续视觉任务。Mamba 复杂度线性且擅长长程依赖，但对雨天低质量表征时难保二维语义并捕捉三维时空关联。

Result: S3VD 包含多尺度语义融合模块 MSSF，将 DINOv2 时间语义先验用于精细特征表示，以弥补 Mamba 一维展平导致的局部语义丢失；以及时空扫描融合模块 STSF，引入解耦门控 DG-Mamba，以两个独立门自适应调节输入片段中前后文，优化帧内与帧间关系建模。在视频去雨基准上达到先进表现，相较 Mamba 基线平均 PSNR 提高 0.84 dB。

Insight: 视频复原中的高效状态空间扫描需保留局部二维语义并分别控制时间前后文；引入预训练视觉语义和双向门控能补足一维展平特性。

Abstract: Heavy rainfall severely degrades outdoor videos by corrupting high-frequency details and introducing motion blur, critically undermining the reliability of visual tasks. Recently, State Space Models (SSMs), particularly Mamba, have emerged as efficient alternatives for vision tasks with their linear complexity and ability to model long-range dependencies. However, when confronted with the poor visual representations in rainy videos, Mamba still faces difficulties in preserving the integrity of 2D spatial semantics and modeling 3D spatio-temporal correlations. To break these limitations, we introduce S3VD, a Semantic-Guidance Spatio-Temporal Scanning framework for video deraining, featuring two key innovations: Multi-Scale Semantic Fusion (MSSF) Module and Spatio-Temporal Scanning Fusion (STSF) Module. The former integrates temporal semantic priors from DINOv2 to guide precise feature representation and counteract the loss of local semantic context inherent to Mamba’s 1D flatten operation, enhancing robustness against extreme degradation. The latter introduces a spatio-temporal scanning mechanism and devises a Decoupled-Gating Mamba (DG-Mamba) layer, which employs two independent gates to adaptively control preceding and subsequent contextual information within the input clip, optimizing intra-frame and inter-frame correlation modeling. Experiments on video deraining benchmarks demonstrate the superiority of S3VD, achieving state-of-the-art performance with an average 0.84 dB PSNR improvement over Mamba-based baselines.

Hongyi Lin, Yiyao Liu, Qi Kang, Heye Huang, Yang Liu

TL;DR: VeriFuse 将 VLM 限定为多车协同三维检测候选间的仲裁器，可选择、修正或拒绝候选，再由确定性约束确定最终几何。

Details

Motivation: 视觉语言模型具备场景语义判断能力，但直接让其回归三维框既不可靠又昂贵；若只让其选择单一来源的结果，又浪费其他车辆或路侧设备提供的证据。

Result: VeriFuse 为车辆端和路侧 proposal 分别生成来源条件几何候选，把原始检测、扰动检测及跨源假设统一成候选池。冻结 VLM 仅可执行三种受限操作：选择合格候选、在物体有证据但几何都不准时修正已有锚点，或拒绝只有路侧支持且不可靠的 proposal。DAIR-V2X 上协同 3D AP50/AP70 为 0.494/0.357；300ms 延迟下车辆侧 BEV AP50 相对下降仅 1.7%。

Insight: VLM 适合用语义推理消解跨来源候选歧义，而最终三维几何由确定性约束处理；将模型权限限制在明确仲裁动作中，有助于平衡准确性、成本和可靠性。

Abstract: Vision-language models (VLMs) have demonstrated strong scene understanding and semantic judgment across diverse tasks, but their appropriate role in cooperative perception remains unclear. Directly asking a VLM to regress 3D detections is unreliable and computationally expensive, whereas using it to select the output of a single source discards useful information from other agents. We introduce VeriFuse, a bounded arbitration framework for vehicle-infrastructure cooperative 3D detection. Each agent first produces detections independently. Around each vehicle and roadside proposal, VeriFuse generates source-conditioned geometric candidates and combines the original detections, their perturbations, and cross-source hypotheses into a unified candidate pool. A frozen VLM then chooses among three admissible actions: SELECT an adequate candidate; REFINE an existing anchor when an object is supported but all candidates are geometrically inadequate; or REJECT an unsupported infrastructure-only proposal. Experiments on the DAIR-V2X dataset show that VeriFuse achieves 0.494/0.357 cooperative 3D AP50/AP70 and limits the relative vehicle-side BEV AP50 drop under a 300 ms delay to 1.7%. Overall, VeriFuse assigns the VLM a clear and constrained role in cooperative perception: semantic reasoning resolves ambiguity among cross-agent hypotheses, while deterministic constraints determine the final 3D geometry.

[13] Cube-Splat: High-Fidelity 360° Gaussian Splatting SLAM via Cubemap Factorization and Adjoint-Consistent Optimization cs.CVPDF

Xiangfei Guo, Hao Shi, Yufan Zhang, Zhonghua Yi, Yongqi Mao

TL;DR: Cube-Splat 将每帧全景图分解为四个共光心虚拟针孔视图，通过伴随映射汇总各面梯度更新同一位姿，并提出合成全景 SLAM 基准。

Details

Motivation: 3D Gaussian Splatting 已支持针孔相机下的稠密视觉 SLAM，但多数流程没有针对 360° 全景图像设计，难保证多视面更新共享位姿时的跨视图几何一致性。

Result: Cube-Splat 将每帧 360° 图像分解为四个固定朝向、共光心的立方体面针孔视图，以前向面为主位姿状态，通过伴随映射累积所有视面梯度，保持一致几何更新。建图模块聚合各面射线以稠密化和优化各向异性高斯。作者还构建 SynPano，可规模生成照片级真实场景、复杂参数化轨迹和多模态真值。在 PALVIO、OmniBlender 和 SynPano 的室内外评测中，跟踪精度及重建保真度均达到先进水平。

Insight: 将全景观测转成共享光心的多个视面，并以单一主位姿协调梯度，可利用多方向图像信息而不破坏整体几何；合成数据集为困难全景 SLAM 评测提供可控真值。

Abstract: Recent progress in 3D Gaussian Splatting (3DGS) has enabled dense visual SLAM with pinhole cameras, yet most pipelines are not designed for panoramic imagery. We present Cube-Splat, the first panoramic GS-SLAM framework that factorizes each 360° frame into a cubemap of four fixed-orientation virtual pinhole views sharing a single optical center. By designating the front face as the primary pose state, we accumulate gradients from all faces via an adjoint mapping, thereby enabling multi-face observations to coherently update a single state while strictly preserving cross-view geometric consistency. Concurrently, our mapping module densifies and optimizes anisotropic Gaussians using aggregated cubemap rays for high-fidelity, dense reconstruction. Furthermore, to rigorously evaluate panoramic SLAM under diverse and challenging conditions, we introduce SynPano, a highly scalable, photorealistic synthetic dataset featuring parameterized complex trajectories and multi-modal ground truth. Extensive evaluations on two public benchmarks (PALVIO and OmniBlender) and our SynPano dataset, collectively encompassing both indoor and outdoor scenes, demonstrate that Cube-Splat achieves state-of-the-art (SOTA) performance in tracking accuracy and reconstruction fidelity. Both the source code and the SynPano dataset are available at https://github.com/guoxf304/CubeSplat.

[14] PrismAlign: Prior-Steered Multi-View VLM Alignment for Hallucination-Robust Table OCR cs.CVPDF

Guangyi Liu, Qianjun Huang, Boyu Hou

TL;DR: PrismAlign 用多视觉语言模型对表格的不同视觉视角进行对齐，并将表格逻辑先验用于结构和单元格内容的贝叶斯一致性校验。

Details

Motivation: 表格抽取经常出现结构错误和语义幻觉；单个模型视角可能产生难以察觉的不一致，而结构对齐与单元格内容对齐也不应混为一谈。

Result: PrismAlign 整合多个 VLM 的不同视觉视角，利用表格逻辑先验评估输出是否合理，并把结构对齐与单元格内容对齐解耦。贝叶斯决策策略根据抽取错误与可计算规则违规之间的相关性最大化对齐准确率。在开源及定制 VLM 评测中，该方法减少幻觉，并在 OmniDocBench 1.5 及 CC-OCR、PureDocBench 的表格类别达到先进表现。

Insight: 表格结构规则可作为模型输出的独立校验信号；多模型多视角证据与贝叶斯规则检查结合，有助把结构错误和单元格内容错误分别定位。

Abstract: Table extraction suffers from frequent structural errors and semantic hallucinations. We propose PrismAlign, a multi-VLM framework aligning diverse visual perspectives to resolve ambiguity. It integrates priors of table logic to assess output plausibility, decoupling structural alignment from cell content alignment. A Bayesian decision strategy maximizes alignment accuracy by exploiting the correlation between extraction errors and computable rule violations. Evaluated on open-source and custom VLMs, PrismAlign reduces hallucinations and achieves state-of-the-art performance on OmniDocBench 1.5, as well as on the table category of CC-OCR and PureDocBench.

[15] Field Tracking of Insects Using a Stereoscopic Event-Based Camera Setup cs.CVPDF

Pratham G. Shenwai, Martin J. Lankheet, John T. Hrynuk, Mandiyam Y. Mahadeeswara, Mandyam V. Srinivasan

TL;DR: 该研究将立体事件相机的异步事件转换成常规视频格式，复用成熟图像分析方法，实现野外快速昆虫的低延迟三维追踪。

Details

Motivation: 在自然环境中高速追踪小型生物有助研究行为与生态，但帧相机时间分辨率有限会产生运动模糊，且视频存储成本较高。

Result: 研究使用事件相机逐像素捕捉亮度变化，再将异步事件转换为常规视频，利用其高时间细节处理昆虫飞行运动，并复用已有图像分析方法。结合立体相机设置后，可在野外连续、低延迟地三维追踪高速目标，减少运动伪影并更准确表示动物运动。方法面向动物行为、生态、农业管理及其他野外高精度追踪任务。

Insight: 事件成像的时间优势可以通过格式转换接入成熟视频分析工具；立体配置进一步提供连续三维轨迹，使高速度行为研究无需完全重写现有处理链。

Abstract: High-speed tracking of small, fast-moving organisms in their natural environments is important to better understand their behavior and ecology. Traditional frame-based imaging suffers from motion blur due to low temporal resolution, and data storage limitations, propelling a search for more adaptive solutions. Event cameras, which capture changes in brightness at pixel level instead of entire frames, have emerged as a promising solution by increasing temporal resolution and data efficiency. Here, we demonstrate the use of event-based imaging with standard video-based processing methods by converting the asynchronous events into conventional video formats, allowing us to leverage the event camera’s enhanced temporal detail to capture intricate insect flight movements and apply established image analysis techniques. Coupling this conversion process with a stereoscopic configuration provides continuous, low-latency, three-dimensional tracking of fast-moving subjects in field conditions. As a result, we substantially mitigate motion artifacts and achieve more accurate representations of animal movements. By making event-based imaging more readily applicable in natural field settings, our method support broader applications across animal behavior and ecological research, agricultural management, and other fields requiring high-fidelity object tracking in the wild.

[16] Hiding in Plain Sight: A Diffusion-based Mitigation of Geolocation Privacy Leakage in Vision-Language Models cs.CV | cs.LGPDF

Yining Wang, Xi Li, Mi Zhang, Xiaohan Zhang, Xiaoyu You

TL;DR: 该方法在扩散模型反向采样时对潜在语义注入针对性扰动，并以 GeoCLIP 为代理攻击信号，降低视觉语言模型从照片推断用户地理位置的能力。

Details

Motivation: 多模态推理模型能从建筑风格、植被和光照等细节推断用户照片的地理位置，形成潜在隐私泄漏。拒答防护可被精心设计的越狱提示绕过，像素空间不可见扰动又常缺乏黑盒迁移性并产生明显伪影。

Result: 作者提出扩散式主动防御，在反向采样时直接扰动扩散模型潜在空间中的高层语义表征，并使用与 GPS 坐标显式对齐的 GeoCLIP 定位并干扰模型利用的地理信号。实验显示该方法提高黑盒迁移能力，同时保持感知图像质量，目标是无缝集成到社交媒体图片分享流程。

Insight: 在高层语义潜在空间干预可针对模型真正使用的地理线索，比只扰动像素更具迁移性；隐私效用权衡通过 GeoCLIP 引导的定向扰动处理。

Abstract: Multimodal large reasoning models (MLRMs) have demonstrated remarkable capabilities in complex visual understanding. However, this very power introduces a critical yet underexplored privacy threat: adversaries can exploit MLRMs to precisely infer users’ geographic locations from casually shared photographs, by performing structured reasoning over subtle visual cues such as architectural styles, vegetation, and lighting conditions. In this work, we present a systematic study of MLRM-driven geolocation privacy leakage. We first reveal that refusal-based safeguards are critically insufficient, as carefully crafted jailbreak prompts can raise model response rates to 100%. We further identify that existing defenses, which inject imperceptible perturbations into shared images, suffer from structural limitations intrinsic to their pixel-space optimization, resulting in degraded black-box transferability and pronounced visual artifacts. Motivated by these findings, we propose a diffusion-based framework that provides targeted, proactive defense against geolocation privacy leakage. By injecting perturbations into the latent space of a diffusion model during reverse sampling, our method operates directly on high-level semantic representations, thereby resolving the effectiveness-utility bottlenecks by construction. We further ground our optimization with GeoCLIP, a model explicitly aligned with GPS coordinates, as a surrogate to pinpoint and disrupt the geographic signals that MLRMs exploit for location inference. This targeted semantic disruption yields significantly stronger black-box transferability while preserving perceptual image quality, offering a seamless integration on social media platforms.

Xinyi Che, Zheng Lian, Kuofei Fang, Xuehao Wang, Xinghai Gao

TL;DR: RobotEQ-Video 将社会主动智能评测从静态图像扩展到视频，并用四层世界状态分类法系统覆盖人类状态、需求与行为得体性。

Details

Motivation: 社会主动智能需评估具身场景中主动帮助是否合乎社交情境。已有研究多依赖静态图像，难利用动态视频中的人类状态线索；自由形式数据收集也难保证场景覆盖全面。

Result: RobotEQ-Video 构建四级由粗到细的世界状态分类体系，含 6 个领域、20 个维度、142 个一级属性和 816 个二级属性。基准包含 2,000 多段视频、10 万多个人类标注及 1.6 万余项行为得体性标签。评测表明现有系统仍不可靠且未达到人类水平；作者还初步探索世界模型对该任务的帮助。

Insight: 视频能提供静态图像缺失的状态变化和需求线索；层级分类提高场景覆盖可控性，使主动协助的社会恰当性可在动态具身情境中系统评估。

Abstract: Social Proactive Intelligence (SPI) extends proactive assistance beyond task completeness to consider social appropriateness in diverse embodied scenarios. However, prior SPI research faces two key limitations. First, existing work focuses on static images, whereas dynamic videos provide crucial cues for inferring human states and needs, offering richer information than isolated images. Second, prior work often relies on free-form data collection pipelines, which fail to guarantee comprehensive coverage of diverse scenarios. To address these gaps, we introduce RobotEQ-Video, shifting the focus from image-centric to video-centric analysis. To ensure comprehensive video coverage, we construct a hierarchical world-state taxonomy organized into a four-level coarse-to-fine structure, comprising 6 domains, 20 dimensions, 142 level-1 attributes, and 816 level-2 attributes. The resulting benchmark comprises 2K+ videos with 100K+ human annotations and 16K+ labels for assessing behavior properness. Benchmark evaluation reveals that current systems remain unreliable and fall short of human performance. We further explore how world models can help tackle this task. This work advances SPI research from static images to dynamic videos and ensures more comprehensive scenario coverage during benchmarking.

[18] JEPA Guided Diffusion: Predictive Vision-Language Conditioning for Generative Traffic Forecasting cs.CVPDF

Trinh Tra Giang Nguyen, Thanh Nguyen Vo, Nguyen Hoai Thuong Bui, Ha Duc Bui

TL;DR: 该交通预测框架冻结 V-JEPA 和 Cosmos，仅训练轻量潜在对齐模块，将预测性场景表征转为扩散视频生成条件，并获 AI City 赛道第三名。

Details

Motivation: 交通预测既要理解场景动态又要合成真实未来观测。扩散视频模型能生成逼真预测，但端到端训练成本高，且容易将场景理解与像素合成耦合。

Result: 框架解耦未来表征学习和视频生成：冻结 V-JEPA 从观测交通视频提取预测性潜在场景动态，再由轻量对齐模块映射到冻结 Cosmos 扩散模块的条件空间，无需重训大型生成模型。在 2026 AI City Challenge Track 5 基准上得分 75.1297，排名第三。

Insight: 预测性世界表征可作为生成模型的语义条件，将场景动态理解与像素合成分开，降低训练成本并保留交通未来预测能力。

Abstract: Accurate traffic forecasting requires both understanding scene dynamics and synthesizing realistic future observations. Recent diffusion-based video generation models produce visually plausible predictions but require expensive end-to-end training and often entangle scene understanding with image synthesis. In this work, we propose a decoupled forecasting framework that separates future representation learning from video generation. A frozen V-JEPA encoder first extracts predictive latent representations from the observed traffic videos, capturing the underlying scene dynamics in a semantic latent space. A lightweight latent alignment module then projects these representations into the conditioning space of a frozen Cosmos diffusion module, enabling future video synthesis without retraining the large generative model. By freezing all foundation models and training only the lightweight alignment module, the proposed framework substantially reduces optimization complexity while preserving forecasting capability. Experimental results on the AI City Challenge 2026 Track 5 benchmark demonstrate that the proposed method achieved a score of 75.1297, ranking third in the competition. These results suggest that predictive world representations learned by V-JEPA can effectively guide downstream video generation, providing a practical and efficient alternative to end-to-end diffusion-based forecasting.

[19] AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agents cs.CV | cs.AIPDF

Seoyeon An, Hyeonseo Jang, Minsu Kim, Chanho Lee, Younghan Park

TL;DR: AgentVidBench 用多跳视频问答及逐步解题轨迹评估 MLLM 智能体的空间、时间和因果推理，发现多轮智能体工作流普遍优于单轮回答。

Details

Motivation: 现有视频理解基准多是场景级问答或全局摘要，只需单步推理；现实视频理解往往需要多跳跨模态推理，而缺少能检查智能体如何获取证据的基准。

Result: AgentVidBench 面向 MLLM 智能体评测空间、时间和因果推理，除问答对外还提供逐步解题轨迹，用来检查智能体是否明确获取支持答案的证据。对 12 种专有及开源模型评测发现，单轮表现有限；将模型接入先进智能体工作流后，准确率和轨迹分普遍提高。作者还提出一个简单且有竞争力的智能体基线。

Insight: 视频答案是否正确不足以说明多跳推理是否基于充分证据；同时评估推理轨迹和证据获取，可更全面衡量智能体视频理解能力。

Abstract: Comprehensive video understanding is crucial for advancing artificial intelligence toward the intricate dynamics of the physical world. While recent advances in Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in video understanding, existing benchmarks remain confined to simple scene-level queries or global summaries that require only single-step inference. Real-world video understanding involves more challenging tasks that require multi-hop multimodal reasoning, and there is a critical absence of video benchmarks equipped to rigorously evaluate these agentic capabilities. To bridge this gap, we introduce AgentVidBench, a multi-hop video question answering benchmark focused on evaluating the spatial, temporal, and causal reasoning capabilities of MLLM agents. Beyond standard question-answer pairs, AgentVidBench provides step-by-step solution traces to support trajectory evaluation that assesses whether agents explicitly acquire the evidence needed to justify their answers. Experiments with 12 proprietary and open-source MLLMs show that single-turn performance remains limited on AgentVidBench, while integrating these models into state-of-the-art agentic workflows generally improves performance with respect to both accuracy and trajectory scores. We further present a simple yet effective agentic strategy that serves as a competitive baseline on AgentVidBench, establishing our benchmark as a holistic testbed for future research on agentic video understanding. Code and datasets are available at https://github.com/krafton-ai/agentvidbench and https://huggingface.co/datasets/agentvidbench/agentvidbench.

[20] SIRA: Reasoning-Aware Surgical Instrument Segmentation via Query-Anchored Alignment cs.CVPDF

Zhibo Zhang, Qijie Wang, Zengqiang Yan

TL;DR: SIRA 把手术器械分割定义为基于术中语境的查询条件推理任务，并用查询锚定的双重对齐将语义与空间分割提示结合。

Details

Motivation: 传统手术器械分割多按类别定位，难利用手术流程语境和任务相关语义；具备推理能力的语义分割还缺少像素级查询接地数据。

Result: 作者构建 SurgRS，含 41,000 组图像-文本对，将实例掩码与结构化问答监督对齐。SIRA 区分目标级和查询级语义，再通过查询锚定双重对齐与视觉特征结合，使查询语义同时对齐空间特征和分割提示，提高掩码预测的语义-视觉一致性。在 SurgRS 上，方法超过已有推理感知分割基线。

Insight: 手术语境中的目标定位应由具体查询条件决定，而不仅是器械类别；对齐查询语义与空间掩码提示有助使分割结果符合操作意图。

Abstract: Surgical instrument segmentation (SIS) plays a critical role in robotic assistance and surgical workflow analysis. However, most existing SIS methods formulate segmentation as a category-driven localization problem, limiting their ability to capture procedural context and task-dependent semantics in surgical workflows. We introduce Reasoning-Aware Surgical Instrument Segmentation (RA-SIS), a task formulation that frames segmentation as query-conditioned inference under surgical context. To benchmark this setting, we construct SurgRS, a surgical reasoning segmentation dataset consisting of 41,000 image-text pairs, which aligns instance-level masks with structured query-answer supervision to enable semantic grounding at the pixel level. Based on SurgRS, we propose Surgical Instrument Reasoning and Segmentation Assistant (SIRA), a multimodal framework that disentangles target-level and query-level semantics and integrates them with visual features through query-anchored dual alignment. By aligning query semantics with spatial features and segmentation prompts, SIRA enhances semantic-visual consistency in mask prediction. Extensive experiments on SurgRS demonstrate improvements over existing reasoning-aware baselines. Code is available at https://github.com/linxir226/SIRA.

[21] A Scene Language Model for Open-Vocabulary Scene Mapping cs.CV | cs.ROPDF

Adam Lilja, Fabio Hübel, Siming He, Junsheng Fu, Claire Tomlin

TL;DR: SceneLM 直接把结构化文本场景图作为唯一持久记忆，由单个 VLM 根据每张新图像增删改对象，在多个定位与检索任务上接近完整系统且记忆更紧凑。

Details

Motivation: 开放词汇三维场景地图需跨视角关联观测并长期保持一致；现有系统依赖复杂手工地图管线，还会存储大量嵌入或图像裁剪，增加持久记忆大小和系统复杂度。

Result: SceneLM 以结构化文本对象列表表示整个场景，作为模型唯一持久状态。每次输入图像，模型读取当前地图并添加、编辑或移除对象；作者通过迭代地图维护监督任务和无人工标注自动数据管线学习此过程。在语言接地检索和定位基准上，其地图性能与由专用感知和几何模块构成的完整系统具竞争力，表示体积小 6–12 倍；四足机器人实验还展示其可在边缘设备在线运行。

Insight: 持久三维地图不一定要存储高维视觉特征；若结构化文本状态足以支撑更新和查询，单个 VLM 可构成更紧凑的在线场景记忆。

Abstract: Open-vocabulary 3D scene mapping aims to build a persistent representation of the objects in an environment. Existing systems typically rely on engineered mapping pipelines to associate observations, merge information across views, and maintain a consistent scene representation over time. Many additionally store feature-rich object representations, such as embeddings or image crops, increasing the size and complexity of the persistent memory. We introduce SceneLM, a Scene-Language Model that directly maintains a textual scene map. The full scene is represented as a structured text list of objects, which serves as the model’s only persistent memory. For each input image, the model reads the current scene state and updates the map by adding, editing, and removing objects. To learn this behavior, we introduce supervision tasks for iterative scene map maintenance together with an automatic annotation pipeline that generates training data from images without human labels. We evaluate SceneLM on both a language-grounded retrieval benchmark and a localization benchmark. Across both benchmarks, the model produces a scene map that achieves competitive performance with complete mapping systems built from dedicated perception and geometric modules while producing a scene representation that is 6-12x more compact. We further show that SceneLM can be run online on an edge device through experiments on a quadruped. These results show that a persistent open-vocabulary 3D scene map can be maintained directly by a single vision-language model using only a lightweight text representation. Training and inference code is available on https://goldengait.github.io/scenelm/.

[22] ME-Dex 1.0: Bringing Heterogeneous Tactile Sensing into World Action Modeling cs.CVPDF

Xuancheng Zhang, Xuetao Liu, Qianying Tang, Jizhe Wang, Zhijing Cheng

TL;DR: ME-Dex 1.0 用视频、触觉和动作专家组成共享注意力的混合 Transformer，并将异构触觉传感映射到统一空间以联合学习世界-动作-触觉模型。

Details

Motivation: 世界动作模型能预测未来视觉状态并生成机器人动作，但触觉能直接提供物理交互信息。既有方法常把触觉仅作为条件输入，却不联合预测未来触觉、视觉观察和动作。

Result: ME-Dex-1.0 由视频、触觉和动作专家构成 Mixture-of-Transformers，全部使用 flow matching 训练，并通过中间层共享注意力让动作生成利用学习到的视觉与触觉动态。规范手模型和统一触觉自编码器将不同身体形态、传感布局的触觉映射到共享空间。为解决成对数据不足，Agentic Tactile Data Engine 在仿真轨迹回放中用力传感器补充 RoboTwin 和 DexJoCo 触觉数据。在 RoboTwin、DexJoCo、ManiFeel 仿真平台及真实机器人评测中，带触觉的夹爪和灵巧手操作表现改善。

Insight: 触觉本身是世界状态的观测，应像视频一样预测其未来变化；统一异构传感布局和专家间共享注意力有助于把触觉动力学纳入动作生成。

Abstract: World Action Models bring the predictive capabilities of video models into robot action generation, providing a rich foundation for modeling future visual states. Tactile sensing complements this foundation with direct measurements of physical interaction. Some existing methods use tactile features as conditioning inputs without jointly predicting future tactile states, visual observations, and actions. Our key insight is that tactile signals, like video, provide observations of the evolving world state and should be modeled as future observations alongside video. We present ME-Dex-1.0 (MachEmbodied-Dex-1.0), a unified World Action Tactile Model for joint visual, tactile, and action learning. ME-Dex-1.0 adopts a Mixture-of-Transformers architecture comprising a Video Expert, a Tactile Expert, and an Action Expert, all trained with flow matching. We use shared attention connects the experts in intermediate layers, allowing action generation to draw on learned representations of visual and tactile dynamics during joint denoising. To support multi-source heterogeneous tactile inputs, a Canonical Hand Model and a Unified Tactile Autoencoder map tactile observations from different embodiments and sensing layouts into shared spatial and latent spaces. To address the limited availability of paired visual, tactile, and action data, we develop the Agentic Tactile Data Engine, an agent-based data production platform. It supplements RoboTwin and DexJoCo with tactile data recorded directly from force sensors during trajectory replay in simulation. Experiments on the RoboTwin, DexJoCo, and ManiFeel simulation platforms, together with real robot evaluations, demonstrate improved manipulation performance using both grippers and dexterous hands equipped with tactile sensing.

[23] CompAdapt: Adaptable Composite Motion Modeling for Physics-Consistent Text-to-Video Generation cs.CVPDF

Haoran Qin, Renlong Wu, Tianyu Huang, Yukang Ding, Hui Li

TL;DR: CompAdapt 把自然语言转为复合运动的结构化物理语义，并用动力学先验匹配支持未见物理环境单样本适配，面向物理一致文本视频生成。

Details

Motivation: 扩散文本视频模型虽能生成逼真连贯视频，却常违反物理规律。现有显式动力学方法多限于单一简单运动、依赖人工参数且难泛化到未见物理定律。

Result: CompAdapt 扩展神经动力学建模以处理耦合运动、多阶段状态转换和多物体碰撞等复合行为；把自然语言提示解析为运动类型、时间关系和初始物理参数。为适配新物理环境，提出动力学感知先验匹配，无需重训核心动力学模块即可单样本适应；物理感知潜特征融合模块改善快速复杂运动下的视觉保真度。物理导向文本视频基准实验显示，CompAdapt 比通用生成模型和物理约束基线更符合物理规律，同时保持画质并适应未见动力学。

Insight: 物理一致性需要表示对象间耦合和阶段关系，而非只约束单一运动；将自然语言解析为结构化物理参数可支持复杂场景控制与快速新环境适配。

Abstract: While diffusion-based text-to-video (T2V) models have demonstrated impressive capability in generating realistic and temporally coherent videos, they often fail to respect fundamental physical dynamics. Although recent physics-constrained methods incorporate explicit dynamics priors to improve physical plausibility, they remain limited to simple single-type motions, depend on manually specified parameters, and struggle to generalize to unseen physical laws. In this work, we propose CompAdapt, a physics-consistent T2V framework for adaptable generation across complex real-world scenarios. It extends neural dynamics modeling beyond single-type motions to encompass composite physical behaviors, including coupled motions, multi-stage transitions, and multi-object collisions. Furthermore, CompAdapt translates natural language prompts into structured physical semantics, enabling end-to-end specification of motion types, temporal relations, and initial physical parameters. To generalize to novel physical environments, CompAdapt introduces dynamics-aware prior matching, achieving one-shot adaptation without retraining the core dynamics module. In addition, a physics-aware latent feature fusion module improves visual fidelity under fast and complex motion. Experiments on physics-focused T2V benchmarks demonstrate that CompAdapt improves physical consistency over both general T2V models and physics-constrained baselines, while preserving high visual quality and adaptability to unseen dynamics. The project page is available at https://makapic.github.io/CompAdapt/ .

[24] SkillIR: Evolving Scene-Aware Skills for Agentic Image Restoration cs.CVPDF

Jie Shao, Shengkai Hu, Xu Zhang, Beihang Song, Yongcheng Jing

TL;DR: SkillIR 将修复经验表示为针对退化情境的局部动作证据，在已验证的残余状态循环中逐步调用工具并更新技能。

Details

Motivation: 现有图像修复智能体常根据原始退化图制定整套工具计划或检索成功轨迹，但一次工具执行会改变残余退化，使后续工具适用性发生变化。

Result: SkillIR 将上下文相关的动作结果整理为场景感知修复技能，记录适用条件、预期效果和可归因失败，而不预先规定完整工具链。检索技能每次仅指导一个有界动作；工具输出先作为候选，只有通过状态转移验证才提交，随后重新评估残余退化。每次 rollout 后，证据用于新建、细化或修补动态技能。合成和真实多重退化数据集实验显示，SkillIR 提升恢复质量，并使工具调用更可靠有效。

Insight: 修复过程是随中间状态改变的决策问题；以可验证的单步动作和动态技能积累替代固定整套计划，更能适应工具执行后退化结构变化。

Abstract: This paper studies agentic image restoration, in which multimodal agents coordinate specialized restoration tools to recover images affected by complex degradations. Existing restoration agents often derive complete tool-use plans from the original degraded image or retrieve previously successful trajectories, providing limited support for adapting individual actions to evolving intermediate restoration states. We find that accepted tool executions can change the residual degradation state and, consequently, the applicability of subsequent tools. To address this issue, we propose SkillIR, a skill-guided framework that represents restoration experience as degradation-centered action evidence rather than complete tool-use trajectories. SkillIR consolidates context-dependent action outcomes into scene-aware restoration skills that characterize applicable conditions, expected effects, and attributable failure cases. Instead of prescribing a complete restoration plan, the retrieved skills guide one bounded action at a time within a verified residual-state loop: each tool output is treated as a candidate, committed only after transition verification, and followed by reassessment of the active residual degradations. After each rollout, the resulting evidence is used to create, refine, or patch dynamic skills, enabling accumulated restoration experience to improve decision-making for subsequent inputs. Experiments on synthetic and real-world multi-degradation datasets demonstrate that SkillIR improves restoration quality and enables more reliable and effective tool use.

[25] MT-WAM: Reorienting the One-Pass Predictive Representation Toward Action Generation cs.CV | cs.ROPDF

Yiguang Yang, Jiankun Peng, Xiaoming Wang, Yiran Zhang, Zhibo Fang

TL;DR: MT-WAM 在视频动作模型原有目标上增加未来二维点轨迹和视觉特征监督，使单次视频表征更服务于动作生成，但推理时不生成未来视频。

Details

Motivation: Fast-WAM 证明视频与动作联合训练能改善控制，而不必在推理时生成未来视频；但未来观测预测未显式优先表示动作控制需要的未来动力学和视觉结构。

Result: MT-WAM 保留原训练目标，额外监督未来二维点轨迹和视觉特征。轻量双流分支由视频骨干末层复制而来，并通过结构注意力掩码阻止跨流干扰；运动 token 为动作专家提供动力学条件，未来视觉特征预测则监督对象与空间结构。在视觉变化下，它学习为动作生成提供更有信息量的上下文，但视觉特征流 token 不进入动作条件。推理时视频和运动缓存每次重新规划仅计算一次，并跳过未来视频预测。无需额外具身策略预训练，LIBERO 成功率为 98.2%，LIBERO-Plus 为 73.7%，比 Fast-WAM 高 23.8 个百分点；RoboTwin 2.0 Clean2Rand 的 Random 成功率从 6.30% 到 19.40%，四个真实任务平均从 67.0% 到 77.8%。

Insight: 动作表征所需的未来信息可通过点轨迹和视觉特征辅助训练注入，而不必在部署时预测像素视频，改善控制同时维持较低推理成本。

Abstract: Fast-WAM shows that video-action co-training improves control without generating future video at inference, making the representation from a single video diffusion Transformer forward central to action generation. However, future-observation prediction does not explicitly prioritize the future dynamics and visual structure needed for control. We present MT-WAM, which retains the original training objectives and adds complementary supervision for future two-dimensional point trajectories and visual features. A lightweight dual-stream branch copied from the video backbone’s final blocks provides target-specific processing, while a structured attention mask prevents cross-stream attention. Motion-stream tokens supply additional dynamics conditions to the action expert. Future visual-feature prediction provides supervision in a feature space that captures object and spatial structure. This supervision trains the video backbone to provide more informative visual context for action generation under changing visual conditions, without adding visual-feature-stream tokens to action conditioning. At inference, MT-WAM uses video and motion caches computed once per replan and skips future-video prediction. Without additional embodied policy pretraining, MT-WAM achieves 98.2% success on LIBERO and 73.7% on LIBERO-Plus, exceeding Fast-WAM by 23.8 percentage points on the latter. On RoboTwin 2.0 Clean2Rand, Random success increases from 6.30% to 19.40%; across four real-world tasks, average success increases from 67.0% to 77.8%.

[26] OpenSAL360: Open-Source Crowdsourcing Platform for Omnidirectional Video Saliency Collection cs.CVPDF

Alexey Bryncev, Andrey Moskalenko, Kira Shilovskaya, Ivan Kosmynin, Dmitriy Vatolin

TL;DR: OpenSAL360 提供无需 VR 头显的开源 360° 视频显著性众包平台，并发布由 2,000 多名参与者标注的 500 段全景视频数据集。

Details

Motivation: 全景视频显著性可用于视口自适应传输、压缩、注视点渲染和网格简化等沉浸媒体任务，但 VR 眼动设备采集成本高、流程复杂，限制数据集扩展。

Result: OpenSAL360 使用标准显示器、鼠标和互联网即可并行收集注视显著性，降低对专业硬件的依赖。作者将采集协议与七个成熟 VR 眼动数据集进行验证，并对界面及前后处理参数做消融。基于该平台采集并公开了 500 段全景视频、2,000 多名众包评审的显著性数据，据作者所知是该领域当时最大数据集。

Insight: 通过用普通屏幕上的交互替代专用头显，360°显著性标注有望规模化；与 VR 眼动数据的协议对照是验证低成本采集可靠性的关键。

Abstract: Omnidirectional video saliency prediction plays an important role in many immersive multimedia applications, including viewport-adaptive streaming and compression, foveated rendering, mesh simplification, perceptual quality assessment. Yet progress in this area remains constrained by the cost and complexity of collecting eye-tracking data with VR headsets, which makes large-scale dataset creation difficult to extend. We present OpenSAL360, the first open-source platform for scalable, low-cost 360° video saliency collection. Unlike conventional VR-based protocols, it requires only a standard screen, mouse, and internet connection, enabling parallel saliency data collection from common crowdsourcing assessors without specialized hardware. We validate our collection protocol against seven well-established VR eye-tracking datasets and conduct ablation studies on key interface, pre-, and post-processing parameters. To demonstrate the effectiveness and scalability of the proposed methodology, we collect and publicly release a saliency dataset covering 500 omnidirectional videos annotated by 2,000+ crowdsourcing assessors, making it, to the best of our knowledge, the largest dataset in this field. We make OpenSAL360 publicly available at https://github.com/msu-video-group/OpenSAL360.

[27] VoxelTTO: Voxel-Aligned Feed-Forward 3D Gaussian Splatting with Test-Time Optimization cs.CVPDF

Yibin Zhao, Yihan Pan, Yangwen Li, Jun Nan, Jianjun Yi

TL;DR: VoxelTTO 将多视图特征融合到全局体素网格再解码 3D 高斯，打破像素与高斯一一对应，并用测试时 LoRA 适配相机位姿。

Details

Motivation: 前馈 3DGS 常回归像素对齐高斯，容易重叠和产生伪影；预测相机位姿不准也会造成新视角合成错位。

Result: VoxelTTO 从任意数量图像及可选相机参数重建场景，先将稠密图像特征聚合到全局体素表示，再从体素特征解码高斯。为利用已知相机参数同时冻结预训练视觉基础模型，方法以位姿监督在测试时优化轻量 LoRA；训练和推理还用随机实体体积渲染替代普通 3DGS 光栅化，以改善几何保真。只更新体素对齐重建模块，训练需 80 GPU 小时。在 Replica、Tanks and Temples 和 DTU 上，新视角 RGB-D 合成和相机位姿估计优于既有方法。

Insight: 体素中间表示可减少直接像素到高斯回归造成的几何重叠；冻结通用视觉骨干而在测试时适配轻量模块，可针对已知位姿条件进行场景级校正。

Abstract: Recent feed-forward 3D Gaussian Splatting (3DGS) methods typically regress pixel-aligned Gaussian primitives, often causing excessive overlap and artifacts, while inaccuracies in predicted camera poses can lead to misalignment in novel-view synthesis (NVS). We present VoxelTTO, a feed-forward framework for reconstructing geometrically accurate 3DGS scenes from an arbitrary number of images and optional camera parameters. VoxelTTO aggregates dense image features into a global voxel representation and decodes Gaussians from voxel features, breaking the pixel-to-Gaussian correspondence. To exploit known camera parameters while keeping the pretrained visual foundation model (VFM) parameters frozen, we introduce test-time optimization (TTO) that adapts lightweight LoRA modules using pose supervision. We further replace vanilla 3DGS rasterization with stochastic solid volume rendering during training and inference, improving geometric fidelity. Training updates only the voxel-aligned Gaussian reconstruction modules, requiring 80 GPU hours. Experiments on Replica, Tanks and Temples, and DTU demonstrate improved RGB-D NVS and camera-pose estimation relative to prior methods.

[28] Adaptive World Memory 3D Foundation Model for Scalable 3D Mapping, Localization, and Rendering cs.CV | cs.ROPDF

Tianchen Deng, Guole Shen, Yilin Shen, Wenhua Wu, Yilin Fang

TL;DR: 该三维基础模型用门控循环世界记忆处理长序列 RGB，结合局部子地图、闭环和全局优化，实现机器人定位、重建及高斯渲染。

Details

Motivation: 近期三维基础模型能从 RGB 图像泛化几何推理，但持久记忆、扩展性和可渲染场景建模仍不足以支撑长期机器人建图。

Result: 模型核心为自适应世界记忆：Transformer 门控更新与测试时的时空约束共同调节记忆传播、状态演化和观测-状态一致性，控制长图像序列中的 token 更新与遗忘。大范围地图由局部子地图组织，并结合渐进建图跟踪、闭环和 SL(4) 全局精化维持局部精度与全局一致性；高斯重建头将记忆增强特征解码为可渲染基元，在单模型中统一位姿估计、稠密点云和照片级渲染。公开基准及多平台自采数据上，轨迹精度、重建完整性和渲染质量优于既有三维基础模型和 SLAM 基线。

Insight: 持续机器人世界模型需要显式控制记忆如何随时间和空间更新、保留或遗忘；将此记忆同时用于定位、几何重建和渲染，可形成统一地图表示。

Abstract: Recent 3D foundation models enable generalizable geometric reasoning from RGB images but remain limited in persistent memory, scalability, and renderable scene modeling. We present a memory-centric 3D foundation model for scalable robotic localization, reconstruction, and Gaussian rendering. Its core is an adaptive world memory mechanism that combines transformer-based gated updates with test-time temporal-spatial regulation. Learned gates control recurrent memory propagation, while temporal state evolution and spatial observation-state consistency regulate token-wise updates and forgetting over long image sequences. To support large-scale mapping, we organize memory into local submaps and integrate progressive mapping and tracking, loop closure, and SL(4)-based global refinement to maintain local accuracy and global consistency. A Gaussian reconstruction head decodes memory-enhanced features into renderable primitives, unifying camera pose estimation, dense point-cloud reconstruction, and photorealistic rendering within a single model. Experiments on public benchmarks and self-collected datasets from diverse robotic platforms demonstrate improved trajectory accuracy, reconstruction completeness, and rendering quality over existing 3D foundation reconstruction and SLAM baselines. These results support adaptive memory as a foundation for persistent robotic world modeling. The dataset and code will be made publicly available at \href{https://github.com/dtc111111/AWM-3DFM}{https://github.com/dtc111111/AWM-3DFM}.

[29] VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration cs.CV | cs.AIPDF

Changbeen Kim, Junwon Chang, Kipyo Kim, Risa Shinoda, Kuniaki Saito

TL;DR: VidOmni-Bench 要求逐句核验密集视频描述中的事件是否有视频证据，覆盖五种复杂度和 4 秒至 90 分钟时长，揭示模型描述和核验均易出错。

Details

Motivation: 视频大语言模型能力提升，但 QA 或与真值描述匹配的基准可能允许模型依靠表面线索作答，且标注不完整，难精细评测视频事件理解。

Result: 基准含 500 段视频，覆盖五种复杂度及 4 秒到 90 分钟时长。作者用不同 Video-LLM 生成密集描述，再由人工核验句子级标签，其中错误事件描述作为困难负例。实验发现：Video-LLM 经常在密集描述中幻觉；作为核验器也难可靠识别听起来合理但实际错误的事件；模型弱点随视频复杂度和时长变化，且各模型的瓶颈不同。

Insight: 核验具体事件是否真实发生，比只评估总体摘要或问答更能揭示视频理解中的幻觉和时间跨度弱点；模型还需同时学会描述和事实核验。

Abstract: While Video Large Language Models (Video-LLMs) have recently demonstrated strong performance, reliably evaluating their fine-grained video understanding remains challenging. Existing benchmarks often rely on question answering or ground-truth caption matching, where models may succeed through superficial cues and incomplete annotations. To this end, we introduce VidOmni-Bench, a benchmark that requires models to verify whether each event in dense video captions is supported by the video. VidOmni-Bench consists of 500 videos spanning five complexity types and diverse durations from 4 seconds to 90 minutes. After collecting videos along these axes, we use diverse Video-LLMs to generate dense captions and obtain human-verified sentence-level labels, where sentences containing incorrect events serve as hard negatives for evaluation. Our experiments on VidOmni-Bench reveal three key findings: (i) Video-LLMs frequently generate hallucinated descriptions in dense video captioning; (ii) they also struggle as verifiers, failing to reliably detect plausible but incorrect event descriptions; and (iii) model weaknesses vary across video complexity and duration, revealing diverse, model-specific bottlenecks in current Video-LLMs.

Hang Cheng, Yan Chen, Mingyu Fan, Long Zeng

TL;DR: RTF 先以类间相似度和类内稳定性筛选三维特征通道，再依据筛选引发的分布变化估计样本级模态可靠性，实现免训练少样本三维识别。

Details

Motivation: 预训练基础模型为三维识别提供丰富多模态先验，但少样本场景中高维特征存在冗余和任务无关噪声，且图像与形状模态的可靠性因样本而异，直接融合会掩盖判别信息。

Result: Refine Then Fusion (RTF) 首先联合建模类间相似性和类内稳定性，找出判别通道，并与预训练模型缓存表征中的领域知识精炼解耦；随后根据特征精炼导致的分布变化估计逐样本模态可靠性，自适应融合。记忆缓存结合实例级支持特征和类别原型来预测查询标签。五个基准实验中，RTF 持续超过单模态、部分融合及轻量适配基线，达到少样本三维识别先进水平；无需梯度优化、额外训练数据、辅助训练或参数更新。

Insight: 少样本多模态识别需先去除特征噪声，再按当前样本评估各模态可靠性；训练免的缓存与自适应融合可避免将不同来源一概等权合并。

Abstract: Recent pre-trained foundation models provide rich multi-modal priors for downstream 3D vision tasks. However, the effectiveness of these representations in few-shot scenarios is limited by two fundamental challenges: High-dimensional features often contain substantial channel redundancy and task-irrelevant noise, while the reliability of different modalities varies across samples. Consequently, direct aggregation of heterogeneous representations overlooks sample-dependent modality reliability and may obscure the discriminative cues essential. To address these limitations, we propose Refine Then Fusion(RTF), a training-free framework for few-shot 3D recognition. RTF first identifies discriminative feature channels by jointly modeling inter-class similarity and intra-class stability, thereby decoupling domain-specific knowledge refinement from the cached representations of pre-trained models. It then introduces a reliability-aware fusion mechanism that estimates sample-wise modality reliability from the distribution shifts induced by feature refinement, enabling adaptive aggregation of multi-modal representations. Furthermore, RTF constructs a memory cache that integrates instance-level support features with class-level prototypes to infer query labels. Extensive experiments on five benchmarks demonstrate that RTF consistently outperforms single-modal baselines, partial-fusion variants, and existing lightweight adaptation methods, achieving state-of-the-art few-shot 3D recognition performance without gradient optimization, additional training data, auxiliary training, or parameter updates.

[31] From Retrieval to Recognition:How Vision–Language Models Become OCR Specialists cs.CVPDF

Yuanxiang Huangfu, Hanmeng Zhong, Linqing Chen, Jeffrey Tiong Jee Hui

TL;DR: 研究发现通用 VLM 的整序列 OCR 头与文本检索/复制头高度重叠，说明 OCR 专门化主要是复用并重分配既有视觉证据读取机制。

Details

Motivation: 通用视觉语言模型获得专门 OCR 能力，是新建读取电路还是复用已有机制尚不清楚；作者研究整序列 OCR 而不是局部答案检索，并以因果干预追踪其机制来源。

Result: 在 GLM-OCR、MinerU2.5 和 PaddleOCR-VL-1.6 中，作者用有证据依据、包含留出因果干预的协议识别稀疏稳定 OCR 头集合。与通用 VLM 中独立识别的文本检索/复制头对比后，两种通用模型的视觉 OCR 头高度重合：未经调优的 top-20 交集分别为 73.3%，全头 Spearman 相关为 0.677–0.886。重叠与因果干预支持整序列 OCR 是稠密、逐步的多模态复制粘贴：反复检索视觉证据并路由到当前输出位置。通用模型专门化后基本保留头身份，每任务 top-20 留存 17–20 个，全头排序相关为 0.874–0.942，但功能和因果强度重新分配。

Insight: OCR 专业化更多改变已有读取头的功能强弱，而非完全创建新电路；模型可通过逐 token 重取视觉证据生成完整文本序列。

Abstract: Does a general vision–language model acquire specialized OCR ability by developing a new reading circuit or by reusing an existing mechanism? We address this question in the setting of full-sequence OCR, rather than local-answer retrieval. Using an evidence-grounded protocol with held-out causal interventions, we identify sparse and stable OCR-head sets in GLM-OCR, MinerU2.5, and PaddleOCR-VL-1.6. We then investigate the mechanistic origin of these OCR heads by comparing them with independently identified textual retrieval/copy heads in general VLMs. Across two general VLMs, visual OCR heads strongly overlap independently identified textual retrieval/copy heads, yielding untuned top-20 intersections of 73.3% and all-head Spearman correlations of 0.677-0.886. The overlap and causal interventions suggest that full-sequence OCR operates as dense sequential multimodal copy-and-paste, repeatedly retrieving visual evidence and routing it to the current output position. Finally, we examine how this shared circuit changes as a general VLM becomes an OCR specialist. Matched base-to-specialized comparisons show that OCR specialization largely preserves head identity, retaining 17-20 of the top 20 heads per task with all-head rank correlations of 0.874-0.942, while redistributing their functional and causal strengths.

[32] HAT: Hypothesis-Anchored Tracking for Video Monocular Spacecraft Pose Estimation cs.CVPDF

André Lopo, Atabak Dehban, Rodrigo Ventura

TL;DR: HAT 利用帧间运动在单目 SLAM 对齐前从多个 CAD 位姿假设中选择一致轨迹，在不回改历史输出的情况下提升航天器视频位姿估计精度和速度。

Details

Motivation: 非合作目标的单目六自由度位姿估计对在轨服务和碎片清除重要，但近似对称航天器易让单帧估计混淆朝向，跟踪过程还可能持续保留错误姿态。

Result: HAT 是因果跟踪框架，先保留多个候选朝向历史，再用相机 SLAM 估计的相对轨迹选择锚定位姿，之后以稀疏锚点和位姿融合输出逐帧估计，初始化后不修改过去结果。只需标定 RGB 序列、度量 CAD 模型和目标图像区域，检测或分割均可提供区域；位姿和 SLAM 网络无需目标域训练。Mega-HAT、Pico-HAT 在 SPARK-2024、SwissCube、SHIRT 和域外 YCB-Video 评测。相较独立 MegaPose，平均姿态误差低 9.4%、持续输入帧率高 3.76 倍；相较独立 PicoPose，分别低 23.9%、帧率高 2.42 倍。

Insight: 时间运动线索可在几何对齐之前消解单帧对称目标的朝向歧义；保留竞争假设并延迟选择，比每帧独立取最高分更稳健。

Abstract: Monocular 6-DoF pose estimation of non-cooperative targets is important for on-orbit servicing and debris removal. A single-image estimator can confuse near-symmetric spacecraft orientations, and tracking can preserve an incorrect pose. We present Hypothesis-Anchored Tracking (HAT), a causal framework that uses inter-frame motion to select among competing CAD-based pose hypotheses before alignment and fusion. Rather than independently choosing the highest-scoring hypothesis in each image, HAT retains competing orientation histories and selects a pose to anchor the relative trajectory estimated by monocular SLAM. Sparse anchors and pose fusion provide per-frame estimates after initialization without revising past outputs. The method requires only a calibrated RGB sequence, a metric CAD model, and target image regions, which can be supplied by detection or segmentation. The pretrained pose and SLAM networks require no target-specific training or fine-tuning. We evaluate two versions, Mega-HAT and Pico-HAT, using MegaPose and PicoPose, on SPARK-2024, SwissCube and SHIRT, with YCB-Video assessing performance outside the space domain. Using one temporal configuration per method, the arithmetic means of the four dataset-wise comparisons show 9.4% lower mean pose error and 3.76 times the sustained input FPS for Mega-HAT relative to independent MegaPose, and 23.9% lower mean pose error and 2.42 times the FPS for Pico-HAT relative to independent PicoPose. Mega-HAT ablations on SPARK and an offline reference examine component contributions and the effect of revising past estimates.

Musa Rochi, Marcel Schubert, Christoph Gebhardt

TL;DR: 该方法将情绪过滤从逐图优化改为一次前向的条件变换预测，在 Android 端实时调整社交媒体视频强度并降低用户报告的唤醒度。

Details

Motivation: 限制使用时长、屏蔽应用或强制休息等常见网络使用干预较强制且容易规避；调整视觉内容的情绪强度可能是较温和的替代方案，但逐图优化要耗时约 80 秒，不适合实时部署。

Result: 模型以 MobileNetV4 为骨干，通过 FiLM 情绪条件输出可微全局变换参数，一次前向计算即可完成情绪调整，从逐图 80 秒降到 3.7 毫秒。54 人用户研究显示，与未编辑视频相比，模型降低了用户报告的唤醒度，效果与灰度健康过滤器相当，同时感知质量评价更高。Android 应用在 Samsung Galaxy S23 上实时处理 Instagram 视频并持续达到 60 fps。

Insight: 学习情绪变换参数可取代高成本的每张图像优化，使内容级干预适用于实时使用；用户研究同时考察了情绪效果和画质接受度。

Abstract: Problematic internet use affects a growing share of the population, yet common interventions, e.g., time limits, blocking, forced breaks, are coercive and easily circumvented. We explore a less restrictive alternative: adapting the emotional intensity of visual content. Prior work has shown that optimization can steer an image’s affective content, but its per-image optimization cost makes it impractical for real-time deployment. We instead learn a model that predicts this transformation in a single forward pass: a MobileNetV4 backbone with FiLM-based emotion conditioning outputs parameters for differentiable global transformations. This replaces prior iterative optimization (80 s per image) with a single 3.7 ms forward pass. In a user study (N = 54), the model reduced viewer-reported arousal relative to unedited images, comparably to the grayscale well-being filter, while being rated higher in perceived quality. We integrate the model into an Android app that adapts Instagram video in real time, sustaining 60 fps on a Samsung Galaxy S23.

[34] Extending Decoupled Attention to Dense Prediction and Masked Training for Multi-Channel Images cs.CVPDF

Umar Marikkar, Sameed Husain, Muhammad Awais, Sara Atito

TL;DR: 该工作通过每通道保留 token 与跨通道融合相分离的注意力更新，扩展 DC-ViT 到稠密预测和独立通道遮罩的多通道图像训练。

Details

Motivation: 多通道成像的各通道代表不同语义信号而非颜色分量。常规 MC-ViT 将每通道 token 拼接后无约束自注意力，可能稀释单通道特征；已有 DC-ViT 又要求每个通道保留相同空间 token，限制掩码训练。

Result: DC-ViT 将通道内更新与通道间更新分开，并在融合前形成每通道表示。为支持各通道独立遮罩，作者通过对各通道保留 patch 做线性指派来恢复对应关系，使解耦注意力能与标准多通道掩码训练结合，而不受相同可见 token 的限制。在三个分类和三个分割基准上，覆盖荧光显微、成像质谱流式细胞术和卫星影像，包含高通道数稠密预测，方法超过最强 MC-ViT 基线。

Insight: 独立多通道的语义需先各自建模、再跨通道融合；用线性指派恢复不同遮罩下的空间对应，可把解耦注意力扩展到更实际的稠密预测训练。

Abstract: Multi-Channel imaging (MCI) data differs fundamentally from natural images, as each channel records a semantically distinct signal rather than a colour band. To adapt vision encoders to MCI data, Multi-Channel Vision Transformers (MC-ViTs) tokenize each channel independently and concatenate the resulting tokens into one sequence, and the channel count is no longer fixed by the architecture. Self-attention is then computed across all channel-patch tokens with no restriction on which channels attend to which, which dilutes the features of individual channels. The Decoupled Vision Transformer (DC-ViT) regulates this by separating updates computed within a channel from updates computed across channels, and by forming a representation per channel before the channels are combined. Its formulation, however, pairs tokens by spatial position, and thus requires the same visible tokens in every channel. Correspondence under independent per-channel masking is recovered by solving a linear assignment between the retained patches of each channel, which allows decoupled attention to be combined with current masked multi-channel training in its standard configuration rather than a restricted one. Across three classification and three segmentation benchmarks spanning fluorescence microscopy, imaging mass cytometry and satellite imaging, including dense prediction at high channel counts, the resulting formulation outperforms the strongest MC-ViT baseline.

[35] Configurable Multi-Stage Vision Pipeline for Crop Disease and Pest Diagnosis cs.CV | cs.CL | cs.LGPDF

Naga Ganesh, Chandrashekar M S, Lakshmi Pedapudi, Aakash Singh, Vineet Singh

TL;DR: 该研究把 Farmer.Chat 作物咨询拆为图像质量、作物识别和病虫害诊断三阶段，对比单 VLM 与专用模型，并发现小型分层视觉模型可超过生产基线。

Details

Motivation: 小农户常只发一张田间照片，缺少症状描述、作物名称甚至文字；照片可能低光、抖动且由低成本手机拍摄。现有生产系统难调整质量阈值、扩展作物或问题类别，也没有置信度拒答界面。

Result: 作者分析来自埃塞俄比亚、印度、肯尼亚和尼日利亚约 116 万张 Farmer.Chat 照片：现生产质量门拒绝其判定样本中的 46.8%，超过四分之一进入诊断流程的照片未返回作物名，标为“病害”的问题中 35.8% 实为可不依赖作物判定的害虫。系统拆成质量门 M0、作物检测 M1 和病虫诊断 M2，比较一次调用的 Qwen3-VL-4B 和 DaViT/YOLO 专家路线。MobileNetV3 质量门以 12ms 达 86.9% F1；同一测试集上 DaViT-Base 作物准确率 95.41%，高于生产基线 91.46%，诊断表现也领先且从不弃答。微调 VLM 的优势是一次调用完成三阶段，并能在图像不足时请求重拍。

Insight: 实际农技咨询应将照片可用性、作物身份和病虫类别分开评估；模块化路线更准且可配置，而统一 VLM 的自然交互及主动请求补充照片仍有价值。

Abstract: Farmer.Chat is Digital Green’s farm advisory service for smallholder farmers. When something looks wrong with a crop, the farmer takes a photograph and sends it, and that photograph is the whole question: no symptom described, no crop named, often no text at all. The service has to determine whether the picture can be used, what crop it shows, and what is wrong with it, from images taken on cheap phones in a field, in poor light and with a moving camera. The system doing this today cannot be adjusted. It has no adjustable thresholds for photograph rejection, crops and problems cannot be added, and there is no confidence cut-off to set. We study about 1.16 million photographs sent to Farmer.Chat from Ethiopia, India, Kenya and Nigeria. The production quality gate rejected 46.8% of the images it judged, over a quarter of those reaching diagnosis returned no crop name, and 35.8% of the labelled problems filed under “disease” are pests, identifiable without the crop. We therefore split the work into three stages: a quality gate (M0), a crop detector (M1), and a disease or pest detector (M2). Route A fills all three with one fine-tuned vision-language model (Qwen3-VL-4B) answering in a single call. Route B fills each with a small specialist model (DaViT, YOLO26). We replace our production GPT-4o quality gate with a small MobileNetV3 gate at 86.9% F1 in 12 ms. On one test set scored the same way for every system, a hierarchical DaViT-Base achieves 95.41% crop accuracy against 91.46% for the production baseline. It also leads on diagnosis and never declines to answer, while every language model in the comparison leaves a large share of rows with no diagnosis. The fine-tuned model retains two capabilities the specialists do not have: one call for all three stages, and a request for a better photograph when the image cannot support an answer.

[36] DRT: Dense Reasoning Trace for Efficient and Grounded Multimodal Reasoning cs.CVPDF

Wan Xu, Yuanfan Guo, Kevin Han, LaLa Chen, Wangmeng Zuo

TL;DR: DRT 用简洁结构化中间轨迹替代冗长自然语言思维链，将视觉观察和逻辑推导分离，并通过参考轨迹核验和结构化 GRPO 降低幻觉。

Details

Motivation: 多模态大模型的思维链仍受限于自然语言表达，产生大量语言开销、稀释信息并削弱视觉接地。

Result: Dense Reasoning Trace 用含简洁中间状态和符号连接器的紧凑结构轨迹表示推理，并显式区分视觉观察与逻辑推断。密集轨迹初始化使模型内化该模式，提高 token 效率并保留视觉证据；轨迹接地强化学习用三视角核验生成参考轨迹，再以结构化奖励的 Trace-Grounded GRPO 鼓励简洁且逻辑接地的推理。在具挑战推理基准上，相较 Qwen3-VL 基线，token 效率提高 5.5 倍、准确率提高 1.3 点。

Insight: 复杂多模态推理不一定需要长篇解释；明确分开观测和推理、用结构连接器表达关系，可减少语言冗余并增强视觉证据约束。

Abstract: Despite the remarkable progress in Multimodal Large Language Models (MLLMs), prevailing Chain-of-Thought (CoT) paradigms remain confined to the natural-language expression space. Consequently, they inherently incur excessive linguistic overhead, leading to information dilution and weak visual grounding. To address this challenge, we propose Dense Reasoning Trace (DRT), a paradigm that departs from natural-language-centered CoT by expressing reasoning as compact structured traces, which include concise intermediate states with symbolic connectors and disentangle visual observations from logical deductions. First, we introduce the Dense Trace Initialization to internalize the DRT reasoning mode into the model, substantially improving token efficiency while preserving visual evidence. To further enable the model to faithfully capture the logical relations within traces, we propose the Trace-Grounded Reinforcement Learning framework, which builds reference traces through a tri-perspective verification pipeline and employs Trace-Grounded GRPO with structured rewards, encouraging the model to generate concise DRT-style traces with reduced hallucination and stronger logical grounding. Extensive experiments on challenging reasoning benchmarks show that DRT achieves 5.5$\times$ token efficiency improvement while improving 1.3 accuracy points over the Qwen3-VL baseline. These findings suggest that complex multimodal reasoning may not require verbose natural-language traces, opening a more efficient path for next-generation MLLMs. Our code and data are available at: https://github.com/HIT-leaderone/DRT

[37] ZYT-World: A Real-Time Controllable World Model for Closed-Loop Autonomous-Driving Simulation cs.CVPDF

Boni Hu, Xiong Wei, Haoming Huang, Yong Huang, Chenbo Wang

TL;DR: ZYT-World 以单步流式生成器实时模拟混合鱼眼和针孔相机驾驶视图，并加入跨轨迹隐式记忆保持重访地点特征。

Details

Motivation: 生产级闭环驾驶模拟需在原生分辨率复现混合相机阵列，同时兼顾逐时刻因果交互、长时域稳定、低延迟和重复地点场景身份保持。

Result: ZYT-World 原生生成四路视场超过 180° 鱼眼视图和三路针孔视图，用投影专属 Plücker 适配器编码相机几何，ego-motion 自适应归一化控制整体运动，像素对齐布局用实例框、朝向和颜色条件化交通参与者。异构训练结合完整相机几何覆盖与高分辨率细节，通过教师强制、因果一致性蒸馏、自 rollout 分布匹配和联合七视角 RigCritic，将 40 步双向教师变为单步逐潜变量流式生成器。19M 参数 TinyVAE、W8A8 量化和推理引擎分别降低解码、骨干和增量执行开销；真实跨轨迹配对训练的隐式记忆插件保持地点证据。内部测试中，单步模型保留教师 90% 以上 PSNR/SSIM，FID、FVD、LPIPS 与教师相差不超过 11%；仅生成器计时比教师快 107.7 倍。

Insight: 驾驶世界模型需同时保留相机阵列几何、因果实时生成与重访地点记忆；单步流式蒸馏和跨轨迹配对记忆分别面向速度与长期场景身份。

Abstract: Generative world models offer controllable and repeatable closed-loop simulation for end-to-end and vision-language-action driving policies, but production deployment exposes three unresolved requirements: faithfully reproducing a mixed fisheye-pinhole rig at native resolutions; reconciling causal, per-timestep interaction with long-horizon stability and low latency; and preserving scene identity when a location is revisited. We present ZYT-World, a single architecture that natively generates four fisheye views with field of view > 180° and three pinhole views. Projection-specific Plucker adapters encode camera geometry, ego-motion adaptive layer normalization provides global motion control, and a lightweight pixel-aligned layout conditions traffic participants and signals through instance-level boxes, headings and colors. Heterogeneous training combines full-rig geometric coverage with high-resolution detail. Teacher forcing, causal consistency distillation, self-rollout distribution matching distillation, and RigCritic transform a 40-step bidirectional teacher into a one-step, per-latent streaming generator, with RigCritic evaluating the seven-view rig jointly. A 19M-parameter variational autoencoder decoder (TinyVAE), W8A8 quantization, and our inference engine reduce decoding, backbone, and incremental-execution costs, respectively. Finally, cross-trajectory pairs derived from real captures train a plug-in implicit-memory module that preserves place-specific evidence. On the internal multi-view test set, the one-step model retains more than 90% of the teacher’s PSNR and SSIM, while FID, FVD, and LPIPS stay within 11% of the teacher. Under the generator-only timing in Figure 2, it is 107.7 times faster than the 40-step bidirectional teacher. TinyVAE decodes 59.8 times faster than Wan. 30s rollouts and cross-trajectory revisits show the intended long-horizon and memory behavior.

[38] Balanced Prompt Adaptation against Entropy-Induced Collapse for Test-Time Binary Segmentation cs.CV | cs.AIPDF

Zhengshan Wang, Joshua Charles Webster-Ford, Yifei Tian, Xinxin Wang, Long Chen

TL;DR: BAPA 通过前景背景等权的高置信锚点和动态文本提示更新，避免测试时二元分割的熵最小化被预测多数类主导并坍塌。

Details

Motivation: 熵最小化常用于测试时适配，但稠密二元分割有成千上万个像素，预测占比大的类别容易主导更新并吞并少数类，最终掩码退化且熵梯度消失。

Result: 作者在共享偏移模型中理论证明该坍塌，并提出平衡锚点提示适配 BAPA。类别平衡锚点从每个预测类别分别选高置信像素，令前景和背景总损失权重相同，防止大区域主导；动态提示适配则在每次更新后刷新锚点，仅优化文本侧提示残差并冻结视觉语言编码器，精化前景-背景边界。在四个领域实验中，BAPA 的平均 Dice 最高；分解消融验证两个模块互补。

Insight: 二元分割的测试时适配需要阻断预测类别面积比例对梯度的支配；平衡采样与只更新提示可在维持预训练稠密视觉表示时调整决策边界。

Abstract: Entropy minimization is a standard objective for test-time adaptation (TTA), but it can fail in imbalanced binary segmentation. Unlike image classification, dense segmentation aggregates thousands of pixel predictions, allowing the larger predicted class to dominate the update, pull minority predictions toward itself, and produce a degenerate mask as predictions saturate and their entropy gradients vanish. We theoretically establish this collapse in a shared-shift model. This analysis motivates Balanced-Anchor Prompt Adaptation (BAPA), which combines two complementary modules. The Class-Balanced Anchors (CBA) module selects high-confidence anchors separately from each predicted class and gives foreground and background equal total loss weight, preventing the larger region from dominating the update. Dynamic Prompt Adaptation (DPA) refreshes these anchors after each prediction update and optimizes only text-side prompt residuals while keeping the vision-language encoders frozen. This prompt-only update refines the foreground-background decision boundary without altering the pretrained dense visual representation. Across experiments from four domains, BAPA achieves the highest mean Dice among the evaluated methods. Factorized ablations further validate the complementary roles of CBA and DPA, supporting balanced prompt adaptation as an effective alternative to entropy minimization for test-time binary segmentation.

[39] Beyond Benchmark Scores: Auditing Medical Vision-Language Models for Chest X-Ray Tuberculosis Screening cs.CV | cs.LGPDF

Mushir Akhtar, M. Tanveer, Mohd. Arshad

TL;DR: 该审计用 12,200 张胸片、五类提示及四数据集检验医学 VLM 结核筛查排序、分数可靠性和阈值迁移，发现结论强烈依赖评估设定。

Details

Motivation: 单一基准分数不能证明医学模型换到不同队列、提示、阴性对照、患病率或阈值后仍可靠；结核筛查尤其需区分真实临床可迁移性与基准适配。

Result: 作者评估 BioMedCLIP、CheXficient、MedSigLIP 和通用 OpenCLIP，在 Montgomery、Shenzhen、TBX11K 和 VinDr-CXR 的 12,200 份记录上使用五类固定提示，形成 244,000 个模型-图像-提示分数。没有模型在所有队列和可靠性标准上领先；48 个多重校正比较中有 21 个因提示族变化导致 AUROC 改变。将健康对照换成其他疾病但无结核对照，四模型 AUROC 下降 0.075–0.306；在 VinDr-CXR 上，模型区分结核与肺炎或肺肿瘤明显差，疾病 AUROC 点估计低于 0.5。TBX11K 训练集按 95% 敏感度选的阈值，仅在 16 个目标评估中 4 个点估计保住该约束。监督源模型在外部队列 AUROC 从验证 0.999 降至 0.629。

Insight: 判别力、分数可靠性和阈值保留是不同可迁移性主张，必须完整披露测试队列和阈值来源；模型检查点本身不能证明具备临床筛查可用性。

Abstract: A medical model’s benchmark score does not establish that the same conclusion holds under a different evaluation. This study tests whether claims about model ranking, score reliability and screening performance survive changes in cohort, prompt, negative spectrum, specified prevalence and operating threshold. We audit three medical vision-language models (BioMedCLIP, CheXficient, and MedSigLIP) and a general-domain OpenCLIP comparator on 12,200 chest radiograph records from four datasets (Montgomery, Shenzhen, TBX11K, and VinDr-CXR). Five fixed prompt families yield 244,000 model–image–prompt scores. No model leads every cohort and reliability criterion. Prompt-family changes alter AUROC in 21 of 48 multiplicity-controlled comparisons. Replacing healthy controls with sick non-tuberculosis controls reduces AUROC by 0.075–0.306 across all four models. On VinDr-CXR, the three medical models distinguish tuberculosis from no-finding controls substantially better than from pneumonia or lung tumor; their AUROC point estimates for both named diseases fall below 0.5. CheXficient has documented VinDr-CXR pretraining exposure, which limits the interpretation of its results. Thresholds chosen for 95% sensitivity on TBX11K training retain that constraint by point estimate in only four of sixteen target evaluations. A five-seed supervised source model reaches 0.999 AUROC on TBX11K validation but 0.629 on each of two external cohorts. Conservative exclusion of perceptual-overlap candidates narrows this gap without closing it. These retrospective, single-task results show that discrimination, score reliability and threshold retention support different portability claims. Evidence for chest X-ray tuberculosis screening should identify the complete evaluation specification rather than attribute clinical portability to a checkpoint alone.

[40] XCalib Depth-Guided Geometric Optimization for Dense Thermal-Visible Video Registration cs.CVPDF

Aurelien Godet, Gabriel Jobert, Mauro Dalla Mura

TL;DR: XCalib 用虚拟针孔相机几何和单目度量深度约束热红外与可见光的稠密配准，并提出适用于跨光谱对齐的归一化边缘相关指标。

Details

Motivation: ADAS 中 RGB 与红外相机光轴和视场不同会产生非均匀视差和重影。基于关键点的方法只拟合全局单应性，无法处理动态深度；不受约束的稠密光流又缺乏结构正则且时序不稳定。

Result: XCalib 是无监督稠密热红外-可见光配准框架。虚拟针孔相机参数化不是绝对度量标定结果，而是几何约束空间；联合优化有效相对位姿、内参和预测的单目度量深度，将位移限制在物理有效投影几何内。作者提出跨光谱结构相似指标归一化边缘相关 NEC。多个公开 ADAS 数据集的定量和定性评估显示，其时间稳定性及对齐精度超过无约束稠密光流基线。

Insight: 把配准搜索限定在相机投影几何允许的位移范围内，可在全局单应性和自由光流之间取得折衷；作者强调方法是几何正则化，而非绝对标定工具。

Abstract: Image registration is a vital preprocessing step in multimodal perception tasks, including image fusion, object detection, and semantic segmentation. In Advanced Driver- Assistance Systems (ADAS), spatial misalignment between visible (RGB) and infrared (IR) cameras -caused by non-coincident optical axes and field-of-view differences- introduces non-uniform parallax and visual ghosting. Classical keypoint-based methods are restricted to global homographies that fail under dynamic depth, while unconstrained dense flow algorithms lack structural regularization and suffer from temporal instability. In this paper, we propose XCalib, an unsupervised dense thermal-visible registration framework that bridges this gap. Rather than serving as an absolute metric calibration tool, XCalib leverages virtual pinhole camera parameterization strictly as a geometric constraint space. By optimizing effective relative pose and intrinsics alongside predicted monocular metric depth, XCalib restricts the search space of spatial displacements to physically valid projection geometries. Our key contributions are: (1) a novel registration paradigm that uses camera parameterization as an implicit regularizer for dense cross-modal warping; (2) Normalized Edges Correlation (NEC), a robust structural similarity metric tailored to cross- spectral alignment; and (3) extensive quantitative and qualitative evaluations across public ADAS datasets, demonstrating superior temporal stability and alignment accuracy over unconstrained dense flow baselines.

[41] VideoReloc: Long-Term Indoor Video Relocalization against a Kilobyte-Scale Semantic Scene Graph cs.CVPDF

Qianru Li, Xuyang Chen, Xuqin Wang, Zhenghao Zhang, Hongyi Luo

TL;DR: VideoReloc 用自适应视频片段积累空间证据，并在片段级验证冲突位姿，以仅 100KB 的类别框地图实现长期室内重定位。

Details

Motivation: 长期室内重定位需在灯光和家具变化后根据紧凑语义地图恢复轨迹。视觉外观法在变化下不稳；逐帧依据对象类别和几何又会因证据稀疏、歧义而失败。

Result: VideoReloc 用里程计累积空间证据，直到满足对象和运动条件后形成自适应长度片段；在整段运行层面汇总相连片段证据，重新核验冲突放置。假设优先的注册从对象三元组提出位姿，并以片段级对象中心及框面验证；方向感知精化用框面、重力和墙体方向解决相机朝向歧义。该方法将重定位视为对时空延展视频查询的验证，只需 100KB 带类别边界框的地图。在 RIO10 和 ReplicaCAD 因果评估中，1m/10° 全帧成功率为 73.5% 和 61.1%；允许片段闭合时升至 90.6% 和 74.8%。

Insight: 紧凑地图的歧义可由视频时间上下文弥补；片段级证据积累和冲突复核比逐帧独立定位更适合长期环境变化。

Abstract: Given a compact semantic scene graph, long-term indoor video relocalization estimates a map-frame trajectory after lighting and furniture changes. Visual methods rely on appearance and become unreliable under these changes; localizing one frame at a time from object classes and geometry instead leaves sparse, ambiguous evidence. We introduce VideoReloc, whose adaptive clips use odometry to gather spatial evidence until object and motion criteria are met, adapting query length to the observed scene. Its run-level decision rechecks conflicting placements using evidence accumulated across connected clips, stabilizing the trajectory beyond adjacent-clip tracking. Hypothesis-first registration proposes poses from object triplets and verifies each using clip-wide object centers and box surfaces. Orientation-aware refinement uses box faces, gravity and wall directions to resolve ambiguity in camera orientation and refine the full pose. This reframes sparse-map relocalization as verification of spatially extended video queries, moving discriminative support from stored appearance to temporal context and permitting a 100 kB map of class-labelled boxes. On RIO10 and ReplicaCAD, the all-frame localization success rate at 1 m/10$^\circ$ is 73.5% and 61.1% under causal evaluation, rising to 90.6% and 74.8% with clip closure. The evaluated per-frame scene coordinate regressors reach up to 47.6% and 49.8%, respectively, with maps of 12.6-42 MB. Project page: https://videoreloc.github.io

[42] PRIME: Perception Feedback with Situational Memory Embeddings in VLA Models cs.CV | cs.ROPDF

Erik Deinzer, Naya Baslan, Luca Paparusso, Narunas Vaskevicius, Peter Knott

TL;DR: PRIME 将过往感知、推理、导航目标和预测行为压缩为情境记忆，并反馈给 VLA 感知查询，使感知关注当前决策相关线索。

Details

Motivation: 自动驾驶 VLA 多按感知、推理、规划的前馈流程运行。即使感知内部存在时间递归，早期视觉处理也常不知道后续推理和导航目标，无法利用先前决策优先关注线索。

Result: PRIME 用跨注意力汇总长度为 L 的历史窗口中感知、推理、导航目标及预测行为的潜在表征，形成情境记忆并条件化 VLA 感知查询，以较小计算开销实现意图驱动注意力。最多增加 2,970 万参数，为 73 亿基座模型的 0.41%。Bench2Drive 闭环基准上驾驶得分 82.47，比 ORION 高 4.73；成功率 60.00%，高 5.38 个百分点，并在使用 Think2Drive 示范训练的已发表 VLA 中取得最高驾驶得分。

Insight: 将下游计划和历史决策反馈给感知模块，可让图像编码更关注当前意图所需的证据，而不只是被动处理每帧。

Abstract: Current Vision-Language-Action (VLA) models for autonomous driving operate primarily through feedforward inference across the perception–reasoning–planning hierarchy. While modern architectures maintain temporal recurrence within the perceptual module, early perception remains blind to downstream reasoning and navigation goals, processing visual inputs agnostically without prioritizing cues informed by prior decisions. To bridge this gap, this paper introduces PRIME, a learned feedback mechanism that conditions the VLA perceptual queries on a novel Situational Memory. By aggregating latent representations of past perception, reasoning, navigation goals, and predicted behaviors across an L-step window via cross-attention, PRIME enables intent-driven perceptual attention at minimal computational cost, adding only a maximum of 29.7M parameters (0.41% of the 7.3B-parameter base model). Evaluated on the Bench2Drive closed-loop benchmark, PRIME achieves a state-of-the-art Driving Score of 82.47 (+4.73 over ORION) and a Success Rate of 60.00% (+5.38 percentage points), the highest reported Driving Score among published VLAs trained on Think2Drive demonstrations.

[43] Info3R: Information-Adaptive Test-Time Training for 3D Reconstruction cs.CVPDF

Sunghyun Baek, Hanna Bae, Minchan Kwon, Junmo Kim

TL;DR: Info3R 在长图像流三维重建中按帧信息量调节测试时状态更新，并在状态过度累积或置信变化时动态重置，缓解在线模型饱和。

Details

Motivation: Transformer 三维重建模型逐渐扩展到在线视频流，但长序列处理常忽略新帧的重要性差异及内部状态信息饱和，导致持续运行时性能退化。

Result: Info3R 提出信息感知状态更新，根据输入帧的冗余和信息量调整状态更新强度；动态状态重置则依据累计状态变化幅度和模型预测置信度触发，并通过锚点到世界对齐恢复状态可塑性。该方法在相机位姿估计、视频深度和三维重建任务上持续改进，并明显缓解长序列性能下降。在 KITTI Odometry 上，平均 ATE 为 LongStream 的 1/1.68。

Insight: 在线重建状态既不能无差别吸收每帧，也不能长期固定；根据帧信息和状态饱和程度调整更新、适时重置，有助于在长视频中持续整合新观测。

Abstract: Transformer-based models have recently achieved strong performance on 3D reconstruction from images, and recent works extend them to process video streams in an online manner for real-world deployment. However, existing methods overlook two key signals when handling long image streams: the importance of each incoming frame and the information saturation of the model’s internal state. In this paper, we propose Info3R, a novel information-adaptive test-time training method for the online 3D reconstruction. We introduce an information-aware state update that modulates the state update strength based on the redundancy and informativeness of each incoming frame. To restore the state’s plasticity – its capacity to incorporate new observations – we propose a dynamic state reset, triggered by the cumulative magnitude of state updates and the model’s prediction confidence and accompanied by an anchor-to-world alignment. Our method achieves consistent improvements on camera pose estimation, video depth estimation, and 3D reconstruction, while substantially mitigating the performance degradation in the long sequence evaluation. Notably, on KITTI Odometry, our method achieves on average 1.68x lower ATE than LongStream, demonstrating its robustness on extended outdoor sequences.

[44] Benchmarking the Explanatory Quality of Open-Weight Vision-Language Models in Face Recognition cs.CV | cs.AIPDF

Laurent Colbois, Sébastien Marcel

TL;DR: 该基准把开源 VLM 人脸验证的解释质量纳入核心评估，分别检查解释是否依赖稳定身份特征、是否忠实于可见图像内容。

Details

Motivation: VLM 可同时输出人脸相似度和自然语言解释，适合需要透明审计的人脸比较场景；但既有评测主要看识别准确率，没有量化解释是否相关、忠实或产生幻觉。

Result: 作者构建结构化解释输出协议，使解释可自动查询和审计，并提出两项标准：相关性要求使用身份稳定的面部特征，忠实性要求描述符合可见图像且不虚构特征。用该框架比较多个开源 VLM 家族的人脸验证准确率与解释质量，发现模型解释仍有明显不足。基准及开源评测工具旨在支持后续可解释人脸识别评估和微调。

Insight: 可解释人脸识别需同时评估“是否认对”和“解释是否依据稳定且真实的视觉证据”；自然语言理由本身不构成透明性保障。

Abstract: Vision-Language Models (VLMs) have recently been proposed as promising tools for face recognition, as they can produce natural language explanations alongside similarity scores. This capability is considered appealing for face comparisons in forensic contexts, which require decisions to be transparent and auditable. However, existing evaluations of VLMs for that use case focus mostly on recognition accuracy, while the validity of generated explanations remains unquantified. In this work, we introduce a benchmarking framework for VLM-based face recognition that treats explanation quality as a core evaluation axis. We propose two criteria that explanations should satisfy: relevance, i.e., reliance on identity-stable facial features; and faithfulness, i.e., alignment with the visible image content without hallucinated features. We jointly develop a methodology enabling the quantification of relevance and faithfulness of evaluated models, based on constraining model outputs to a structured explanation format that supports automated querying and auditing. Using this framework, we benchmark several families of open-weight VLMs, jointly evaluating face verification accuracy and explanation quality. Our results highlight remaining shortcomings of produced explanations, and emphasize the need for such explanation quality metrics to get a complete picture of model performance. The proposed benchmark and open-source evaluation harness provide a foundation for proper benchmarking and future fine-tuning of explainable face recognition systems.

[45] OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation cs.CVPDF

Wenxue Li, Peiyan Guan, Haoyang Jiang, Junxian Cai, Hualuo Liu

TL;DR: OmniVBench 覆盖七类、十八种细粒度参考到视频控制任务，并发布 34 万训练样本的 Omni-R2V 数据集，以因子级清单评估参考元素是否正确保留和路由。

Details

Motivation: 参考到视频生成正扩展到多类型参考和复杂组合，但既有基准类型有限、主要评整体一致性，未检查各参考因素是否被正确保留、解耦并绑定目标；训练资源也稀缺。

Result: OmniVBench 覆盖内容、运动、风格、结构、叙事及多参考等七个任务族、十八项细粒度任务，用 12,172 条案例专属清单检查参考因素是否忠实保留、正确解耦并绑定目标，以及是否按指令实现。Omni-R2V 数据集主要由大规模专业视频素材处理而成，含 34 万训练样本及多种参考组合，并配套任务特定参考-目标构造流程。先进开源和闭源模型评测显示各任务家族和评估维度仍有明显差距。

Insight: 整体视觉相似度不足以说明复杂参考控制成功；应逐因素检查目标内容、动作、风格及多参考关系是否按指令保留并正确作用。

Abstract: Reference-to-video (R2V) generation is evolving toward increasingly general and versatile reference control, giving rise to the emerging paradigm of omni R2V generation. However, existing benchmarks fall short of these emerging capabilities: their test cases cover limited reference types and compositions, and their evaluation protocols largely assess holistic reference consistency, overlooking whether reference factors are properly preserved, disentangled, and routed. Meanwhile, the high cost of constructing omni R2V training data makes suitable training resources scarce. To address these gaps, we introduce OmniVBench and the Omni-R2V Dataset for evaluating and training omni R2V models. OmniVBench expands R2V evaluation across broader reference types, fine-grained control tasks, and richer reference compositions, covering 7 task families and 18 fine-grained tasks spanning content, motion, style, structure, narrative, and multi-reference settings. We introduce factor-grounded evaluation with 12,172 case-specific checklist items, assessing whether intended reference factors are faithfully preserved, correctly disentangled and bound to their targets, and properly realized according to the instruction. We further introduce the Omni-R2V Dataset, bringing industrial-grade training resources for diverse R2V tasks to the broader research community. Drawing primarily on a large-scale corpus of professional video footage, it comprises 340K processed training samples spanning diverse reference types and multi-reference compositions. We develop task-specific pipelines for reference-target pair construction, offering a practical and scalable recipe for omni R2V data construction. Extensive evaluation of advanced open- and closed-source R2V models reveals clear performance gaps across task families and evaluation dimensions on OmniVBench, highlighting remaining limitations of current R2V models.

[46] MintAct: A Unified Visual Agent for Digital Environments cs.CVPDF

Mingfei Gao, Rui Tian, Haiming Gang, Bohan Zhai, Le Zhang

TL;DR: MintAct 提供 2B、4B、8B 视觉语言模型统一处理界面接地、跨设备多步导航和视觉工具使用，并在 OSWorld-Verified 达到 48.9。

Details

Motivation: 数字环境智能体需要识别界面元素、执行跨移动端/桌面/网页的多步操作并调用视觉工具；分别训练各域专用模型会造成能力割裂。

Result: MintAct 将界面接地、多步导航和视觉工具使用统一在一系列 VLM 中，并精心设计环境、数据与训练流程，使其在这些能力上达到领域专用模型水平。为训练构建可扩展环境和 RL 基础设施，可在异构后端并发托管数百实例以收集轨迹和在线强化学习；异步训练框架显式控制跨域数据分布，并适应噪声反馈和离策略偏移。多个基准评测中，该模型在相近规模模型中达到先进表现，OSWorld-Verified 得分 48.9。

Insight: 统一数字环境智能体依赖跨域环境和数据分布控制，而不只是扩大模型；异步 RL 基础设施使同一模型能从多种界面交互中共同学习。

Abstract: We present MintAct, a family of vision-language models that unifies UI grounding, multi-step navigation across mobile, desktop, and web, and visual tool use, trained at 2B, 4B, and 8B scales. Through careful design of our environments, data, and training recipes, MintAct models match the performance of per-domain specialists across all of these capabilities. To enable this, we develop a scalable environment and reinforcement learning (RL) infrastructure. On the environment side, we host hundreds of concurrent instances across heterogeneous per-domain backends, serving both trajectory data collection and online RL. To enable efficient and scalable RL training, an asynchronous framework keeps explicit control over the cross-domain training distribution and remains stable under noisy environment feedback and off-policy drift. Experimental results show that MintAct achieves state-of-the-art performance (48.9 on OSWorld-Verified) across a wide range of benchmarks at comparable model sizes.

cs.CL [Back]

[47] HERMES: Contrast-Aware Knowledge Graph Reasoning from Clinical Notes for Patient Outcome Prediction cs.CL | cs.LGPDF

Gia-Bach Nguyen, Hoang-Ha Nguyen, Tuan-Cuong Vuong, Trang Mai Xuan, Duy Quoc Ngo

TL;DR: HERMES 从临床文本构造带时间关系和治疗变化的个性化知识图，再用图注意力网络预测住院死亡与 30 天再入院风险。

Details

Motivation: 临床预测通常依赖结构化病历时间序列和操作编码。近期方法开始使用非结构化临床记录，但将其编码为平面序列可能丢失叙述中的关系和时间结构。

Result: HERMES 只使用临床文本，通过大语言模型引导抽取个性化知识图，并以对比逻辑建模显式捕捉时间动态、治疗失败和结局变化；再用图注意力网络从图结构中汇总患者表示。在 MIMIC-III 和 MIMIC-IV 的院内死亡与 30 天再入院预测实验中，HERMES 持续超过强文本单模态基线。

Insight: 显式临床关系和时间建模能保留平面文本编码容易丢失的患者病程结构；治疗失败和变化也可作为预测信息，而不只是背景文本。

Abstract: Clinical predictive models often rely on structured Electronic Health Record data, such as time-series and procedure codes. While recent approaches have begun leveraging unstructured clinical notes, they typically encode them as flat sequences, which may lose explicit relational and temporal structure present in clinical narratives. In response, we propose HERMES, a graph-based framework that operates exclusively on clinical text while preserving clinical relationships. This approach builds on two key ideas. First, personalized Knowledge Graphs (KGs) are constructed through Large-Language-Model-guided extraction from clinical notes with Contrastive Logic Modeling that explicitly captures temporal dynamics and treatment failures and changes in outcomes. Second, a Graph Attention Network synthesizes patient representations through graph-based learning over the KGs. Experiments on MIMIC-III and MIMIC-IV for in-hospital mortality and 30-day readmission prediction show that HERMES consistently outperforms strong text-only baselines. Our findings demonstrate that explicit relational modeling with Contrastive Logic Modeling significantly advances predictive performance.

[48] Reviser: Revision-Capable Text Generation via Autoregressive Cursor Actions cs.CLPDF

Sean Diab

TL;DR: Reviser 将文本生成建模为对可修改画布执行插入、光标移动或停止的自回归动作序列，允许非单调生成和回头修订。

Details

Motivation: 能插入或修改早期内容的生成方式更灵活，但许多非自回归或编辑式方法需多轮整序列计算；固定从左到右解码则不能回改早期选择。

Result: Reviser 是仅解码器 Transformer，每步预测一个 INSERT(token)、MOVE(Δ) 或 STOP 动作，对编辑历史而非最终文本顺序自回归。延续任务的 arena 评估中，它明显优于 SEDD 和 MDLM；生成轨迹统计确认模型频繁后退和在画布中间插入，而非只是模拟末尾追加。与同规模自回归基线相比，100M 和 300M 规模表现具竞争力；在共同 FLOPs 统计口径下，推理计算也低于代表性多轮精炼和扩散基线。

Insight: 把生成顺序与文本最终排列解耦，并将光标编辑纳入下一动作预测，可实现真正的非单调生成，同时维持简单统一的动作接口。

Abstract: Revision-capable generation is appealing because it can insert or revise earlier content, but many non-autoregressive and edit-based approaches obtain this flexibility through repeated sequence-level computation. We propose Reviser, a decoder-only Transformer that generates a response as a sequence of cursor-relative actions on a mutable canvas. At each step, Reviser predicts exactly one action token: INSERT(token), MOVE($Δ$), or STOP, and is autoregressive over edit-history actions rather than final text order. This design enables genuinely non-monotonic generation while preserving a simple next-action interface. On a continuation benchmark, Reviser is strongly preferred to SEDD and MDLM in our arena evaluations, and trajectory statistics confirm that the model performs frequent backward moves and mid-canvas insertions rather than merely emulating end-append decoding. Against size-matched autoregressive baselines, Reviser is competitive at both the 100M and 300M scales. Under our shared FLOPs convention, Reviser also requires substantially less inference compute than representative multi-pass refinement and diffusion-style baselines.

[49] A Generative Grammar Underlying the Voynich Manuscript, the Pastiche Hypothesis: Evidence from Large Language Models cs.CLPDF

Nicolas Turenne

TL;DR: 这项跨学科研究结合概率语法、语音分解和图文分析，认为伏尼契手稿符号呈字母式结构，整体可能是结合自然语言规律与草药知识的仿作系统。

Details

Motivation: 伏尼契手稿以未解读文字书写，文本统计似自然语言，植物插图又类似中世纪草药书；研究尝试以新转写语料结合定量建模和多模态分析考察其结构来源。

Result: 作者用位置相关概率文法建模词和字母分布，比较印欧、闪米特和亚洲语言的语音模式，并通过大语言模型图文对齐寻找植物对应。结果显示符号更像字母而非音节单元，词长分布则接近音节结构；语音模式更接近希伯来语或阿拉伯语等辅音密集语言。模型复现 Zipf 分布并发现重复词首字母序列概率极低，支持对自然语言的结构化模仿；植物图像与地中海传统的 Pseudo-Apuleius 草药书存在较强对应。

Insight: 综合文本概率结构和插图传统，作者支持一种兼具语言规则与草药知识的生成系统假说；这些结果属于证据支持的解释，并未解密手稿内容。

Abstract: Background: The Voynich Manuscript is a fifteenth-century codex written in an unknown script whose content remains undeciphered. Previous studies suggest that its statistical properties resemble those of natural languages, while its illustrations - primarily plants - recall medieval herbals. Methods: We present a multidisciplinary analysis combining probabilistic modeling, phonetic decomposition, rare-event detection, and multimodal image analysis, based on a newly transliterated corpus. Word- and letter-level distributions are modeled using position-dependent probabilistic grammars, while phonetic patterns are compared across Indo-European, Semitic, and Asian languages. Image-text alignment methods based on large language models are applied to identify potential botanical correspondences. Results: The results indicate that Voynich symbols behave as letters rather than syllabic units, while word-length distributions resemble syllabic structures. Phonetic analyses show closer alignment with consonant-heavy languages such as Hebrew or Arabic than with Indo-European languages. Probabilistic modeling reproduces Zipf-like distributions and reveals extremely low probabilities for repeated initial-letter sequences, indicating a structured imitation of natural language. Image analysis suggests strong correspondences between Voynich plant illustrations and those found in Pseudo-Apuleius herbals from the Mediterranean tradition, consistent with an imitation of medieval medicinal books. Perspectives: These findings support the hypothesis that the Voynich Manuscript follows a structured generative system combining linguistic regularities and herbal knowledge, and demonstrate the value of integrating probabilistic and AI-assisted approaches in the analysis of historical manuscripts.

[50] COAL-SQL: Coverage-Guided Augmentation and Failure-Driven Learning for Text-to-SQL Post-Training cs.CL | cs.DBPDF

Qifeng Cai, Xuanguang Pan, Hao Liang, Chang Xu, Wentao Zhang

TL;DR: COAL-SQL 将 SQL 结构覆盖引导的数据增强与失败驱动训练结合，在有限后训练样例下提升文本到 SQL 的执行准确率。

Details

Motivation: 开源大语言模型生成复杂真实 SQL 需要任务后训练，但现有数据集未覆盖所有 SQL 结构，增强方法也少识别具体结构缺口；单独 SFT 或 RL 又难动态回应训练时暴露的弱点。

Result: COAL-SQL 包含覆盖引导增强 CGA 和失败驱动学习 FDL。CGA 用贪心选择找出原数据缺少的 SQL 结构并生成互补示例。FDL 保留 GRPO 为主要优化目标，同时对累积未解样例提供针对性监督：步骤级使用强 LLM 对已验证推理轨迹做 SFT，轮次级根据失败检索结构相关样例进行练习。仅使用 12,600 个不同后训练样例，模型在 BIRD 开发集执行准确率达 64.9%，超过相近规模训练基线。

Insight: 后训练数据既要补全能力结构覆盖，又要根据具体失败动态补练；将可验证推理轨迹与结构相似例结合，使监督直接对应模型当前短板。

Abstract: Text-to-SQL translates natural-language questions into executable SQL queries, but open-source large language models still require task-specific post-training for complex, real-world SQL generation. Effective post-training requires both training data that cover the capabilities demanded by the target task and a learning strategy that enables the model to acquire them. Existing datasets provide valuable supervision but incompletely cover SQL structures, while augmentation methods typically expand data without identifying structural gaps. Moreover, supervised fine-tuning (SFT) or reinforcement learning (RL) alone cannot dynamically address weaknesses exposed during training. We propose COAL-SQL, a unified framework combining Coverage-Guided Augmentation (CGA) and Failure-Driven Learning (FDL). CGA uses greedy selection to identify SQL structures missing from the original dataset and constructs complementary examples, improving structural coverage. FDL retains GRPO as the main optimization objective while supplying targeted supervision for unsolved examples. At the step level, it applies SFT to verified reasoning traces generated by a strong LLM for accumulated failures. At the epoch level, it retrieves structurally related examples based on accumulated failures to create targeted practice, helping the model acquire the corresponding SQL capabilities. With only 12,600 distinct post-training examples, COAL-SQL achieves 64.9% execution accuracy on the BIRD development set and outperforms baselines trained at comparable scale. The code is available at https://github.com/TechNomad-ds/COAL-SQL.

[51] PhysioBench: A Unified Benchmark for Physiological Signal Question Answering cs.CL | cs.LGPDF

Mengxuan Li, Junfa Chen, Jinze Xia, Yundan Chen, Lixin Fan

TL;DR: PhysioBench 将 22 个公开生理信号数据集整合成 30 种任务、6,140 万问答样例，用自然语言统一评测多类模型的生理信号理解。

Details

Motivation: 生理信号支持多种临床和监测任务，但基础模型通常需逐任务适配。自然语言可统一描述不同预测目标，当前模型能否跨信号模态遵循指令尚缺全面评估。

Result: PhysioBench 从 22 个公开数据集协调标注，构造 30 项任务、6,140 万问答；每对问答均对应具体信号片段并可追溯到来源标注。作者在三种互补设置下评估 21 个模型，包括大语言、视觉语言、时间序列语言和生理信号基础模型。没有模型能在不同信号模态和任务上持续保持强表现；自然语言支持统一预测接口，但性能对问题措辞敏感。

Insight: 统一语言接口并未自动消除不同生理信号和任务间的能力差距；样例级来源可追溯与问题措辞敏感性评估为后续模型分析提供基础。

Abstract: Physiological signals support diverse clinical and monitoring tasks, yet existing physiological signal foundation models typically require task-specific adaptation for each task. Natural language provides a common interface for specifying different prediction objectives, but the ability of current models to follow such instructions across physiological signal modalities remains insufficiently evaluated. To address this gap, we introduce PhysioBench, a unified benchmark for physiological signal question answering. PhysioBench harmonizes annotations from 22 public datasets into 61.4 million questions across 30 tasks. Each question-answer pair is grounded in a signal segment and traceable to its source annotation. We evaluate 21 representative models, including large language models, vision-language models, time-series language models, and physiological signal foundation models under three complementary settings. The results show that none of the evaluated models achieves consistently strong performance across physiological signal modalities and tasks. The incorporation of natural language supports unified prediction across tasks, although performance remains sensitive to question formulation. Beyond these findings, PhysioBench offers an extensible platform for fine-grained analysis and future research on physiological signal understanding. Our codes are available at https://github.com/Leanna97/PhysioBench.

[52] VISPATH: Visual-Intent-Guided Path Reasoning for Multimodal Knowledge Graph Question Answering cs.CL | cs.LGPDF

Jinke Wu, Zhengpin Li, Mengzhe Jia, Yang Li, Wentao Zhang

TL;DR: VISPATH 在多模态知识图谱问答的每一跳重新计算视觉意图，并按问题及当前路径状态扩展和剪枝证据链，避免多跳后退化为纯文本图搜索。

Details

Motivation: 现有多模态知识图谱问答通常只在起始实体定位或检索时使用图像，后续多跳推理退化成文本图搜索，无法利用中间步骤变得重要的多模态线索。

Result: VISPATH 先将多模态定位与图结构线索结合以确定起始实体；随后根据输入、问题和当前部分路径重新计算逐跳多模态意图，以引导路径发现。对候选路径按其与问题、推理草图及逐跳意图的一致性进行链级剪枝，并检查证据是否足以回答。作者构建 VISPATH-Bench，覆盖知识图谱上 2–4 跳问答。多项实验中持续优于强基线；以 GPT-4o 为骨干时，在该基准上平均准确率相对提升 10.6%，两跳推理提升 13.1%，超过 GPT-5.4 的对应结果。

Insight: 多模态证据不应只用于问题开头的实体链接；随着推理路径变化重新评估视觉意图，才能将视觉信息持续带入多跳知识推理。

Abstract: Knowledge graph question answering (KGQA) enables models to answer natural-language questions through structured graph reasoning and has achieved substantial progress across many benchmarks and applications. Recently, multimodal KGQA (MM-KGQA) has attracted increasing attention because many questions require jointly using multimodal inputs and KG evidence. However, existing MM-KGQA methods typically use multimodal information only for starting entity grounding or evidence retrieval, after which multi-hop reasoning degenerates into text-only graph search. As a result, they cannot exploit multimodal cues that become important at intermediate hops. To address this limitation, we propose VISPATH, a visual-intent-guided path reasoning framework for MM-KGQA. VISPATH first identifies a reliable starting entity by combining multimodal grounding with graph-structural cues. It then performs intent-guided path discovery by recomputing hop-specific multimodal intent from the input, question, and current partial paths, so that each expansion is guided by the current reasoning state. The discovered paths are further refined through reasoning-chain pruning, which evaluates candidate paths as complete evidence chains based on their consistency with the question, reasoning sketch, and hop-specific intent. Finally, VISPATH checks whether the selected evidence is sufficient for answer generation. We further construct VISPATH-Bench, a benchmark for evaluating multimodal multi-hop reasoning over KGs, covering questions that require two to four hops over KG paths. Extensive experiments on VISPATH-Bench and three additional multimodal QA benchmarks show that VISPATH consistently outperforms strong baselines. Notably, with GPT-4o as the backbone, VISPATH surpasses GPT-5.4 on VISPATH-Bench, achieving a 10.6% relative improvement in average accuracy and a 13.1% improvement at 2-hop reasoning.

[53] Reading Less While Writing: A Closed-Form Bandwidth Dial for Streaming Multimodal Decoders cs.CL | cs.LGPDF

Yasir Mehmood, Kashif Javed

TL;DR: ZENDAYA 用单一连续参数控制流式多模态解码器可见的输入前缀，使离线与实时生成成为同一调度族，并直接对应平均读取比例和延迟预算。

Details

Motivation: 传统视频或音频到文本解码器需读完整输入后才输出词，实时场景无法等待全片结束。wait-k 等固定规则虽能流式运行，却对每个输出词都等待相同 token 数，不适应输入长度和速度。

Result: ZENDAYA 以参数 γ 控制可见源前缀，使其按生成进度并依据模型预测的输入长度闭式增长；平均每个输出词消耗的源输入比例约为 1/(1+γ)，因此同一参数可解释为延迟和读取预算。作者证明，任意预先固定且单调非降的调度下，已输出 token 不会依赖尚未到达输入，该性质对任意权重及异步无界流都成立。从头训练的 29M 参数紧凑解码器覆盖两种模态、Charades-STA、ActivityNet Captions 和 LibriHeavy 三语料；读取更少源内容仍匹配或超过固定调度，低延迟时提升最大，三个语料的流式 METEOR 改善均显著。

Insight: 减少当前输出可见的输入有时反而改善文本，因为过多源信息会稀释模型对自身输出的注意力；单一连续旋钮同时提供可验证因果性和可解释的读取预算。

Abstract: A decoder that turns video or audio into text conventionally consumes the entire input before emitting a word. Offline this is merely more than the task requires; live it is impossible, since a caption cannot wait for a match to end. Streaming systems bolt on a fixed rule such as wait-$k$, which waits for the same number of input tokens before every word, regardless of the input’s length or pace. We replace the fixed offset with ZENDAYA, a schedule governed by a single continuous parameter $γ$. It makes the visible source prefix a closed-form function of generation progress, scaled by the input’s own predicted length, so an ordinary offline decoder and a real-time streaming decoder become two endpoints of one family rather than separate models. The same scalar fixes, in closed form, the mean fraction of source consumed per emitted word, $\bar{E}(γ) \approx 1/(1+γ)$, making it at once a latency dial and an interpretable budget. We prove a structural dependency theorem: under any schedule fixed in advance and non-decreasing, no emitted token can depend on input that has not yet arrived. The guarantee holds for trained and untrained weights alike, and extends to unbounded streams under arbitrary asynchronous arrival. The empirical result is counterintuitive: seeing less can produce better text, because a flood of source dilutes attention exactly when the model has the least of its own output to anchor on. Trained from scratch across two modalities and three public corpora (Charades-STA, ActivityNet Captions, LibriHeavy), a compact 29M-parameter decoder matches or beats the fixed schedule while reading less of the source, with the sharpest gains at the lowest latencies, where a fixed offset collapses. Streaming METEOR gains are statistically significant on all three corpora.

[54] Boosting Deepresearch and LongContext Ability with Self-Generated Deepresearch Rollouts Traces cs.CLPDF

Zihan Wang, Hao Wang, Boyuan Jiang, Yiqun Zhang, Shi Feng

TL;DR: DR-to-Long 把深度研究智能体的网页轨迹转换为长上下文问答，再以 LongQA-RL 中间训练后继续深度研究强化学习，提升研究和长文档证据整合表现。

Details

Motivation: 深度研究智能体多轮搜索和访问网页会快速拉长上下文。即使经深度研究强化学习，剩余错误中仍有 61.6% 与长上下文理解不足有关，包括长文幻觉和跨文档证据整合失败；仅增大上下文长度不足以解决训练数据缺口。

Result: DR-to-Long 将强化学习轨迹中的搜索历史、网页、证据片段和答案监督转为 LongQA 样例，把紧凑摘要替换为对应 URL 的完整网页内容，在不新增人工标注的情况下形成更长多文档上下文。DLD-RL 先做短期 DR-RL 收集轨迹，转为 LongQA 零标注样例，再经 LongQA-RL 加强长上下文能力，最后继续完整 DR-RL。实验相较标准 DR-RL，在三个深度研究基准提高 7.3%，在三个长上下文基准提高 13.5%。

Insight: 研究轨迹天然包含查询、来源和答案之间的证据关系，可再利用为长上下文训练数据；交替训练长文证据整合和端到端研究能力比单纯延长输入更有效。

Abstract: Deepresearch (DR) agents interact with real-world web environments through multi-turn search and visit, causing their contexts to grow rapidly over time. We observe that, even after DR Agentic Reinforcement Learning (DR-RL), 61.6% of the model’s remaining prediction errors can still be attributed to insufficient long-context understanding, including longcontext hallucination and failures in cross-document evidence integration. It motivates us to further break the bottleneck of DR-RL by strengthening the model’s long-context ability. However, effective LongContext training requires more than simply increasing context length. To bridge the data gap, we propose `DR Rollouts to LongContext-QA (DR-to-Long)’. The method repurposes DR-RL trajectories, which naturally contain search histories, visited webpages, evidence snippets, and final-answer supervision. It then replaces the compact snippets and webpage summaries in each trajectory with the full contents of their corresponding URLs, producing substantially longer multi-document contexts while preserving the original evidence relationships. Building on DR-to-Long, we introduce DLD (DR -> LongQA -> DR)-RL. DLD-RL first performs a short DR-RL stage to collect rollout trajectories, which are then converted into LongQA instances at zero annotation cost. The model is subsequently optimized with LongQA-RL to strengthen LongContext ability, followed by full DR-RL to continue improving its DR capability. Experiments show that DLD-RL outperforms standard DR-RL by 7.3% on three Deepresearch benchmarks and improves performance by 13.5% on three long-context benchmarks.

[55] Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Models cs.CL | cs.LGPDF

Polina Tsvilodub, Max Höth, Michael Franke, Björn Deiseroth, Carina Kauf

TL;DR: 研究将人类在不可回答问题上的有限推理努力作为原则，设计鼓励高效判断可答性的 GRPO 奖励，使推理模型更会拒答且思维链更短。

Details

Motivation: 大推理模型虽擅长许多任务的正确作答，却常不知道何时应拒答。与人类对照显示，人类在不可答任务上的思考投入不超过可答任务，而模型却在不可答提示上生成更长思维链，浪费计算资源。

Result: 作者提出受资源理性认知启发的新 GRPO 奖励，鼓励模型先判断问题是否提供足够信息。用该奖励微调多个 4B 推理模型后，平均拒答表现提高 12.8%，同时保留作答能力；平均 CoT 长度缩短 44%。

Insight: 遇到信息不足时，提升可靠性不应要求模型继续无尽推理，而应学会尽早识别可答性边界；高效推理奖励可同时改善拒答质量和 token 成本。

Abstract: While modern large reasoning models (LRMs) excel at providing correct answers in many tasks, we provide additional evidence for the observation that they often struggle with a critical capability: knowing when to abstain from answering. We analyze this gap by comparing LRM behavior to results from a human study, revealing that human reasoning effort on unanswerable tasks is upper-bounded by answerable tasks, whereas LRMs waste computational resources by generating longer Chains of Thought (CoTs) on unanswerable than on answerable prompts. To overcome this inefficiency, we take inspiration from a resource-rational perspective on human cognition and introduce a novel GRPO reward that encourages efficient reasoning about whether the task contains all the information needed to solve it. Fine-tuning several 4B LRMs with this reward leads to human-like abstention performance gains (+12.8% on average) while retaining answering capabilities and boosting the models’ efficiency (44% shorter CoTs on average).

[56] Enhancing Audio Reasoning via Semantic Summary Prediction cs.CL | cs.SD | eess.ASPDF

Francesco Bonzi, Pooneh Mousavi, Cem Subakan, Mirco Ravanelli

TL;DR: SPARE 通过在推理开始前将目标语义摘要注入音频模型潜在空间，缓解显式思维链使模型偏离音频输入的问题。

Details

Motivation: 大型音频语言模型可以处理复杂问答，但显式思维链有时会比直接回答准确率更低。作者假设长推理序列会让模型注意力偏离原始音频。

Result: SPARE 将注册 token 与最终结论的 Sentence-BERT 嵌入用余弦相似度损失对齐，在推理开始前以目标语义引导模型潜在状态。SALMONN 在 MMAU 和 MMAR 上的实验显示，该方法改善零样本推理，并使早期注意力更集中于音频输入，且不增加推理成本。

Insight: 用紧凑语义目标预先锚定推理方向，可能比追加长文本思维链更能保留对音频证据的关注；该方法在保持推理成本不变的同时提升了零样本效果。

Abstract: Large Audio Language Models (LALMs) perform well on complex question answering but often show a reasoning gap, where explicit Chain-of-Thought (CoT) reduces accuracy compared to direct answers. We hypothesize that long reasoning sequences shift attention away from the audio input. To address this, we propose SPARE (Semantic Prediction for Audio REasoning), which introduces a register token aligned with the final conclusion using a cosine similarity loss with a Sentence-BERT embedding. This conditions the model’s latent space with the target semantic goal before reasoning begins. Experiments on MMAU and MMAR with SALMONN show improved zero-shot reasoning and stronger early attention to audio without additional inference cost.

[57] MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs cs.CL | cs.CVPDF

Yueming Lyu, Yilian Shi, Haoxiang Tan, Linzhuang Zou, Qihao Wang

TL;DR: MME-Safety 用风险场景、伤害严重度和模态隐蔽性四维标注，并分层评估回应可靠性、实际风险暴露及防御结构完整性。

Details

Motivation: 多模态大语言模型的跨模态能力带来单模态安全过滤难覆盖的复杂漏洞。现有基准缺少细粒度意图相关标注，并依赖单一指标，难全面评估鲁棒性。

Result: 作者提出经过严格核验的 MME-Safety，使用四维标注体系刻画风险场景、伤害严重程度和各模态隐蔽方式，并以分层框架评估基础响应可靠性、实际风险暴露和防御行为结构完整性。对 17 个先进多模态模型的零样本评测形成全面安全画像，分析了跨模态输入配置及思维链推理所涉及的安全影响。

Insight: 多模态安全评测需要同时说明风险意图、严重度、隐蔽模态和防御行为是否完整；单一越狱成功率不能反映不同跨模态攻击路径及推理方式的风险。

Abstract: While Multimodal Large Language Models (MLLMs) show remarkable advancements, their cross-modal capabilities introduce complex vulnerabilities that easily bypass unimodal filters. Existing benchmarks lack fine-grained intent-related annotations and rely on unidimensional metrics, hindering comprehensive robustness evaluation. To address this, we propose MME-Safety, a rigorously verified benchmark featuring a unique four-dimensional annotation schema that categorizes risk scenarios, harm severity, and modality-specific stealth levels. Furthermore, we introduce a hierarchical evaluation framework to assess fundamental response reliability, actual risk exposure, and the structural integrity of defensive behaviors. Extensive zero-shot evaluations across 17 state-of-the-art MLLMs provide a comprehensive safety profile of current multimodal systems. Our analysis systematically investigates cross-modal input configurations and uncovers safety implications associated with Chain-of-Thought (CoT) reasoning. These multifaceted findings underscore the urgent need for robust, reasoning-aware safety alignment in the multimodal landscape.

[58] Hallucination-R1: Robustness-Oriented Paraphrase Generation for Factual Consistency cs.CL | cs.AIPDF

Wenhan Yu, Wenxin Wu, Hao Wang, Lei Sha

TL;DR: Hallucination-R1 生成语义等价但更能暴露事实不一致的改写问题，以两阶段优化平衡语义保真和挑战性，并可构造鲁棒微调数据。

Details

Motivation: 模型可能对原始事实问题答对，却在语义等价的改写下答错，暴露对语义不变性缺乏稳定性。普通改写要么与原题太相似、监督信号弱，要么变化过大、破坏语义等价。

Result: Hallucination-R1 分两阶段训练改写生成器：先稳定保持语义且具多样性的改写，再奖励能揭示下游问答模型事实一致性退化的改写。在 SimpleQuestions、PopQA 和 TruthfulQA 上，该方法获得较强一致性-多样性折衷，并在多个模型家族和数据集暴露鲁棒性问题。分析表明问题不只是表面词汇变化或语义漂移；轻量微调实验还显示，用其生成数据训练能提高改写条件下鲁棒准确率。

Insight: 鲁棒性改写需保持事实语义不变，同时足以挑战模型的事实表征；这种数据可用于发现潜在不稳定性，也能用于鲁棒训练。

Abstract: Factual hallucination is commonly defined by incorrect factual outputs. We study a paraphrase-induced hallucination setting, where a model answers a factual question correctly in its original form but generates an incorrect answer under a semantically equivalent paraphrase. Such inconsistencies expose latent factual instability under semantic invariance. However, general-purpose paraphrases are often insufficient as robustness-oriented supervision: near-copy paraphrases provide weak signals, while overly diverse paraphrases may break semantic equivalence. In this paper, we propose HALLUCINATION-R1, a robustness-oriented paraphrase generation framework that learns to produce semantically faithful yet robustness-challenging paraphrases for factual consistency. Through two-stage optimization, it first stabilizes meaning-preserving and diverse paraphrasing, then rewards paraphrases that reveal factual consistency degradation in downstream QA models. Experiments on SimpleQuestions, PopQA, and TruthfulQA show that HALLUCINATION-R1 achieves a strong consistency–diversity trade-off and exposes robustness failures across multiple model families and datasets. Further analyses indicate that these failures are not reducible to surface-level artifacts or semantic drift, but reveal non-trivial factual instability under meaning-preserving variation. A lightweight fine-tuning study also shows that HALLUCINATION-R1-generated data improves robust accuracy under paraphrase variations, suggesting its utility for robustness-oriented training. Our code and models are publicly available at https://github.com/yuwenhan07/Hallucination-R1.

[59] When Does Reasoning Help in Machine Translation? A Hierarchical Analysis of LRM Reasoning Traces cs.CLPDF

Yuxiang Liu, Jiaming Luo, Eleftheria Briakou, Colin Cherry

TL;DR: 研究跨模型、语言、领域和数据集分析机器翻译推理轨迹，并提出层级元摘要自动归纳推理结构，发现最佳推理语言和长度均依模型及任务而变。

Details

Motivation: 大推理模型越来越在机器翻译中生成中间推理轨迹，但何时推理有帮助或有害仍不清楚，也缺少对轨迹结构的系统分析。

Result: 作者研究推理语言、长度和结构，发现最佳推理语言因模型而异，推理长度与翻译质量呈非单调关系，轨迹中还反复出现功能模式。层级元摘要 HMS 在不预设分类体系的情况下诱导粗细粒度推理结构，发现共同组织包含理解/规划、翻译/起草、润色/验证，同时不同领域存在变化。

Insight: 机器翻译推理不适合统一规定输出语言或思维链长度；控制策略应依据模型、长度及任务中的功能模式分别调整。

Abstract: Large Reasoning Models increasingly use intermediate traces for machine translation, but it remains unclear when such reasoning helps or hurts. We analyze reasoning traces across models, languages, domains, and datasets, focusing on reasoning language, length, and structure. We find that the best reasoning language is model-specific, reasoning length has a non-monotonic relationship with quality, and traces exhibit recurring functional patterns. To uncover these patterns, we introduce Hierarchical Meta-Summarization (HMS), a scalable framework that induces coarse- and fine-grained reasoning structures without predefined taxonomies. HMS reveals a shared organization–understanding/planning, translating/drafting, and refining/verifying–alongside domain-specific variation. Our results suggest that MT reasoning should be controlled in a model-aware, length-aware, and pattern-aware manner rather than uniformly encouraged.

[60] ArenaFlow: From Trajectory Ranking to Hierarchical Credit Propagation for Open-Ended Agent RL cs.CLPDF

Qiang Zhang, Ruixue Ding, Fanrui Zhang, Xi Chen, Boli Chen

TL;DR: ArenaFlow 将开放式智能体强化学习中的锦标赛轨迹排序转化为关键步骤优势传播和技能效用更新，生成可复用策略先验。

Details

Motivation: 开放式智能体任务解法多样，可靠标量奖励难获取。成对比较可缓解奖励区分坍塌，但仍把丰富反馈压成整条轨迹分数，忽视关键中间步骤，也难将成功策略固化成可复用技能。

Result: ArenaFlow 用锦标赛相对排序构造轨迹级奖励，并为比较加入结构化反思评估，识别关键成功步骤、可复用策略技能及检索技能使用归因。步骤层面根据锦标赛存活深度把轨迹优势传播到高置信关键步骤，精准优化局部推理；技能层面从群体使用归因估计技能效用，维护全局技能记忆并按效用更新、剪枝和检索。高效技能还作为未来探索的策略先验。实验验证其在开放式智能体任务上的效果。

Insight: 轨迹排名提供可靠的相对奖励后，仍需把信用传回决定成败的具体步骤并沉淀有效技能；层级传播连接了整轨迹比较与局部可复用行为。

Abstract: Reinforcement learning has substantially improved large language model (LLM) agents in verifiable domains, but remains difficult to apply to open-ended agent tasks, where solutions are diverse and reliable scalar rewards are hard to obtain. Recent pairwise evaluation methods alleviate reward discrimination collapse by replacing pointwise scoring with relative preferences. However, they still compress rich comparative feedback into a single trajectory-level reward, obscuring decisive intermediate steps and preventing successful behaviors from being consolidated into reusable skills. We propose ArenaFlow, a hierarchical credit propagation framework for open-ended agent reinforcement learning. ArenaFlow leverages tournament-based relative ranking to derive trajectory-level reward signals. Each comparison is further equipped with structured reflective evaluation, which reveals three types of supervision: pivotal success steps, reusable strategy skills, and usage attribution of retrieved skills. At the step level, ArenaFlow propagates trajectory-level advantages to high-confidence pivotal steps according to tournament survival depth, enabling more targeted optimization of local reasoning behaviors. At the skill level, ArenaFlow estimates skill utility from group-level usage attribution and maintains a global skill memory through utility-aware updating, pruning, and retrieval. The resulting high-utility skills further serve as policy priors for future exploration. Extensive experiments validate ArenaFlow’s effectiveness on open-ended agent tasks.

[61] Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction cs.CL | cs.CV | cs.MM | cs.SDPDF

Qi Chen, Yunfei Chu, Haolin He, Yifan Yang, Zihan Liu

TL;DR: ODU-Bench 评测多模态模型能否在音视频与对话上下文中判断用户是否真的提出需求并推断意图，发现当前模型常漏掉上下文信息或误触发。

Details

Motivation: 自然音视频交互允许用户通过语音和视觉与助手沟通，但现有基准主要评回答质量，较少检查更基础的问题：模型是否先正确理解用户的真实需求。真实表达常不完整、含糊或受噪声影响，且类似请求的语句未必真的在向助手提出要求。

Result: 作者将全模态需求理解定义为从交互流中判断需求是否存在并结合多模态和对话上下文推断意图，覆盖单轮和多轮五个维度。ODU-Bench 由挑战分类、分类指导的视频生成、真人录制互动以及基于媒体的标注和人工核验构成。14 个原生 MLLM 中，即使最强 Gemini 3.1 Pro 也只恢复 44.7% 必须从视觉、音频或对话推断的关键信息；14 个模型中 11 个在非需求场景的误触发率高于 50%。

Insight: 助手在生成回答前必须先判断是否存在真实请求，并整合跨模态上下文；高误触发率说明仅优化回答质量不足以确保交互得体。

Abstract: Natural audio-visual interaction is emerging as an important interface for AI assistants, allowing users to communicate through speech and vision rather than carefully composed text prompts. However, existing benchmarks of interactive capabilities still focus primarily on response quality, leaving a more fundamental question underexplored: can a model correctly infer the user’s underlying demand from complex multimodal interaction? Real-world user demands are often underspecified in speech and must be inferred from multimodal cues and dialogue history. This inference is further complicated by ambiguous or disfluent expression and noisy acoustic environments. Conversely, request-like speech may not constitute a demand to the assistant, leading to false triggers. We establish Omni Demand Understanding (ODU) as a distinct multimodal contextual inference problem: given an interaction stream, a model must detect whether a user demand is present and infer intent from multimodal and conversational context. ODU evaluates this capability along five dimensions, covering both single-turn and multi-turn interactions. We construct ODU-Bench using a challenge-driven taxonomy, taxonomy-guided agentic video generation, and human-recorded interactions, followed by media-grounded annotation and human verification. We evaluate 14 native MLLMs. Even the strongest, Gemini 3.1 Pro, recovers only 44.7% of key information that must be inferred from visual, acoustic, or conversational context. Moreover, 11 of the 14 models exhibit false-trigger rates above 50% on non-demand scenarios. These results reveal a systematic capability gap in current MLLMs’ ability to infer contextual user demands. We hope ODU can establish the evaluation of a previously underexplored yet essential capability in multimodal interaction: correctly understanding user demands before generating an appropriate response.

[62] MIRAGE: Multi-Perspective Creative Language Model Reasoning with Reinforcement Learning Guidance cs.CLPDF

Arash Lagzian, Srinivas Anumasa, Dianbo Liu

TL;DR: MIRAGE 以视角选择器挑选有效概念路径、推理器依次求解并在必要时汇总多视角答案，提升数学、科学和逻辑任务的推理准确率。

Details

Motivation: 大型语言模型在复杂数学、科学与逻辑任务上仍有困难。人类可灵活切换代数、概率等思考视角，作者将这种认知灵活性用于推理时搜索。

Result: MIRAGE（多视角推理时智能体引导探索）由两部分组成：Selector 优先选择有效概念视角，Reasoner 依次尝试求解直至得到有信心的答案，否则综合多个视角。GSM8K、MATH500、MMLU-Pro 和 Game-of-24 实验中，方法持续超过 Chain-of-Thought 和多样化提示集成，且推理额外开销较小。

Insight: 将不同概念路径作为推理时的可选视角，可在单一路径失败时扩展搜索；信心足够时提前停止，有助在准确率和额外推理成本间折衷。

Abstract: Recent advances in Large Language Models (LLMs) have revolutionized artificial intelligence and how human interact with AIs. Despite impressive advancements, LLMs struggle with complex mathematical, scientific, and logical tasks. Inspired by human cognitive flexibility - our ability to dynamically switch mental perspectives - we propose MIRAGE (Multi-perspective Inference-time Reasoning via Agent-Guided Exploration), a novel inference-time creative thinking framework. MIRAGE includes a Selector that prioritizes effective conceptual perspectives (e.g., algebraic, probabilistic) and a Reasoner that sequentially solves tasks until a confident solution emerges, otherwise aggregating multiple perspectives. Tested on GSM8K, MATH500, MMLU-Pro, and Game-of-24 benchmarks, MIRAGE consistently outperforms methods like Chain-of-Thought and diverse prompting ensembles, significantly boosting accuracy with minimal inference overhead, providing a scalable solution for practical applications.

[63] When Steering Fails in Latent Reasoning: A Latent-to-Language Transition Gap cs.CL | cs.AIPDF

Gaoxiang Huang, Lei Qi

TL;DR: 研究发现对连续潜在思维进行激活引导，虽能改变潜在表征，却远不如对显式思维链的同等干预影响语言输出，提出潜在到语言的转移鸿沟。

Details

Motivation: 激活引导已被用于显式 CoT 推理控制，研究者尝试扩展到潜在 CoT；但模型在连续思维中的干预是否能传递到最终语言生成仍不明确。

Result: 作者发现，即使隐藏表征移动幅度相近，对连续思维的 steering 对后续语言输出作用仍显著弱于显式 CoT steering。任务信息仍可从连续思维中识别，因此提出“潜在到语言转移鸿沟”假说。另两项结果支持该解释：跨越潜在空间到语言的边界时输出分布骤变，且任务相关方向对潜在 CoT 的双向控制力远弱于显式 CoT。

Insight: 隐藏状态中存在可读出的任务信息，不代表操纵该信息就能稳定控制语言答案；潜在思维与语言解码之间的接口应成为未来 steering 方法的设计和评测重点。

Abstract: Activation steering has become a widely used approach for controlling language models during explicit chain-of-thought (CoT) reasoning, motivating its extension to latent CoT. However, we find that steering continuous thoughts produces substantially weaker effects on subsequent language generation than steering explicit CoT, even when the hidden representations are moved by comparable amounts. We first show that task information remains identifiable in continuous thoughts. Hence, we hypothesize a \textbf{latent-to-language transition gap}, in which an intervention effect in latent space fails to transfer to language generation. Two further results support this hypothesis: the output distribution changes abruptly at the transition boundary, and task-related directions exert much weaker bidirectional control in latent CoT than in explicit CoT. These findings identify the transition interface as a central target for evaluating and designing future latent-steering methods.

[64] TrialAtlas: Multi-Agent Research Organization for Clinical Trial Design and Optimization cs.CLPDF

Jiacheng Lin, Zifeng Wang, Zheng Chen, Erick Scott, Ziwei Yang

TL;DR: TrialAtlas 用多智能体协作分析临床试验设计、监管先例及开发风险，并以 FDA Complete Response Letters 构建评测基准。

Details

Motivation: 约九成进入临床开发的药物最终失败。临床开发规划需要临床科学、统计、监管和竞争情报专家联合收集证据并评估成功概率，过程费时且主观。

Result: TrialAtlas 是带记忆的多智能体研究组织，协调文献综合、竞争试验情报、监管先例分析及试验设计和风险整合推理，并从历史临床试验和监管结局（含 NDA）学习经验。TrialAtlasBench 从 291 份 FDA 完全回应函构建，覆盖试验设计缺陷检测、可操作改进建议和技术/监管成功预测。缺陷检测 F1 为 50.0%，高于最强基线 6.1 点；成功预测 balanced accuracy 为 85.3%、F1 为 84.7%，相较最佳基线分别提高 6.7 点和 12.0 个 Cohen κ 点。专家认为系统提出的问题中 86.4% 有效，高于 OpenAI DeepResearch 的 83.1% 和 Gemini DeepResearch 的 59.3%。

Insight: 临床开发决策需要跨领域证据和历史监管经验，单次摘要不足以支持规划；专门智能体分工与结果记忆有助于提供可审查的风险分析。

Abstract: Nearly 90% of drugs entering clinical development ultimately fail, despite billions of dollars in investment. Pharmaceutical companies therefore rely on clinical development planning (CDP) and probability of technical and regulatory success assessment to anticipate development risks, yet these decisions remain labor-intensive and subjective, requiring experts across clinical science, statistics, regulatory affairs, and competitive intelligence to jointly acquire, synthesize, and reason over heterogeneous evidence. Here, we introduce TrialAtlas, a memory-augmented multi-agent research organization for CDP that mirrors this collaborative process by coordinating specialized agents for literature synthesis, competitive trial intelligence, regulatory precedent analysis, and integrated reasoning over trial design and development risk. TrialAtlas further learns from historical clinical trials and regulatory outcomes, including prior New Drug Applications (NDAs), to ground its decisions in accumulated development experience. To evaluate these capabilities in an authentic regulatory setting, we introduce TrialAtlasBench, constructed from 291 FDA Complete Response Letters and spanning three practical tasks: detecting trial design deficiencies, recommending actionable design improvements, and predicting technical and regulatory success. TrialAtlas achieves an F1 score of 50.0% for deficiency detection, outperforming the strongest baseline by 6.1 points, and reaches 85.3% balanced accuracy and 84.7% F1 for prediction of technical and regulatory success, improving over the best baselines by 6.7 points in balanced accuracy and 12.0 points in Cohen’s kappa. In expert evaluation, 86.4% of TrialAtlas-generated concerns were judged valid, compared with 83.1% for OpenAI DeepResearch and 59.3% for Gemini DeepResearch.

[65] DiaVLo: Diagnosing Behaviours of Vision-Language Models cs.CL | cs.AIPDF

Lorenzo Corti, Jie Yang

TL;DR: DiaVLo 结合人工策划与 VLM 生成构造期望及实际行为规范，并估计影响模型行为的关键概念，用于诊断视觉语言模型对齐问题。

Details

Motivation: 视觉语言模型需在可靠部署中展现期望行为并避免有害行为，但现有识别模型行为的方法仍少，难定位模型如何组织和优先考虑概念。

Result: DiaVLo 通过人工策划和 VLM 生成能力构建模型期望行为与观测行为的规范，发现潜在失配；还提供因果估计以识别最影响行为的概念。在多个开源 VLM 的分类和生成设置中，行为标签与模型性能相关，并为量化表现提供语境。框架识别出明显对齐及失配行为，也发现模型感知、组织和优先级安排概念的模式。

Insight: 仅观察最终正确率难以说明模型遵循什么行为规范；把期望和实际行为明确对照，并估计关键概念因果影响，可提供更具诊断性的对齐分析。

Abstract: Vision-language models (VLMs) rely on storing and transferring appropriate information across their sub-components. Verifying that the VLMs exhibit desired behaviours, while avoiding harmful ones, is central to their reliable deployment. Yet, methods that identify VLM behaviours remain scarce. We present DiaVLo, a diagnostic framework that leverages human curation and VLMs’ generation capabilities to construct specifications of desired and observed VLM behaviours, surfacing potential misalignments. Beyond this, DiaVLo also provides causal estimates to identify the most influential concepts steering VLM behaviours. We evaluate DiaVLo on several open-source VLMs under both classification and generation conditions. Our experiments show that DiaVLo produces behaviour labels that correlate with model performance and provide context for measured performance. DiaVLo surfaced behaviours that are clearly aligned and misaligned, alongside patterns in how VLMs perceive, organise, and prioritise concepts.

cs.AI [Back]

[66] Offline Multimodal Large Language Models for Decision Support in Air Operations cs.AI | cs.CLPDF

Joao P. A. Dantas, Jelton A. Cunha, Gabriel Dietzsch

TL;DR: 论文介绍可在隔离环境运行的多模态检索增强决策支持系统，支持依据原始来源可追溯的文本和图像手册问答，并进行小规模空军分析员试点。

Details

Motivation: 空中行动分析需要在网络受限和严格安全环境中结合复杂条令与图像，分析任务时间紧，而研究人员常无法访问外部计算资源。

Result: 作者描述一套无需互联网的模块化 RAG 架构，处理技术手册文本和图像，并让自然语言交互可追溯至原始条令来源。巴西空军四名图像分析员参加试点，包含电子目标识别条令知识测验及无 AI 手工制作侦察目标报告的认知负荷评估。手动任务精神需求为 6/7、努力为 5/7；系统知识测验得分与人类同为 8/10，但用时 7.1 分钟，人类平均 26.5 分钟。作者将其定位为后续人工和 AI 辅助流程系统比较的基线。

Insight: 离线部署和来源可追溯对高安全环境至关重要；该试点显示耗时节省潜力，但样本只有四名分析员，仍需更全面的流程评估。

Abstract: Air operations rely on complex rules, established procedures, and time-critical analysis under limited connectivity and strict security constraints. In such environments, analysts must combine written doctrine with images, often without access to external computing resources. This paper studies offline large language models as decision support tools, deployed in isolated and restricted environments to give analysts access to doctrinal knowledge that remains traceable to its original sources through natural language interaction. We describe a modular retrieval-augmented architecture suitable for operation without Internet connectivity, supporting both text and image input from technical manuals. As a first step toward evaluating this architecture, we report a pilot study with four image analysts of the Brazilian Air Force, combining (i) a doctrinal knowledge assessment based on their electronic-target identification doctrine, comparing human and proposed system performance on the same test, and (ii) a measurement of the cognitive workload involved in manually producing a reconnaissance target report (Relatório de Missão de Reconhecimento - REMIR) without AI assistance. The results show a demanding manual task, especially in terms of mental demand (6.0/7) and effort (5.0/7), while the proposed system matches the human score (8/10) and completes the assessment in 7.1 minutes (compared to a human average of 26.5 minutes), establishing a baseline for future AI-assisted evaluation. Finally, we describe a future evaluation protocol to systematically compare manual and AI-assisted workflows.

[67] MIST: Multimodal Survival Prediction with Genomic-Guided Histology Attention cs.AI | cs.CVPDF

Muhammet Sami Yavuz, Sabri Mustafa Kahya, Richard R. Chen, Jana Lipkova, Benedikt Wiestler

TL;DR: MIST 让基因组 token 查询紧凑病理图像上下文，在生存预测前用组织学信息丰富分子表示，并以特征遮罩、切片 dropout 和对比对齐训练。

Details

Motivation: 全切片图像和基因组谱包含互补预后信息，但外部队列域移位及计算复杂度使多模态融合困难；只在最终阶段拼接两种模态可能错过跨模态交互。

Result: MIST 将基因组特征表示为 token，让其在生存预测前查询由基础模型提取的紧凑组织学上下文。训练结合离散时间生存目标、基因组特征遮罩、全切片图像 dropout 和配对图像-基因组对比对齐。在结肠、肾、肺和胶质母细胞瘤四个外部评估中，主要比较下 MIST 的外部 C-index 优于标准融合基线。

Insight: 先用形态学上下文丰富分子表示，比在两类特征分别编码后末端拼接更主动地利用跨模态联系；紧凑病理 token 也控制了全切片计算量。

Abstract: Multimodal survival models can combine complementary prognostic information from whole-slide images and genomic profiles, but effective fusion remains challenging amid external cohort shift and computational complexity. To address these challenges, we propose MIST, multimodal survival prediction with genomic-guided histology attention. MIST represents genomic features as tokens and allows them to query compact foundation-model-derived histology context tokens before survival prediction. This design enriches molecular information with histology context rather than merging separately encoded modalities only at the final stage. Training combines discrete-time survival prediction with genomic feature masking, WSI dropout, and paired WSI-genomics contrastive alignment. Across four external evaluations in colon, renal, lung, and glioblastoma cohorts, MIST improves external C-index over standard fusion baselines in the primary comparisons. These results support genomic-guided histology attention as a compact and effective strategy for multimodal oncology outcome prediction. Our code is available at https://github.com/samiyavuuz/MIST .

cs.LG [Back]

[68] BI-Agent and BI-Bench: Towards Automating End-to-End Business Intelligence cs.LG | cs.AI | cs.CL | cs.DBPDF

Chuxuan Hu, Yeye He, Penny Zhou, Wee Hyong Tok, Daniel Kang

TL;DR: BI-Agent 将商业智能流程拆成表搜索、连接和转换等子任务并调用专门工具；BI-Bench 显示前沿模型端到端准确率不足一半，工具增强和领域后训练可大幅改善。

Details

Motivation: Power BI、Tableau 等业务分析流程要求用户先找相关表、转换数据、建立连接，再回答业务问题，准备过程复杂耗时。作者评测 LLM 能否端到端免除这些手工步骤。

Result: 作者从公开真实 BI 项目收集数据，并从用户仪表盘人工抽取问题及真值答案，构成首个系统评测端到端 BI 能力的 BI-Bench。即使前沿模型准确率也低于 50%。工具增强 BI-Agent 将流程拆为结构化数据搜索、连接和转换等子任务，并协调专门数据管理方法；另从真实项目合成训练轨迹，使用 SFT 和 RL 后训练。BI-Agent 相较原生 LLM 准确率最高提升 40 个百分点，后训练版本再取得最高 30 点增益。

Insight: 复杂 BI 不是单次生成答案，而是跨表检索、数据准备和分析的工具工作流；工具增强推理与真实项目轨迹后训练需要结合。

Abstract: Business intelligence (BI) is a cornerstone of enterprise decision-making and is widely used by enterprise users in software such as Power BI and Tableau. In traditional BI workflows, users need to prepare data by (1) identifying relevant tables, (2) performing data transformations, and (3) building join relationships, before they can (4) answer their business questions. These steps can be complex and time-consuming, making BI challenging. Given the strong capabilities of large language models (LLMs) in working with data, we study their ability to answer BI questions end-to-end, without requiring users to manually perform the tedious preparation steps. To do this, we harvest a large collection of real-world BI projects from public sources, and manually extract pairs of (questions, ground-truth answers) from real user dashboards. The resulting benchmark, BI-Bench, is the first benchmark to systematically study LLMs’ ability on end-to-end BI. We find that even frontier LLMs perform poorly on BI-Bench, with less than 50% accuracy. To address their limitations, we design a tool-augmented BI-Agent that decomposes BI workflows into subtasks on structured data, such as search, join, and transform, and orchestrates specialized data management methods across BI stages. Furthermore, we develop a post-training framework that synthesizes training trajectories from real BI projects, enabling BI-Agent to be further post-trained using both supervised fine-tuning (SFT) and reinforcement learning (RL). BI-Agent achieves substantial accuracy gains of up to 40 percentage points with vanilla LLMs, and post-trained BI-Agent yields gains of up to 30 points. Our results highlight the importance of combining tool-augmented reasoning with domain-specific post-training in complex BI workflows, and point to promising directions for future research.

cs.RO [Back]

[69] WM-VS: Progress-Aligned World Models for Closed-Loop Visual Servoing cs.RO | cs.CVPDF

Guanzhong Sun, Junyi Ma, Yixuan Zhou, Yuxuan Wu, Yanzi Miao

TL;DR: WM-VS 用目标中心的有符号伺服坐标监督动作条件世界模型，并以未来误差收缩训练反应式控制器，在真实 7DoF 视觉伺服中实现稳定闭环修正。

Details

Motivation: 视觉伺服模型需预测动作是否会减小任务误差，而不仅是动作是否看起来合理；常规世界模型的预测目标与闭环控制目标并不一致。

Result: WM-VS 从离线目标区域 DINOv2 对应关系定义包含平移、尺度和面内旋转的四维伺服坐标。第一阶段使动作条件潜在转移与该坐标对齐；第二阶段冻结世界模型，用动作模仿、后果监督和短时想象 rollout 训练偏好误差收缩的关节速度策略。部署仅需 RGB 和反应式控制，无在线轨迹优化。真实 7DoF 系统 30 次试验全部在 30 步内将角点 RMSE 降至初值 10% 以下，25/30 次（83.33%）在最后有效帧仍满足标准；去掉未来误差对齐后保持率降到 26.67%。学习进度信号与训练/控制未用到的 AprilTag 误差相关系数为 0.8778。

Insight: 世界模型若直接对齐未来任务误差收缩，学到的动作后果就能服务反复视觉修正；外部 AprilTag 误差验证说明控制进度信号并非只与训练标签自洽。

Abstract: Closed-loop visual servoing requires predictions that indicate whether an action reduces task error, not only whether the action is plausible. We call this gap the prediction-control mismatch and introduce WM-VS, a target-centric progress-aligned world-model framework for closed-loop visual servoing. Offline target-region DINOv2 correspondences define a signed four-dimensional servo coordinate for translation, scale, and in-plane rotation. Stage 1 aligns action-conditioned latent transitions with this coordinate; Stage 2 freezes the world model and trains a reactive joint-velocity policy with action imitation, consequence supervision, and short imagined rollouts that favor error contraction. Deployment is RGB-only and reactive, without online trajectory optimization. On a real 7-DoF eye-to-hand system, WM-VS reaches a corner RMSE no larger than 10 percent of its initial value in 30/30 trials and retains this criterion at the final valid frame in 25/30 (83.33 percent). Removing future-error alignment reduces retention to 26.67 percent. The learned progress signal agrees with an external AprilTag corner error not used for training or control (mean Spearman rho = 0.8778). Without retraining, two unseen 3D targets achieve translation-error reductions of 86.48 percent and 90.27 percent and rotation-error reductions of 70.01 percent and 65.70 percent. These results link progress-aligned action consequences to repeated closed-loop correction and transfer. Code and data will be released as open source.

[70] FOCAL-VLA: Subtask-Guided Geometry Distillation and Implicit World Modeling for Vision-Language-Action Models cs.RO | cs.AI | cs.CVPDF

Zhiyuan Gao, Di Wen, Yanxiang Zhan, Mohammad Khoshnazar, Jeroen Schäfer

TL;DR: FOCAL-VLA 将 VGGT 子任务相关几何特征蒸馏与 Track4World 隐式动力学表征结合，提升精细和长程机器人操作，无需推理时调用这些教师模型。

Details

Motivation: 直接从当前二维观测映射动作的 VLA 缺少空间和时间理解。全场景几何监督及未来状态预测可能引入大量与当前交互无关的信息，分散模型对关键几何和动力学的学习。

Result: FOCAL-VLA 用子任务相关图像区域特征对齐 VGGT 几何潜变量，将几何知识蒸馏到 VLA；再通过当前和未来示范帧的 Track4World 特征进行隐式世界建模，捕捉当前交互未来三维演化。两种互补表示共同指导动作生成，但推理时无需运行 VGGT 或 Track4World。模拟基准和真实操作实验均显示超过基线。

Insight: 几何和未来状态监督应聚焦当前子任务相关区域与交互，而非无差别建模全场景；教师只在训练阶段提供结构化表示，部署仍保持高效。

Abstract: Vision-language-action (VLA) models built on pretrained vision-language models have demonstrated strong performance across diverse robotic manipulation tasks. However, VLA models that directly map current 2D observations to actions often lack sufficient spatial and temporal understanding, limiting their performance in precise and long-horizon manipulation. Recent methods enhance VLA models through geometric supervision and future-state prediction across the entire scene. However, these methods can suffer from redundant scene information, distracting the model from learning the geometry and dynamics relevant to the current interaction. To address this issue, we propose FOCAL-VLA, a framework that combines subtask-guided geometry distillation with implicit world modeling to learn representations of current spatial structure and future interaction dynamics. To focus geometric learning on the current subtask, we transfer geometric knowledge from VGGT to the VLA model by aligning geometry latents with features from subtask-relevant image regions. To capture the future 3D evolution of the current interaction, we incorporate implicit world modeling using Track4World features from current and future demonstration frames. The two complementary representations jointly guide action generation without running VGGT or Track4World at inference time. Experiments show that FOCAL-VLA outperforms baselines on both simulation benchmarks and real-world manipulation tasks. Project website: https://zhiyuan-gao.github.io/FOCAL-VLA/.

[71] VLA-Scope: Shift-Aware Failure Prediction for Vision-Language-Action Models cs.RO | cs.AI | cs.CV | eess.SYPDF

Kaiwen Zhu, Dongfang Liu, Liangkai Liu

TL;DR: VLA-Scope 先识别输入分布偏移类型，再结合动作前缀及执行进度逐步更新失败风险，避免仅凭 OOD 标志预测机器人任务失败。

Details

Motivation: VLA 遇到分布外输入时可靠性可能下降，但部分 OOD 状态仍能成功，因此只检测输入是否分布外不足以预测执行失败。

Result: VLA-Scope 两阶段运行：先用图像和语言池化表示检测 OOD 并分类偏移类别；对 OOD 输入，再将类别、动作前缀和执行进度特征输入跨偏移共享的逻辑回归，随执行推进更新风险。OpenVLA 在 10 个 LIBERO-Spatial 任务、留一组交叉验证中，OOD 检测 ROC-AUC 为 0.9454，偏移分类准确率 91%。在 1,400 条 OOD rollout 上，执行 60 个动作后的失败预测 AUC 为 0.8497，高于不含进度特征时的 0.7906，也超过所评 ActProbe 和 SAFE-MLP 基线。

Insight: 输入偏移与执行失败并非同义；结合实际动作和执行进展更新风险，可在任务过程中区分虽新颖但仍可完成的状态与真实失败轨迹。

Abstract: Vision-language-action (VLA) models map visual observations and natural-language instructions to robotic actions, but distribution shifts can compromise their reliability. Because these models may still succeed under out-of-distribution (OOD) conditions, detecting OOD inputs alone is insufficient to predict execution failure. In this paper, we introduce VLA-Scope, a two-stage framework that combines input-shift characterization with execution history to predict failure during OOD rollouts. The first stage uses pooled image and language representations to detect OOD inputs and classify their shift categories. For inputs flagged as OOD, the second stage combines the predicted category, action-prefix features, and execution progress features. A logistic regression model shared across shift categories updates failure risk as execution proceeds. We evaluate the framework with OpenVLA on ten LIBERO-Spatial tasks using leave-one-group-out cross-validation. OOD detection achieves a ROC-AUC of 0.9454, and shift classification achieves 91% accuracy. Evaluated independently of the OOD gate on all 1,400 OOD rollouts, the failure predictor achieves a ROC-AUC of 0.8497 after 60 executed actions, compared with 0.7906 without execution progress features. It also achieves a higher ROC-AUC than the evaluated ActProbe and SAFE-MLP baselines. These results suggest that combining action features with temporally aggregated execution step representations improves failure prediction under input shifts.

[72] ProTracer: Proprioception-Guided Failure Diagnosis in Robot Manipulation cs.RO | cs.CVPDF

Chang Dong, Mehdi Hosseinzadeh, King Hang Wong, Lingqiao Liu, Francois Fraysse

TL;DR: ProTracer 将本体感受信号转换为结构化描述，与 VLM 视觉推理结合，训练免地完成机器人操作失败检测、分类、解释和起始时刻定位。

Details

Motivation: 机器人失败分析不仅要判断任务是否失败，还需分类原因、生成解释并定位执行首次偏离正确轨迹的时刻；单靠视频可能难以精确找到动作状态变化。

Result: ProTracer 使用机器人本体动力学定位时间上有信息量的动作边界，并将丰富状态信号转为结构化自然语言描述，与视觉观察联合交给现有 VLM 分析，无需额外模型训练。作者还提出 FailTime，含同步视觉和本体观察，可评估常规失败诊断及失败起点定位。实验显示，ProTracer 在两类任务上均有较强表现，说明本体感受推理有助于细粒度时间诊断。

Insight: 本体状态提供动作变化的时间精度，视频模型提供多模态语义解释；将二者对齐可同时定位偏离起点并解释失败原因。

Abstract: This paper presents a comprehensive framework for robot manipulation failure analysis that includes binary failure detection, failure categorization, explanation generation, and the additional capability of failure onset localization, which aims to identify the earliest moment at which a robot execution deviates from a valid task-completion trajectory and is ultimately followed by task failure. To address these tasks, we propose ProTracer, a training-free framework that leverages existing Vision-Language Models (VLMs) together with proprioceptive signals for failure analysis. Our method uses proprioceptive dynamics to identify temporally informative action boundaries and converts richer robot-state signals into structured natural-language descriptions that can be jointly analyzed together with visual observations by the VLM. This design combines the temporal precision of proprioceptive signals with the multimodal reasoning capabilities of modern VLMs without requiring additional model training. We further introduce FailTime, a benchmark with synchronized visual and proprioceptive observations for evaluating conventional failure diagnosis tasks as well as failure onset localization. Experiments demonstrate that ProTracer achieves strong performance across both conventional failure diagnosis tasks and the newly introduced failure onset localization task, highlighting the importance of proprioceptive reasoning for fine-grained temporal failure analysis.

[73] GALA: Geometry-Aware Latent Action Modeling for Vision-Language-Action Model Pretraining across Embodiments cs.RO | cs.CVPDF

Yichen Liu, Puzhen Yuan, Xiang Zhu, Yanjiang Guo, Jianyu Chen

TL;DR: GALA 将场景级视觉潜在动作与三维末端执行器几何动作结合，并通过 UEMR 保留手指细节及跨具身语义，用于多具身 VLA 预训练。

Details

Motivation: 从多具身数据训练大规模 VLA 时，各末端执行器动作空间异构。现有基于图像的潜在动作模型难捕捉人手和灵巧机器人手指级几何变化；直接加点云又可能造成细粒度表示缺少跨具身共享语义。

Result: GALA 用统一末端运动表示 UEMR 兼顾动作细节与跨具身泛化，融合捕捉场景动态的视觉潜在动作和捕捉共享细粒度手部关节变化的几何潜在动作。该表示支持包括无动作标签第一人称人类视频在内的多具身数据 VLA 预训练。在精细运动探测、跨具身检索和下游 VLA 实验中显示有效，RoboCasa-GR1 成功率为 68.3%，真实世界成功率为 75.5%。

Insight: 跨具身学习需要同时保留场景变化和末端精细几何运动；统一动作表示可将两类互补信息用于机器人策略预训练。

Abstract: Learning large-scale vision-language-action (VLA) models from multi-embodiment datasets remains challenging due to heterogeneous action spaces across end effectors. Although latent action models (LAMs) can learn embodiment-agnostic action representations from diverse video data, existing image-based LAMs often fail to capture fine-grained end-effector articulation, particularly finger-level geometric changes in human and dexterous robot hands. To address this limitation, we propose GALA, a Geometry-Aware Latent-Action modeling framework that augments image-based latent actions with 3D end-effector geometric motion. However, naively incorporating point clouds yields fine-grained action representations with limited shared semantics, hindering cross-embodiment pretraining. To address this issue, we introduce the Unified End-effector Motion Representation (UEMR), which preserves fine-grained motion information while improving the cross-embodiment generalizability of latent actions. Building upon UEMR, GALA combines visual latent actions that capture scene-level dynamics with geometric latent actions that capture shared fine-grained end-effector articulation, providing effective supervision for VLA pretraining from multi-embodiment data, including action-free ego-centric human videos. Experiments on fine-grained motion probing, cross-embodiment retrieval, and downstream VLA evaluation demonstrate GALA’s effectiveness in modeling generalizable fine-grained motions across embodiments, achieving 68.3% RoboCasa-GR1 success rate and 75.5% real-world success rate. Code, appendix, and demos are available at https://puzhenyuan.github.io/GALA-website/.

cs.SE [Back]

[74] GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions cs.SE | cs.AI | cs.CLPDF

Xinyu Che, Yunfei Ge, Shihao Li, Yanchen Liu, Hang Yan

TL;DR: GameLogicBench 在 Godot 游戏项目的每个模拟 tick 检查玩法规则，覆盖 72 个任务和 1,451 个测试用例，并用变异验证评测器能区分正确实现。

Details

Motivation: 编程智能体可修改大型项目代码，但游戏开发要求落实持续运行中的玩法规则。重放固定案例、评分视频或让另一模型评审无法保证在多种场景的整个执行过程均无规则违反，也难保证结果可复现。

Result: 基准的自动评测器逐 tick 检查游戏规则，并须接受同一任务的不同正确实现、拒绝删去所需能力的变异实现。72 个 Godot 任务覆盖独立机制、系统交互和仓库级功能；403 个手工场景经随机种子扩展为 1,451 个用例。20 组模型与脚手架中最佳运行解决 52.78% 任务；在 Claude Code 下，任务范围越大完成数越少。多数失败提交可以运行，但规则实现错误。未用变异验证的评测器会错误接受提交；网络开放时还观察到智能体从公开仓库复制代码。

Insight: 可靠的智能体评测不仅要自动检查运行行为，还要用变异测试验证检查器确实能抓错；外部代码访问也会影响评测结果和能力归因。

Abstract: Coding agents can modify and test code across large software projects. Game development is a domain where agents must implement gameplay rules. A game can end in a valid state even after violating its rules during the run. Current game-development benchmarks replay fixed examples, score videos, or ask another model to judge the result. However, no existing benchmark checks game rules throughout execution across varied evaluator-selected scenarios while ensuring exactly reproducible verdicts. We introduce GameLogicBench, a benchmark of 72 gameplay-logic tasks in Godot projects. An automated evaluator checks each game’s rules at every simulation tick. Across 403 hand-designed scenarios, seeded parameter variations produce 1,451 test cases. To ensure that the evaluator measures behavior rather than implementation choice, it must accept different correct implementations for each task while rejecting mutants, implementations with one required capability removed. The tasks span isolated mechanics, multi-system interactions, and repository-scale features. Across 20 combinations of language models and scaffolds, the best observed run solves 52.78% of tasks. Under Claude Code, all twelve models solve fewer tasks as task scope expands from isolated mechanics, through interacting systems, to repository-scale features. Agents inspect code more often and make more tool calls on repository-scale tasks than on isolated-mechanic tasks. Most unsuccessful submissions are runnable, but implement some required game behavior incorrectly. We compared versions of our benchmark evaluator built with and without validation using mutants. Without this validation, incorrect agent submissions passed. A separate analysis finds agents copying code from public repositories when network access is open. Reliable evaluation thus depends both on what the tests reject and on what external code agents can access.

eess.AS [Back]

[75] The Spoken Wikipedia Presentation Corpus eess.AS | cs.CL | cs.MM | cs.SDPDF

Thomas Ranzenberger, Steffen Freisinger, Tobias Bocklet, Korbinian Riedhammer

TL;DR: Spoken Wikipedia Presentation Corpus 为口语维基语料增加自动生成的配套幻灯片，构成多模态 ASR 资源；音频单模态模型强于当前全模态零样本提示。

Details

Motivation: 幻灯片可提供语音识别所需的视觉上下文，但可对齐的大规模演示、文本和音频数据较少。

Result: 语料以 LLM 切分文章章节，再通过混合流程生成幻灯片：模型规划标题、要点、总结和插图描述，规则系统选择布局、主题和样式，视觉语言模型将幻灯片文字抽取为 Markdown。多种 ASR 和口语模型评测中，最佳模型仅用音频时平均 micro-WER 为 10.23%、micro-CER 为 6.48%；英语错误最低，其次德语和荷兰语，低资源语言表现下降。尽管音频基线较强，全模态模型零样本提示仍有困难；对齐的幻灯片、文本和音频数据显示跨模态上下文有提升识别的潜力。

Insight: 幻灯片可作为语音内容的语义上下文，但现有模型尚不能通过简单零样本多模态提示稳定利用它；新语料为跨模态 ASR 训练提供对齐数据。

Abstract: We present the Spoken Wikipedia Presentation Corpus, an extension of the Spoken Wikipedia Corpora featuring LLM-generated slide decks for multimodal ASR. Slides are created from LLM-segmented sections using a hybrid pipeline that combines LLM-based content planning with rule-based design decisions. For each section, an LLM generates a slide title, bullet points, a takeaway message, and a visual description that is used to create an illustration. Rule-based matching then selects layouts, themes, and styles to produce the final slides. A vision LLM extracts slide text as Markdown. We evaluate multiple ASR and spoken language models (SLMs). The best model achieves an average micro-WER of 10.23% and an average micro-CER of 6.48% on audio-only inputs. English yields the lowest error rates, followed by German and Dutch, while performance declines across lower-resource languages. Although audio-only baselines are strong, multimodal zero-shot prompting of omni models remains challenging. The aligned slide, text, and audio data show a strong potential to improve recognition through cross-modal context.

eess.IV [Back]

[76] MultiHU-TD: Multifeature Hyperspectral Unmixing Based on Tensor Decomposition eess.IV | cs.CV | eess.SPPDF

Mohamad Jouni, Mauro Dalla Mura, Lucas Drumetz, Pierre Comon

TL;DR: MultiHU-TD 将丰度和多模态特征建模为高阶张量，以低秩张量分解完成可解释高光谱解混，并把丰度和为一约束融入 ADMM 优化。

Details

Motivation: 高光谱解混将混合像素表示为纯材料及其丰度权重，但仅用光谱特征常不够，需利用场景其他特征；将多模态多特征高光谱数据表示为高阶张量时，矩阵模型不足，现有张量分解又缺少通用框架及结果解释。

Result: 作者提出多特征高光谱解混框架 MultiHU-TD，基于低秩张量分解，在交替 ADMM 优化中加入丰度和为一约束，并提供数学、物理和图示解释及与扩展线性混合模型的联系。附加特征包括数学形态学，并将邻域 patch 方法纳入该框架。真实高光谱图像实验展示模型及结果的可解释性；提供 Python 和 MATLAB 实现。

Insight: 把额外空间特征作为高阶张量维度统一处理，并保留丰度物理约束，可以同时利用多特征信息与可解释解混结果。

Abstract: Hyperspectral unmixing allows representing mixed pixels as a set of pure materials weighted by their abundances. Spectral features alone are often insufficient, so it is common to rely on other features of the scene. Matrix models become insufficient when the hyperspectral image (HSI) is represented as a high-order tensor with additional features in a multimodal, multifeature framework. Tensor models such as canonical polyadic decomposition allow for this kind of unmixing but lack a general framework and interpretability of the results. In this article, we propose an interpretable methodological framework for low-rank multifeature hyperspectral unmixing based on tensor decomposition (MultiHU-TD) that incorporates the abundance sum-to-one constraint in the alternating optimization alternating direction method of multipliers (ADMM) algorithm and provide in-depth mathematical, physical, and graphical interpretation and connections with the extended linear mixing model. As additional features, we propose to incorporate mathematical morphology and reframe a previous work on neighborhood patches within MultiHU-TD. Experiments on real HSIs showcase the interpretability of the model and the analysis of the results. Python and MATLAB implementations are made available on GitHub.

[77] Uncertainty-driven training for three-dimensional calibrated lung nodule classification eess.IV | cs.CVPDF

Giuseppe Tripodi, Alessandro De Rosis, Saleh Rezaeiravesh

TL;DR: 该框架用 MC Dropout 或证据深度学习的不确定性调节 CT 肺结节分类损失，在相近分类性能下显著改善概率校准。

Details

Motivation: 三维 CT 肺结节分类需要不仅预测正确，还要给出可靠概率，以支持临床信任和部署决策。

Result: 验证集不确定性估计用于重加权训练损失，促使模型关注困难或不可靠类别。作者比较 Monte Carlo Dropout 与证据深度学习，并在 LIDC-IDRI 临床队列和 NoduleMNIST3D 上使用 ResNet、DenseNet、EfficientNet、ViT 和 Swin。分类性能与常规训练相近，但 LIDC-IDRI 的 ECE 最多降低 65%。EDL 在浅层网络上具竞争力且只需单次推理，MCD 对较深网络更稳健；不确定性训练对卷积骨干收益更一致，EDL 在 ViT 上退化。事后温度缩放在所有配置中都很有效。

Insight: 分类性能与概率校准应分开报告；显式不确定性训练的收益依架构而异，而简单温度缩放本身可能已具竞争力。

Abstract: In this work, we present an uncertainty-driven training framework for three-dimensional computed tomography (CT) lung nodule classification, where validation-based uncertainty estimates guide loss reweighting to enhance predictive performance and probability calibration. Two Uncertainty Quantification (UQ) methods are considered: Monte Carlo Dropout (MCD) and Evidential Deep Learning (EDL). Both provide per-class uncertainty estimates that modulate the loss and encourage focus on hard or unreliable classes. The framework is evaluated with ResNet, DenseNet, EfficientNet, Vision Transformer (ViT), and Swin Transformer backbones on two datasets: the clinical LIDC-IDRI cohort and the NoduleMNIST3D benchmark. Uncertainty-driven training achieves classification performance similar to conventional training while substantially improving calibration, with an expected calibration error (ECE) reduced by up to 65% on LIDC-IDRI. EDL attains competitive performance on shallower architectures with single-pass inference, whereas MCD is more robust on deeper networks. Analysis across architectural families reveals that uncertainty-driven training benefits convolutional backbones more consistently than transformer-based architectures: EDL in particular degrades on ViT, suggesting that the Dirichlet evidence parameterisation may interact unfavourably with attention-based architectures at lower input resolutions. A posteriori temperature scaling proves highly effective across all configurations, indicating that a simple scalar calibration can be competitive even without explicit uncertainty-aware training. Our results indicate that integrating UQ into the training loop can significantly improve probabilistic calibration and support more trustworthy deployment of three-dimensional medical imaging models.

[78] Adaptive Color Grading eess.IV | cs.CVPDF

Trevor D. Canham, Abhijith Punnappurath, Michael S. Brown

TL;DR: Adaptive Color Grading 开源可独立控制阴影、高光等影调区域的调色工具，并收集视频帧阈值数据；KNN 在预测紧凑参数上优于所比较的端到端增强方法。

Details

Motivation: 摄影、绘画和电影后期需要独立调节不同亮度区域。现有调色工具以强度阈值分割影调区，若要自动化创意风格化，需要学习少数关键阈值而非拟合完整图像映射。

Result: 作者开发开源调色工具，并用其标注大规模视频帧影调区域阈值；比较从业者经验规则及机器学习建模策略。K 近邻成为有效阈值预测方法，在图像增强比较中超过先进端到端方法。工具和数据公开。

Insight: 对创意调色等高维输出任务，学习紧凑的核心控制参数可能比直接预测全图更有效；该工作将从业者操作抽象为可监督影调阈值。

Abstract: Independent control of tonescale regions (e.g., shadows, highlights) is essential for painters, photographers and cinematographers to bring 2D images to life. In image manipulation software this is most directly addressed by color grading modules, which use intensity thresholds to segment distinct illumination regions for local manipulation. In this work we develop an open source color grading tool and use it to annotate a large dataset of video frames with tonescale region thresholds. Using these thresholds we conduct modeling experiments with strategies based on both practitioners’ conventional wisdom and machine learning. Results show that K-nearest neighbors is an effective prediction strategy, outperforming state-of-the-art end-to-end methods for image enhancement. This outcome demonstrates the benefit of focusing on a compact set of core parameters when modeling creative stylization processes. Our adaptive color grading interface and data are available at https://github.com/SamsungLabs/adaptive-color-grading.

Table of Contents

cs.CV [Back]

[1] MemeTAG: Keyword-Driven Meme Classification through Tag Embedding Reconstruction cs.CV | cs.MMPDF

[2] TAPe+ML: A Compact Structured Representation for Multi-Task Computer Vision cs.CV | eess.IVPDF

[3] 4DGS-Fixer: Generative Sparse-View 4D Gaussian Splatting with Iterative Refinement Guided by Video Diffusion Priors cs.CVPDF

[4] MarsFM: Shading-Regularized Flow Matching for Martian Relief Estimation cs.CVPDF

[5] Fragment-Aware Vision Transformers for Fresco-Fragment Style Classification cs.CV | cs.LGPDF

[6] VGGT-CAD: Reconstructing Parametric CAD 3D Model with Geometric Grounding cs.CVPDF

[7] Multi-viewpoint Geo-localization with Event Cameras cs.CV | cs.ROPDF

[8] Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing cs.CVPDF

[9] Beyond Exact Match: Task-Aware GRPO for Cross-Domain PCBA Visual Question Answering cs.CV | cs.AIPDF

[10] Combining Object Detection with Geometry-Aware Clustering to Distinguish Overlapping Plants in UAV Imagery cs.CVPDF

[11] S3VD: Semantic-Guidance Spatio-Temporal Scanning for Video Deraining cs.CVPDF

[12] VeriFuse: Bounded Vision-Language Arbitration and Reason-Guided Refinement for Cooperative 3D Perception cs.CVPDF

[13] Cube-Splat: High-Fidelity 360° Gaussian Splatting SLAM via Cubemap Factorization and Adjoint-Consistent Optimization cs.CVPDF

[14] PrismAlign: Prior-Steered Multi-View VLM Alignment for Hallucination-Robust Table OCR cs.CVPDF

[15] Field Tracking of Insects Using a Stereoscopic Event-Based Camera Setup cs.CVPDF

[16] Hiding in Plain Sight: A Diffusion-based Mitigation of Geolocation Privacy Leakage in Vision-Language Models cs.CV | cs.LGPDF

[17] RobotEQ-Video: A Video-Centric Benchmark for Social Proactive Intelligence with World-State Taxonomy cs.CV | cs.HCPDF

[18] JEPA Guided Diffusion: Predictive Vision-Language Conditioning for Generative Traffic Forecasting cs.CVPDF

[19] AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agents cs.CV | cs.AIPDF

[20] SIRA: Reasoning-Aware Surgical Instrument Segmentation via Query-Anchored Alignment cs.CVPDF

[21] A Scene Language Model for Open-Vocabulary Scene Mapping cs.CV | cs.ROPDF

[22] ME-Dex 1.0: Bringing Heterogeneous Tactile Sensing into World Action Modeling cs.CVPDF

[23] CompAdapt: Adaptable Composite Motion Modeling for Physics-Consistent Text-to-Video Generation cs.CVPDF

[24] SkillIR: Evolving Scene-Aware Skills for Agentic Image Restoration cs.CVPDF

[25] MT-WAM: Reorienting the One-Pass Predictive Representation Toward Action Generation cs.CV | cs.ROPDF

[26] OpenSAL360: Open-Source Crowdsourcing Platform for Omnidirectional Video Saliency Collection cs.CVPDF

[27] VoxelTTO: Voxel-Aligned Feed-Forward 3D Gaussian Splatting with Test-Time Optimization cs.CVPDF

[28] Adaptive World Memory 3D Foundation Model for Scalable 3D Mapping, Localization, and Rendering cs.CV | cs.ROPDF

[29] VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration cs.CV | cs.AIPDF

[30] Refine Then Fusion: Training-Free 3D Point Cloud Adaptation with Priority Refinement and Multi-Modal Knowledge Fusion cs.CVPDF

[31] From Retrieval to Recognition:How Vision–Language Models Become OCR Specialists cs.CVPDF

[32] HAT: Hypothesis-Anchored Tracking for Video Monocular Spacecraft Pose Estimation cs.CVPDF

[33] Learned Parametric Emotion Editing: Real-Time Affective Filtering for On-Device Social Media Video cs.CV | cs.HCPDF

[34] Extending Decoupled Attention to Dense Prediction and Masked Training for Multi-Channel Images cs.CVPDF

[35] Configurable Multi-Stage Vision Pipeline for Crop Disease and Pest Diagnosis cs.CV | cs.CL | cs.LGPDF

[36] DRT: Dense Reasoning Trace for Efficient and Grounded Multimodal Reasoning cs.CVPDF

[37] ZYT-World: A Real-Time Controllable World Model for Closed-Loop Autonomous-Driving Simulation cs.CVPDF

[38] Balanced Prompt Adaptation against Entropy-Induced Collapse for Test-Time Binary Segmentation cs.CV | cs.AIPDF

[39] Beyond Benchmark Scores: Auditing Medical Vision-Language Models for Chest X-Ray Tuberculosis Screening cs.CV | cs.LGPDF

[40] XCalib Depth-Guided Geometric Optimization for Dense Thermal-Visible Video Registration cs.CVPDF

[41] VideoReloc: Long-Term Indoor Video Relocalization against a Kilobyte-Scale Semantic Scene Graph cs.CVPDF

[42] PRIME: Perception Feedback with Situational Memory Embeddings in VLA Models cs.CV | cs.ROPDF

[43] Info3R: Information-Adaptive Test-Time Training for 3D Reconstruction cs.CVPDF

[44] Benchmarking the Explanatory Quality of Open-Weight Vision-Language Models in Face Recognition cs.CV | cs.AIPDF

[45] OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation cs.CVPDF

[46] MintAct: A Unified Visual Agent for Digital Environments cs.CVPDF

cs.CL [Back]

[47] HERMES: Contrast-Aware Knowledge Graph Reasoning from Clinical Notes for Patient Outcome Prediction cs.CL | cs.LGPDF

[48] Reviser: Revision-Capable Text Generation via Autoregressive Cursor Actions cs.CLPDF

[49] A Generative Grammar Underlying the Voynich Manuscript, the Pastiche Hypothesis: Evidence from Large Language Models cs.CLPDF

[50] COAL-SQL: Coverage-Guided Augmentation and Failure-Driven Learning for Text-to-SQL Post-Training cs.CL | cs.DBPDF

[51] PhysioBench: A Unified Benchmark for Physiological Signal Question Answering cs.CL | cs.LGPDF

[52] VISPATH: Visual-Intent-Guided Path Reasoning for Multimodal Knowledge Graph Question Answering cs.CL | cs.LGPDF

[53] Reading Less While Writing: A Closed-Form Bandwidth Dial for Streaming Multimodal Decoders cs.CL | cs.LGPDF

[54] Boosting Deepresearch and LongContext Ability with Self-Generated Deepresearch Rollouts Traces cs.CLPDF

[55] Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Models cs.CL | cs.LGPDF

[56] Enhancing Audio Reasoning via Semantic Summary Prediction cs.CL | cs.SD | eess.ASPDF

[57] MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs cs.CL | cs.CVPDF

[58] Hallucination-R1: Robustness-Oriented Paraphrase Generation for Factual Consistency cs.CL | cs.AIPDF

[59] When Does Reasoning Help in Machine Translation? A Hierarchical Analysis of LRM Reasoning Traces cs.CLPDF

[60] ArenaFlow: From Trajectory Ranking to Hierarchical Credit Propagation for Open-Ended Agent RL cs.CLPDF

[61] Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction cs.CL | cs.CV | cs.MM | cs.SDPDF

[62] MIRAGE: Multi-Perspective Creative Language Model Reasoning with Reinforcement Learning Guidance cs.CLPDF

[63] When Steering Fails in Latent Reasoning: A Latent-to-Language Transition Gap cs.CL | cs.AIPDF

[64] TrialAtlas: Multi-Agent Research Organization for Clinical Trial Design and Optimization cs.CLPDF

[65] DiaVLo: Diagnosing Behaviours of Vision-Language Models cs.CL | cs.AIPDF

cs.AI [Back]

[66] Offline Multimodal Large Language Models for Decision Support in Air Operations cs.AI | cs.CLPDF

[67] MIST: Multimodal Survival Prediction with Genomic-Guided Histology Attention cs.AI | cs.CVPDF

cs.LG [Back]

[68] BI-Agent and BI-Bench: Towards Automating End-to-End Business Intelligence cs.LG | cs.AI | cs.CL | cs.DBPDF

cs.RO [Back]

[69] WM-VS: Progress-Aligned World Models for Closed-Loop Visual Servoing cs.RO | cs.CVPDF

[70] FOCAL-VLA: Subtask-Guided Geometry Distillation and Implicit World Modeling for Vision-Language-Action Models cs.RO | cs.AI | cs.CVPDF

[71] VLA-Scope: Shift-Aware Failure Prediction for Vision-Language-Action Models cs.RO | cs.AI | cs.CV | eess.SYPDF

[72] ProTracer: Proprioception-Guided Failure Diagnosis in Robot Manipulation cs.RO | cs.CVPDF

[73] GALA: Geometry-Aware Latent Action Modeling for Vision-Language-Action Model Pretraining across Embodiments cs.RO | cs.CVPDF

cs.SE [Back]