cs.CV [Total: 120]
cs.CL [Total: 32]
cs.AI [Total: 12]
cs.CR [Total: 1]
cs.GR [Total: 1]
cs.HC [Total: 1]
cs.IR [Total: 1]
cs.LG [Total: 13]
cs.RO [Total: 13]
cs.SD [Total: 2]
eess.AS [Total: 1]

cs.CV [Back]

[1] SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows cs.CV | cs.AIPDF

Albert Gao, Bing Xue, Andrea Zanette

TL;DR: 提出 SLVR，以规划、定位、证据选择和推理整合等有类型潜变量组织视觉推理监督，在避免推理时生成文本 CoT 的同时提升多模态推理表现。

Details

Motivation: MLLM 可能依赖语言先验而忽略图像证据；文本 CoT 可引导逐步查找视觉证据，但自回归生成增加成本。既有潜在推理方法对中间状态编码内容控制有限，难以分别监督规划、定位与证据选择。

Result: SLVR 先通过屏蔽泄露答案的文本并区分正确答案与视觉上合理的干扰项，训练模型依赖图像；随后将推理组织为规划、定位、证据选择和整合潜阶段，分别用计划、框、视觉证据与最终理由监督。基于 Qwen2.5-VL-7B，在多模态推理基准上持续提升：MMVP 绝对提高 9.4 分，BLINK Relation 提高 14.2 分，V*、MathVista、ChartQA 也有改善。

Insight: 类型化的潜变量阶段提供了比无结构隐状态更直接的功能监督，又避免推理时解码完整文本推理链。摘要未列出其它基准分数及推理延迟节省量。

Abstract: Multimodal large language models (MLLMs) often answer visual reasoning questions by relying on linguistic priors rather than task-relevant visual evidence. Textual chain-of-thought reasoning can partially mitigate this issue by encouraging models to decompose visual questions into intermediate evidence-seeking steps, but generating these steps autoregressively increases inference cost. Latent reasoning avoids explicit rationale generation, but existing approaches provide limited control over what intermediate states encode, making it difficult to impose separate supervision for planning, grounding, and evidence selection. We propose Structured Latent Visual Reasoning (SLVR), a training framework that bridges explicit chain-of-thought and latent reasoning by organizing multimodal reasoning into typed latent stages for planning, grounding, evidence selection, and reasoning integration. SLVR first trains the model to rely on the image by masking answer-revealing text and contrasting the correct answer with visually plausible distractors. It then organizes reasoning into latent stages for planning, grounding, evidence selection, and integration, supervising each stage with the corresponding signal: plans, boxes, visual evidence, and final rationales. This gives latent reasoning an explicit functional structure while avoiding generated textual chains at inference time. Built on Qwen2.5-VL-7B, SLVR improves consistently across multimodal reasoning benchmarks, with absolute gains of +9.4 on MMVP and +14.2 on BLINK Relation, as well as improvements on V*, MathVista, and ChartQA. These results suggest that structured latent supervision can improve fine-grained visual reasoning without the decoding overhead of textual CoT. Project page is available \href{https://bogao-code.github.io/SLVR/}{here}.

[2] A Camera-Native Stereo VR180 Dataset cs.CV | cs.MMPDF

Linxuan Lu

TL;DR: 发布原生相机采集的立体 VR180 数据集，保留鱼眼视频、相机 RAW 和镜头标定，并比较原生鱼眼与半等距编码的视口画质和码率。

Details

Motivation: 公开 VR180 资源多来自网络平台，已被未知流程拼接、投影和压缩，且缺少镜头标定，难以研究原始相机内容、投影与编码影响。

Result: 数据集由两台 Blackmagic URSA Cine Immersive 相机实拍，含 1,211 个样本：636 段立体视频（总长 2,220.8 秒，多为 90 fps）和 575 张立体静帧。提供相机原生 Blackmagic RAW、左右眼独立原生鱼眼 HEVC（每眼 8160×7200）、半等距 HEVC（每眼 7200×7200）、厂商镜头标定、转换工具及 AI 生成场景/视觉挑战标注。在 24 个片段、双眼、四个码率点和九个视角的 x265 重编码比较中，同等视口质量下原生鱼眼编码在所有 24 个片段均比半等距需要更多码率，中位数多 38%，且差异出现在整个视场。

Insight: 提供相机原始数据和标定使 VR180 研究不再受平台预处理未知影响。码率结论针对 x265 和所测 24 个视频片段及视角，不能直接推断其他编解码器。

Abstract: Immersive VR180 video is increasingly produced with professional stereo fisheye cameras, yet public VR180 research resources are mostly collected from online platforms such as YouTube: already stitched, projected and compressed by unknown pipelines, and without lens calibration. We present a firsthand-captured stereo VR180 dataset recorded with two Blackmagic URSA Cine Immersive cameras. It contains 1,211 samples – 636 stereo video clips (2,220.8 s, mostly 90 fps) and 575 stereo stills – each released as camera-native Blackmagic RAW, separate-eye native fisheye HEVC (8160x7200 per eye) and half-equirectangular HEVC (7200x7200 per eye), together with the factory lens calibration, portable fisheye/half-equirectangular conversion tools and AI-generated scene and visual-challenge annotations. Re-encoding the released fisheye and half-equirectangular renders with x265 over 24 clips, both eyes, four rate points and nine viewing directions, native-fisheye coding needed more bitrate than half-equirectangular coding at equal viewport quality for all 24 clips (median +38%), in every part of the field of view. Data: https://huggingface.co/datasets/lulinxuan/VR180 ; code: https://github.com/lulinxuan/vr180-dataset-tools

[3] Seeing Through the Glare: A Multi-Source Benchmark and Ocular-Adaptive Pixel MeanFlow for Eyeglass Reflection Removal cs.CVPDF

Tao Liu, Youwei Pang, Kailai Zhou, Jiaming Zuo, Hanqi Liu

TL;DR: 构建眼镜反光去除基准 OcuBench，并提出 OcuFlow，一步像素 MeanFlow 配合眼部自适应表征，兼顾严重遮挡处重建与可见细节保留。

Details

Motivation: 眼镜反光从轻微光照污染到遮挡眼部不等，修复需准确选择性去除并自然补全，同时不能改变身份和面部外观；现有数据集涵盖的反光条件有限。

Result: OcuBench 含 10,280 对可控合成样本、732 对真实输入伪配对样本和 458 张独立真实测试图像，支持配对评估与超出合成监督的测试。OcuFlow 结合几何自适应表示和一步 pMF 聚焦反光遮挡眼区，并用原生分辨率频率保持生成保留可靠可见细节。跨多种反光条件实验报告去除质量、眼部保真度和效率均持续占优；盲评中获得 67.32% 选择率，为次优方法的 6.2 倍。

Insight: 数据同时包含合成配对、真实伪配对和独立实景测试，能部分检验合成训练向真实输入的泛化。用户研究显示主观偏好优势，但摘要未给参与人数或置信区间。

Abstract: Eyeglass reflection removal is important across smartphone imaging, video conferencing, and other face-centric visual applications. The task is challenging because reflections range from mild photometric contamination to severe ocular occlusion, requiring selective correction and plausible reconstruction without altering identity or natural appearance. Existing datasets cover limited reflection conditions, constraining generalization to complex real-world scenes and systematic evaluation. We introduce \textbf{OcuBench}, a multi-source benchmark comprising 10,280 controllable synthetic pairs, 732 real-input pseudo-pairs, and 458 independent real-world test images, supporting both paired evaluation and assessment beyond generated supervision. We further propose \textbf{OcuFlow}, an ocular-adaptive pixel MeanFlow (pMF) framework for efficient, detail-preserving restoration. It combines geometry-adaptive representation with one-step pMF to focus reconstruction on reflection-obscured ocular regions, together with native-resolution frequency-preserving synthesis to retain reliable observed details. Experiments across diverse reflection conditions demonstrate that OcuFlow achieves consistent advantages in reflection removal quality, ocular fidelity, and efficiency. In a blind user study, it receives $67.32%$ of selections, $6.2\times$ the next-best share. Both the code and dataset will be released.

[4] VICO: Visual Environments Co-Evolving for Vision-Language Model Reasoning cs.CV | cs.AIPDF

Meng Lu, Ligeng Zhu, Olivia Xiao, Yuchen Zhuang, Zihan Wang

TL;DR: 提出 VICO，让 VLM 策略与图像侧环境改写器共同进化，按策略通过率调节视觉推理任务难度，并减少对额外人工标注的依赖。

Details

Motivation: 可验证奖励 RL 通常在静态视觉任务上训练；随着 actor 变强，部分任务过易、部分任务仍不可解，导致学习信号失衡。作者主张训练环境也应跟随模型能力演化。

Result: VICO 联合训练 actor 和 EnvRewriter。后者改写场景图、图表数据或受保护区域掩码等可验证图像结构并重新渲染，依据 actor 通过率奖励校准难度，生成标签有效样例且不需新增人工标注。九个涵盖数学推理和视觉 grounding 的多模态基准上，VICO-8B 在分布外任务较基础模型最高高 5.0%；较最强自进化和文本编辑协同进化基线分别高 4.3% 和 8.4%；与图表专用 RLVR 方法表现相当，但使用标注样本少 16–160 倍。

Insight: 将图像环境的难度作为训练中可适配变量，可避免静态语料逐渐偏离学习前沿。效果依赖能够正确编辑并重渲染图像结构的任务类型，摘要未列各基准具体分数。

Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a standard recipe for post-training vision-language models (VLMs), but it typically assumes a static training environment. As the actor improves, fixed tasks drift out of its learning frontier: many become trivial, others remain unsolvable; and the learning signal collapses. We argue that VLM post-training should evolve the visual environment alongside the actor, not just the actor itself. We propose VICO, a co-evolutionary framework in which an actor and an Environment-as-Rewriter (EnvRewriter) are trained jointly: the EnvRewriter edits verifiable image-side structures, such as scene graphs, chart tables, or protected region masks, and re-renders them to produce label-valid training samples whose difficulty is calibrated to the actor’s current ability through a pass-rate-based reward. This loop continuously realigns task difficulty with actor capability without any additional human annotation. Across nine multimodal benchmarks spanning mathematical reasoning and visually grounded understanding, VICO-8B improves over its base model by up to +5.0% on out-of-domain tasks, surpasses the strongest self-evolution and text-editing co-evolution baselines by +4.3% and +8.4% respectively, and stays comparable to chart-specialized RLVR methods using 16-160 times fewer labeled samples. By shifting from human-labeled supervision to image-editing co-evolution, VICO offers a scalable path beyond static-corpus RLVR for visual reasoning.

[5] SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages cs.CVPDF

Raja Kumar, Rajat Koner, Ritwick Chaudhry, Zhuowei Li, Nishant Sankaran

TL;DR: 提出 SPLIT-RL，将视觉语言推理拆成视觉感知和语言推理阶段训练，并用 CLA-GRPO 为视觉主张分配细粒度优势。

Details

Motivation: 传统 RLVR 以单条思维链最终答案奖励同时训练视觉推理与语言推理，CoT 中每个 token 得到相同序列级优势，无法区分具体错误来自视觉感知还是语言推理。

Result: SPLIT-RL 分开训练 VR 与 LR；每组 rollout 在单一能力上变化，因此组相对优势可隔离对应能力，并使用阶段专属奖励。CLA-GRPO 将视觉阶段轨迹分成原子视觉主张，按视觉类型分组后计算主张级优势。训练结束后推理与 GRPO 相同，仅需一次 CoT 调用。Qwen3-VL 2B 至 30B-A3B 和 InternVL3.5-8B 上，平均准确率较 GRPO 提高 1.4–6.1 个百分点。oracle 诊断发现，只有答案级奖励的 GRPO 未改善感知，SPLIT-RL 则改善视觉与语言两种能力。

Insight: 阶段化奖励和主张级信用分配让视觉事实错误得到直接监督，同时保留标准单次 CoT 推理方式。摘要未提供逐模型的能力分解分数。

Abstract: Vision-Language (VL) reasoning requires a model to both extract relevant and accurate information from an image (visual reasoning, VR), and to infer the answer from it (language reasoning, LR). Reinforcement learning with verifiable rewards typically trains both through a single chain-of-thought with a final-answer reward. This gives every CoT token the same sequence-level advantage, failing to distinguish capability specific errors. We propose SPLIT-RL, a staged post-training approach that trains VR and LR in disjoint phases. Because a group’s rollouts differ along one capability at a time, the group-relative advantage isolates it, and each phase is optimized using phase-specific reward. We further introduce Claim-Level Advantage (CLA-GRPO), which decomposes VR-phase rollouts into atomic visual claims and provides a fine-grained advantage at claim level based on visual-type group formation. Although trained in two phases, trained policy is evaluated like GRPO model, with a single CoT call at inference time. Under this protocol, SPLIT-RL improves average accuracy over GRPO by 1.4-6.1 points across Qwen3-VL models from 2B to 30B-A3B and InternVL3.5-8B. Evaluating each capability using an oracle based diagnostic shows that answer-only GRPO leaves perception unchanged, whereas SPLIT-RL improves both VR and LR.

[6] Fluid-Gen-Zero: Grounding Pretrained Video Generators in Physics without Training cs.CV | cs.GRPDF

Hong Huang, Yuqiu Liu, Chenyu You, Daniel Martin, Chuhang Zou

TL;DR: 提出 Fluid-Gen-Zero，将模拟器中的流体物理信号注入冻结视频生成器的潜空间，免训练生成更符合物理的流体—物体交互视频。

Details

Motivation: 预训练视频模型外观生成能力强，但常难正确呈现流体与物体间物理运动。作者希望把物理推理和外观合成分开，直接复用物理模拟器与现有视频基础模型。

Result: 两级代理工作流包含生成阶段规划：VLM 理解意图与模拟 rollout、组织生成片段；以及潜空间引导：用区域感知 latent wrapping 把模拟信号注入去噪。框架可插接当前视频基础模型。作者提出流体—物体交互视频基准，在基于 CogVideoX 的 Tora、以及基于 Wan 的 VACE 和 WanMove 上，物体轨迹误差降低 26.7%–81.5%，流体端点误差 fEPE 降低 67.9%–84.0%，感知质量大体保持。人评中，同骨干方法的偏好率为 55.1%–74.4%，相较模拟器类方法为 90.4%–94.2%。

Insight: 模拟器负责动力学、生成模型负责外观，区域化潜空间桥接可在无需训练新模型时提升物理对齐。不同骨干的数值范围较宽，用户偏好比较也因对照组而不同。

Abstract: We present Fluid-Gen-Zero, a training-free framework for physics-aware fluid-object interaction video generation that decouples physical reasoning from appearance synthesis. Our key insight is to delegate motion dynamics to a physics simulator while preserving the appearance modeling capacity of pretrained video generators. We bridge these two domains through a two-level agentic workflow: generation-time planning, where a vision-language model (VLM) agent interprets intent and the simulation rollout to organize generation clips, and latent-space guidance, which injects simulation signals into denoising through region-aware latent wrapping. This plug-and-play design is compatible with current video foundation models. We further introduce a benchmark for fluid-object interaction video generation. Across Tora (CogVideoX-based), VACE and WanMove (Wan-based), Fluid-Gen-Zero consistently improves simulation alignment, reducing object trajectory error by 26.7%-81.5% and fluid fEPE (fluid flow endpoint error) by 67.9%-84.0%, while largely preserving perceptual quality. In a human preference study, raters favor Fluid-Gen-Zero in 55.1%-74.4% of same-backbone comparisons across three backbones, and in 90.4%-94.2% of comparisons against simulation-based methods. Code and data will be released upon acceptance.

[7] Less from More: Reinforcing Sparse Video Reasoning from Dense References cs.CVPDF

Wenfang Sun, Yingjun Du, Cees G. M. Snoek

TL;DR: 提出 SAVER，以训练阶段的密集视频观察指导稀疏帧推理，使用时序定位奖励让模型在有限帧预算下保留关键时间证据。

Details

Motivation: 视频模型常假设观测帧越多推理越可靠；作者认为真正重要的是在稀疏时间证据下学会可靠推理，而不只是高效处理更多帧。

Result: SAVER 在强化后训练中联合优化配对的密集/稀疏视频视图，用 grounding 奖励和可靠性门控参考奖励鼓励稀疏视图预测保留任务相关时序证据。训练仅使用 1,250 个随机时间定位样例，不使用视频问答标注。跨三个时间定位和六个视频问答基准、不同帧预算，表现均持续提升；使用显著更少帧时，可匹配或超过密集帧 Qwen3.5 基线。

Insight: 密集视图只作训练时证据参考，学到的定位能力迁移到稀疏推理，并扩展到问答任务。摘要未列各帧预算对应的分数。

Abstract: Video-language models commonly assume that more temporal observations lead to more reliable reasoning. We question this assumption and argue that the key challenge is not merely processing more video frames efficiently, but learning to reason reliably under limited temporal evidence. We propose SAVER, a dense-to-sparse post-training framework that uses dense video views as training-time references for sparse-frame inference. During reinforcement post-training, paired dense and sparse views are optimized with grounding rewards and a reliability-gated reference reward, encouraging sparse view predictions to preserve task-relevant temporal evidence. Notably, SAVER is trained only on 1,250 randomly sampled temporal grounding examples, without using any video question answering annotations. Across three temporal grounding benchmarks and six video question-answering benchmarks, SAVER consistently improves performance across frame budgets. In particular, SAVER can match or surpass dense-frame Qwen3.5 baselines while using substantially fewer frames. These results show that temporal grounding can serve as an effective evidence-localization proxy for learning sparse video reasoning that transfers to broader video understanding tasks.

[8] LVSPM: Long Sequence View Synthesis and Pose Estimation Model cs.CVPDF

Xi Chen, Yachi Zhang, Linghao Chen, Minghua Liu, Hao Su

TL;DR: 提出 LVSPM，联合估计相机位姿与新视角图像，只需 RGB 和位姿监督，并通过测试时训练扩展到数百视图和大场景。

Details

Motivation: 从未标定图像集合恢复位姿并合成新视角通常依赖难以取得的稠密 3D 真值，长序列下模型也需保持可扩展性。

Result: LVSPM 仅以 RGB 图像和位姿监督训练，不需要稠密 3D 标注，并采用测试时训练层适配最多数百个输入视图。在 RealEstate10k、Co3Dv2、DL3DV 上，输入 16–256 视图时位姿估计超过 VGGT，严格阈值处优势尤其明显。对于更多视图覆盖更大场景的实用新视角协议，其无位姿质量达到 SOTA，PSNR 甚至高于依赖位姿模型；随场景变大仍保持高质量，而基线明显退化。摘要未提供逐项指标数值。

Insight: 位姿估计和合成质量共同评估可检验模型在长序列中的扩展性；文中以测试时训练适应更多视图，且不需要稠密几何标签。两类结果采用不同评测设定。

Abstract: We present LVSPM, a generalizable model that jointly estimates camera poses and synthesizes novel views from uncalibrated image collections. Trained with only RGB images and pose supervision, LVSPM avoids dense 3D ground truth and employs test-time training (TTT) layers to scale seamlessly to hundreds of input views. On RealEstate10k, Co3Dv2, and DL3DV, LVSPM surpasses VGGT in pose estimation across 16-256 views, with especially large margins at strict thresholds. For novel view synthesis under a practical protocol where more views cover larger scenes, LVSPM achieves state-of-the-art pose-free quality—surpassing even pose-dependent models in PSNR—and still maintains high quality as scene scale grows, while baselines collapse. The code is available at https://burningdust21.github.io/Projects/LVSPM .

[9] Mid-Training Language Models on Raw Video cs.CV | cs.AI | cs.LGPDF

Jaedong Hwang, Xiaoqian Shen, Ernie Chang, Changsheng Zhao, Chong Zhou

TL;DR: 研究仅用无字幕原始视频进行语言模型视觉中期训练：让模型预测连续帧视觉 token，并检验图像/视频能力与文本能力的影响。

Details

Motivation: 多模态 LLM 主要由配对图文或标注视频训练，原始网络视频很少用于已有语言模型的继续训练。作者检验无字幕、无文本损失的原始视频能否作为有效中期训练数据。

Result: 作者以连续视觉 token 表示帧，让模型预测下一个视觉 token；对 Qwen3-1.7B 使用 YT-Temporal-1B 原始片段中期训练，并对中期训练与未训练模型施加相同图文指令微调，以隔离中期训练因素。中期训练模型在四个视频基准平均高 2.9 分，在覆盖感知、文档和图表任务的十个图像基准平均高 5.1 分。尽管中期阶段没有文本，14 个文本基准平均分仍为 48.9，对照 48.0。训练曲线显示图像/视频收益在 30% 训练内出现后趋于平台，变化小于 0.5 分。预测字幕未超过预测下一视觉 token，说明可保持纯自监督且避免自动字幕的计算与噪声。

Insight: 原始视频的下一视觉 token 预测可作为已有 LLM 的自监督中期训练信号，并把收益迁移到图像理解，未损害文本评测。实验证据基于 Qwen3-1.7B 和所测数据集，摘要未列具体基准名称。

Abstract: Multimodal large language models learn mostly from paired image-text data or annotated video, and raw web video is rarely used to further train an existing language model. We study whether raw video, with no captions and no text loss, can serve as mid-training data for a pretrained language model. Frames are encoded into continuous visual tokens, and the language model learns to predict the next visual token. We mid-train Qwen3-1.7B on raw clips from YT-Temporal-1B and then apply the same image-text instruction tuning to it and to the model without mid-training, so that the two differ only in mid-training. The mid-trained model scores 2.9 points higher on average across four video benchmarks and 5.1 points higher across ten image benchmarks, spanning perception, document, and chart tasks. Text performance is preserved even though mid-training includes no text, with an average of 48.9 across 14 text benchmarks compared with 48.0 for the model without mid-training. Analyses across training show that the image and video gains emerge within 30% of training and plateau thereafter, varying by less than 0.5 points. Predicting captions fails to outperform next-visual-token prediction, demonstrating that video mid-training can remain purely self-supervised without the computational overhead or labeling noise of automated captioning.

[10] Region-Aware CLS Token Augmentation for Fine-Grained Image Retrieval cs.CV | cs.LGPDF

Ian de Holanda Cavalcanti Bezerra, Vivek Trivedy, Lucas Pascotti Valem, Longin Jan Latecki

TL;DR: 提出面向细粒度图像检索的区域感知 CLS token 增强方法，将 DINOv2-reg 语义 token 与自动匹配的局部 patch 区域结合，形成轻量多向量表示。

Details

Motivation: 图像检索常用 ViT 单个全局 CLS 描述符压缩整幅图像语义，细粒度检索中这可能丢失局部对象和部件信息。保留所有 patch 向量又会带来较大存储成本。

Result: 方法针对 CLS 和四个 register token 各匹配一个空间 patch“伙伴”，以 N×N patch 区域构建局部 ROI token；利用 DINOv2-reg register 自发学习的对象/部件表示，不需要外部框或显著性模块。再用类似 ColBERT 的逐 token 对齐多向量检索。实验显示 register token 提供补充 CLS 的细节，自动池化 ROI token 可提高细粒度区分；少量多向量 token 的检索效果优于 DINOv2-reg 单向量基线，同时可用于大规模搜索。

Insight: 局部语义区域可由语义 token 与空间表示自动匹配得到，在无需密集存储全部 patch 特征的情况下提升细节匹配。摘要没有提供检索数据集、指标数值或存储量。

Abstract: Image retrieval methods often rely on a single global semantic descriptor extracted from an image, e.g., the [CLS] token in vision transformers. However, trying to squeeze all the semantic information of an image into a single descriptor can hurt downstream retrieval performance, especially for fine-grained retrieval tasks. In this work, we augment the semantic tokens in the newer visual transformers, the global [CLS] token and the four register tokens, with a carefully selected collection of spatial tokens, aiming to capture the spatial region representation that characterizes the contents captured in each of the semantic tokens. We leverage the DINOv2-reg model, which includes register tokens that emergently learn object and part-based representations. For each “cue” token ([CLS] and each register token), we find a “buddy” image patch token and extract an N x N patch region to produce a set of localized ROI tokens. Our approach automatically captures important regions of interest without any external bounding boxes or saliency modules, purely by matching semantic tokens with their spatial representation regions. Furthermore, we incorporate these tokens into a multi-vector retrieval framework inspired by ColBERT, enabling fine-grained matching via a per-token alignment mechanism while avoiding the large storage cost of keeping all patch embeddings. Through extensive experiments, we find that (1) register tokens encode useful fine-grained details that can complement the [CLS] token; (2) automatically pooled ROI tokens further improve fine-grained discrimination; and (3) multi-vector retrieval with a small set of tokens improves over a DINOv2-reg single-vector baseline while remaining tractable for large-scale search. The code is available at https://github.com/IdhcbIan/Augmenting_CLS_with_ROI_tokens.

[11] Omni-Diffusion-Distill: Few-Step Distillation of Unified Multimodal Diffusion Large Language Models cs.CV | cs.LGPDF

Hong Huang, Chenhongyi Yang, Junzhe Sun, Animesh Sinha, Wuyang Chen

TL;DR: 提出 Omni-Diffusion-Distill，两阶段统一多模态扩散模型蒸馏，在离散 token 空间同步压缩图像生成和多模态理解，并以专门正则抑制退化。

Details

Motivation: 统一多模态扩散 LLM 同时具备图像生成与理解，但迭代解码常需数十至数百次前向。既有少步蒸馏偏重图像或文本单一能力，难将完整离散多模态扩散模型压成高效学生。

Result: 第一阶段重放缓存教师轨迹训练学生跳过解码步，第二阶段在学生自身 rollout 中间状态上细化；图像、文本生成与理解都在离散 token 空间对齐。成对碰撞惩罚减少并行文本解码重复，熵匹配指导缓解图像生成拟合锐化教师分布导致的熵塌缩。图像生成步数由 128 降到 8，多模态理解由 512 降到 64，墙钟速度分别提升 18.2×、21.2×。对应预算下，文生图 GenEval 为 0.828、DPG-Bench 为 83.0；理解的 MM-Vet GPT judge 为 20.0、COCO captioning 为 57.2，后者为教师同等步数 28.4 的两倍。

Insight: 统一蒸馏需要分别处理并行文本重复与图像生成熵塌缩，才能同时保留两类能力。这里多个指标采用不同基准和评分方法，不能直接横向比较。

Abstract: Unified multimodal diffusion large language models (dLLMs) offer a single architecture for both image generation and multimodal understanding, but their iterative decoding requires tens to hundreds of forward passes. Existing few-step distillation methods largely focus on either image generation or text generation, making it unclear how to compress a fully discrete multimodal dLLM into a single efficient student while preserving both generation and understanding. We introduce Omni-Diffusion-Distill, a unified two-stage distillation framework that retains strong generation and understanding capabilities while substantially reducing the inference cost of a unified multimodal dLLM. Omni-Diffusion-Distill aligns the distillation of both generation and understanding, for both images and text, in the discrete token space. In the first stage, the student is trained to skip decoding steps by replaying cached teacher trajectories, and in the second stage the student is refined on intermediate states along its own rollouts. We further remedy two sources of degradation in unified distillation with a pairwise collision penalty that reduces repetition under parallel text decoding, and entropy-matched guidance that prevents entropy collapse caused by fitting the sharpened teacher distribution in image generation. Omni-Diffusion-Distill achieves state-of-the-art trade-offs between decoding efficiency and generation and understanding performance for multimodal dLLMs, reducing image generation from 128 to 8 decoding steps and multimodal understanding from 512 to 64, giving 18.2x and 21.2x wall-clock speedups. Under these budgets, it scores 0.828 on GenEval and 83.0 on DPG-Bench for text-to-image generation, while reaching GPT judge scores of 20.0 on MM-Vet and 57.2 on COCO captioning (twice the teacher’s 28.4 at the same steps) for multimodal understanding.

[12] Rendering-Free Lookahead for Question-Guided Active Vision cs.CV | cs.ROPDF

Koya Sakamoto, Daichi Azuma, Shuhei Kurita, Naoya Chiba, Yusuke Iwasawa

TL;DR: 提出 Rendering-Free Lookahead（RFL），训练时利用 3D Gaussian Splatting 教师渲染未来视图并蒸馏其可回答性，部署时无需渲染即可选择相机动作。

Details

Motivation: 主动视觉机器人需移动相机以显露当前视角看不到的任务相关信息；对于视角依赖问答，模型不仅要理解当前图像，还要预测哪些未观测视图有助于回答。

Result: RFL 将视图可回答性定义为 VLM 对某视角是否足以回答问题的估计，并用预测未来可回答性排序相机候选动作。训练时，特权教师在 3DGS 场景中渲染候选未来视图，由冻结 VLM 计算一步和两步目标；两阶段蒸馏后，学生根据问题、近期观测和候选动作预测动作价值。部署不需未来视图渲染。在 377 个未见环境 E3VS-Bench 测试 episode 上，相同 VLM 的直接动作基线相比，RFL 平均裁判分提高 43%。

Insight: 把昂贵的未来视图渲染限于离线教师训练，部署时用学生预测动作价值，转移特权视觉前瞻能力。效果针对未见环境基准和同一 VLM 对照，摘要未报告逐任务结果。

Abstract: Active robot vision requires controlling the camera to reveal task-relevant information that is hidden from the current viewpoint. For example, determining what is inside a box may require raising the camera and looking down into it. For viewpoint-dependent question answering, the challenge is to select camera motions that expose the visual evidence needed to answer the question. Although vision-language models (VLMs) can interpret observed images, selecting such motions requires anticipating the usefulness of unseen views. We quantify this usefulness as answerability, a VLM’s estimate that a view suffices to answer the question, and present Rendering-Free Lookahead (RFL), a viewpoint-selection policy that ranks candidate camera motions by predicted future answerability. RFL transfers visual lookahead from deployment to offline training. At training, a privileged teacher renders candidate future views in 3D Gaussian Splatting (3DGS) scenes and uses a frozen VLM to compute one- and two-step answerability targets. Through two-stage distillation, a student learns to predict these action values from the question, recent visual observations, and a candidate camera motion. At deployment, RFL uses these predicted values to select camera motions without rendering future views. On 377 E3VS-Bench test episodes in unseen environments, RFL improves the mean judge score by 43% over a direct-action baseline using the same VLM. These results support learning camera-control policies from privileged visual lookahead for viewpoint-dependent question answering.

[13] Transforming Image Editors into Video Editors cs.CVPDF

Feng Wang, Zijie Li, Ceyuan Yang, Alan Yuille, Peng Wang

TL;DR: 提出 Anchor-based Video Editing（AVE），将视频编辑拆成稀疏关键帧编辑与时序传播，用现成强图像编辑器加运动引导图生视频模型完成。

Details

Motivation: 图像编辑器在语义理解、画面质量和指令遵循方面进步明显，视频编辑仍更困难且成本高。作者希望避免训练一个端到端整体视频编辑器。

Result: AVE 两阶段工作流先选取关键帧，由强图像编辑器完成组合编辑；随后运动引导的图像到视频扩散模型将修改后关键帧作为固定锚点生成视频。在 IVEBench 和 VIE-Bench 上，方法在指令遵循、时间一致性和内容保真方面表现强。消融发现最终视频编辑质量与底层图像编辑器质量高度相关，提示可依靠更强图像编辑基础和轻量迁移推动视频编辑。摘要未提供具体指标。

Insight: 稀疏关键帧锚定能直接继承图像编辑器能力，同时把时间传播交给视频生成器。基于该分解，改进上游图像编辑器可能比从头训练视频编辑器更有效；结论仍基于两个基准。

Abstract: Recent image editing systems have achieved impressive semantic understanding, visual fidelity, and instruction-following ability, while video editing remains substantially more difficult and costly. In this paper, we present a simple alternative to end-to-end video editing: instead of training a monolithic video editor, we transform a strong image editor into a video editor through anchor-based generation. Our key insight is that video editing can be decomposed into two subproblems: editing a sparse set of keyframes and propagating those edits across time. Based on this observation, we propose Anchor-based Video Editing (AVE), a two-stage framework in which a powerful image editor first performs composed editing on selected keyframes, and a motion-guided image-to-video diffusion model then generates the final video by treating the edited keyframes as fixed anchors. This design directly inherits the strengths of modern image editors while avoiding expensive end-to-end video editing training. Experiments on IVEBench and VIE-Bench show that AVE achieves strong performance in instruction following, temporal consistency, and content fidelity. Further ablations reveal that final video editing quality is strongly correlated with the quality of the image editor, suggesting that future progress in video editing may come from stronger image editing foundations and lightweight transfer to video. Code is available at https://github.com/wangf3014/AVE.

[14] Expression-Diverse References for Identity-Preserving Video Generation cs.CVPDF

Tianwen Fu, Wenbin Teng, Gonglin Chen, Junyi Ouyang, Haolin Xiong

TL;DR: 研究表达变化对身份保持视频生成与身份相似度评估的影响，构建多表情身份参考图库，并将其用于扩展 Stand-In 条件输入。

Details

Motivation: 单张中性肖像只呈现一种面部状态；表情变化会以身份相关方式改变外观，使未见表情下的身份难以从单参考确定。中性参考相似度也会因强表情而下降，即便生成/真实图像属于同一人。

Result: 作者先用受控照片和 MEAD 视频量化人脸识别相似度随表情强度的变化，再构建紧凑且表情多样的参考图库，以更稳健评估表情动作下的身份相似度，并分别报告轻微、强烈和极端表情表现。生成方面，将 Stand-In 扩展为接受多表情参考，并从训练视频筛选一致且多样的人脸裁剪；仅有单张肖像时，用预训练面部重演模型合成额外参考。在受控基准上，真实和合成参考集在三种表情强度下均超过所测基线，极端表情时增益最大。

Insight: 身份评估应控制表情强度，并用多样表情参考减轻单一中性图像的欠定性。合成参考也能改善身份保持，但摘要未给出表情相似度数值或生成质量权衡。

Abstract: Identity-preserving video generation aims to maintain a subject’s identity while synthesizing realistic videos. Yet a single reference portrait captures the subject’s appearance under only one facial configuration. As expressions change, facial appearance can vary in highly identity-specific ways, leaving the subject’s appearance under unseen expressions underdetermined by the reference alone. This expression-dependent variation also complicates evaluation: similarity to a neutral reference may decrease under strong expressions even for real images of the same person. We investigate this limitation from both generation and evaluation perspectives. First, we quantify how face-recognition similarity varies with expression intensity using controlled photographs and MEAD videos. We then construct a compact yet expressive reference gallery that captures diverse expression-dependent facial configurations. Matching against this gallery provides a more robust measure of identity similarity under expressive motion. To further expose performance degradation with expression intensity, we report identity similarity separately for mild, intense, and extreme expressions. For generation, we extend Stand-In to condition on our expression-diverse reference sets and develop a data-curation pipeline that extracts consistent yet diverse face crops from training videos. In practical settings where only a single portrait is available, we construct the reference set by synthesizing additional expressions with a pretrained facial reenactment model. On our controlled benchmark, both real and synthesized reference sets outperform the evaluated baselines in identity similarity across all three expression-intensity regimes, with the largest improvements for extreme expressions.

[15] Learning What to Trust in Multimodal Learning under Noisy Supervision cs.CV | cs.LGPDF

Jiashuo Zou, Xiaobo Xia

TL;DR: 提出 REFINE 多模态标签噪声检测框架，联合 fused 和单模态表示为每类选择可靠子空间，并据判别方向筛选干净监督样本。

Details

Motivation: 现实多模态分类往往缺少高质量标签；传统噪声标签样本选择方法主要针对单模态，未充分利用多模态信息。

Result: 论文先分析表征结构与噪声检测能力的关系，再构建判别特征方向以区分目标类与背景类，为每个类别选择噪声检测表现更好的可信表示空间。在各可信空间中，依据实例表示与判别特征向量的对齐度筛选样本，再合并各空间选出的子集，用更干净监督更新多模态分类器。作者在多种任务上的广泛实验报告 REFINE 优于基线方法；摘要没有列数据集、噪声率或指标数值。

Insight: 不同类别可能需要不同的表示空间判断标签可靠性；融合与单模态视角互补，筛选后再更新分类器可减轻错标影响。方法效果幅度和噪声设定需查看完整实验。

Abstract: Multimodal classification processes and relates information from multiple modalities to achieve more accurate predictions. However, existing methods typically rely on high-quality ground-truth labels, which are difficult to obtain in real-world scenarios. While sample-selection methods for learning with noisy labels aim to identify correctly labeled examples from noisy data, traditional methods primarily focus on unimodal settings and fail to exploit multimodal information fully. This motivates us to build a more reliable noise detector in multimodal learning. To this end, we theoretically analyze the relationship between representation structure and noise detection capability. Based on this analysis, we propose REFINE, which is a multimodal label-noise detection framework that jointly uses fused and unimodal representations for label-noise detection. Specifically, REFINE constructs discriminative eigenvectors through discriminative analysis of the target and background classes and selects trusted representation spaces with better noise detection capability for each class. Within each trusted space, REFINE measures the alignment between each instance representation and the discriminative eigenvectors. It then combines the subsets selected from these spaces. The combined set provides cleaner supervision for updating the multimodal classifier, thereby reducing the influence of mislabeled examples during training and improving model generalization. Extensive experiments across diverse tasks demonstrate REFINE’s superiority compared to baseline methods. The source code will be publicly available.

[16] AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understanding cs.CV | cs.AIPDF

Tianhui Cai, Xinglong Sun, Chao Fang, Zhenxin Li, Rui Song

TL;DR: 提出 AffordDrive3D，将驾驶相关可行驶区域和碰撞风险区域与未来场景几何联合建模，为世界动作模型提供规划相关语义和空间结构。

Details

Motivation: 世界动作模型通常预测 RGB 外观，部分工作加入几何提升空间理解；但稠密几何覆盖全场景，却未标出对自车动作最关键的区域。驾驶策略还需预测可安全行驶区域与碰撞风险。

Result: AffordDrive3D 以 VLM 骨干预测直接影响自车运动的可行驶区域和碰撞关键区域，并从 RGB 世界模型潜变量预测未来几何。在 NAVSIM 上达到 91.3 PDMS 和 89.9 EPDMS，作者报告为 SOTA。

Insight: 把与行动相关的可供性区域和对应未来几何一起预测，避免让规划器从整场景几何中自行找出安全通行线索。摘要未给出相较基线具体差值。

Abstract: World-action models have recently improved autonomous driving by jointly learning future scene prediction and trajectory generation. Most existing approaches model the future primarily through RGB appearance, and recent works have begun to incorporate geometric prediction to improve spatial understanding. However, dense geometry describes the spatial layout of the entire scene without indicating which parts are most relevant to the ego vehicle’s action. For driving, the model must also identify and anticipate where it can safely move and which regions may pose collision risks. Jointly modeling action-relevant regions and future geometry can provide the policy with both driving-relevant cues and their corresponding spatial structure. We therefore propose AffordDrive3D, an affordance- and geometry-aware world-action model that jointly learns future action-relevant regions and spatial structure. In order to capture the scene semantics and driving context needed for driving affordance prediction, we build AffordDrive3D on a VLM backbone to forecast drivable areas and collision-critical regions that directly affect ego motion, while predicting future geometry from RGB world-model latents. On NAVSIM, AffordDrive3D achieves state-of-the-art performance with 91.3 PDMS and 89.9 EPDMS, demonstrating the effectiveness of jointly modeling future affordances and geometry for trajectory planning.

[17] Refine Connections, Close the Gap: A Reliable Enhancement Framework for Driving Scene Topology cs.CVPDF

Xiaoqi Wang, Dingyi Zhaung, David Paz, Wenbin He, Yucai Bai

TL;DR: 提出 TopoEnhance，将驾驶场景拓扑增强建模为受噪真值图恢复，使模型校正不可靠道路连通关系，并同时评估连续拓扑分数和离散决策图。

Details

Motivation: 自动驾驶地图元素检测已较成熟，但车道及交通元素的连接推理仍远低于检测结果所允许的理论上限。常见方法阈值化连续连接分数得到决策图，但分数不一定符合真实逻辑连通概率，易出现错误连接或漏连；现有基准也较少衡量决策所需的离散拓扑。

Result: TopoEnhance 将拓扑增强定义为去噪图重建：训练模型从随机扰动的真值图恢复结构一致性，以修复逻辑冲突和不可靠连边。跨多种基线的实验显示，TOP 连续拓扑指标和作者适配的离散连接 Topology Jaccard Similarity（TJS）均持续提升。框架与方法来源无关，可增强多种 SOTA 基线，无需重新训练原始模型。摘要未列具体分数或接近理论上限的数值。

Insight: 输出决策可用的离散连通图与优化连续打分是两个目标，评测需要覆盖二者。把增强视作图去噪允许在不重新训练源方法的情况下做后处理。

Abstract: In autonomous driving, understanding scene topology - the connectivity between lanes and traffic elements - is critical for safe path planning and motion control. While current methods excel at detecting individual map elements, their connectivity reasoning often falls short of its theoretical potential, leaving a significant performance gap relative to the theoretical upper-bound achievable given the underlying detections. Furthermore, the decision-ready topology graphs passed to downstream tasks often remain unreliable. Current approaches typically derive connectivity by thresholding continuous topology scores; however, these scores often fail to reflect the true logical likelihood of connectivity, resulting in false positives or missing connections. Existing benchmarks further overlook this issue by primarily evaluating continuous metrics, rather than assessing the discrete connectivity required for decision-making. To bridge these gaps, we propose TopoEnhance, a novel topology enhancement framework designed to unlock the latent potential of existing methods and improve the reliability of decision-ready topology. We formulate topology enhancement as a denoising-based reconstruction process, where the model learns to recover structural consistency from stochastically corrupted ground-truth graphs. This formulation enables the model to resolve logical inconsistencies and rectify unreliable connections, producing robust discrete topology graphs that closely approach theoretical maximum performance. Extensive experiments across different baselines show that TopoEnhance consistently improves both continuous topology metrics (TOP score), and discrete connectivity measured by our adapted Topology Jaccard Similarity (TJS) metric. As a flexible, source-agnostic framework, TopoEnhance delivers substantial gains across diverse state-of-the-art baselines without requiring retraining.

[18] Continuous Ground-Truth Construction and a Recovery Policy for Air–Water Robotic Tracking cs.CV | cs.AIPDF

Jiangong Xiao, Zhe Sun, Kanzhong Yao, Yuanbo Bi, Haofei Zhao

TL;DR: 为跨水面空气—水下跟踪构建运动捕捉监督的连续真值集，并提出由置信度触发的模板恢复策略以减轻气泡、折射和遮挡造成的跟踪丢失。

Details

Motivation: 空气—水下界面存在水花、气泡、折射、反光和突变外观，会使视觉观测失效。此时仅凭图像标注难反映目标物理位置，在线跟踪中受损观测又会污染运动估计和外观模板。

Result: 标注流程同步相机帧与运动捕捉位姿，投影已知目标几何，并用介质条件残差修正水下投影，再人工审阅，得到仅供评估的跨介质测试集，共 22,346 帧。CMRP 以置信度触发模板选择，向 MixFormerV2 输入固定初始模板、触发前窗口最优模板和 Kalman 引导的触发帧图像裁剪及相应权重，不重训视觉骨干。准确率评估 Macro Success AUC 为 49.90，比官方 MixFormerV2 高 2.95 分。在选定跨介质转换和遮挡恢复片段上，相对官方更新覆盖率从 47.91% 升到 50.43%；对成功恢复视频，平均丢失到恢复延迟从 55.3 帧降到 49.3 帧。

Insight: 连续物理位姿真值帮助避免把视觉盲区误标为目标消失；固定、近期优质及触发帧模板可在置信度下降时提供多种恢复线索。局部恢复指标在筛选片段上统计，应与全测试集 AUC 分开理解。

Abstract: Visual tracking across the air-water interface is challenged by splashes, bubbles, refraction, reflections, and abrupt appearance changes that can temporarily invalidate observations. This setting poses two coupled difficulties: first, for evaluation, image-only annotation cannot reliably describe the target’s physical location during visual blindness; second, for online tracking, corrupted observations can contaminate motion estimates and appearance templates. We address the first difficulty with a construction pipeline that synchronizes camera frames with motion-capture poses, projects known target geometry, corrects underwater projection with a medium-gated residual, and subjects the annotations to manual review. This yields an evaluation-only cross-medium test set of 22,346 frames. We further introduce a Cross-Medium Recovery Policy (CMRP) centered on confidence-triggered template selection. It supplies MixFormerV2 with the fixed initial template, a window-best pre-trigger template, and a trigger-frame Kalman-guided image crop, together with their associated weights, without retraining the visual backbone. In the accuracy evaluation, CMRP achieves 49.90 Macro Success AUC, 2.95 points above MixFormerV2 Official. On selected cross-medium transition and occlusion-recovery intervals, CMRP increases MixFormerV2 tracking coverage from 47.91% to 50.43% relative to Official updating, while mean loss-to-recovery latency over successfully recovered videos decreases from 55.3 to 49.3 frames.

[19] TKCAM: Text and Keyframe to Camera Trajectory Generation cs.CVPDF

Haozhe Yang, Zhiyang Dou, Zekai Gu, Cheng Lin, Wenping Wang

TL;DR: 提出 TKCAM，以文本和稀疏时间关键帧为条件生成相机轨迹，并构建 RealEstate10K-Cap 与跨域评测及通用 CLaTr 评估器。

Details

Motivation: 高质量、可控相机运动对 AI 电影制作、视频合成和三维理解都重要；用户除了文字描述，也可能希望在指定时间提供图像观察来约束轨迹。

Result: TKCAM 用位置、速度和连续旋转表示组成 12 维运动特征，再由残差向量量化器离散为层级运动 token。两阶段掩码 Transformer 学习重建和细化 token，并以自注意力和交叉注意力融合多模态条件。用户可提供自由文本和部分时间戳 RGB 帧作为稀疏视觉锚点，引导中间轨迹。作者整理 RealEstate10K-Cap 文本—相机数据集，并建立跨域基准和 Universal CLaTr Evaluator。实验报告在 Fréchet 距离、文本—动作匹配和 R@K 检索指标上优于近期 SOTA；另分析时间平滑度与跨域泛化。摘要未列具体分数。

Insight: 稀疏视觉关键帧为相机轨迹提供局部时间锚定，文本控制其整体语义；层级 motion token 负责建模轨迹动力学。摘要未说明数据规模和具体指标值。

Abstract: Generating high-quality and controllable camera motion is essential for AI-assisted cinematography, video synthesis, and 3D scene understanding. We introduce TKCAM, a Text- and Keyframe-conditioned CAMera-motion synthesis framework based on generative masked modeling. We represent camera dynamics using a 12-dimensional kinematic feature comprising position, velocity, and a continuous rotation representation and discretize them into hierarchical motion tokens via a Residual Vector Quantizer (RVQ). A two-stage masked transformer architecture then learns to reconstruct and refine these tokens, utilizing explicit self- and cross-attention modules for multimodal conditioning. A central feature of our framework is sparse visual keyframe conditioning: users can provide free-form text prompts together with RGB observations at selected timestamps, which provide temporally localized visual guidance for generating coherent in-between trajectories. Furthermore, to advance evaluation standards, we curate RealEstate10K-Cap, a large-scale text-camera dataset, and establish a cross-domain benchmark with a Universal CLaTr Evaluator. Extensive experiments demonstrate that TKCAM surpasses recent state-of-the-art baselines on Fréchet distance (FID), text-motion matching scores, and retrieval metrics (R@K), while additional analyses evaluate temporal smoothness and cross-domain generalization. Code is available at https://github.com/linearalgebrayhz/TKCAM.

Mengping Dong, Jinbao Li, Fei Li

TL;DR: 提出 DiscoVL，以正交对抗正则化和多分支低秩残差对齐解耦视觉—语言表示子空间，提升 VLM 的跨域、base-to-novel 和少样本泛化。

Details

Motivation: 预训练视觉语言模型适配新下游任务时，参数高效提示学习可能产生不一致表征，并难以应对语义分布偏移；传统三元组损失也可能将特征过度拉向类别中心。

Result: DiscoVL 设计多分支低秩残差对齐器，将表示分解为子空间，并在每层视觉与文本流之间双向传递反馈；对抗三元组损失加入正交正则，防止类别中心坍缩。15 个基准上的评估显示，其在 base-to-novel 泛化、跨数据集评估和少样本学习中持续优于 SOTA。摘要未列模型、指标或提升幅度。

Insight: 跨模态对齐需要保持子空间结构并避免表示向类中心坍缩；低秩双向交互与正交约束分别处理模态交互不足和泛化退化。

Abstract: Pre-trained vision-language models excel across varied perception tasks, but adapting them to novel downstream settings without sacrificing generalization remains non-trivial. Existing parameter-efficient prompt learning method often yields inconsistent representations and fails to account for semantic distribution shifts. In this work, we present DiscoVL, a disentangled cross-modal representation learning framework that couples orthogonal adversarial regularization with structured cross-modal alignment for vision-language models. To address the insufficient cross-modal interaction, our DiscoVL designs a multi-branch low-rank residual aligner that decomposes representations into subspaces and enables bidirectional cross-modal feedback between visual and textual streams at each layer. Furthermore, while conventional triplet constraints overfit features to class centroids, we design an orthogonal regularization for adversarial triplet loss, which prevents centroid collapse and substantially boosts generalization. Evaluations on 15 benchmarks demonstrate that DiscoVL delivers consistent improvements over state-of-the-art methods for base-to-novel generalization, cross-dataset evaluation, and few-shot learning

[21] Improving Image-Based Nutrition Estimation Through Multimodal Food-Item Verification and Recovery cs.CV | cs.AI | eess.IVPDF

Jingbo Yue, Bruce Coburn, Jinge Ma, Jui-Feng Chi, Fengqing Zhu

TL;DR: 提出单图营养估算中的多模态食物核验与遗漏恢复流程，分别检查食物身份和区域是否支持份量估计，并至多触发一次定向补检。

Details

Motivation: 单张图像营养估算可能静默漏掉可见食物；即使识别正确，提出的区域也可能不适合估算份量，导致后续营养值错误。

Result: 框架用 MLLM 清点图中食物，分别核验食物身份和候选 2D 区域是否适合份量估计；全图审查根据核验结果找出未解决缺口或遗漏项，最多触发一次定向恢复。恢复区域在不提供恢复提示的情况下重新核验，再汇总成最终食物清单供营养估算，无需任务专用微调。在共同有效输出样本的配对评估中，各测试设置下食物级 grounding 都改善质量估算准确度；相较适配检索基线，能量准确率也提升，身份精确率与召回率同时改善。恢复后的视觉覆盖率在推理阶段单独评估，不使用真值标注。

Insight: 将“识别是否正确”和“区域是否能支持份量估计”分开核验，可以更明确地定位误差并只对缺失内容补查。摘要未报告具体精度提升数值，视觉覆盖评估也与有标注的配对评估分开。

Abstract: Single-image nutrition estimation can fail silently when visible foods are missed. Even when a food is correctly identified, its proposed region may not support portion estimation. We propose a framework that uses multimodal large language models (MLLMs) to inventory visible foods and separately verify food identity and whether each proposed 2D region supports portion estimation. One whole-image review uses these verification results to identify unresolved gaps and omitted foods, triggering at most one targeted recovery pass. Recovered regions are re-verified without access to the recovery prompt, then reconciled into a final item set for nutrition estimation. The framework requires no task-specific fine-tuning. Matched evaluation on common valid-output samples shows that item-level grounding improves mass accuracy across all tested settings and energy accuracy relative to an adapted retrieval baseline, with item-identity precision and recall also improving, while post-recovery visual coverage is assessed separately at inference time without ground-truth annotations.

[22] SP-DocReader: Difference-Aware Self-Play for Precise Document OCR cs.CV | cs.AIPDF

Wenjie Liao, Xiaohui Song, Liangjie Zhao, Haonan Lu

TL;DR: 提出 SP-DocReader，以参考/生成转录差异为目标进行 OCR 自博弈训练，通过未匹配 token 的上下文打分和定点似然监督修复 SFT 后残余错误。

Details

Motivation: 输入和训练预算有限时，VLM 页级转录仍不准确；监督微调后剩余的局部差异可作为重点训练信号，而不是均匀更新所有 token。

Result: Reading Discrepancy Masking 用最长公共子序列对齐参考与生成 token，并将未匹配位置连同完整条件前缀一起打分；Focused Fidelity Loss 对未匹配的真值位置增加直接负对数似然监督。训练只更新 OCR 模块，骨干冻结。作者推导组合梯度，区分相对分数优化与直接监督效果。相较 SFT-2，SP-DR-3 在两个骨干上都降低 Vary-600K 字符错误率；Qwen3-VL-4B 上字符错误率约降低 54%，DocVQA 平均归一化 Levenshtein 相似度提高 3.7 分。

Insight: SFT 后自博弈聚焦剩余转录差异，可更有效利用有限训练预算；该方法同时兼有差异排序信号和直接纠错监督，且冻结主干。摘要未列两个骨干各自的绝对错误率。

Abstract: Accurate page transcription remains difficult for vision language models under limited input and training budgets. We present SP-DocReader, a self-play framework for optical character recognition (OCR) that targets residual errors after supervised fine-tuning. Reading Discrepancy Masking aligns reference and generated model tokens through a longest common subsequence, then scores unmatched positions with their full conditioning prefixes. Focused Fidelity Loss adds direct negative log-likelihood supervision at unmatched ground-truth positions. Only the OCR module is trained, while the backbone remains frozen. We derive the combined gradient to distinguish relative score optimization from direct supervision. Compared with SFT-2, SP-DR-3 reduces Vary-600K character error rate on both backbones. On Qwen3-VL-4B, it reduces character error rate by approximately 54 percent and improves DocVQA Average Normalized Levenshtein Similarity (ANLS) by 3.7 points. These results show the value of focusing self-play training on the discrepancies that remain after supervised fine-tuning.

[23] VAMR: Multi-Question Agentic Reasoning for Efficient Long-Form Video Understanding cs.CV | cs.AIPDF

Runquan Gui, Hanzhu Chen, Zehao Wang, Hanxin Zhu, Xin Li

TL;DR: 提出 VAMR，让同一视频的多个问题共享一条工具使用轨迹和分层记忆，并用问题时域策略优化训练在不同轮次完成的问题。

Details

Motivation: 长视频研究常为每个问题各自启动视频探索和记忆构建，重复消耗帧与推理轮数，错失联合收集证据和逐步形成共享视频理解的机会。

Result: VAMR 每轮由持久策略为一个或多个未解决问题调用工具，也可在证据充足时提交部分答案。问题条件视觉感知一次获取多个问题的细粒度线索；分层多问题记忆将可复用上下文整合成共享视频故事，同时保存各问题独立证据。监督微调初始化交互协议后，QHPO 用问题级 critic 评估每个活动问题价值，将问题优势对齐到直接服务它的轮次，再聚合优化共享策略。在 LVBench、Video-Holmes、LongVideoBench 上总体准确率最高、迭代方法中推理轮数最少。LVBench 准确率 62.1%，比 VideoARM 高 4.3 个百分点，同时推理轮数减少 85.9%、处理帧数减少 61.4%。

Insight: 多个问题共享探索轨迹可以共同积累证据，但各问题仍需保留单独证据链；QHPO 将不同问题在不同时间完成这一特性纳入信用分配。

Abstract: Long-form video understanding often involves multiple questions about different aspects of the same recording. Yet existing video agents typically process each question through an isolated tool-use trajectory. This repeatedly restarts video exploration and memory construction, missing opportunities to acquire evidence jointly and progressively build a shared understanding that supports the complete question set. We introduce \textbf{VAMR} (\textbf{V}ideo \textbf{A}gent for \textbf{M}ulti-Question \textbf{R}easoning), which coordinates all questions about a video through one shared tool-use trajectory. At each round, a persistent policy model can invoke tools for one or more unresolved questions and submit answers for questions with sufficient evidence. Question-conditioned visual perception retrieves fine-grained clues for several questions in one call, while layered multi-question memory integrates reusable context into a shared video story and preserves separate evidence for individual questions. After supervised fine-tuning initializes this interaction protocol, we propose question-horizon policy optimization (\qhpo) to optimize shared trajectories in which questions progress and finish at different rounds. Specifically, a question-level critic estimates the value of each active question, while round alignment maps each question advantage to the rounds that directly serve it before the aligned advantages are aggregated to optimize the shared actor. Across LVBench, Video-Holmes, and LongVideoBench, VAMR achieves the highest accuracy overall and the fewest reasoning rounds among iterative methods. On LVBench, it reaches 62.1% accuracy, exceeding VideoARM by \textbf{4.3} points while reducing reasoning rounds and processed frames by \textbf{85.9%} and \textbf{61.4%}.

[24] A Unified Score Matching Paradigm for Video Anomaly Detection and Anticipation cs.CV | eess.IVPDF

Congqi Cao, Zhenhe Liang, Hanwen Zhang, Yifan Zhao, Qinyi Lv

TL;DR: 提出 Uni-DSM，以统一去噪分数匹配框架同时进行视频异常检测与提前预警，使用输出差异构造预警监督而不依赖未来真值帧。

Details

Motivation: 生成式异常检测方法常受局部异常模式限制；视频异常预警比事后检测更具挑战，依赖真实未来帧的对比推理方式无法直接用于预警。

Result: Uni-DSM 使用共享噪声条件分数 Transformer、场景嵌入和运动加权，在同一分数建模框架中以不同推理/监督方式实现检测与预警。检测端 ADSM 通过自回归去噪逐步累积异常证据，以识别视觉线索之外的局部模式；预警端增加轻量解码器和自蒸馏 SDSM，根据输出差异构造监督，不需不可用的未来真值。作者在多个基准上报告 VAD 和 VAA 均达 SOTA，同时保持较高效率；摘要未列数据集或分数。

Insight: 通过同一数据分布分数建模框架统一事后检测与事前预测，且以输出差异替代未来标签监督。具体部署时提前量和效率数据需查看完整实验。

Abstract: Video anomaly detection (VAD) is a fundamental and safety-critical task in computer vision. Recent generative approaches detect anomalies from a distributional perspective, but remain limited by local anomaly modes. Meanwhile, video anomaly anticipation (VAA), as a proactive extension beyond post-hoc detection, introduces additional challenges. In particular, the contrastive inference paradigm in VAD, which relies on ground-truth frames, is not applicable to VAA, hindering its development. To address these challenges, we propose a unified score-driven framework, termed Uni-DSM, based on denoising score matching (DSM), which models anomaly patterns through likelihood estimation and score functions over the learned data distribution. Within this unified framework, we adopt a shared noise-conditioned score transformer backbone with scene-dependent embeddings and motion-aware weighting for distribution-level modeling. Instead of introducing separate architectures, Uni-DSM unifies VAD and VAA through different inference and supervision paradigms built upon the same score-based formulation. For VAD, we instantiate an autoregressive denoising score matching (ADSM) mechanism, which progressively accumulates anomalous evidence via autoregressive denoising, enabling enhanced perception of local modes beyond visual cues. For VAA, we extend the same architecture by incorporating a lightweight auxiliary decoder and a novel self-distilled denoising score matching (SDSM) mechanism. By constructing supervision from output discrepancies instead of relying on unavailable future ground truth, our method achieves efficient training suitable or early anomaly anticipation. Extensive experiments on multiple benchmark datasets demonstrate state-of-the-art performance in both VAD and VAA while maintaining high efficiency, establishing a unified and scalable pipeline from anomaly detection to anticipation.

[25] Multimodal Remote Sensing Image Registration: A Comprehensive Review, Challenges and Prospects cs.CVPDF

Zhiqiang Han, Yuanxin Ye, Qiuyun Wu, Jinhao Chen, Bai Zhu

TL;DR: 系统综述多模态遥感图像配准，按区域、特征和深度学习三类流程梳理方法、数据集、瓶颈与发展方向。

Details

Motivation: 多模态遥感配准是影像融合、变化检测和目标识别等协同处理的基础，但传感器几何、辐射机理、成像平台和环境造成的辐射、几何、尺度、视角及时间差异，使高精度鲁棒匹配困难。

Result: 文章按配准流程将方法归为区域基、特征基和深度学习三类，介绍各类核心原理、代表算法、优缺点；总结遥感多模态公开数据集及适用场景，并分析高精度配准研究瓶颈与未来方向。摘要未给出新的算法或实验结果。

Insight: 这是领域综述而非新方法论文，主要价值是按流程整合方法与数据资源，强调多源差异对稳健配准的影响。

Abstract: Multimodal remote sensing image registration is a crucial prerequisite for the collaborative processing and downstream application of remote sensing data, such as image fusion, change detection, and target recognition. However, significant variations in radiometry, geometry, scale, viewpoint, and time often exist between multimodal images. These differences, driven by varying sensor geometries, physical radiation mechanisms, imaging platforms, and environmental disturbances, pose severe challenges to achieving high-precision, robust registration. This paper systematically reviews the progress of mainstream multimodal remote sensing image registration methods. Based on their registration pipelines, existing approaches are categorized into three main types: region-based, feature-based, and deep learning-based methods. We detail the core principles, representative algorithms, advantages, and limitations of each category. Additionally, we summarize publicly available multimodal image datasets in the remote sensing domain, analyzing their specific characteristics and applicable scenarios. Finally, we highlight current bottlenecks in high-precision registration research and outline future development trends. This review aims to provide a comprehensive reference and valuable insights for researchers in related fields.

[26] MATE4D: Matrix-Guided Editable 4D Generation from a Single Image cs.CVPDF

Xiaotian Chen, Dongfu Yin

TL;DR: 提出 MATE4D，从单张图像生成可编辑动态 4D 场景：用文本控制背景构建时空多视图图像矩阵，再优化 3D 高斯并加入轻量形变动画。

Details

Motivation: 从单一视角恢复动态 3D/4D 内容仍困难，因为输入图像提供的结构线索有限、运动证据不足，导致几何与时间稳定性欠佳。

Result: 方法通过文本引导背景编辑构建时空多视图图像矩阵，为视角、外观和运动提供一致监督；再以这些合成观察优化 3D Gaussian 基元，并用轻量形变模块驱动动画。生成场景几何保持更忠实、时序更平滑、背景编辑更一致，减少上下文歧义和运动伪影。在 Objaverse-XL 和 Diffusion4D 上，作者报告视觉质量、效率和可控性均优于强基线，支持 AR/VR 内容创作；摘要没有提供具体指标。

Insight: 多视角加时间维合成观察补足单图缺失的视角和动作监督，文本编辑背景有助于减少上下文歧义。摘要未说明动态几何精度或不同场景类别的分项表现。

Abstract: Generative models have rapidly pushed content creation be-yond 2D imagery toward dynamic 3D and 4D scene synthesis. Yet pro-ducing realistic and temporally stable 4D content from a single image is still difficult because one view provides limited structural cues and weak motion evidence. We introduce MATE4D, a framework that converts one input image into editable dynamic 4D content. Our method constructs a spatio-temporal multi-view image matrix with text-guided background manipulation, delivering coherent supervision over viewpoint, appear-ance, and motion. These synthesized observations are used to optimize 3D Gaussian primitives, which are then animated through a lightweight deformation module to form a 4D representation. The resulting scenes preserve geometry more faithfully, maintain smoother temporal behavior, and keep background edits more consistent, reducing context ambiguity and motion artifacts. Experiments on Objaverse-XL and Diffusion4D show that MATE4D outperforms strong baselines in visual quality, effi-ciency, and controllability, supporting practical AR/VR content creation.

[27] IntactWorld: Joint World Modeling with Intact Features cs.CVPDF

Boming Tan, Xiangdong Zhang, Yan Xia, Qi Zhu, Deyi Ji

TL;DR: 提出 IntactWorld，以不压缩的视觉特征进行联合世界建模，并在中间层预测干净特征规避高维空间流场预测的流形优化困难。

Details

Motivation: 逼真的视频生成并不必然意味着理解现实世界逻辑。将多样世界知识纳入模型时，计算开销和维度对齐往往促使特征压缩，导致结构信息损失；直接在高维未压缩空间预测速度场又面临数据流形差距。

Result: IntactWorld 使用完整特征开展联合世界建模，在中间层预测干净特征 x₀ 而不是流速度 v，以缓解高维流形优化瓶颈。为降低引入完整世界知识的开销，Full-to-Compact 训练范式以精炼 CLS token 替代原始完整特征，使用单分支指导；空间显存占用降低 11.4%，推理延迟降低 43.8%。综合评测显示在 VBench 2.0 上较已有基线高 2.46 分。

Insight: 保留结构化完整特征用于表征学习，并将精炼 CLS 用于紧凑指导，试图兼顾世界知识与部署效率。VBench 2.0 增益为 2.46 分，摘要未提供其他基准表现。

Abstract: While recent video generation models synthesize highly realistic visuals, they lack a genuine understanding of intrinsic real-world logic. Existing methods attempt to understand the world by internalizing diverse world knowledge, yet constrained by computational overhead or dimensionality alignment, their learning processes inevitably compress features, causing a severe loss of structural information. To address this, we propose \textbf{IntactWorld}, a \textbf{Joint World Modeling Architecture} utilizing uncompressed \textbf{Intact Features}. Since data naturally reside on a low-dimensional manifold within a high-dimensional space, predicting the flow velocity $v$ within this uncompressed high-dimensional space induces a severe manifold gap. To successfully eliminate this optimization bottleneck, our framework instead predicts the clean feature $x_0$ at intermediate layers. Furthermore, to mitigate the computational overhead of incorporating complete world knowledge, we introduce a \textit{Full-to-Compact Training Paradigm}. By replacing raw full features with highly refined CLS tokens, this paradigm enables efficient single-branch guidance, reducing spatial memory consumption by 11.4% and cutting inference latency by 43.8%. Extensive evaluations demonstrate the effectiveness of IntactWorld, outperforming established baselines by 2.46 points on the VBench 2.0 benchmark.

[28] 3DTexMOR: 3D Gaussian Multi-Object Removal via Texture-Space Inpainting cs.CVPDF

Kunxin Guang, Yonghao Zhao, Jian Yang, Beibei Wang

TL;DR: 提出 3DTexMOR，在跨视图共享纹理空间进行多物体移除修补，再投影结果监督 3D Gaussian 几何与外观补全。

Details

Motivation: 现有 NeRF/3DGS 场景物体移除常逐张 2D 图像修补以指导 3D 完成；复杂多物体布局限制每视图可见上下文，不同视角的不一致修补还会产生冲突监督和模糊重建。

Result: 3DTexMOR 将多视角观测聚合成共享纹理图，在纹理空间补全缺失区域，再投影回相机视角监督高斯场景完成。为避免视角相关高光与反射影响，方法分解外观并聚合视角无关内禀属性，而非直接融合 RGB；还加入几何正则化约束补全区域形状。在多物体移除实验中，补全视觉合理且达到 SOTA；相比现有方法，PSNR 提高 5.8 dB，LPIPS 至少降低 22%。

Insight: 共享纹理空间汇集视角互补上下文，减少逐视图修补不一致；内禀属性聚合降低高光反射干扰。摘要未给出数据集和具体场景设置。

Abstract: 3D object removal aims to remove target objects from reconstructed scenes and complete the geometry and appearance of occluded regions. Existing NeRF- and 3DGS-based methods typically inpaint 2D images to guide 3D completion. However, complex multi-object layouts limit the surrounding context visible in each view, making 2D inpainting prone to artifacts. Inconsistent completions across views also introduce conflicting supervision and blurry reconstructions. We propose 3D Gaussian Multi-Object Removal via Texture-Space Inpainting (3DTexMOR). Our key idea is to perform inpainting in a unified texture space shared by all views. By combining complementary observations, this space provides richer context for recovering missing regions and promotes cross-view appearance consistency. We aggregate multi-view observations into texture maps, inpaint the missing regions, and reproject the completed maps into camera views to supervise Gaussian scene completion. To avoid the influence of view-dependent highlights and reflections, we decompose appearance and aggregate view-independent intrinsic attributes instead of RGB colors. We further introduce geometrically regularized Gaussian completion to constrain the geometry of the completed regions. Extensive experiments demonstrate visually plausible completions and state-of-the-art multi-object removal performance, improving PSNR by 5.8 dB and reducing LPIPS by at least 22% compared with existing methods.

[29] Dissecting Representation Structure in Vision Transformers: A Rigorous Architectural Study cs.CV | cs.LGPDF

Kim-Cuc Nguyen, Ngai-Man Cheung

TL;DR: 系统分析 ViT 中各模块产生的特征信息与泛化表现，并提出结构指标作为架构选择代理，用于识别更准确且计算成本不增的 ViT。

Details

Motivation: 理解 ViT 表示结构及其泛化行为对架构设计重要，但既有研究没有隔离分析模块级特征，也较少检验特征交互对性能预测的作用。

Result: 作者跨多个架构尺度进行特征信息分析，发现初始化时存在会产生冗余的特征坍缩，并提出缓解方法；以熵和最小特征值量化信息，发现二者可预测泛化；token 空间特征比 embedding 空间更忠实；线性子模块产生的特征对预测泛化表现也很关键。基于这些发现构造的代理使相关性排序较既有基线提升 18%–48%，可有效识别准确率更高且计算量更低或相近的 ViT 架构。

Insight: 仅检查最终表示不足以评估架构，初始化冗余、token 层特征和线性子模块信号都与泛化预测有关。摘要没有给出具体架构及数据集清单。

Abstract: Representation structure is crucial for understanding Vision Transformer (ViT) architectures and their generalization behavior. However, prior studies neither isolate nor analyze module-level features nor investigate how their interactions contribute to performance estimation. In this work, we conduct the first rigorous analysis of feature information across diverse architectural scales, empirically uncover the relationship between ViT representation and generalization behavior, and leverage these insights to guide efficient ViT design. Our contributions are fivefold: Across diverse architectural scales, 1) We identify feature collapse at initialization, which leads to redundancy, and propose a reduction scheme to mitigate this issue. 2) We quantify feature information using entropy and the minimum eigenvalue, demonstrating that these metrics serve as reliable indicators for generalization prediction. 3) We show that feature in the token space provides a more faithful representation than those in embedding space. 4) We discover an unexpected finding: features produced by linear submodules within ViT layers are critical for the prediction of generalization performance. 5) Our proposed proxy improves the correlation ranking by 18-48% over prior baselines and can effectively identify ViT architectures that achieve higher accuracy at lower or comparable computational cost.

[30] GATOR: Generative and Agentic 3D Object Reconstruction From Casual Images cs.CV | cs.ROPDF

Qirui Wu, Stan Birchfield, Hesam Rabeti, Angel X. Chang, Bowen Wen

TL;DR: 提出 GATOR，从一张或多张随手拍图像重建带纹理、相对场景位姿的完整 3D 物体，并通过多模态代理观察—编辑—渲染循环细化几何和纹理。

Details

Motivation: 从稀疏且不确定的随手图像重建完整场景对齐物体，需要推断遮挡区域表面并估计物体相对场景位姿；生成资产还需与场景上下文保持一致。

Result: 局部模态混合器先融合 patch 对齐的 RGB、目标掩码和点图特征，再进行跨视角推理，以保留场景信息同时区分物体与背景。文本类别名和字幕通过分阶段适配器补充结构、几何和外观条件。生成资产初始化多模态代理，代理通过观察引导的编辑—渲染—审查循环细化形状与纹理。在合成对象、杂乱桌面和室内场景上，作者报告几何/外观保真和稀疏观测位姿恢复表现强，并通过时间预算对比和场景级仿真展示效率与仿真可用性；摘要没有列具体指标。

Insight: 生成初始化负责从稀疏观察补全，代理再围绕特定实例迭代修改，形成生成式重建与场景验证闭环。摘要未说明各类场景的分数或完整重建时间。

Abstract: Reconstructing complete, scene-aligned 3D objects from casual images requires integrating sparse, uncertain observations and inferring surfaces hidden by occlusions. We present GATOR, a generative and agentic framework that recovers textured object assets and their scene-relative pose from one or more images. Our local modality mixer couples patch-aligned RGB, target-mask, and pointmap features before cross-view reasoning, preserving scene context while distinguishing the target from its surroundings. Text-guided semantic conditioning complements these spatial cues with category names and object captions through stage-specific adapters for structure, geometry, and appearance generation. The generated asset initializes a multimodal agent, providing instance-specific geometry and pose for targeted structural and texture refinement through an observation-guided edit-render-review loop. Across synthetic objects, cluttered tabletops, and indoor scenes, GATOR achieves strong geometric and appearance fidelity while recovering scene-relative pose from sparse observations. Time-budget comparisons and scene-level simulation further demonstrate the reconstruction efficiency and simulation readiness. Project page: https://research.nvidia.com/labs/lpr/gator/

[31] MiniVer-V: Identifying Minimal Sufficient Evidence for Short Video Verification cs.CV | cs.CL | cs.MMPDF

Leran Chen, Lingnan Kong, Zile Cai

TL;DR: 提出 MiniVer-V 和两层短视频核查框架，按证据充分性选择最小支持证据集，并在证据不足时明确弃答。

Details

Motivation: 短视频事实核查若把所有证据都交给核查器会引入噪声，按主题相关性选证又会把“相关”误当作“足以支持判断”；真正需要判断某证据子集是否足以可靠定论且无冗余。

Result: MiniVer-V 含 195 个短视频、支持/驳斥/证据不足三类判定，以及 5,510 个多模态证据单元（关键帧、语音转录、网络检索来源）。两层框架先用内部证据评估声明与视频是否一致，再结合外部佐证判断事实真伪。充分性驱动贪心搜索逐步收集证据至阈值，耗尽候选池仍不足时输出 insufficient，而非强制判定。Claude Sonnet 4 平均只用 4.5 个证据单元（全集的 16%），Macro-F1 为 0.510，与全证据基线 0.518（平均 27.7 个证据）统计上无显著差异；相较不允许弃答的同一搜索方法，证据不足识别显著改善。GPT-5.5 上复现效率结果，开源 Qwen2.5-72B 上仅部分成立。消融显示外部证据对事实判定不可或缺，视频内部证据则提供声明—视频一致性 grounding。

Insight: 证据高效核查依赖显式建模充分性和弃答，而不是仅检索更多相关材料；若证据真正不足，强制给出结论会降低可靠性。

Abstract: A core challenge in short-video fact-checking is identifying which evidence is sufficient to support a verification conclusion. Existing approaches either give the verifier all available evidence, introducing noise, or select evidence by topical relevance, which conflates relatedness with sufficiency. We identify evidential sufficiency as the selection criterion: whether a subset of evidence is adequate to support a confident verdict without redundancy. We introduce MiniVer-V, a benchmark of 195 short videos with three-way verdict annotations (supported, refuted, insufficient) and 5,510 multimodal evidence units spanning visual keyframes, speech transcripts, and web-retrieved external sources. We propose a two-layer verification framework that separates claim-video consistency, assessed from internal evidence, from factual verdict determination, which additionally requires external corroboration. On top of it, a sufficiency-driven greedy search assembles evidence until a sufficiency threshold is met and outputs insufficient when the candidate pool is exhausted, rather than forcing a verdict. With Claude Sonnet 4, the method reaches a Macro-F1 of 0.510 using 4.5 evidence units on average (16% of the full evidence set), statistically indistinguishable from the full-evidence baseline (0.518 with 27.7 units), while significantly improving recognition of insufficient cases over the same search without abstention. The efficiency result replicates with GPT-5.5 and holds only partially with an open-weight Qwen2.5-72B verifier. Ablations show that external evidence is indispensable for factual determination, while internal video evidence grounds the verdict in claim-video consistency. These findings suggest that evidence-efficient verification is achievable, and that explicit abstention is needed when evidence is genuinely inadequate.

[32] V-CoLA: Vision Token Compression with Linear Attention cs.CV | cs.LGPDF

Hao Jiang, Yiru Mao, Tianpeng Bu, Hao Zhou, Hongtao Duan

TL;DR: 提出 V-CoLA，面向线性注意力 VLM 的免训练视觉 token 压缩，以独特性重要度选择关键 token 并自适应合并，兼容分块并行计算。

Details

Motivation: 视觉 token 往往占 VLM 输入序列主体，压缩可减少计算；但混合架构引入线性注意力后，为 softmax 注意力设计的压缩方法泛化不佳，注意力与相似度方案都会出现明显性能下降。

Result: V-CoLA 使用独特性意识的重要度准则识别关键视觉 token，并通过自适应合并进行压缩；实现层面专门适配线性注意力分块并行。在多个基准上，保留原模型 99.5% 性能时只需 50.0% 视觉 token；压缩到 12.5% 时仍超过 88.0% 性能。Prefill 加速为 1.86×–6.15×。

Insight: 线性注意力需要专门考虑 token 的信息独特性及分块执行模式，直接迁移 softmax 注意力压缩会损害性能。不同保留率对应的具体速度/性能搭配摘要未详列。

Abstract: Vision-language models (VLMs) have demonstrated impressive capabilities but suffer from substantial computational overhead, as vision tokens dominate the input sequence. This motivates vision token compression as a key direction to alleviate the burden. However, with the emergence of hybrid architectures incorporating linear attention (\eg, Qwen3.5), prior methods designed for softmax attention struggle to generalize. Our analysis reveals that both attention- and similarity-based approaches suffer notable performance degradation, underscoring the urgent need for compression methods tailored to this regime. To this end, we propose \textbf{V-CoLA}, an efficient training-free token compression framework specifically designed for linear attention. V-CoLA introduces a novel \textit{uniqueness-aware importance criterion} for identifying critical vision tokens, coupled with an \textit{adaptive token merging strategy} that performs compression. All components are optimized at the implementation level to remain compatible with the chunk-wise parallelism of linear attention, ensuring strong practical value. Extensive experiments across multiple benchmarks demonstrate the superiority of V-CoLA: it achieves 99.5% of the original performance with only 50.0% of vision tokens, and over 88.0% with as few as 12.5%, while delivering a 1.86$\times$ to 6.15$\times$ prefill speedup.

[33] Spatial-Frequency-Aware Implicit Neural Representation of Multidimensional Signals via MLP-KAN Fusion cs.CVPDF

Wen Yan, Ligen Shi, Jun Qiu, Haimiao Zhang, Lina Wu

TL;DR: 提出结合 MLP 与 KAN 分支的空间—频率感知隐式神经表示，并以小波域融合和频带分离正则促进两类网络互补建模。

Details

Motivation: 基于 MLP 的隐式神经表示存在频谱偏置，倾向拟合低频而遗漏高频细节。傅里叶特征等方法虽能缓解，但依赖敏感手工调参且可能产生频谱伪影。

Result: MLP 分支负责平滑、低频结构，KAN 分支补充局部变化与细节；两支输出经过离散小波变换分解为系数，相应系数相加后逆变换重建。频带分离正则抑制 MLP 高频响应和 KAN 低频响应，鼓励分工互补。实验覆盖 1D 信号、2D 图像、3D 体数据和符号距离函数、视频及 4D 光场；作者报告各类评估信号的重建保真度均有改善，但摘要未列数值。

Insight: 以变换域融合和显式频带约束分配 MLP/KAN 的频率职责，减少靠手工频率编码调参。跨多种信号的结果支持方法通用性，但摘要没有给各任务基线和误差。

Abstract: Implicit Neural Representations (INRs) have emerged as a compelling paradigm for modeling multidimensional signals by mapping continuous coordinates to signal values. However, Multi-Layer Perceptrons (MLP)-based INRs inherently suffer from spectral bias, which favors low-frequency components and suppresses the reconstruction of essential high-frequency details. While existing techniques, such as Fourier feature mappings, mitigate this issue, they often rely on sensitive manual tuning and are prone to spectral artifacts. In this paper, we propose a spatial-frequency-aware INR framework that combines an MLP branch with a Kolmogorov-Arnold network (KAN) branch for complementary frequency-oriented modeling. The MLP branch provides a low-frequency-oriented representation of smooth structures, whereas the KAN branch complements localized variations and fine details. To coordinate the two branches, we integrate the discrete wavelet transform (DWT) and inverse discrete wavelet transform (IDWT) into the output fusion stage. The outputs of the two branches are decomposed into wavelet coefficients, and the corresponding coefficients are additively fused before inverse wavelet reconstruction. A wavelet-domain band-separation regularization further penalizes high-frequency responses in the MLP branch and low-frequency responses in the KAN branch, thereby encouraging complementary frequency-oriented behavior. Experiments on 1D signals, 2D images, 3D volumes and signed distance functions, videos, and 4D light-fields demonstrate the applicability of the proposed representation across the evaluated signal modalities. Results demonstrate improved reconstruction fidelity across the evaluated signal modalities.

[34] iCATS: Fast Video Generation via Interaction-Aware Sparse Attention and Timestep-Adaptive Sparsity cs.CVPDF

Chengfeng Han, Baole Ai, Xianlu Bian, Jie Yao, Zilong Huang

TL;DR: 提出 iCATS，免训练稀疏注意力方案，以查询—键交互估计重要性、按去噪信噪比动态调整稀疏度，并通过尾部合并提升 GPU 利用率。

Details

Motivation: DiT 稀疏注意力通过只计算重要 query-key 区域加速，但掩码近似会损害生成质量。既有重要度估计常分别聚类 query 与 key，且固定 top-p 稀疏度未考虑不同去噪步对近似误差的容忍度。

Result: iCATS 发现按 query-key 点积交互聚类比独立相似度聚类更准确，并将估计改写成低成本二次型；用 SNR 引导的稀疏调度取代固定 top-p；以 tail-merging 减少不规则聚类尺寸导致的填充开销。在 HunyuanVideo-T2V-13B 上加速 2.03×、PSNR 31.017 dB；Wan2.1-T2V-14B 上加速 1.55×、PSNR 29.301 dB，作者报告效率—质量折中达到 SOTA。

Insight: 重要度建模、去噪时刻稀疏度控制和硬件执行共同决定实际速度与画质，单独设计稀疏掩码并不足够。

Abstract: Training-free sparse attention offers a practical acceleration solution to Diffusion Transformers (DiTs) via reducing computations without fine-tuning. It typically involves estimating the importance of query-key regions and deriving sparse masks to compute only the important candidates, which inevitably introduces approximation errors that may degrade generation quality. To better balance the efficiency-quality trade-off, we propose iCATS, integrating improved importance estimation and sparse mask construction with an efficient hardware execution strategy. Specifically, for importance estimation, unlike previous works that perform independent clustering over query and key tokens based on feature similarity to estimate attention scores, iCATS demonstrates that clustering based on query-key dot-product interactions is more accurate and further reformulates this objective as a simple quadratic form for low-cost computation. For sparse mask construction, instead of using a fixed top-p rule, we observe that tolerance to sparse approximation errors varies across denoising timesteps and therefore introduce an SNR-guided sparsity schedule to adjust sparsity dynamically, leading to higher accuracy. Finally, for hardware execution, we devise a tail-merging strategy to reduce padding overhead caused by irregular cluster sizes, improving GPU kernel utilization. Extensive experiments show that iCATS achieves $2.03\times$ acceleration with 31.017 dB PSNR on HunyuanVideo-T2V-13B and $1.55\times$ acceleration with 29.301 dB PSNR on Wan2.1-T2V-14B, delivering a state-of-the-art efficiency-quality trade-off.

[35] Efficient Multi-Granularity Knowledge Transfer for Radiology Report Generation cs.CVPDF

Xubin Zhong, Zheyu Zhang, Wenjian Qin, Ning Wen

TL;DR: 提出 EMGKT，以全局医学知识表示、细粒度教师—学生蒸馏及疾病专家分类器增强放射报告生成，并在推理时丢弃外部先验以降低成本。

Details

Motivation: 从 X 光生成放射报告需要识别疾病并以专业方式描述；现有方法可能忽略关键区域的医学知识提取和聚合，而这些能力需要多个粒度层次的知识。

Result: EMGKT 用医学 VLM 编码全局知识嵌入提供上下文，并增加细粒度知识蒸馏任务：教师嵌入使用额外先验，学生从教师表示学习。推理时仅用学生嵌入增强细粒度知识，丢弃教师嵌入，因此不需额外先验、计算开销很低。另以疾病嵌入初始化多个诊断专家分类器，以处理不同粒度特征。两个公开数据集、多种基线上实验显示该模块有效且可迁移至多数现有方法；摘要未报告具体指标。

Insight: 训练时使用额外知识监督、推理时只保留学生表示，可将细粒度知识迁移到轻量部署路径。摘要未说明两个数据集名称及具体成本。

Abstract: Radiology report generation can automatically generate clinical descriptions from X-ray images, thereby significantly improving the efficiency of radiologists. This task is challenging because it requires medical knowledge to accurately identify diseases and describe them in a professional manner. However, existing methods often overlook the importance of enhancing medical knowledge in describing pivotal areas, a capability that requires models to effectively extract and aggregate knowledge at multiple levels of granularity. Accordingly, we herein propose a novel and compact Efficient Multi-Granularity Knowledge Transfer (\textbf{EMGKT}) method to address the above issues. First, we encode global knowledge embeddings using a medical vision-language model, which provides contextual medical knowledge. Moreover, we devise a novel Fine-Grained Knowledge Distillation (FGKD) training task which efficiently extract fine-grained knowledge. Specifically, the FGKD training task contains teacher embeddings and student embeddings. Teacher embeddings are encoded using extra priors; while student embeddings are learned from the teacher embeddings through knowledge distillation. During inference, the student embeddings are used to enhance fine-grained knowledge while the teacher embeddings are discarded, resulting in negligible computational costs and no need for extra priors. Finally, we further develop a mixture of disease diagnosis expert classifiers to enhance knowledge extraction. The classifiers are initialized using disease embeddings and are modeled as different experts to address various granularity features. Notably, \textbf{EMGKT} can be efficiently applied to most existing methods. Extensive experiments are conducted on two widely-used public datasets and various baselines, which demonstrates the effectiveness and transferability of \textbf{EMGKT}.

[36] Collaboratively Guided Adversarial Robust Distillation with Teacher-Favorable Examples cs.CVPDF

Zhi Li, Haowei Liu, Hongchen Yang, Xiaoxuan Wang, Song Gao

TL;DR: 提出 CGARD，在相同扰动邻域内联合优化学生对抗样本与教师协同样本，利用教师更偏好的例子改善对抗鲁棒蒸馏。

Details

Motivation: 对抗蒸馏将高容量教师的鲁棒性迁移到小学生，但既有方法主要用干净或对抗样本上的教师预测监督，对扰动邻域内教师更有利的样本探索不足。

Result: CGARD 联合生成学生对抗样本和教师协同样本；后者受约束，使教师在其上的交叉熵不高于干净输入。再结合教师协同引导与对抗教师监督完成鲁棒知识迁移。在 CIFAR-10、CIFAR-100 上进行白盒评估和额外黑盒迁移评估，结果显示相较强对抗蒸馏基线，鲁棒性持续提升；摘要未提供具体准确率或攻击成功率。

Insight: 同一扰动邻域内同时考虑学生暴露的困难样本和教师可稳定处理的样本，有助于将教师知识传给紧凑学生。摘要未说明教师、学生架构或具体扰动预算。

Abstract: Adversarial distillation transfers robustness from high-capacity teachers to compact students. Existing adversarial distillation methods mainly use teacher predictions on clean or adversarial examples to supervise student learning. However, teacher-favorable supervision within the perturbation neighborhood remains underexplored in adversarial distillation. We therefore propose Collaboratively Guided Adversarial Robust Distillation (CGARD), which jointly optimizes distinct student-adversarial and teacher-collaborative examples within the same perturbation neighborhood. The teacher-collaborative example is constrained to incur no greater cross-entropy loss under the teacher than the clean input. CGARD combines collaborative teacher guidance with adversarial teacher supervision to improve robust knowledge transfer. Experiments on CIFAR-10 and CIFAR-100, including white-box evaluation and additional black-box transfer evaluation, demonstrate consistent robustness improvements over strong adversarial distillation baselines.

[37] PathLang: A Language-Centered Benchmark for Vision-Language Models in Computational Pathology cs.CVPDF

Fanqi Cheng, Kuo Gong, Shangke Liu, Beidi Zhao, Junchao Zhu

TL;DR: 提出 PathLang，临床病理语言敏感性零样本基准：固定切片、标签和图文评估方向，只改变诊断措辞，测量病理 VLM 对临床等义表达的鲁棒性。

Details

Motivation: 既有病理 VLM 基准多用固定封闭提示或只改通用模板，将语言视为恒定因素；临床诊断措辞实际上会随报告风格、术语和具体程度变化。

Result: PathLang 涵盖零样本图文分类/对齐、跨模态检索、语义等价改写及长度/报告风格鲁棒性、提示集成，以及四种竞争结构不同的开放词汇诊断检索任务。全部提示和候选集经六位持证病理学家审核。九个 VLM、五个公共数据集、四种器官的实验发现，临床等义改写会显著影响表现，语义竞争方式也造成大幅差异；图文对齐质量不必然转化成类别间可分性。

Insight: 诊断语言本身是病理 VLM 的重要变化轴，评测需控制影像和标签、单独改变措辞；只测固定提示可能掩盖临床语言变化下的脆弱性。

Abstract: Pathology vision-language models (VLMs) have shown strong visual perception ability, but their robustness in the language domain remains poorly characterized. Existing pathology VLM benchmarks largely rely on canonical closed-set prompts or perturb only generic templates, treating language as a fixed evaluation component rather than a variable axis of model behavior. In clinical practice, however, diagnostic language varies across reports, institutions, and candidate diagnoses. We introduce PathLang, a language-centered and clinically grounded zero-shot benchmark. PathLang holds the underlying slides, ground-truth labels, and image-text evaluation direction fixed while systematically varying only the diagnostic language, so that performance differences reflect how a diagnosis is phrased rather than what is imaged. The language variation follows how pathologists actually rephrase diagnoses (terminology, specificity, and reporting style), and all prompts and candidate pools are validated by six board-certified pathologists. PathLang covers four task families: (1) zero-shot classification with image-text alignment analysis, (2) cross-modal retrieval, (3) paraphrase robustness, including semantic-equivalence paraphrases, length and reporting-style variation, and prompt ensembling, and (4) open-vocabulary diagnosis retrieval over four candidate pools with distinct forms of semantic competition. Across nine VLMs and five public datasets spanning four organs, we find that performance is highly sensitive to clinically equivalent paraphrases, varies substantially across forms of semantic competition, and that image-text alignment quality does not necessarily translate into inter-class separability. We release the prompt corpus, candidate pools, pre-computed text embeddings, and evaluation code at https://anonymous.4open.science/r/PathLang.

[38] FloorSAV: Elucidating Spatial Audio-Visual Context with 2D Floormap for AV-LLMs cs.CV | cs.LGPDF

Kyeong-Rae Kim, Sungnyun Kim, Tae-Hyun Oh

TL;DR: 提出 FloorSAV，用动态 2D 平面图将点云、相机轨迹、空间音频和语义地标同步提供给 AV-LLM，以加强具身场景中的全局几何推理。

Details

Motivation: 动态第一人称环境中的三维空间推理对具身智能重要，但 AV-LLM 缺少从原始视听流中显式整合全局几何的机制；现有方案可能需要昂贵微调，或未充分发挥跨模态推理能力。

Result: FloorSAV 将 3D 点云、相机轨迹、空间音频线索和语义对象地标渲染为动态 2D floor map，与第一人称视频同步输入 AV-LLM，并通过平面图解释指导共同推理。作者构建 SAVED-Bench，涵盖动态相对方位、区域和路径推理问答。FloorSAV 在 SAVED-Bench 和 SAVVY-Bench 多项空间任务上改善 AV-LLM 表现；使用真值 floor map 的研究显示，若空间信息准确，提升潜力更大。摘要未报告具体分数。

Insight: 把三维几何与空间音频显式压缩为同步平面图，便于 AV-LLM 在单次推理中联合利用视觉、听觉和几何信息。真值图结果提示感知/重建质量会影响最终推理。

Abstract: While 3D spatial reasoning in dynamic egocentric environments is crucial for embodied intelligence, audio-visual large language models (AV-LLMs) lack explicit mechanisms to process and internalize global geometry directly from raw sensory streams. Existing approaches either require costly fine-tuning or underutilize the model’s cross-modal reasoning capacities. In this paper, we propose FloorSAV, a novel framework that explicitly grounds spatial audio-visual context by rendering a dynamic 2D floormap. By integrating 3D point clouds, camera trajectories, spatial audio cues, and semantically grounded object landmarks, we inject this floormap into the AV-LLM as a synchronized stream with an egocentric video. AV-LLMs utilize their multi-modal capabilities to jointly reason over visual, auditory, and geometric cues in a single inference with floormap interpretation guidance. We further introduce SAVED-Bench (Spatial Audio-Visual Egocentric Benchmark with Dynamic Agents), constructing essential tasks of spatial capability in real-world scenarios: dynamic relativity, regional, and path reasoning QAs. FloorSAV improves AV-LLMs’ spatial reasoning on various tasks from both SAVED-Bench and SAVVY-Bench. Studies with ground-truth floormaps demonstrate the substantial potential of FloorSAV with accurate spatial information.

[39] Point-Focused Attention Meets Context-Scan State Space: Robust Biological Visual Perception for Point Cloud Representation cs.CVPDF

Kanglin Qu, Pan Gao, Qun Dai, Yuanhao Sun

TL;DR: 提出 PointLearner，以仿生的“聚焦局部、扫描全局”设计联合建模点云局部几何和长程上下文，并在多项点云任务上报告 SOTA 鲁棒表现。

Details

Motivation: 点云表征需同时捕获精细局部结构和全局依赖。普通全局建模难兼顾非均匀采样的局部细节与场景整体语义，论文借鉴视觉注视和扫视的分工。

Result: Point-focused attention 通过局部邻域与空间下采样特征间的竞争归一化注意力模拟中央凹聚焦；下采样特征由可学习诱导点池化，诱导点数量可调并直接与点云交互，以适应非均匀分布。Context-scan state space 借助 Hilbert 曲线引导的双向 S6 扫描，模拟眼球扫视并推断场景整体语义和空间布局。作者报告其在多项点云任务上鲁棒且达到 SOTA；摘要没有列出数据集和数值。

Insight: 两阶段处理分别负责局部几何细节和全场景上下文，仿生结构让两者协同而非仅依赖单尺度注意力。

Abstract: Synergistically capturing intricate local structures and global contextual dependencies has become a critical challenge in point cloud representation learning. To address this, we introduce PointLearner, a point cloud representation learning network that closely aligns with biological vision which employs an active, foveation-inspired processing strategy, thus enabling local geometric modeling and long-range dependency interactions simultaneously. Specifically, we first design a point-focused attention, which simulates foveal vision at the visual focus through a competitive normalized attention mechanism between local neighbors and spatially downsampled features. The spatially downsampled features are extracted by a pooling method based on learnable inducing points, which can flexibly adapt to the non-uniform distribution of point clouds as the number of inducing points is controlled and they interact directly with point clouds. Second, we propose a context-scan state space that mimics eye’s saccade inference, which infers the overall semantic structure and spatial content in the scene through a scan path guided by the Hilbert curve for the bidirectional S6. With this focus-then-context biomimetic design, PointLearner demonstrates remarkable robustness and achieves state-of-the-art performance across multiple point cloud tasks.

[40] EgoPhys: Estimating Peak Contact Force and Mechanical Work from Egocentric Manipulation Video cs.CVPDF

Zhuo Dong, Jianhua Yang, Haohao Li, Yumeng Zhao, Keji He

TL;DR: 提出 EgoPhys，仅用 RGB 第一人称操作视频预测峰值接触力与机械功，并以接触感知空间聚合和目标专属多专家时序路由建模不同物理量。

Details

Motivation: 铰接物体操作的物理理解既需估计接触瞬间的峰值力，也需估计整个接触期间力与运动的耦合做功；前者是短暂事件，后者依赖持续时序，因此不能只从全局外观推断。

Result: EgoPhys 包含 CASA，融合外观与几何特征、突出交互线索；TMTR 使用语义、事件和运动时序专家，并分别路由预测力和功。在 Hoi! 测试集上，峰值力 MAE 为 5.205±0.584 N，机械功 MAE 为 0.894±0.081 J，作者称显著优于既有方法。

Insight: 同一 RGB 操作视频中的接触力和机械功具有不同时间尺度，应分别聚合和建模。评估在 Hoi! 测试划分上进行，摘要未提供不同物体或接触类型的误差。

Abstract: Physically grounded manipulation of articulated objects requires understanding both the maximum forces encountered during contact and the work performed as their parts move. Peak contact force and mechanical work quantify these complementary aspects, but estimating them from egocentric video is challenging because physical interaction cues are local and indirect. Moreover, peak force is associated with brief contact events, whereas mechanical work depends on force-motion coupling throughout the contact duration. To address these challenges, we propose EgoPhys, an RGB-only framework comprising Contact-Aware Spatial Aggregation (CASA) and Target-Specific Multi-Expert Temporal Routing (TMTR). CASA integrates appearance and geometry features to emphasize interaction-relevant cues, while TMTR models semantic, event, and motion cues with specialized temporal experts and routes them separately for force and work prediction. On the test split from Hoi! dataset, EgoPhys substantially improves predictions of peak force and mechanical work, achieving MAEs of (5.205 \pm 0.584) $N$ and $0.894 \pm 0.081$ $J$, respectively.

[41] CRISP: Fixing Flying Pixels in Latent LiDAR Generation via Diffusion Decoding cs.CV | cs.AIPDF

Andrea Ceron, Michael Schmidt, Alvaro Marcos-Ramiro, Sebastian Schmidt, Benjamin Busam

TL;DR: 提出 CRISP，以像素空间扩散解码器替换 LiDAR 生成管线中的 VAE 解码器，针对径向深度边缘模糊导致的漂浮点问题提升几何质量。

Details

Motivation: 潜空间 LiDAR 流程中的卷积 VAE 会模糊尖锐径向深度断层，反投影后形成位于不同表面之间的 flying pixels。作者将其定位为可单独替换的解码瓶颈。

Result: CRISP 由与骨干无关的潜变量适配器、DiT 去噪器和支持掩码预测器组成；固定编码器与潜变量生成器，仅替换视频 VAE 或 LiDAR 专用解码器。在 KITTI-360、SemanticKITTI 和 nuScenes 上，跨冻结骨干平均 FSVD/FPVD 降低 50.5%；通用视频 VAE 的降幅分别达 71%/74%。在 LiDAR 专用 LiDM 上 FRID 降低 71%，深度断层处收益最大。将同一解码器零样本替换到预训练 LiDM 世界模型后，FSVD 进一步改善 15.5%，缩小仿真到真实差距。

Insight: 点云生成质量可能被潜空间解码器的深度边缘模糊限制；无需改动潜变量生成器，仅更换解码器即可带来显著改善。摘要未解释各质量指标的定义细节。

Abstract: Latent LiDAR pipelines suffer from flying pixels: convolutional VAEs blur sharp radial depth discontinuities, yielding edge depths that back-project to points floating between surfaces. We identify this as a major, directly correctable decoder bottleneck and introduce CRISP: a pixel-space diffusion decoder with a backbone-agnostic latent adapter, DiT-based denoiser, and support mask predictor. CRISP replaces video-VAE and LiDAR-native decoders alike while keeping the encoder and latent generator fixed. Across KITTI-360, SemanticKITTI, and nuScenes, replacing only the decoder reduces FSVD/FPVD by 50.5% on average across frozen backbones; for generic video VAEs, the reductions reach 71%/74%. On the LiDAR-native LiDM backbone, FRID drops by 71%, with the largest gains at depth discontinuities. In a pretrained LiDM world model, the same zero-shot replacement improves FSVD by 15.5%, narrowing the sim-to-real gap.

[42] Rethinking Contrastive Loss in CLIP Post-training: A Complementary Framework with Frozen Text Encoder cs.CV | cs.AIPDF

Zidan Wang, Yaqian Li, Xiaokai Zhang, Kaiwen Long, Kun He

TL;DR: 重新审视 CLIP 后训练对比损失，提出 ComCLIP：冻结文本编码器，以适当温度的 InfoNCE、原模型锚定和 DINOv2 关系蒸馏轻量优化图像编码器。

Details

Motivation: 近期研究认为小批量 CLIP 后训练使用标准对比损失会灾难性遗忘，因此转向蒸馏。作者发现问题可能主要来自对比温度过大，而非负样本不足。

Result: 作者分析 InfoNCE 梯度随温度变化，发现设置足够小的温度后，对比式后训练能改善而非损害预训练 CLIP。ComCLIP 单 epoch 训练，冻结文本编码器，使改进后的视觉编码器可直接替换且不增加架构和推理成本；同时使用适温对比损失、相对原 CLIP 的 MSE 锚定损失和 DINOv2 关系蒸馏。多随机种子下，零样本分类匹配 CLIP-Refine；ViT-B/16 线性探测为 48.99，对照 42.28；ViT-L/14 MMVP 为 24.20，对照 19.01。CLIP-Refine 在图文检索仍更强。将 ComCLIP 视觉编码器直接接入 LLaVA-1.5-7B，不重对齐 projector 或 LLM，八个 VLM 基准总体无净变化，说明下游兼容性保留。

Insight: 合适的温度可让 InfoNCE 继续用于 CLIP 后训练；锚定和关系蒸馏辅助保持表示。增强主要体现在视觉迁移和 MMVP，不意味着图文检索也超过 CLIP-Refine。

Abstract: CLIP serves as a foundational vision-language model and the de facto vision encoder for downstream VLMs such as LLaVA. Post-training offers a lightweight route to refine CLIP, but recent work argues that the standard contrastive loss is unsuitable for post-training due to catastrophic forgetting under small batches, motivating designs that abandon the contrastive objective in favor of distillation. We revisit this premise and find that, for the InfoNCE objective, the reported forgetting is driven primarily not by insufficient negatives but by an inappropriate magnitude of the contrastive temperature $τ$: with $τ$ set sufficiently small, contrastive post-training improves rather than degrades the pretrained CLIP, which we explain through the temperature dependence of the InfoNCE gradient. Building on this finding, we propose \textbf{ComCLIP}, a lightweight single-epoch post-training recipe that freezes CLIP’s text encoder—so the refined vision encoder is a drop-in replacement with unchanged architecture and inference cost—and trains the vision encoder with a properly-tempered contrastive loss, an MSE anchoring loss against the original CLIP, and a relational distillation loss from DINOv2. Over multiple seeds, ComCLIP matches the self-distillation baseline CLIP-Refine on zero-shot classification while significantly improving the transferability of visual features, measured by linear probing ($48.99$ vs.\ $42.28$ on ViT-B/16), and on ViT-L/14 it also improves MMVP over CLIP-Refine ($24.20$ vs.\ $19.01$); CLIP-Refine remains stronger on image-text retrieval. Used as a drop-in vision encoder for LLaVA-1.5-7B without re-aligning the projector or LLM, ComCLIP yields no net change across $8$ VLM benchmarks, i.e., the refinement does not break downstream compatibility. Code and models are available at https://github.com/showstarpro/ComCLIP.git.

[43] FastJEV: Understanding Redundancy for Compact JEV Inference cs.CVPDF

Jie Ma, Jie Gao, Yihang Liu, Zhike Qiu, Junle Li

TL;DR: 提出 FastJEV，通过共享上下文锚点、候选前缀状态复用和决策引导剪枝降低选项式 JEV 模型的候选评估深度，无需额外训练且保留所有候选。

Details

Motivation: JEV 直接给候选打分，虽可只编码一次共同上下文，但候选评估仍可能重复 token 历史、复制推理状态并运行完整骨干。复用计算还需保存后续分支需要的状态，因此计算共享与状态存储需协同设计。

Result: FastJEV 共享递归初始状态并省去未使用的末端缓存，再通过候选前缀共享保留中间分支状态；另用小规模无标签数据上的相对分数变化进行决策引导剪枝。三种 OmniJev 模型、五个公开基准及重建的 LIBERO-10 离线问题共六组评测中，选定剪枝预算下候选深度减少 43.75%–45.83%，平均保留原任务分数 93.66%–97.52%。受控实验分析候选重叠和分支结构对复用开销的影响，并发现其实现中的候选前缀共享虽可减少重复计算，却可能增加延迟。

Insight: JEV 加速需将候选结构、状态存储粒度和执行调度共同考虑；减少 FLOPs 不一定降低真实时延。

Abstract: JEV models make multimodal decisions by directly scoring candidates. Although the common context is encoded once, candidate evaluation can still repeat matching token histories, duplicate inference states, and execute the full backbone. In this paper, we study these sources of redundancy and present FastJEV for compact candidate evaluation. We jointly organize history reuse and state storage, since sharing computation requires preserving states for later branches. We first introduce shared context anchoring to reuse recurrent initial states and omit unused final recurrent caches. We extend this reuse through candidate prefix sharing, retaining the intermediate states needed by subsequent branches. To further reduce the depth of these paths, we apply decision guided pruning based on relative score changes measured on a small unlabeled set. Our method retains full context encoding and all candidates without additional training. We evaluate FastJEV across three OmniJev model sizes on five public benchmarks and reconstructed LIBERO-10 offline questions. At the selected pruning budgets, the complete method reduces candidate depth by 43.75% to 45.83%, while retaining 93.66% to 97.52% of the original task scores on average across the six evaluation sets. Through controlled experiments, we show how candidate overlap and branching structure affect the execution cost of history reuse. In our implementation, candidate prefix sharing can reduce repeated computation while increasing latency. These findings motivate designing sharing granularity and execution schedules together for efficient JEV inference.

[44] GroundSight at GroundLM 2026 Shared Tasks: GoldenViewVQA cs.CVPDF

Kun Wang, Yupeng Hu, Ruping Cao, Hao Liu, Zhiran Li

TL;DR: 提出 CoVeR-VQA，一个无需训练的多阶段答案与支持视角校验框架，参加 GroundLM 2026 GoldenViewVQA 驾驶场景问答任务。

Details

Motivation: GoldenViewVQA 不仅要回答驾驶场景问题，还要找出提供证据的相机视角，因此答案正确与证据定位同样重要。

Result: 框架从 GPT-5.6 零样本预测出发，依次使用 Gemini-3.6-Flash 按视角核验、Claude-Opus-5 结合先验联合核验，以及利用共享多视图场景的语义问题组和验证集先验进行跨划分组级核验。官方测试集四阶段流程联合准确率 84.75%，比 GPT-5.6 零样本高 13.56 个百分点；答案准确率 94.92%、视角准确率 86.44%。提交结果经过两次基于评估器反馈的事后修正后，联合准确率为 88.14%。误差分析显示剩余错误主要来自支持视角定位。

Insight: 多视图问答的瓶颈更多在证据视角定位而非单独生成答案；报告需区分官方四阶段结果与额外评估器反馈后的最终提交分数。

Abstract: GoldenViewVQA requires models to jointly answer driving-scene questions and identify the camera view containing the supporting visual evidence, making precise evidence localization as important as answer correctness. We present \textbf{CoVeR-VQA}, a training-free multi-stage verification and correction framework for grounded multi-view VQA. Starting from GPT-5.6 zero-shot predictions, CoVeR-VQA progressively applies view-specific verification with Gemini-3.6-Flash, prior-guided joint verification with Claude-Opus-5, and cross-split group-level verification that exploits semantically filtered question groups from shared multi-view scenes and validation-derived prior knowledge. On the official GoldenViewVQA test set, the four-stage CoVeR-VQA pipeline achieves 84.75% Joint Accuracy, improving the GPT-5.6 zero-shot baseline by 13.56 percentage points, while reaching 94.92% Answer Accuracy and 86.44% View Accuracy. The final submitted run achieves 88.14% Joint Accuracy after two additional evaluator-informed post-hoc corrections. Our analysis shows that supporting-view localization remains the primary source of residual errors, highlighting the importance of explicit evidence verification for reliable multi-view multimodal reasoning.

[45] Missing Modality-Aware Calibration for Trustworthy Brain Tumor Segmentation cs.CV | cs.AIPDF

Sol Lee, Hyunji Kim, Sungrae Hong, Donghee Han, Mun Yi

TL;DR: 提出缺失模态感知局部温度缩放 MMA-LTS，为脑肿瘤分割预测学习逐体素空间校准温度，专门处理不同 MRI 模态缺失组合带来的置信度偏差。

Details

Motivation: 临床 MRI 常因扫描失败或协议差异缺失模态。现有方法虽能在缺模态下维持分割准确率，却常忽略置信度可靠性；缺模态数目并不能简单代表难度，具体缺少哪些模态会造成组合特异且空间异质的校准误差。

Result: MMA-LTS 是后处理逐体素置信度校准方法，通过可学习模态可用性 token 和逐体素难度分数估计空间自适应温度场。在 BraTS 2020 和 FeTS 2024 上，作者报告其在多种缺模态情景下改善校准，同时保持 SOTA 模型的分割准确率。摘要未列校准指标或各模态组合的具体结果。

Insight: 临床缺模态场景需依据缺失组合及空间位置校准置信度，而非统一按可用模态数量调温度；该方法是后处理，不改变分割模型精度。

Abstract: Multimodal brain tumor segmentation typically leverages multiple MRI modalities, yet incomplete modality acquisition is common in clinical practice due to protocol heterogeneity and scan failures. Although recent methods maintain segmentation accuracy under missing modality conditions, they frequently overlook prediction reliability, leading to miscalibrated confidence estimates that hinder clinical adoption. Existing calibration techniques are largely modality-agnostic or assume that prediction difficulty decreases monotonically as additional modalities become available. However, in brain tumor segmentation, prediction difficulty depends primarily on which modalities are absent rather than how many, leading to combination-specific and spatially heterogeneous calibration errors. To address this, we propose Missing Modality-Aware Local Temperature Scaling (MMA-LTS), a post-hoc voxel-wise confidence calibration method. It estimates a spatially adaptive temperature field conditioned on a modality-availability learnable token and a voxel-wise difficulty score. Experiments on BraTS 2020 and FeTS 2024 show that MMA-LTS improves calibration while preserving the segmentation accuracy of state-of-the-art models across diverse missing-modality scenarios, thereby enhancing trustworthiness toward clinical deployment.

[46] Fresco++: Frequency-Guided and Canonical-Consistent Optimization for Fine-Grained Head Avatar Modeling cs.CVPDF

Shikun Zhang, Yong Li, Yiqun Wang, Qiuhong Ke, Cunjian Chen

TL;DR: 提出 Fresco++，用频率渐进课程稳定头部细节恢复，并以规范表面区域共识聚合跨视角观测，改善头像重建和视角一致性。

Details

Motivation: 头像重建通常逐视角拟合，易过早追逐不稳定高频细节，并在不同视角间产生局部外观不一致。

Result: Fresco++ 在频率维度先稳定低频结构、再加入高频约束，以恢复精细面部和头发细节并避免早期放大伪响应。跨视角方面，Canonical Group Consensus 按共享规范表面区域关联局部观测；几何及可见性筛选去除不可靠数据，再在特征空间聚合其余证据以监督当前渲染，不依赖特定图像参数化，也不需额外渲染辅助视图。NeRSemble 实验显示重建质量和跨视角一致性改善；多种头像表示上的评估也支持方法通用性和可迁移性。

Insight: 频率课程解决优化先后顺序，规范空间共识解决多视角证据匹配，二者从粗到细稳定重建。摘要未提供量化分数。

Abstract: We propose Fresco++, a unified optimization framework for fine-grained and view-consistent head avatar reconstruction. Head avatar optimization is typically driven by per-view image supervision, which can lead to premature fitting of unstable high-frequency details and inconsistent local appearance across viewpoints. Fresco++ addresses these challenges by regulating both the progression of visual detail and the formation of cross-view supervision during optimization. For frequency-aware optimization, a progressive curriculum first stabilizes low-frequency structures and then introduces high-frequency constraints to recover fine facial and hair details without amplifying spurious responses at early stages. For cross-view optimization, we introduce Canonical Group Consensus, which associates local observations through shared canonical surface regions and establishes correspondence across different viewpoints. Geometric and visibility-aware screening removes unreliable observations, while the remaining multi-view evidence is aggregated in feature space to form a consensus target for supervising the current rendering. This design enforces local consistency without relying on a specific image-space parameterization and avoids additional rendering of the auxiliary view. Together, the frequency curriculum and canonical consensus provide stable optimization from coarse structures to fine details while maintaining coherent appearance across viewpoints. Extensive experiments on NeRSemble demonstrate improved reconstruction quality and cross-view consistency, while evaluations across diverse avatar representations further confirm the generality and transferability of Fresco++.

[47] EchoDiST: Self-distillation-based joint learning for diffusion-conditioned echocardiographic myocardial motion estimation cs.CVPDF

Feiyue Qi, Xingyue Wei, Jianwen Luo

TL;DR: 提出 EchoDiST，无需位移真值的超声心肌运动估计框架，以自蒸馏联合优化有限标注解剖分割和扩散条件运动网络。

Details

Motivation: 超声心肌运动估计受图像伪影、信息有限、散斑去相关影响，真实位移场标注又稀缺。解剖引导可提供结构信息，但常依赖专家标注心肌分割。

Result: EchoDiST 的自蒸馏联合学习分割与运动估计；此处无监督指不使用真值位移场，而不代表完全没有解剖标注。扩散条件在训练阶段使用随机扰动，推理只需一次确定性前向，不进行迭代反向扩散。三项超声数据集评估中包含两个外部跨视角/跨数据集测试。与七种代表性学习方法相比，EchoDiST 在解剖对齐、心肌应变评估及由运动导出的功能和心动周期评估上均持续改善，所测任务和数据集上的差异具有统计显著性；摘要未给数值。

Insight: 训练时利用随机扩散条件提升运动估计，推理成本保持单次前向；联合自蒸馏利用有限解剖监督，不需要位移真值。

Abstract: Motion estimation in echocardiography is essential for quantitative assessment of cardiac function and myocardial mechanics, but remains challenging due to image artifacts, limited image information, speckle decorrelation, and the scarcity of ground-truth displacement fields. Anatomy-guided approaches can provide structural information, yet often rely on expert-labeled myocardial segmentations. We propose EchoDiST, a framework for unsupervised echocardiographic myocardial motion estimation that integrates self-distillation-based joint learning with a diffusion-conditioned motion estimation network. Here, unsupervised motion estimation refers to learning without ground-truth displacement fields. The self-distillation strategy jointly optimizes anatomical segmentation and myocardial motion estimation under limited anatomical annotations. Diffusion-based conditioning is used during training with stochastic perturbations, while inference requires only a single deterministic forward pass without iterative reverse-diffusion sampling. EchoDiST was evaluated on three echocardiographic datasets, including two external test datasets under cross-view and cross-dataset settings. Compared with seven representative learning-based methods, EchoDiST consistently improved anatomical alignment, myocardial strain assessment, and motion-derived functional and cardiac-phase assessment. These gains were statistically significant across the evaluated tasks and datasets. Overall, EchoDiST provides an effective approach for reliable myocardial motion estimation under limited anatomical supervision and supports downstream quantitative assessment of cardiac function.

[48] SAGE: Sink-Aware Guided Emphasis for Visual Grounding in Vision-Language Decoders cs.CV | cs.CLPDF

Jeonghyo Song, YoungJoon Yoo

TL;DR: 提出 SAGE，识别视觉语言解码器早晚层的提示不变注意力沉降区域（PIS），并用 ROI 掩码把注意力引向查询相关图像区域以改善 grounding。

Details

Motivation: VLM 解码器注意力可能压制视觉证据并加剧幻觉。视觉注意力沉降常被视为统一现象，但其层间行为可能不同，简单整体处理会忽略提示相关的视觉—语言对齐。

Result: 作者发现跨提示的早期与晚期解码层会将注意力塌缩到相同少数图像区域，称为 Prompt-Invariant Sinks；中间层则更受提示条件控制并驱动图文对齐。SAGE 用 CLIP、ViT、DINOv3 等标准视觉骨干产生 token 对齐 ROI 掩码，轻量引导解码器从 PIS 转向查询相关区域。在多种视觉编码器与 decoder-only LLM 组合上，报告视觉 grounding 改善、幻觉减少，并在公开下游 VLM 基准（包括细粒度判别）取得稳定收益。

Insight: 注意力沉降存在层依赖：早晚层的提示无关聚焦与中间层的提示条件对齐需要区别处理。摘要没有列具体基准分数或幻觉指标。

Abstract: Recent large vision-language models (VLMs) pair a visual encoder with a large language model (LLM) and perform well on diverse image-text tasks, yet their reliability is often limited by decoder attention pathologies that suppress visual evidence and exacerbate hallucinations. In this paper, we revisit visual attention sinks and uncover a structured, layer-dependent behavior: across prompts, early and late decoder layers exhibit prompt-invariant attention collapse onto the same few image regions, which we term PIS (Prompt-Invariant Sinks), whereas mid layers become prompt-conditioned and drive vision-language alignment. This split suggests that treating sinks as a uniform effect is incomplete. Building on this insight, we propose SAGE (Sink-Aware Guided Emphasis), a lightweight intervention that steers decoder attention away from PIS and toward query-dependent regions of interest (ROIs) using token-aligned ROI masks derived from standard vision backbones such as CLIP, ViT, and DINOv3. Evaluated on diverse vision-encoder + decoder-only LLM VLM families, SAGE improves visual grounding, reduces hallucinations, and yields consistent gains across public downstream vision-language benchmarks, including fine-grained visual discrimination settings where localized evidence is crucial, when instantiated with backbone-derived ROI masks.

[49] Learning to Retrieve: Internalizing Memory Retrieval for Video World Models cs.CVPDF

JiaKui Hu, Tailai Chen, Yuqi Pan, Xuerui Qiu, Jialun Liu

TL;DR: 提出 L2R，将历史记忆检索内化到视频世界模型的生成状态中，通过相机条件检索门和 3D 重可见性触发器决定检索什么、何时检索。

Details

Motivation: 视频世界模型根据相机轨迹生成可探索且 3D 一致的场景视频，长程生成易发生场景漂移。外部记忆虽可检索之前见过的内容，但记忆路径独立于模型生成动态，模型无法内在学会何时、检索什么。

Result: L2R 将模型持久内部状态复用为历史记忆；相机条件检索门依据当前视角选择相关历史信息，检索触发器决定检索时机。触发器使用 3D 重可见性信号监督：先前观察到的内容重新进入当前视野时触发检索，否则保留现有上下文。在多个基础模型和相机重访基准上，L2R 提升长期场景一致性，无需外部记忆库或 3D 条件。

Insight: 历史检索可作为生成模型内部行为学习，而非单独外接数据库；重可见性信号将相机回访与记忆调用联系起来。摘要未提供一致性指标数值。

Abstract: Video world models aim to generate explorable, 3D-consistent scene videos conditioned on camera trajectories. Existing approaches often rely on external memory systems that explicitly retrieve previously observed content to mitigate scene drift during long-horizon generation. However, these auxiliary memory pathways operate outside the model’s internal generative dynamics, preventing the model from intrinsically learning when and what historical information should be retrieved. We propose to internalize memory retrieval into the generation process, allowing retrieval to emerge as an intrinsic behavior of the video world model rather than relying on an external memory system. Based on this principle, we introduce \textbf{Learning-to-Retrieve (L2R)}, which repurposes the model’s persistent internal state as a memory for historical context. A camera-conditioned retrieval gate selectively accesses relevant historical information from this state, determining \textit{what to retrieve}, while a retrieval trigger determines \textit{when to retrieve}. We further supervise the trigger with a 3D re-visibility signal, activating retrieval when previously observed content re-enters the current view while otherwise preserving the existing context. Together, these components enable the model to intrinsically acquire memory retrieval behavior and incorporate relevant historical observations into generation without a separate retrieval pathway. Across multiple base models and camera-revisit benchmarks, L2R improves long-term scene consistency while eliminating the need for an external memory bank or 3D conditions. https://jkhu29.github.io/l2r

[50] Conditional Residual Prediction: Improving Autoregressive Video Diffusion without a Bidirectional Teacher cs.CV | cs.AI | cs.LGPDF

Bowen Zheng, Zhiguang Liu, Jiarong Ou, Rui Chen, Tianyang Hu

TL;DR: 提出 Conditional Residual Prediction（CRP），让因果视频扩散模型先根据当前输入预测，再只用历史增加残差，从而减少对真值历史的依赖及自回归误差传播。

Details

Motivation: 因果视频扩散适合流式和长视频生成，但同规模下常弱于双向模型，许多方法依赖预训练双向教师。作者从图像模型初始化独立训练因果视频模型，发现使用真值历史训练会使模型过度依赖历史，推理时自身生成误差随时间累积。

Result: CRP 规定模型先在不看条件时预测目标，历史条件只能在此基础上追加残差；对视频历史而言，模型先尽力从当前观察预测每块，仅在当前不足时使用过去信息。受控实验中，CRP 几乎弥合与同设置双向模型之间 6.14 分的差距。扩展后训练得到 2B 参数 Optica，可自回归生成 5 秒 480p 视频，仅用约 1,500 万训练视频，在 VBench 达到 82.78。

Insight: 通过限制历史只承担当前帧无法预测的补充信息，可降低模型对无误差真值上下文的依赖，使因果模型无需双向教师也能提升质量。

Abstract: Causal video diffusion models generate video autoregressively, which suits streaming, interactive, and long-video generation. Under standard training, however, they often yield lower generation quality than bidirectional models of the same size. Many existing approaches address this gap by initializing from or distilling a pretrained bidirectional teacher. We instead train a causal model from an image-model initialization, with no bidirectional video model at any stage. Because this path requires neither a large bidirectional teacher nor a complex distillation pipeline, it is simpler and more scalable. On this path, we find that a causal model trained on ground-truth history becomes strongly dependent on it, so that at inference errors in its own generated history propagate forward. We hypothesize that much of this dependence is unnecessary, because the current input already determines much of what the history provides. We propose Conditional Residual Prediction (CRP), a simple recipe for reducing a model’s reliance on a condition: the model first predicts the target without the condition, and the condition may only add a residual on top of this prediction. Applied to history, CRP makes the model predict each chunk from the present as far as it can and use the past only for what the present cannot supply. In controlled experiments, CRP nearly closes the 6.14-point gap to a bidirectional model trained under the same setup. Scaling this recipe, we train Optica, a 2B-parameter causal video model that autoregressively generates 5-second 480p videos and reaches 82.78 on VBench with only about 15M training videos.

[51] Stop My Dancing! Understanding, Detecting and Attributing Motion-Aware Deepfake Videos cs.CV | cs.CRPDF

Fazhong Liu, Yan Meng, Tian Dong, Guoxing Chen, Haojin Zhu

TL;DR: 构建首个运动感知深伪（MAD）视频基准，并提出 MoDA，融合空间语义和频域取证特征检测、归因由姿态引导扩散模型生成的人体运动深伪。

Details

Motivation: 姿态引导扩散现已能合成人体全身运动视频，产生新的运动感知深伪类别并获得大量观看。需要针对该类视频理解其伪造线索，建立检测和生成器归因方法。

Result: 基准含超过 150 万帧，包含 1,363 个真实视频和 6 个可控生成器生成的 30,122 个合成视频，附真实扰动和开放世界划分。分析发现，受生成器有限输入帧约束，运动边界需预测和模拟连贯运动，产生高频伪影和模型特有频谱指纹。MoDA 通过跨域对齐和多尺度聚合结合空间语义与隐写分析频域特征，检测准确率为同分布 94.8%、跨数据集 89.1%，较此前方法提高 10%–25%，生成器归因准确率 91.5%。对两种未见商业平台的 200 个片段零样本准确率 81.94%；开放互联网收集的 1,200 个未见 MAD 片段（共 5.5 万帧）检测准确率 78.13%。白盒、灰盒、黑盒自适应攻击下性能相对稳定，而基线快速下降。

Insight: 生成过程在运动边界留下空间与频谱线索，可同时支持检测和源模型归因；开放互联网数据和未见商业平台结果提供了一定开放世界迁移证据。

Abstract: Pose-guided diffusion models can now synthesize entire human figures in motion, spawning a new class of deepfakes: Motion Aware Deepfake (MAD) that have already reached hundreds of millions of viewers. To better understand this emerging threat, we construct the first MAD-specific benchmark and measurement framework, containing over 1.5 million frames that mix 1,363 real and 30,122 synthetic videos from six controllable generators, with realistic perturbations and open-world evaluation splits. Then, we dissect MAD and discover that, despite their global coherence, these videos betray faint yet reliable cues: because the model relies on limited input frames for motion synthesis, it must predict and simulate coherent movement at motion boundaries, thereby producing high-frequency artifacts along with model-specific spectral fingerprints. Based on the observations obtained from analysis on dataset, we propose MoDA, the first defense framework tailored to detect and attribute MAD videos. MoDA couples spatial semantics with steganalysis-rich frequency features via cross-domain alignment and multi-scale aggregation, achieving 94.8% in-distribution and 89.1% cross-dataset detection accuracy gains of 10% to 25% over prior work and 91.5% model attribution accuracy. MoDA achieves 81.94% accuracy on 200 clips produced by two unseen commercial MAD platforms, indicating promising zero-shot transfer, and 78.13% detection accuracy on 1,200 unseen MAD video clips (55k frames in total) collected from the open Internet. Under white-box, gray-box, and black-box adaptive attacks, MoDA maintains relatively stable detection and attribution performance while the accuracies of the baselines drop rapidly.

[52] Parametric Trajectory Distillation for Few-Step Video Generation cs.CVPDF

Lan Feng, Peter Karkus, Maximilian Igl, Julius Berner, Yuxiao Chen

TL;DR: 提出参数化轨迹蒸馏 PTD，让少步视频生成学生以多项式参数化教师轨迹片段，并沿自身预测路径学习，适配模型容量与轨迹曲率。

Details

Motivation: 视频扩散/流模型需多次顺序前向，少步蒸馏可降低成本，但学生需在很少计算下复现教师迭代生成。直接模仿高噪声阶段高度弯曲的教师轨迹可能超出学生容量并损害细节。

Result: PTD 用多项式表示教师轨迹片段，让学生在自身预测路径上接受教师指导；学习到的曲率可随骨干预测能力调整。曲率头仅用于训练，推理保持原骨干结构。在 Wan2.1-14B 上，四步 PTD 在相同训练设置下较表现最强的纯轨迹基线 PDD 显著提升动态质量和自然度；在 33B 音视频 MiniMax-H3 上，LoRA 训练 PTD 较 LightX2V Turbo 提高多样性和自然度。盲评中，PTD 对 PDD 和 LightX2V Turbo 的偏好率分别为 55.1% 和 63.4%。

Insight: 让学生拟合自身可表达的轨迹曲率而非硬追踪教师的高曲率路径，有助于保留动态和多样性。摘要未提供具体自动质量指标分数。

Abstract: Video diffusion and flow models require many sequential evaluations, making generation computationally expensive. Few-step distillation reduces this cost but poses a capacity allocation problem: a student must match the teacher’s iterative generation with far less sequential computation. Existing trajectory methods ask the student to reproduce teacher transitions that are highly curved at high noise, which can exceed its capacity and degrade fine detail. We introduce Parametric Trajectory Distillation (PTD), which lets the student parameterize teacher trajectory segments as polynomials and learn from teacher guidance along its own predicted path. PTD is designed to let the learned curvature adapt to the backbone’s predictive capacity, preserving motion and diversity. The curvature head is used only in training; inference keeps the original backbone architecture. On Wan2.1-14B, four-step PTD sets a new state of the art for trajectory distillation, significantly improving dynamic quality and naturalness over PDD, the best-performing trajectory-only method on this model, under the same training setting. On the 33B audio-video MiniMax-H3, LoRA-trained PTD significantly improves diversity and naturalness over the state-of-the-art LightX2V Turbo. Blinded human votes give PTD 55.1% and 63.4% preference shares against PDD and LightX2V Turbo. Project page: https://alan-lanfeng.github.io/PTD/.

[53] MSGAT: Multi-Head Spiking Graph Attention with Similarity-Space Fusion for Image-Text Retrieval cs.CV | cs.AIPDF

Xintao Zong, Wenxuan Liu, Jianhao Ding, Zhaofei Yu, Tiejun Huang

TL;DR: 提出 MSGAT 和 Sim-Fuse，以多头脉冲图注意力建模图文细粒度关系，并在相似度空间融合离散稀疏脉冲特征与连续全局表征。

Details

Motivation: 脉冲神经网络可通过稀疏事件驱动计算降低能耗，但图文检索需捕捉跨模态语义结构，脉冲表示稀疏使对齐困难。既有方法偏局部对齐，依赖软标签且缺少结构/多粒度关系。

Result: MSGAT 以动态多头脉冲图注意力捕捉互补关系并执行脉冲驱动图推理和聚合。由于细粒度脉冲特征是稀疏离散的、全局表示是连续的，Sim-Fuse 在相似度空间融合粗细粒度匹配关系，避免直接混合异质特征。Flickr30K、MSCOCO 上，匹配设置下优于 ANN 方法及既有 SNN 检索基线。仅用两个时间步时，SNN 表现与 ANN 相当或更优，理论模块级能耗降低 55%。

Insight: 相似度空间融合可减少离散脉冲与连续特征直接融合的干扰，同时保留多粒度匹配；能耗数字是理论模块级估算，不是整机能耗实测。

Abstract: Spiking neural networks (SNNs) offer an energy-efficient computing paradigm through sparse event-driven computation, showing great potential for efficient multimodal learning. However, applying SNNs to high-level multimodal tasks, such as image-text retrieval (ITR), remains challenging, since sparse spike representations make it difficult to capture semantic structures required for cross-modal alignment. Existing spiking ITR methods rely on local alignment and additional soft-label supervision during training, while lacking awareness of structural and multi-granularity relationships. To address these issues, we propose a Multi-head Spiking Graph Attention Network (\textbf{MSGAT}) for structural modeling and equip it with dynamic attention heads to capture complementary relational patterns and enable spike-driven graph reasoning and aggregation. However, within a two-branch multi-granularity fusion framework, the fine-grained spike representations generated by MSGAT are sparse and discrete, whereas the global representations are continuous, making conventional feature-level fusion susceptible to interference across heterogeneous representations. Therefore, we introduce \textbf{Sim-Fuse}, a similarity-space fusion alignment strategy integrating coarse- and fine-grained matching relations while avoiding direct fusion of heterogeneous representations. Experiments on Flickr30K and MSCOCO show our method outperforms ANN methods under matched settings and existing SNN retrieval baselines. Moreover, with only two time steps, our SNN achieves comparable or superior performance to its ANN counterpart while reducing theoretical module-level energy by 55%. The code is provided in the Supplementary Materials.

[54] Hankel Subspace Self-Supervised Learning for Parallel MRI Reconstruction cs.CVPDF

Mingyu Hu, Siquan Zhu, Xijun Zhong, Qiegen Liu

TL;DR: 提出 HSSRecon，以物理采集单元划分和重复 Hankel 副本归一化避免 MRI 自监督数据泄漏，并在 Hankel 子空间学习结构算子、在原始 k 空间强制数据一致。

Details

Motivation: 并行 MRI 欠采样重建是病态逆问题；多线圈重复解剖信息与重叠窗口局部 k 空间重复性可辅助恢复，但若在 Hankel 提升后划分训练/目标，原始样本可能同时出现于两侧，造成泄漏。

Result: HSSRecon 在 Hankel 提升前按物理采集单元划分数据，并对重叠窗口产生的重复副本做 multiplicity normalization。网络学习紧凑复值 Hankel 子空间算子，而非直接预测缺失数据；最终在原始 k 空间变量上使用共轭梯度并施加硬数据一致性。理论分析覆盖物理组划分和副本归一化，并建立正定性、解唯一性、硬数据一致性及有限步共轭梯度误差界。在 fastMRI 脑部数据的三种对比度、三种采样掩码共六种聚合条件下，PSNR、SSIM、NMSE 表现有竞争力；摘要没有提供具体值。

Insight: 把 Hankel 域结构学习与物理 k 空间数据一致性分开，既利用重复结构也避免提升域随机拆分泄漏。

Abstract: Parallel magnetic resonance imaging reconstruction is an ill-posed inverse problem under undersampling. Multi-coil acquisition and Hankel lifting expose complementary repeated information: observations of the same anatomy across coils and repeated local k-space neighborhoods in overlapping windows. These dependencies guide recovery of missing k-space data. However, splitting lifted Hankel entries for self-supervision can place the original sample in both input and target, causing data leakage. We propose Hankel Subspace Self-Supervised Reconstruction (HSSRecon), a scan-specific reconstruction framework for parallel magnetic resonance imaging. HSSRecon partitions data by physical acquisition units before Hankel lifting and applies multiplicity normalization to repeated Hankel copies in overlapping windows. Rather than learning a mapping that directly predicts missing data, the network learns a compact complex-valued Hankel subspace operator. Reconstruction is performed over the original k-space variables using a conjugategradient solver with hard data consistency. This design separates structural learning in the Hankel domain from data consistency in the physical domain: the former exploits multi-coil and local Hankel correlations, while the latter solves over unacquired degrees of freedom. We provide theoretical analyses of physicalgroup splitting and multiplicity normalization, and establish positive definiteness, uniqueness, hard data consistency, and a finite-step conjugate-gradient error bound for the system. On fastMRI brain data with three contrasts and three sampling masks, HSSRecon achieves competitive peak signal-to-noise ratio, structural similarity, and normalized mean squared error across six aggregated conditions.

[55] SV-TAD: Native Sparse Convs for Efficient Temporal Action Detection cs.CVPDF

Ricardo Pizarro, Roberto Valle, José M. Buenaposada, Luis M. Bergasa, Luis Baumela

TL;DR: 提出 SV-TAD，以原生稀疏二维卷积让视频时序动作检测适配器直接处理动态剪枝 token 集合，避免稀疏 token 重新稠密化带来的计算浪费。

Details

Motivation: 冻结大规模 ViT 并训练轻量卷积适配器虽参数高效，但不降低推理计算，长视频扩展性有限。token 剪枝可减少注意力计算，却会破坏卷积模块所需网格结构，稠密重建抵消加速收益。

Result: 作者引入原生稀疏 2D 卷积，使适配器首次可直接高效处理动态剪枝 token 集。集成到 SV-TAD 后，VideoMAEv2-L 计算量最多降低 64%，推理速度达到 2.2×，并在 THUMOS-14 和 ActivityNet-1.3 上保持 SOTA 准确率。扩展到 InternVideoNext-L 后，以约一半计算量超过先前 SOTA；稀疏表示还支持辅助任务 token，改善 ATTACH 细粒度装配检测。

Insight: 稀疏化要与下游模块的数据结构相兼容；直接稀疏卷积避免恢复密集网格的额外成本。

Abstract: To adapt billion-parameter Vision Transformers for long-video understanding, recent methods freeze the backbone and train lightweight convolutional modules. While effective for parameter-efficient training, existing adapters do not reduce inference-time computation, leaving scalability with respect to video length largely unaddressed. Token selection can reduce attention cost by pruning redundant tokens, but it breaks the spatial grid structure required by convolutional adapters. This forces an expensive dense reconstruction, nullifying much of the potential speedup. We address this by introducing native sparse 2D convolutions, a primitive that allows these adapters, for the first time, to operate directly and efficiently on dynamically pruned token sets. We integrate this primitive into SV-TAD, an adapter framework for temporal action detection, reducing VideoMAEv2-L computation by up to 64% and achieving 2.2x faster inference, while maintaining state-of-the-art accuracy on THUMOS-14 and ActivityNet-1.3. When scaled to InternVideoNext-L, our approach surpasses the previous state of the art at roughly half its computational cost. Moreover, the sparse formulation naturally supports auxiliary task tokens, which improves fine-grained assembly detection on ATTACH.

[56] Beyond Report Imitation: Clinically Aware Multi-Image Ultrasound Report Generation from Visible Evidence cs.CV | cs.AIPDF

Yuchen Yang, Xin Wang, Lufan Wang, Yinghong Pan, Yujuan Feng

TL;DR: 提出 CAMEO，以可见证据为约束生成多图像超声报告，分阶段学习超声图文基础、多视图证据 grounding 和临床错误导向偏好对齐。

Details

Motivation: 存档关键帧只覆盖动态超声检查的一部分，直接模仿完整临床报告会要求模型生成图像中无法核验但对完整检查可能合理的内容，造成监督与可见证据错位。

Result: CAMEO 三阶段训练：学习超声视觉语言基础；将可信可见报告要点蒸馏为多图像 QA 和报告监督；再用临床错误导向偏好对进行对齐。作者基于 USReport 构建 USReport-Distilled，含 17,670 个证据 grounding 配对图像训练实例，以及含 21,869 对偏好的 USReport-Pref；另使用 25,631 个 PubMedVision-US 超声指令样本做领域适配和多图像指令微调。在主要 USReport-Distilled 基准上，较 EchoVLM，BLEU-1 从 0.25 升至 0.40、ROUGE-1 从 0.28 升至 0.45、METEOR 从 0.27 升至 0.43，ClinicalScore 从 55.02 升至 74.20。

Insight: 超声报告生成应受当前可见影像证据约束，不能把完整检查报告中的合理信息直接当作单帧/有限视图监督。论文同时提出证据对齐数据和临床结构化指标。

Abstract: Generating ultrasound reports from multiple images requires aggregating clinical evidence across views, yet archived key frames capture only part of the dynamic examination. Raw-report imitation is therefore misaligned with visual supervision: content that is clinically valid for the full examination may be unverifiable from the images available to a model. This gap creates a clinical behavior alignment problem. A model must preserve visible findings, avoid diagnostic reversals and unsupported completion, and not collapse into conservative templates. We propose CAMEO, a Clinically Aware Multi-image Evidence-grounded Orchestration framework for ultrasound report generation. Stage I learns ultrasound visual-language primitives; Stage II performs Cross-View Evidence Grounding by distilling trusted visible report points into multi-image QA and report-style supervision; and Stage III performs Clinically Aware Preference Alignment using clinical-error-oriented preference pairs. From USReport, we construct USReport-Distilled with 17,670 evidence-grounded paired-image training instances and USReport-Pref with 21,869 preference pairs; we additionally use 25,631 PubMedVision-US ultrasound instruction samples for domain adaptation and multi-image instruction tuning. On the primary USReport-Distilled benchmark, CAMEO improves over EchoVLM from 0.25 to 0.40 BLEU-1, 0.28 to 0.45 ROUGE-1, and 0.27 to 0.43 METEOR, while raising ClinicalScore from 55.02 to 74.20. These results underscore the value of evidence-grounded supervision, clinically aware alignment, and clinically structured evaluation for reliable ultrasound report generation.

[57] TAM: Task-Aware Memory Distillation for Efficient Spatiotemporal Prediction cs.CV | cs.AIPDF

Yuqi Li, Xiaoqin Feng, Fan Xu, Weilun Feng, Chuanguang Yang

TL;DR: 提出 TAM，将冻结教师的时空预测知识组织成有界可检索历史，再按任务选择历史参考对学生蒸馏，训练结束后不增加学生推理开销。

Details

Motivation: 常规蒸馏逐样本匹配教师输出或特征，没有充分利用跨样本预测结构；要利用该结构，表示与历史参考需反映具体任务的动态。

Result: TAM 的记忆条目可编码潜特征、预测变化或光流残差，并按任务选择历史参照。学生要么匹配教师在共享参照上的相似度分布，要么回归受观测条件的残差原型；辅助目标与监督预测及常规蒸馏结合。教师、记忆和适配器只在训练时使用，推理端学生保持不变。评估涵盖视频预测、天气预报和交通流预测、多种师生配置；四次配对运行均值中，相较对应 KD 基线，六个视频集 SSIM 全部提升、五个 MSE 降低。平均配对 MSE 降幅最高：KittiCaltech 1.86%，gSTA 教师的 WeatherBench 1.93%，TaxiBJ 1.01%。

Insight: 历史教师监督能利用不同样本间的预测关系，且不把教师或记忆引入学生推理过程。报告的收益跨任务，但提升较温和且基于配对运行均值。

Abstract: Knowledge distillation enables efficient spatiotemporal prediction by transferring knowledge from an accurate teacher to a compact student. However, matching outputs or features independently for each sample leaves cross-sample predictive structure underused. Exploiting this structure requires representations and historical references that reflect the dynamics of each task. We propose TAM, a Task-Aware Memory Distillation framework that organizes a frozen teacher’s knowledge into a bounded, retrievable history. Memory entries encode latent features, forecast changes, or flow residuals, while task-specific selection rules identify relevant historical references. The student either matches the teacher’s similarity distribution over shared references or regresses observation-conditioned residual prototypes. These objectives complement supervised prediction and conventional distillation. The teacher, memory, and auxiliary adapters are used only during training, leaving student inference unchanged. We evaluate TAM on video prediction, weather forecasting, and traffic flow prediction across multiple teacher-student configurations. Averaged over four paired runs, adding TAM improves SSIM on all six video datasets and reduces MSE on five relative to the corresponding KD baselines. Mean paired MSE reductions reach 1.86% on KittiCaltech, 1.93% on WeatherBench with a gSTA teacher, and 1.01% on TaxiBJ. These results demonstrate the utility of historical teacher supervision across distinct forecasting tasks without additional student inference cost.

[58] DisFace3DNet: Explainable Facial Attractiveness Prediction via 3D Component Disentanglement cs.CVPDF

Fenggui Rao, Yan Luximon, Jie Zhang

TL;DR: 提出 DisFace3DNet，通过 3D 部件解耦从整体吸引力评分学习七项部件参考分数，以量化形状、皮肤、头发、光照、背景、表情和姿态的贡献。

Details

Motivation: 面部吸引力预测通常只给整体评分，形状、外观及拍摄条件各自作用不清；缺少人工部件分数也限制了可解释监督。

Result: 方法以整体评分和辅助弱语义监督学习七个部件分数，结合专门 3D 表示和图像线索训练身份、皮肤、头发、光线、背景、表情和姿态路由，再用受约束拟合将五个静态与两个带符号动态分数组合为整体评分。SCUT-FBP5500 五折平均 Pearson 相关系数为 0.8904±0.0063；部件项可在数值精度内重构所有留出整体预测。皮肤、头发和脸型产生最大部件预测变化。人评支持脸型、皮肤、头发分数方向，表情一致性较弱。

Insight: 模型将整体预测分解为可量化部件贡献，但这些贡献是从整体评分与弱语义监督推导，不是独立人工部件真值。人评支持程度因属性而异。

Abstract: Facial attractiveness prediction usually assigns one overall rating, leaving the roles of shape, appearance, and viewing conditions implicit. We propose DisFace3DNet, which uses 3D component disentanglement to learn seven component reference scores from overall ratings with auxiliary weak semantic supervision, without human-labeled component targets. Designated 3D representations and image cues feed jointly learned routes for identity, skin, hair, light, background, expression, and pose. A constrained fit then combines five static and two signed dynamic scores into the overall rating, exposing each component’s numerical contribution and supporting component-specific comparisons across images. On SCUT-FBP5500, DisFace3DNet achieves a Pearson correlation of $0.8904\pm0.0063$ (mean $\pm$ standard deviation across five folds) with average human ratings; its component terms reconstruct every held-out prediction to numerical precision. Skin, hair, and facial shape account for the largest component-wise prediction variation. Human evaluation supports the score directions for facial shape, skin, and hair; expression agreement is weaker. DisFace3DNet thus connects overall prediction to quantitative analysis of the facial and contextual cues entering each estimate.

[59] Perceptually Grounded and Semantics-Aware Evaluation for Holistic Co-Speech Gesture Generation cs.CVPDF

Nick Milkin, Lanmiao Liu, Esam Ghaleb, Asli Ozyurek, Zerrin Yumak

TL;DR: 提出联合感知和语义感知的共语手势生成评测框架，验证 13 项客观指标与 101 名人类评分的相关性，并构造五种目标特定综合指标。

Details

Motivation: 整体且语义一致的共语手势生成进步快，但客观指标与人类感受不总一致，语义合适度也难量化。

Result: 基准整理 13 项指标，涵盖分布相似度、几何保真、运动学、跨模态同步和语义适切性；新增 Semantic Gesture Preservation（SGP）衡量生成动作保留真值语义手势的程度，并用多模态 LLM 扩增 BEAT2 标注。101 名参与者从拟人度、运动多样性、无动画错误、语音时序和内容匹配五方面评审生成手势。与 Semantic Score 不同，SGP 与语音感知的人类判断有选择性关联。作者据此构建五种目标专属综合指标，在全部五个主观维度上提高感知对齐，动画错误缺失和内容匹配的增益最大。

Insight: 单一指标不一定代理人类感知；指标需在主观评价上验证，互补客观信号的组合可更好匹配特定质量维度。

Abstract: Holistic and semantics-aware co-speech gesture generation has advanced rapidly, yet evaluation remains behind: objective metrics do not consistently reflect human perception, and semantic appropriateness remains difficult to quantify. We present a perceptually grounded and semantics-aware benchmark that combines standardized model comparison, human-centered metric validation, and fine-grained semantic evaluation. We first curate a list of 13 objective metrics covering different aspects, including distributional similarity, geometric fidelity, kinematic quality, cross-modal synchrony, and semantic appropriateness. For the semantic-appropriateness category, we propose a new metric, Semantic Gesture Preservation (SGP), which measures how far semantic gestures in the ground truth are preserved in the generated gestures. For this, we augment the BEAT2 dataset’s annotations using a multi-modal LLM. We then conduct a perceptual study where 101 participants score generated gestures among five dimensions, including human-likeness, motion diversity, absence of animation errors, speech timing and content match. We systematically analyze objective metric–subjective score correlations. Unlike Semantic Score (SC), which shows no significant association with the evaluated perceptual dimensions, SGP is selectively aligned with speech-aware human judgments. We construct five target-specific composite metrics aligned with the subjective dimensions. These composites improve perceptual alignment across all five dimensions, with the largest gains for absence of animation errors and content match, indicating that complementary objective signals can better approximate human judgments than individual metrics alone. Overall, our results show that objective metrics require validation against subjective evaluations.

[60] VESSI - VLM-Enhanced Support for Surveillance and Investigations cs.CVPDF

Saverio Cavasin, Pietro Tedeschi, Mattia Tamiazzo, Alessandro Brighente, Simone Milani

TL;DR: 提出 VESSI，以 VLM 对监控视频进行提示式询问，将显著视觉内容转成文本描述，并提供无参考标签的 CMUS 评估指标支持运维筛查。

Details

Motivation: 传统自动监控系统缺少语义理解模块，难以支持全面态势感知和事后取证，降低事件理解能力并增加人工分析负担。

Result: VESSI 用四种 SOTA 模型测试视频问询框架；由于多数监控数据集无标签，另提出 Composite Model Utility Score（CMUS）评估模型。评测中最可靠模型在超过 66% 的视频里标记潜在相关活动，同时审阅时间减少超过 85%。无参考的 CMUS 模型排序与普通视频 CMUS 重现一致，并匹配基于 5,909 张人工参考帧得到的假阳性率排序，支持该分数在所测设置中的运维使用。

Insight: 该系统面向辅助分析员筛查，不是自动定性事件；CMUS 试图在缺少标注时排序模型，但其一致性验证限于本实验视频和人工帧参考。

Abstract: Automated video surveillance analysis has become a critical component of intelligence infrastructures and Law Enforcement agencies. Traditional systems lack the semantic module for comprehensive situational awareness and forensic tasks, limiting their ability to interpret events meaningfully or support post-incident investigations. This slows operational insight and increases the burden on human analysts. Recent advances in Vision-Language Models (VLMs) offer promising pathways to bridge this gap. To address this, we propose VLM-Enhanced Support for Surveillance and Investigations (VESSI), a VLM-based framework designed to enhance automated video surveillance analysis through prompt-driven interrogation of video sequences where salient visual features are converted into textual descriptions. We test our framework with four state-of-the-art models. Since most datasets for this task are unlabeled, we also propose the Composite Model Utility Score (CMUS) to assess VLM performance. Experimental results show that our solution substantially improves the analysis capabilities of human operators and enhances the flexibility of automated surveillance systems. In our evaluation, the most reliable model flagged potentially relevant activity in more than 66% of the videos while reducing review time by more than 85%, offering a practical balance between selectivity and efficiency. The model ordering produced by the reference-free CMUS evaluation was reproduced by the normal-video CMUS evaluation and matched the false-positive-rate ordering obtained from 5,909 manually referenced frames. This agreement supports the operational use of the score within the evaluated setting.

[61] Towards Unified Evaluation of Prompt Enhancers for Video Generation cs.CVPDF

Yawen Shao, Yubo Zhu, Ziyun Dai, Zixun Fang, Kai Zhu

TL;DR: 提出 PEBench，直接评测视频生成提示增强器在文本、图像和参考视频条件下的改写质量，而无需反复生成成片。

Details

Motivation: 提示增强器把简短指令和多模态参考转换为结构化电影式计划；现有评估需渲染视频，计算与人工成本高、迭代慢，还会把增强器质量与下游生成器行为混淆。

Result: PEBench 含 1,100 个专家审核案例、1,005 个视觉素材和 35 种细粒度任务，涵盖时间、电影语言、视听与多参考条件。证据 grounding 评估结合模态感知事实抽取和 24 项量表。对代表性开闭源方法的分析显示，提示增强正从细粒度描述扩写转向保留意图的电影式规划；字幕重建与前向细化方法在电影覆盖、语义保真或内部连贯性方面各有优势。人类验证发现，PEBench 评分与专家对增强提示的判断，以及 Wan3.0、MiniMax-H3 生成视频的质量判断高度一致。

Insight: 在提示层评估可避免将增强器与视频生成器的影响混在一起；人类一致性结果表明提示质量评分能反映下游效用，但不同增强方法呈互补而非单一排序。

Abstract: Modern video generators can realize increasingly complex visual narratives, positioning the prompt enhancer (PE) as a critical bridge from concise user instructions and multimodal references to structured cinematic plans. However, existing PE evaluation relies on rendered videos, imposing substantial computational and human costs, slowing PE training and iteration, and conflating PE quality with downstream generator behavior. To address this gap, we introduce PEBench, the first unified benchmark for direct PE evaluation across text-to-video, image-to-video, and reference-to-video prompt enhancement. It comprises 1,100 expert-verified cases and 1,005 visual assets, spanning 35 fine-grained tasks with diverse temporal, cinematic, audiovisual, and multi-reference requirements. In addition, we develop PEBench evaluation, an evidence-grounded framework that combines modality-aware fact extraction with rubric-based assessment across 24 criteria. Our systematic evaluation of representative open- and closed-source PE methods reveals an emerging shift from fine-grained descriptive expansion toward intent-preserving cinematic planning, while the caption-reconstruction and forward-refinement methods show complementary strengths in cinematic coverage and semantic fidelity or internal coherence, respectively. Human validation shows that PEBench scores align closely with expert judgments of enhanced prompts and downstream videos from Wan3.0 and MiniMax-H3, indicating that prompt-level evaluation reliably reflects downstream utility.

[62] Streaming-Aware Diffusion for Real-Time Video Super-Resolution via Cross-Step Attention cs.CVPDF

Harris Partaourides, Sotirios Chatzis

TL;DR: 提出流式感知视频超分框架，以跨步注意力复用相邻帧及扩散步骤的中间特征，并通过耦合轨迹调度在降低复杂度时保持时间连贯。

Details

Motivation: 实时视频超分要求高时空保真和严格低延迟；扩散模型的反复采样成本高，而只使用预训练单图像模型又缺少显式时序协调。

Result: Cross-Step Attention 在相邻帧和扩散步骤间复用去噪中间特征，不做显式时间建模；Trajectory-Coupled Diffusion Scheduling 对齐相邻扩散状态，为跨步条件提供更干净中间表示。流式管线逐帧传播潜变量，将复杂度由 N 帧、S 步的 O(N·S) 降为 O(N+S)。在 REDS4 和 YouHQ40-Test 上提升感知质量与时间真实感，同时维持逐帧稳定；冷启动后以 512×512 分辨率超过 40 FPS。

Insight: 通过跨帧共享去噪特征和对齐扩散轨迹实现流式提速，无需显式时序模块；报告速度包含冷启动后的运行表现。

Abstract: Real-time video super-resolution requires high spatio-temporal fidelity under strict latency constraints, challenging diffusion models due to their iterative sampling cost and limited temporal coordination. We propose a streaming-aware framework that adapts pretrained single-image latent diffusion models for efficient video super-resolution (VSR) by exploiting the sequential structure of video streams. Our Cross-Step Attention mechanism reuses intermediate denoising features across adjacent frames and diffusion steps, enabling temporal information exchange without explicit temporal modeling. We further introduce Trajectory-Coupled Diffusion Scheduling, which aligns adjacent diffusion states and provides cleaner intermediate representations for cross-step conditioning, improving temporal coherence. These components are integrated into a streaming inference pipeline that incrementally propagates latent states across frames, reducing the effective computational complexity from $O(N \cdot S)$ to $O(N + S)$ for $N$ frames and $S$ diffusion steps. Experiments on REDS4 and YouHQ40-Test demonstrate improved perceptual quality and temporal realism while maintaining frame-wise stability. Our method achieves over 40 FPS at $512 \times 512$ resolution after cold start, enabling real-time VSR without explicit temporal modeling.

[63] From Video Clips to Creation Trajectory: Sora100K for AI-Native Video Creation cs.CVPDF

Sicong Yang, Ruihuan Yang, Jian Lu, Jianfei Yuan, Xiaodong Cun

TL;DR: 提出 Sora100K，将 AI 原生视频创作表示为文本生成、单轮编辑和多轮编辑组成的结构化轨迹数据集，而非孤立视频片段集合。

Details

Motivation: AI 原生创作正从单段视频生成转向反复编辑工作流，但现有数据仍以孤立视频片段为主，没有连接生成与后续编辑阶段。

Result: Sora100K 将创作过程拆为三类结构：文生视频记录作为根节点、单轮编辑作为边、多轮编辑作为完整轨迹；VLM 为生成根标注语义，为编辑边标注操作。严格构建流程重建源视频到编辑结果的谱系、编辑顺序和中间视频状态。作者用 LTX-2 做轻量适配检验监督价值，发现视觉质量、多镜头生成及镜头间一致性提升；连续轮次评估显示，遵循多轮编辑指令仍有困难。

Insight: 保留来源和编辑序列可支持工作流级生成/修改研究；实验也显示多轮指令跟随尚未解决。

Abstract: AI-Native video creation is shifting from isolated video clips toward iterative video creation workflows. However, existing datasets remain largely video clips, representing video generation and editing as separate tasks rather than connected stages of a video creation workflow. In this paper, we introduce Sora100K, a dataset that represents the AI-Native video creation workflow as a structured video creation trajectory. Specifically, we first identify video creation trajectories and decompose them into three subsets according to their structural roles: text-to-video generation records as roots, single-turn video editing records as editing edges, and multi-turn video editing records as complete trajectories. Then, we use a VLM to assign semantic annotations for generation roots and editing-operation annotations for editing edges. A strict construction pipeline further reconstructs source-to-edit lineage, editing order, and intermediate video states while ensuring data quality. Finally, we perform lightweight adaptation on LTX-2 models to assess the supervision value of Sora100K. The results show improvements in visual quality, multi-shot generation, and cross-shot consistency, while successive-turn evaluation reveals that following multi-turn editing instructions remains challenging. Sora100K establishes a new data foundation for AI-Native video creation beyond isolated video clips and toward structured video creation trajectory. The dataset and supplementary materials are publicly available at https://huggingface.co/datasets/ysicong/Sora100K.

[64] Memory Forcing: Attendable Mid-Horizon History for Streaming Video Generation cs.CVPDF

Jiaming Zhang, Xinyu Wang, Huafeng Shi, Gangshan Wu, Limin Wang

TL;DR: 提出 Memory Forcing，在固定 KV 缓存中分设 sink、archive 和 working 区，保存近期动作与中程事件，缓解流式自回归视频生成的中程遗忘。

Details

Motivation: 少步自回归视频扩散常只在固定大小 KV 缓存中保留开头和最近帧；事件一旦移出窗口，后续生成就无法关注，造成中程遗忘和场景/主体不连续。

Result: Archive 与 Working Banks 在固定内存下分别保留多样中间事件和近期运动。由于绝对时间索引可能超出训练范围，Bank-aware RoPE 在注意力时重新分配位置索引，使各缓存区保持可区分。在 1.3B 模型上，Memory Forcing 在长片段领先；在报告全部四种时长的方法中，从 5 秒到 60 秒性能降幅最小，并能保持主体和场景离开后重返的连续性。同一设计扩展至 Wan2.2 5B，生成的视频物理合理性、真实感和动态性更好；作者称其为该 forcing 线路首个公开 5B 模型。

Insight: 在固定缓存大小下，不同历史区域需分配不同保留策略和位置编码；该设计专门处理离开近期窗口后再出现的中程事件。

Abstract: Autoregressive video diffusion enables causal video streaming without a bidirectional pass over the full clip, but existing few-step systems usually retain only the opening and most recent frames in a fixed-size KV cache. Once an event leaves this window, later frames can no longer attend to it, a failure we term mid-horizon forgetting. We present Memory Forcing, a few-step streaming method that preserves this missing history without increasing the cache size. Its Archive & Working Banks partition the cache into sink, archive, and working regions, retaining diverse intermediate events alongside recent motion under fixed memory. Because absolute temporal indices drift outside the training range, Bank-aware RoPE reassigns indices at attention time so each bank remains distinguishable. At 1.3B, Memory Forcing leads on longer clips, shows the smallest drop from 5s to 60s among methods reporting all four lengths, and preserves subjects and scenes through leave-and-return. The same design scales to Wan2.2 5B, producing more physically plausible, realistic, and dynamic videos and, to our knowledge, the first public 5B model on this forcing line.

[65] Skill-V: Verifiable Self-Evolving Skill Library for Interactive Agents cs.CVPDF

Jie Ma, Zhipeng Qian, Yufei Ma, Zihan Liang, Jiayi Ji

TL;DR: 提出 Skill-V，把交互技能表示成可证伪、带版本的契约，并根据任务失败新增技能、依据契约—结果分歧修订旧技能，同时过滤当前不适用技能。

Details

Motivation: 自进化技能库常通过积累新知识改进，失败可能产生新技能，但存量技能很少根据新证据重新检查；累积本身不保证可靠，已存技能也可能误设适用边界。

Result: Skill-V 将语义意图与可观察行为标准绑定成有版本、可证伪的技能契约。任务失败触发技能新增，契约评价与结果不符则引导边界修订；修订需保留受保护语义约束，并在历史回放的量表—结果指标上满足非退化条件。适用性过滤器会排除被高置信判为当前任务不适用的候选。在 ALFWorld 和 WebShop 上成功率分别为 95.3% 和 85.9%，技能库比只增不改的基线更紧凑；过滤减少误调用，结果 grounding 的修订能纠正边界而不损害历史证据上的表现。

Insight: 可靠技能进化同时需要新增、验证、修订和择时使用；历史回放与非退化约束防止新经验破坏已有能力。

Abstract: Interactive agents can turn experience into reusable skills, yet existing self-evolving skill libraries primarily improve by accumulating new knowledge. Failures may lead to new skills, while previously stored skills are less often revisited as new evidence arrives. However, growth alone does not ensure reliability, as a retrieved skill may be inapplicable under the current task conditions, and an existing skill may encode a mis-specified operational boundary. Reliable skill evolution therefore requires not only adding knowledge, but also testing and revising what is already stored. We introduce Skill-V, a verifiable self-evolving skill library. To make stored knowledge testable, we propose representing skills as versioned, falsifiable contracts that link semantic intent to observable behavioral criteria. We use environment outcomes to drive library evolution. Specifically, task failures motivate skill addition, while disagreements between contract evaluations and task outcomes guide revisions to existing skill boundaries. To validate these revisions, we require them to preserve protected semantic constraints and satisfy non-regression criteria for rubric-outcome metrics on historical replay evidence. Finally, we employ an applicability-aware filter to exclude candidates judged confidently inapplicable to the current task. Across ALFWorld and WebShop, Skill-V achieves success rates of 95.3% and 85.9%, respectively, while maintaining a more compact skill library than growth-oriented baselines. Applicability-aware filtering reduces incorrect skill invocations, and outcome-grounded revisions correct mis-specified skill boundaries without degrading performance on previously observed evidence. These results show that reliable skill evolution requires more than accumulating experience: the library must learn which knowledge to retain, when to revise it, and when it should be applied.

[66] Phase-aware video generation for physics-grounded dynamics and interactions cs.CVPDF

Jingfeng Ou, Kun Wang, Rui Zhao, Jingwei Guan, Limin Wang

TL;DR: 提出 PAVG，以固体/气体双分支和时空交叉注意力显式建模不同相态运动及其耦合，并构建含 70 多万条轨迹的模拟训练语料。

Details

Motivation: 固—气动力学视频生成需同时呈现不同相态的独特运动规律，以及相之间物理交互造成的耦合响应。

Result: PAVG 使用双分支分别建模固体与气体动力学，再以时空交叉注意力捕捉相间物理交互，使模型既保留各相运动特征又生成物理一致的相互响应。作者构建模拟语料，含超过 700,000 条覆盖固体、气体及固—气交互场景的物理轨迹。多项实验显示，相较现有方法，生成视频在运动遵循性、物理合理性和视觉质量方面改善；摘要未列量化分数。

Insight: 分离建模不同相态再显式融合，比把所有运动模式混入单一生成表示更有利于保留相态特性并表达交互。

Abstract: Generating physically plausible videos for solid-gas dynamics is challenging as different phases exhibit distinct dynamics yet remain coupled through physical interactions. We present PAVG, a Phase-Aware Video Generator for solid-gas dynamics and interactions. It employs a dual-branch architecture to explicitly model the distinct dynamics of solids and gases, while spatiotemporal cross-attention captures their physical interactions. This design enables PAVG to preserve phasespecific motion characteristics while producing physically consistent responses across phases. To facilitate this task, we further construct a simulation corpus comprising over 700K physical trajectories across diverse solid, gas, and solid-gas interaction scenarios. Extensive evaluations demonstrate that our PAVG produces videos with improved motion adherence, physical plausibility, and visual quality compared with existing approaches.

[67] Seek-and-View Reasoning for Multi-View Spatial Understanding cs.CVPDF

Qixiang Chen, Cheng Zhang, Fucai Ke, Chi-Wing Fu, Jianfei Cai

TL;DR: 提出 Vantage，一个无需训练、模型无关的多视角空间推理框架，先规划与问题相关视角，再借助 3D 基础模型补充几何证据。

Details

Motivation: 现有多视角空间推理常仅处理稀疏固定视角，VLM 只能在这些输入视图中理解场景、推断空间关系，导致跨视图对应不稳及几何到语言的瓶颈。

Result: Seek-and-View 方法先定位隐含的跨视图证据，选择与问题相关的视角。Vantage 由问题分析/视角规划阶段和几何证据增强阶段组成，将 VLM 与 3D 基础模型结合，综合生成并纳入空间证据。六个 VLM、五个基准上的综合实验显示，无需微调即可持续提升表现；作者称它减少了对语言式跨视图对齐的依赖。摘要未列具体提升分数。

Insight: 空间推理可以通过主动寻找有用视角补充证据，而不是让模型只在现成稀疏图像间进行隐式语言对齐。

Abstract: Existing approaches to multi-view spatial reasoning operate largely on sparse input views. Vision-language models (VLMs) are thus restricted to understand a scene and infer spatial relations within these fixed views, leading to fragile cross-view alignment and geometry-to-language bottleneck. To address these issues, we formulate a novel Seek-and-View reasoning approach to find implicit cross-view spatial evidence by locating a question-relevant view to support the spatial reasoning. To realize this approach, we propose Vantage, a training-free model-agnostic reasoning framework that pairs a VLM with a 3D foundation model: a viewpoint-grounded reasoning stage for question analysis and view planning, followed by a geometry-grounded evidence augmentation stage to effectively synthesize and incorporate visual evidence into the final reasoning. Comprehensive experiments on six VLMs demonstrate consistent improvements on five benchmarks without fine-tuning. Overall, by revealing spatial evidence through view-grounded reasoning, Vantage can largely reduce reliance on language-based cross-view alignment and improve multi-view spatial understanding. Our code is available at https://github.com/q1xiangchen/Vantage.

[68] From Pixels to Structure: Lightweight Vision-Language Models for Document OCR and Structured JSON Extraction cs.CV | cs.AIPDF

Uddipan Basu Bir, Vincent Christlein, Andreas Maier, Mathias Zinnen

TL;DR: 比较八种不超过 7B 参数的开源 VLM 在三所大学遗产馆藏中的 OCR 到结构化 JSON 能力，并分析微调、图像预处理和多阶段训练的影响。

Details

Motivation: 闭源大型 VLM 虽在文档理解上表现强，但商业 API 带来数据自主、持续费用和算力环境负担；遗产数字化还涉及历史手写、珠宝/史前/建筑等专业术语和非标准版式，需抽取高维结构字段。

Result: 研究要求模型从文档图像提取文本并生成符合 schema 的 JSON，以便自动验证与下游使用；按零样本、少样本和微调设置评估 CER、ANLS* 和 mAP-F1。相对微调基线，分别研究超参数优化、照明平坦化/去噪/CLAHE 等传统预处理和多阶段训练的独立效果，并比较馆藏专用微调与单个多数据集检查点的权衡。作者认为经过精心适配的 7B 以下模型可成为隐私可控、性能较强的替代方案，并提供面向遗产机构的实践建议；摘要未报告最终量化指标。

Insight: 对遗产文档，模型大小不是唯一因素，数据集专用适配、传统图像预处理和多馆藏联合训练均需纳入权衡；联合训练可用一个检查点覆盖多馆藏，但可能改变各馆藏间表现。

Abstract: While massive, closed-source Vision-Language Models (VLMs) set strong benchmarks for document understanding, their dependence on commercial APIs limits adoption in institutional archives due to data autonomy concerns, recurring costs, and the environmental footprint of hyperscale computing. This is especially acute in heritage digitization, where documents include historical handwriting, domain-specific terminology (e.g., jewelry, prehistory, architecture), and non-standard layouts requiring high-dimensional structured extraction. We present a comparative study of eight open-source lightweight VLMs (up to 7B parameters) for Optical Character Recognition (OCR)-to-structure across three university heritage collections. Given a document image, models must extract text and generate schema-compliant JSON, enabling automatic validation and downstream use. We evaluate models under a constraint-aware protocol across zero-shot, few-shot, and fine-tuning settings, measuring extraction fidelity and structured-output quality using Character Error Rate (CER), Approximate Normalized Levenshtein Similarity (ANLS*), and mean Average Precision F1 (mAP-F1). Against a fine-tuning baseline, we further test the independent impact of (i) hyperparameter optimization, (ii) classical image preprocessing (illumination flattening, denoising, and CLAHE), and (iii) multi-stage training. Finally, we analyze the trade-off between dataset-specific fine-tuning and a single multi-dataset checkpoint, where joint training enables one model to operate across collections but can shift performance between datasets. Overall, we show that carefully adapted VLMs with up to 7B parameters can provide a sustainable, private, high-performing alternative to manual transcription or commercial black-box systems, and we offer actionable guidance for heritage institutions seeking institution-controlled OCR-to-JSON extraction.

[69] From Suppression to Repair: Mitigating Object Hallucination in Large Vision-Language Models via Localized Distribution Alignment cs.CV | cs.AIPDF

Chen Zhao, Xingping Dong, Jiachun Shi, Liang Peng, Chong Wang

TL;DR: 提出 ResOT，一种免训练的推理时对象幻觉修复方法：在低维残差子空间内用高斯最优传输将幻觉分布对齐至忠实表征分布。

Details

Motivation: 抑制隐藏表示中的幻觉方向是一种直观做法，但相关方向也可能包含有用信息，整体删除会削弱多模态能力。

Result: ResOT 将主要幻觉方向投影到忠实子空间之外，形成低维干预残差空间；再用高斯最优传输对齐幻觉与忠实分布，得到尽可能少改动原表征的修复目标。推理时按 token 自适应控制状态向 OT 目标移动的幅度。三个代表性 LVLM 实验显示，方法显著降低对象幻觉，同时改善图像描述质量和多个基准上的多模态能力。摘要未给出模型、基准或量化分数。

Insight: 从“删除可疑方向”转向“把表示局部修复到忠实分布”，有望保留与幻觉特征重叠的有用信息；方法无需训练，效果范围需结合完整基准结果评估。

Abstract: Object hallucination remains a major obstacle for large vision-language models (LVLMs) to generate reliable content. An intuitive mitigation strategy is to suppress hallucination-related components in hidden representations. However, these components may also contain useful information, and suppressing them can weaken the model’s multimodal capabilities. In this paper, we propose ResOT, a training-free method that repairs representations at inference time through localized distribution alignment. Specifically, ResOT projects dominant hallucinated directions away from the faithful subspace, forming a low-dimensional residual subspace for intervention. Within this subspace, ResOT uses Gaussian optimal transport (OT) to align the hallucinated distribution with the faithful one. The resulting map defines repair targets with minimal changes to the original representations. At inference, ResOT adaptively controls how far each token state moves toward its OT target. Experiments on three representative LVLMs show that ResOT substantially reduces object hallucination while improving image caption quality and multimodal performance across multiple benchmarks. Code will be released.

[70] Open-Vocabulary Audio-Visual Event Localization via Complex-Valued Fusion cs.CV | cs.LG | cs.MM | cs.SD | eess.IVPDF

Anirudh Praveen, Koteswar Rao Jerripothula, Pratik Joshi, Aveen Dayal, Neela Sawant

TL;DR: 提出复值融合的开放词汇音视频事件定位方法，将视觉/文本及音频/文本相似度表示为复数，再以复值网络学习跨模态融合。

Details

Motivation: 开放词汇事件定位需在视频各片段识别训练未见事件。现有流程用冻结多模态模型计算视觉—文本和音频—文本两项余弦相似度，再以固定规则压成一个分数，难学习两种证据间关系。

Result: 方法为每种模态补充伴随流：视觉用 iHSV 虚部，音频用 CycleGAN 翻译的相位谱；原始表征为实部，形成复值相似度后由复值神经网络融合。视觉和音频编码器冻结，只训练时序注意力块及融合网络。在 OV-AVEBench 未见类划分上，Acc/Seg-F1/Event-F1 分别为 66.5/59.1/54.1%，较此前微调基线高 1.6/4.1/6.6 个百分点。修改后的 AVE 数据集上达到 60.7/51.9/50.4%，同样达 SOTA；双流变体也显著优于基线。

Insight: 复数实虚分量给视觉、音频相似度提供额外相位/互补信息，学习式融合优于固定标量规则。摘要报告了两个基准及其对应指标。

Abstract: Open-Vocabulary Audio-Visual Event Localization (OV-AVEL) labels each video segment with an event class, including classes that were never seen during training. The dominant pipeline uses a frozen multimodal foundation model (e.g. ImageBind) to embed the visual frame, the audio mel-spectrogram, and each candidate class name into a shared space, then computes two cosine similarities for each segment against each class: visual-text and audio-text. Existing methods then collapse this pair into a single scalar score with a fixed rule (geometric mean, weighted average) before taking the argmax. Instead, we compute complex-valued similarities and learn their fusion using a complex-valued neural network (CVNN). Each modality’s standard representation becomes the real part of our pipeline, and a paired companion stream supplies the imaginary part. We use imaginary part of iHSV for visual modality and CycleGAN-translated phase spectrogram for audio modality as these companion streams. This results in two complex similarities, which are then fused. While the vision and audio encoders remain frozen, only the temporal-attention blocks and the fusion CVNN are trained. The four-stream complex architecture sets a new state of the art on both OV-AVEL benchmarks. On the open (unseen-class) split of OV-AVEBench we reach 66.5/59.1/54.1% Acc/Seg-F1/Event-F1 (+1.6/+4.1/+6.6 over the previously reported fine-tuned baseline), with consistent gains for seen classes as well. We also modify AVE dataset for this task and observe that our architecture reaches 60.7/51.9/50.4% Acc/Seg-F1/Event-F1, achieving state-of-the-art OV-AVEL results on it as well. We also propose a two-stream alternative, which also sees great improvements over the baseline.

[71] VEDJE: Video-Efficient Discriminative Joint Encoder for Scalable Video-Text Retrieval cs.CVPDF

Shahaf Wagner, Gabriele Serussi, Dan Ben Ami, Tomer Galanti, Chaim Baskin

TL;DR: 提出 VEDJE，将视频特征压缩到可复用的小型缓存并利用特征变化预测辅助训练，以提高可扩展视频—文本检索效率。

Details

Motivation: 检索相似场景时需区分不同事件，联合匹配有助于准确性，但为每个查询反复处理丰富视频表征成本高。

Result: VEDJE 在采样帧内压缩特征，并将其分别存入可重复使用的缓存；训练时用特征变化预测作为辅助监督，提升压缩缓存检索而不增加查询时开销。在 MSR-VTT、MSVD、DiDeMo、ActivityNet 上，两个检索方向的 R@1 均优于匹配的第一阶段检索器。使用微调 VideoCLIP-XL 第一阶段时，MSR-VTT 文本到视频 R@1 为 59.8；VideoPrism 配置将每视频缓存压缩四倍至 12 KiB，文本到视频召回仅降低 0.2 分。

Insight: 将一次编码的紧凑视频证据缓存后供多个查询复用，可把准确检索与低查询成本结合；辅助特征变化预测只在训练阶段提供信号。

Abstract: Finding the right video often requires distinguishing similar scenes in which different events occur. Joint matching improves retrieval, but processing rich video representations for each query is costly. VEDJE compresses features within sampled frames while keeping their representations separate in a reusable cache. Feature-change prediction supplies an auxiliary training signal that improves retrieval from the compressed cache without adding work at query time. On MSR-VTT, MSVD, DiDeMo, and ActivityNet, VEDJE improves R@1 over matched first-stage retrievers in both retrieval directions. On MSR-VTT, it reaches 59.8 text-to-video R@1 with a fine-tuned VideoCLIP-XL first stage. In the VideoPrism configuration, shrinking the per-video cache fourfold to 12 KiB preserves text-to-video recall within 0.2 points. These results show that accurate video search can operate on compact evidence, encoded once and reused as new queries arrive.

[72] Relative Patch Response Learning for Generalizable AI-Generated Image Detection cs.CVPDF

Tianyu Wang, Ouxiang Li, Yanbin Hao, Zhenhua Tang, Shuo Wang

TL;DR: 提出相对 patch 响应学习 PRL，通过比较配对真实/生成混合视图中相同 patch 分数变化训练 AI 图像检测器，处理自注意力导致的局部来源标签失准。

Details

Motivation: AI 生成图像检测器需泛化到未见生成器和扰动。配对真实/生成图像可控制内容偏差，但若混合替换部分 patch，自注意力会让 patch 特征受其他区域来源影响，因此给单个 patch 标注真实/生成标签并不准确。

Result: PRL 不直接给 patch 打来源标签，而比较同一 patch 在两种配对混合视图中的响应变化。相对响应目标比较来源变化 patch 与仅因上下文变化而响应的来源不变 patch；参考一致性目标要求未变来源 patch 组整体移动；面积排序目标要求生成内容占比更大的视图平均 patch 分数更高。八个标准和三个野外基准的平均平衡准确率分别比最佳已有方法高 4.3% 和 5.9%。

Insight: 局部 patch 特征受全局上下文影响时，学习相对响应比绝对来源标签更稳健；三个目标分别处理上下文漂移、参照稳定性和生成区域面积差异。

Abstract: Generative models can now synthesize highly realistic images, simultaneously increasing the risks of misinformation and visual forgery. Therefore, detecting AI-generated images becomes more essential, and a reliable detector must generalize to unseen generators and stay robust to unseen perturbations in the wild. Existing detectors are typically trained on either independently collected real and generated images or aligned real-generated pairs designed to mitigate content bias. Building on aligned pairs, recent methods form a mixed view by replacing some patches of the real image with their generated counterparts. However, we find that self-attention lets real and generated patches interact, so the feature of each patch no longer reflects its own source alone. This contextual shift makes a per-patch source label an imprecise target. To this end, we propose Relative Patch Response Learning (PRL). Instead of labeling each patch, PRL compares the same patch across two mixed views of an aligned pair and learns from its patch response, the change of its score between the views. (i) To give precise supervision under the contextual shift, a relative response objective measures the responses of source-changed patches against those of source-unchanged patches, which respond to the shift alone. (ii) To provide a reliable reference for the shift, a reference coherence objective keeps each group of source-unchanged patches moving as a whole. (iii) Since the two views contain different amounts of generated content, an area ranking objective asks the view with the larger generated area to have a higher mean patch score. Extensive experiments demonstrate the superior performance of PRL, which surpasses the best prior methods by 4.3% and 5.9% in average balanced accuracy across eight standard and three in-the-wild benchmarks, respectively.

Jingyi Pan, Dan Xu, Qiong Luo

TL;DR: 提出 FreeInpaint，从无位姿多视图掩码图像前馈补全 3D 场景，并以可学习掩码注意力保护可靠观测、以支持 token 为严重遮挡区注入外观证据。

Details

Motivation: 3D 场景修补要恢复编辑/遮挡区域几何和纹理一致性，但现有方法一般依赖准确标定的相机位姿，不适合随手拍真实场景且增加预处理成本。

Result: FreeInpaint 扩展 3D 基础模型，使其从参考视图将掩码区域传播到其他无位姿视图，同时保留模型恢复相机位姿和场景几何的能力。可学习掩码注意力在处理掩码输入时保持可靠观测的空间锚点，并允许掩码区域在深层逐步吸收上下文；严重遮挡时，Support Token Refinement 将扩散生成的支持证据以置信度加权辅助 token 注入，同时保留原空间锚定。多样数据集实验报告补全质量优于基线，无需预计算相机位姿且推理速度快；摘要未列数值。

Insight: 该方案把位姿恢复和场景补全放入同一前馈模型，并将辅助生成证据作为补充 token 而非替换空间锚点；摘要没有量化精度与速度。

Abstract: 3D scene inpainting aims to recover missing or occluded regions in edited 3D scenes, while ensuring geometric and textural consistency. Existing approaches, however, typically require accurately calibrated camera poses, which restricts their applicability in casual, in-the-wild scenarios and introduces additional preprocessing overhead. To overcome this limitation, we present FreeInpaint, a novel feed-forward framework that generates complete and 3D-consistent scenes directly from unposed multi-view images with masked regions. At its core, FreeInpaint extends a 3D foundation model to propagate masked regions from a reference view to other unposed views, bridging 3D reconstruction and scene inpainting while preserving the model’s native ability to recover camera poses and scene geometry. Our method addresses two key challenges in adapting feed-forward 3D foundation models to masked inputs. First, masked regions can corrupt cross-view correspondence reasoning, degrading pose estimation and geometry recovery. To address this, we introduce a Learnable Mask Attention mechanism that preserves the spatial anchoring of reliable observations while allowing masked regions to progressively absorb useful context in deeper layers. Second, under severe occlusions, a single forward pass often lacks sufficient appearance evidence for high-fidelity completion. Therefore, we propose a Support Token Refinement strategy, which injects diffusion-generated support evidence as confidence-weighted auxiliary tokens to refine under-observed regions while preserving the original spatial anchor. Extensive experiments across diverse datasets demonstrate that FreeInpaint achieves superior inpainting quality, eliminating the reliance on pre-computed camera poses while keeping a fast inference speed. The project page is https://rorisis.github.io/FreeInpaint/.

[74] Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations cs.CV | cs.AIPDF

Shuran Ma, JiaLe Li, Yuxin Dong, Shan Zheng, Qingyun Jiang

TL;DR: 提出 AIMS，在解码时按查询自适应协调视觉、预填文本和已生成上下文的影响，以免单一模态增强方法遗漏文字等其他证据来源。

Details

Motivation: 多数免训练 LVLM 幻觉缓解依赖对比解码或增强视觉证据，但不同上下文来源可能需要动态平衡；模型能否按当前问题调节各来源贡献是关键。

Result: 作者分析发现 LVLM 存在内在视觉关注倾向，可用于自适应视觉引导，文本上下文也可参与降低幻觉。AIMS 为视觉、预填文本和已生成上下文构造紧凑原型，估计它们与当前查询的亲和度，以确定逐注意力头引导权重；再将多来源引导方向作用于查询表示。该方法无需训练或额外前向。在多个 LVLM 和解码策略上的实验显示，可有效缓解对象幻觉，同时保留有竞争力的通用多模态能力。

Insight: 幻觉控制可依据查询动态整合多类上下文，而不是一概提升视觉权重。AIMS 通过轻量原型计算实现来源协调，摘要未提供幻觉指标分数。

Abstract: Hallucination remains a significant challenge in Large Vision-Language Models (LVLMs). Existing training-free methods generally mitigate hallucinations through contrastive decoding or visual enhancement, often increasing the relative influence of visual evidence during generation. This raises a fundamental question: Can LVLMs dynamically regulate the contributions of different context sources to suppress hallucinations? In this work, we investigate and quantify how LVLMs coordinate multiple context sources during decoding and examine how this intrinsic behavior can guide hallucination mitigation. We find that LVLMs exhibit an intrinsic vision-attending tendency that can guide adaptive visual steering, while textual contexts can also contribute to hallucination mitigation. Motivated by these findings, we propose AIMS (Adaptive Information Multi-source Steering), a lightweight training-free framework that adaptively coordinates visual, prefilled textual, and generated contexts during decoding. Specifically, AIMS constructs compact prototypes for the three context domains and estimates their affinities with the current query to determine head-wise steering weights. The resulting multi-source steering direction is applied to the query representation, enabling adaptive context integration without additional model training or auxiliary forward passes. Extensive experiments across multiple LVLMs and decoding strategies demonstrate that AIMS effectively mitigates object hallucination while maintaining competitive general-purpose multimodal capabilities.

[75] Revisiting Identity and Spectra Dispersion in Media-Bridged Time Series Forecasting: Linking Multivariate Signals and Narrative Flows cs.CV | cs.LGPDF

Jierui Lei, Wenjian Zhang, Qingyi Yang, Yuyang Hong, Fangzheng Chen

TL;DR: 提出 MIDAPN，将多变量数值序列和预对齐叙事流统一到同一时空预测骨干，用媒体通用图关系与自适应频谱卷积处理身份和时间尺度差异。

Details

Motivation: 媒体桥接时间序列预测从传统多变量数据扩展至含文本辅助的多模态场景，但现有模型常依赖特定关系、融合和时序模块，难以形成数值与叙事流共用的预测骨干。

Result: MIDAPN 包含媒体身份感知图 MIDAG，以静态本质、动态行为和潜在共性表示变量身份及跨媒体依赖；上下文身份调制 CIM 进一步改善判别式聚合。Spectral Prism Convolution（SPConv）进行层级时间分析，平衡粗趋势与细节；自适应搜索指导配置尺度高效结构用于时间维重构。评估比较 16 种 SOTA TSF 模型，覆盖 13 个多变量和 12 个多模态数据集；另与 14 个时间序列基础模型及融合预训练语言模型进行长上下文比较。作者报告 MIDAPN 持续领先且兼容广泛共享骨干，摘要未列具体指标分数。

Insight: 媒体身份拆解与频谱时间建模被设计为解耦互补模块，以统一不同数据类型的预测范式。广泛数据集与模型比较支持通用性，但摘要未提供逐项结果。

Abstract: Media-bridged time series forecasting is expanding to encompass traditional “multivariate” and emerging “multimodal” (e.g., through textual assistance). Existing Time Series Forecasting (TSF) models still rely on paradigm-specific relation, fusion, and temporal modules, hindering a common forecasting backbone across numerical and pre-aligned narrative-flow settings. To explore this, we propose the Multimedia Identity-Aware Prism Network (MIDAPN), a unified spatiotemporal forecasting backbone based on media-general graph adaptation and automatic temporal learning: (1) Following media pre-alignment, our Multimedia Identity-Aware Graph (MIDAG) revisits identity through static essence, dynamic behavior, and latent commonality, inducing affinities that extend variable-specific dependencies across media. Contextual Identity Modulation (CIM) further refines discriminative aggregation. (2) We develop Spectral Prism Convolution (SPConv) to automatically perform hierarchical temporal analysis, balancing coarse trends and fine-grained details. Meanwhile, its Adaptive Search Guidance configures a scale-efficient architecture for temporal-dimension reconstruction. These decoupled yet synergistic components jointly address media identity disentanglement and temporal-scale mismatch. Comprehensive evaluations involving 16 SOTA TSF models across 13 “multivariate” and 12 “multimodal” datasets, alongside targeted long-context comparisons against 14 time series foundation models and fused pretrained language models, demonstrate MIDAPN’s consistent superiority and broad shared backbone compatibility. The code is available at \href{https://github.com/leijieruilq/MIDAPN/tree/main}{https://github.com/MIDAPN}.

[76] Does Target Alignment Mean Target Recovery? An Evidence-Ladder Study of Adversarial Claims on Contrastive Encoders cs.CVPDF

Tao Yang, Jianying Zhou

TL;DR: 研究对抗图像的 VLM 内部目标对齐分数能否预测独立模型是否恢复目标语义，并用预注册证据阶梯评估六种编码器。

Details

Motivation: 视觉语言对抗攻击常将图像推向文本目标，再用被攻击模型自身相似度上升证明攻击成功；但受害模型空间里的目标对齐是否意味着独立模型也能识别目标并不清楚。

Result: 作者构造测量工具，包括攻击几何外部的监督判定器、真实目标混合对照、目标打乱负例，以及基于 50% 目标图混合的参考水平。两个预注册实验在三个扰动预算下比较六种对比编码器。鲁棒训练编码器 FARE、TeCoA、PMG、TRADES 比标准 CLIP/SigLIP 能迁移更多独立证据；八个对比均在 bootstrap 最低 p 值阈值下显著。但没有任何单元达到混合图参考水平，最佳三个仅落在其复制区间内，因此实际目标恢复尚无法定论。鲁棒编码器内部，样本级对齐增益与证据增益相关 ρ=0.24–0.51；标准 CLIP 内相关接近零；跨编码器没有单调关系。

Insight: VTS 只在固定鲁棒编码器内提供有限证据，不能跨编码器直接把内部相似度当作目标语义恢复；作者建议采用更严格报告协议。

Abstract: Adversarial attacks on vision-language models optimize an image toward a text target, then cite the attacked model’s similarity score as evidence of success. We ask whether that score - victim-space target alignment (VTS) - predicts recovery of the target by an independent model. We first build a measurement instrument: supervised judges outside the attacked geometry, real-target blend controls, shuffled-target negatives, and a reference level derived from a 50% target-image blend. Two preregistered studies then compare six contrastive encoders under a matched attack at three perturbation budgets. Robustly trained encoders (FARE, TeCoA, PMG, TRADES) transfer substantially more independent evidence than vanilla CLIP or SigLIP; all eight contrasts reject at the bootstrap floor. However, no cell reaches the blend-derived reference level. The three best cells fall within its replication band, leaving practical recovery undecided. Within robust encoders, per-sample alignment gain correlates with evidence gain ($ρ= 0.24-0.51$); within vanilla CLIP the correlation is consistent with zero. Across encoders we find no monotone alignment-evidence relation. VTS is therefore informative only within a fixed robust encoder, and we provide a reporting protocol in its place.

[77] Look Where You Can: Active View Selection for CAD Reconstruction under Occlusion cs.CVPDF

Kartik Bali, Mahish Guru, Yiderigun Borjigin, Alexandra Starostina, Christian J. Cyron

TL;DR: 提出 SightCAD，将可行视角选择作为受场景遮挡约束的 CAD 重建问题，并联合训练视角策略与生成可执行 CadQuery 的 VLM。

Details

Motivation: CAD 重建常假设可从任意角度观察物体，但真实物品可能贴墙、卡角或落地，场景会遮挡大量视角，剩余视角的信息量也不相同。

Result: SightCAD 在真实室内场景中嵌入标准 CAD 基准物体，按物理可见性限制离散视球中的视角。可学习策略选择 K 个可行视图供 VLM 生成可执行 CadQuery，并以执行实体的几何保真为奖励；由于奖励经过离散视角、文本生成和 CAD 内核执行，作者联合训练视角选择器与 VLM。策略优于随机、均匀、覆盖贪心和表面积最大化方法，在 K=1–5 时比 SA-max 最多高 6.4 IoU 点。DeepCAD/Fusion360 场景嵌入遮挡渲染上，完整系统较最佳外部基线有效 mIoU 分别高 21 和 17 点；还在 T-LESS 和 MP6D 工业零件实图上优于基线，可执行程序比例最高且无效代码率≤1.5%。

Insight: 应优化“从可观察角度获取信息”与“依据视图生成几何”这一整体链路，而不是假设视图可自由选择。实图基准包含领域规范化处理，需与合成遮挡设置区别看待。

Abstract: CAD reconstruction methods assume a luxury reality rarely grants: unrestricted visual access to the object, photographed from any desired angle. Real objects, however, are scene-embedded, bolted against walls, wedged into corners, resting on floors, where the scene renders much of the view sphere unreachable and the remaining views unequally informative. We introduce \textbf{SightCAD}, a framework for parametric CAD reconstruction that treats view feasibility as a first-class constraint. In this work we consider objects from standard CAD benchmarks embedded in realistic indoor scenes with physically derived visibility constraints over a discrete view sphere. A learned view selector must choose $K$ feasible views for a vision–language model (VLM) that generates executable CadQuery code, scored by geometric fidelity of the executed solid. Because reward arrives only after discrete view selection, autoregressive generation, and CAD-kernel execution, we propose a joint training paradigm in which the view selector and the CAD-generation VLM are trained together against this reward. The learned selection policy departs sharply from random, uniform, and coverage-greedy alternatives, outperforming surface-area maximization (SA-max) by up to $6.4$ Intersection-over-Union (IoU) points across budgets $K\in{1,\dots,5}$. The full system surpasses strong external baselines on scene-embedded, occluded multi-view renders of DeepCAD and Fusion360 objects ($+21$ and $+17$ effective-mIoU points over the best baseline, respectively), as well as on test-time domain-canonicalized real images from the industrial T-LESS benchmark and on both synthetic and real images from the MP6D industrial metal-parts benchmark, while producing the highest rate of executable programs of any method compared (invalid-code rate ${\leq}1.5%$).

[78] Right Screen, Wrong Transition: World Models as Verifiers for GUI Agents cs.CVPDF

Jiaming Zhang, Xuan Wang, Fuyao Zhang, Yang Cao, Lingjuan Lyu

TL;DR: 提出 LGWM，以表征空间预测 GUI 下一屏幕作为安全世界模型，并用向量差直接检测动作—状态转移是否异常及其潜在危害。

Details

Motivation: GUI 安全取决于动作后的转移，而非单看屏幕：登录页在点击登录后是正常的，在点击查看订单后则可能表示攻击。文本/图像世界模型需第二个模型比较预测与观察，增加时延和误判。

Result: LGWM 是 action-conditioned、无解码器世界模型，在 185 万真实 GUI 转移上无需语义标注训练，直接预测下一屏幕表示。验证可简化为向量比较，同一信号也用于判断不一致是否有害。RSWT-BENCH 让凭据屏幕出现在合法与劫持转移之后，仅看屏幕的检测器按设计接近随机。免训练评分达到 0.987 AUC，单次判定 17ms，与最强闭源 VLM 相当；比生成式 GUI 世界模型高约 10 个 AUC 点，延迟低三个数量级以上。残差方向对有害/良性违规的 AUC 为 0.953，而提示式 VLM 接近随机。

Insight: 直接在观测表征空间预测未来状态可同时做低时延转移验证和危害分类，世界模型除模拟/规划外还可承担安全验证。

Abstract: A login screen that appears after a tap on Sign in is expected; the same screen after a tap on View order is an attack. For GUI agents, safety is therefore a property of the transition rather than of the screen, and a monitor that inspects only screens can be defeated by reusing a legitimate one. Judging a transition requires an expectation of what should have followed the action. Existing GUI world models provide one, but they output it as text, code, or images, so checking it against the observed screen requires a second model to judge the two. We argue that a world model meant for verification should instead predict in the space in which observations are encoded, and present LGWM, a decoder-free, action-conditioned world model that predicts the representation of the next screen directly, trained without semantic annotation on 1.85M real GUI transitions. Verification reduces to a vector comparison, and the same signal reveals whether a mismatch is harmful. We evaluate on RSWT-BENCH, a diagnostic where each credential screen appears under both a legitimate and a hijacked transition, so detectors that see only the screen are at chance by construction. The training-free score reaches 0.987 AUC at 17 ms per decision, on par with the strongest closed-source VLMs and about ten AUC points above generative GUI world models at over three orders of magnitude lower latency. The residual direction reaches 0.953 AUC at separating harmful from benign violations, where prompted VLMs are near chance. Further analyses show that the prediction is a usable future state rather than an anomaly score. World models have mostly served as simulators or planners; our results point to a third role, verification, for which predicting in representation space is the natural design.

[79] Learning Which Correspondences to Trust: Confidence-Weighted Event-Camera Localization in LiDAR Maps cs.CV | cs.ROPDF

Panagiotis Kiousis, Kuangyi Chen, Jun Zhang, Friedrich Fraundorfer

TL;DR: 提出 CELL，以端到端位姿可微概率 PnP 学习逐对应置信度，并用于事件相机对 LiDAR 地图定位中的流监督、测试选择及边缘精化。

Details

Motivation: 事件相机在预建 LiDAR 地图中的定位可先估计深度渲染图与事件图之间的稠密光流，再用诱导的 3D–2D 对应求位姿。现有方法依赖几何共识，却未建模单条对应的可靠性及其对位姿约束力。

Result: 按逐对应误差监督置信度会有深度偏差：小像素误差多出现在远距离，未必提供强位姿约束。CELL 使用可微概率 PnP，通过 log-partition 项鼓励产生约束更好的位姿分布，端到端学习置信度。置信度用于解耦训练中的光流加权、测试时概率对应选择，以及结合边缘概率进行最终边缘匹配精化；另提出部分补全深度表示，在不跨越大缺口幻觉补值的前提下增加信号。在 M3ED、DSEC 上，大多数评估序列超过 LEAR 基线，平移中位误差最高降低 26.9%，旋转中位误差最高降低 15.8%。

Insight: 对应可靠性不能仅用像素误差衡量，还应直接评估它对位姿可辨识性的贡献；概率 PnP 可把这一目标纳入置信度学习。

Abstract: Localizing an event camera against a pre-built LiDAR map can be cast as dense optical-flow estimation between a rendered depth view and an event image, followed by a Perspective-n-Point (PnP) solver over the induced 3D-2D correspondences. Existing pipelines rely on geometric consensus during pose estimation, but do not explicitly model the reliability or pose informativeness, i.e., how strongly a correspondence constrains the camera pose, of individual correspondences. We show that the natural way to learn it – using the per-correspondence error to constrain the learning of confidence – suffers from a depth-dependent bias: small pixel errors reside predominantly at large depths and do not lead to high pose informativeness. Instead, in our method (CELL), we learn a per-correspondence confidence end-to-end through the pose, using a differentiable probabilistic PnP whose log-partition term encourages weight configurations that yield a better-constrained pose distribution. The learned confidence is used in three ways: (i) it reweights the flow supervision in a decoupled training scheme that keeps pose gradients out of the flow/edge backbone; (ii) it drives a probabilistic correspondence selection at test time; and (iii) together with the network’s edge-probability it weights a final edge-matching refinement. We further design a partial-completion depth representation that adds signal without hallucinating across large gaps. On M3ED and DSEC our full system improves over the LEAR baseline on the majority of the evaluated sequences: it reduces the median translation error by up to 26.9% and the median rotation error by up to 15.8%.

[80] FearCaut-Qwen: Affective Steering in a Vision-Language Model Shifts the Decision Criterion for Hazard Assessment cs.CVPDF

Xiaoshan Zhou

TL;DR: 研究 Qwen2.5-VL 灾后建筑危险评估中的保守判定标准，并通过机制解释定位恐惧相关表征，以激活 steering 调整红色危险预测倾向。

Details

Motivation: 灾后 VLM 可用于损坏评估，但可能不愿判定危险建筑，导致召回率低，即便总体准确率看起来尚可。作者用信号检测理论区分感知能力与决策标准，并检验是否能因果调节过度保守标准。

Result: SeisMLLM-1K 上的 Qwen2.5-VL-7B 两阶段系统只标出 27.0% 真正不安全建筑且从不误报红色，标准指标 c=+1.354，判别能力 d′=1.521。作者在情绪丰富自然场景中定位恐惧相关通路，以稀疏神经元敲除和分布式 steering 在留出情绪数据上做因果验证，再将方向注入建筑任务。恐惧方向使红色召回率升至 75.7%（p<0.001），减去同方向则完全抑制红色；范数匹配随机方向和快乐方向无显著效果。机制表现为标准转移到 c=-1.515，判别能力没有改善（d′=-0.493）。

Insight: 激活干预改变了愿意判红的阈值，而不是增强危险辨别能力；召回提升伴随决策标准大幅变化，不能解读为视觉感知更准。

Abstract: Vision-language models (VLMs) show great potential for damage assessment after a disaster, but a recurring deficiency is that they are reluctant to declare a hazard; that is, recall is low even when overall accuracy appears adequate. This study examines that deficiency by using signal detection theory to decompose the decision behavior into perceptual capability and decision-criterion placement. We then propose a novel method for correcting the over-conservative decision policy, inspired by the finding that fear makes humans risk-averse, and ask whether an affective representation associated with fear can be causally manipulated to similarly alter a VLM’s decision tendency. Using mechanistic interpretability, we localize a causally implicated affective circuit in the model and use activation steering to manipulate it while observing the effect on downstream prediction. The method is tested on a two-stage SeisMLLM pipeline built on Qwen2.5-VL-7B-Instruct, which flags only 27.0% of genuinely unsafe buildings on the SeisMLLM-1K test split and never issues a false Red, an SDT criterion of c = +1.354, despite adequate evidence quality (d’ = 1.521). An affective direction is localized on emotion-rich natural scenes, causally validated by sparse-neuron knockout and distributed steering on held-out emotion data, and then injected into the building task. Fear-direction injection raises Red recall to 75.7% (p<0.001), and subtracting the same direction suppresses Red predictions entirely, whereas norm-matched random and matched happiness directions show no significant effect. The mechanism is a shift in criterion (c=-1.515) while discrimination is not improved (d’=-0.493). These results show that VLM decisions can be adjusted at inference time without retraining and demonstrate how mechanistic interpretability can be used to diagnose and control VLM decision behaviors in engineering applications.

[81] DataVista: Diagnosing Multimodal LLMs on Data Video Understanding cs.CV | cs.AI | cs.CLPDF

Yupeng Xie, Zhenyang Wang, Jiayi Zhu, Yinghao Tang, Zhouan Shen

TL;DR: 提出 DataVista，评估多模态模型对动态数据视频中的图表读取、跨时序证据整合与叙事理解，并揭示当前模型在因果和叙事结构问题上的瓶颈。

Details

Motivation: 数据视频把数据可视化与视频叙事结合，需读取动画图表、跨图表和时间整合证据，并理解叙事安排和视觉设计传达的信息。现有基准分别针对普通视频或静态图表，尚未系统评估该任务。

Result: DataVista 含 961 个真实数据视频及 6,775 个评测问题，按数据感知、时间推理和叙事理解三层能力组织，覆盖五个主题领域和十种细粒度题型。对 19 个主流 MLLM 评测中，最佳 Gemini-3.1-Pro 总准确率 70.0%，仍低于人类专家；模型在因果推理和叙事结构上最弱。增加帧数和字幕主要帮助数据感知及时间推理，对叙事理解提升有限。响应分析还发现图表读取、证据判断和指令理解等常见错误。

Insight: 多帧和字幕改善低阶感知与时序整合，但不能自动解决高层叙事理解；基准将数据视频能力分层诊断。

Abstract: Data video is a media form that integrates data visualization with video narrative, widely adopted in news reporting and business analysis. Compared with general video understanding, data video understanding places greater emphasis on accurately reading data from animated charts, integrating evidence across charts and time, and understanding how narrative organization and visual design communicate information. Yet existing benchmarks target either general videos or static charts, and data video understanding has not been systematically evaluated. We present DataVista, the first benchmark for data video understanding, containing 961 real-world data videos and 6,775 evaluation questions organized under a three-level progressive capability framework (data perception, temporal reasoning, narrative understanding) with 10 fine-grained question types across five topic domains. Systematic evaluation of 19 mainstream MLLMs shows that the best-performing model, Gemini-3.1-Pro, achieves 70.0% overall accuracy, still far below human expert performance, with models performing worst on Causal Reasoning and Narrative Structure. Increasing frame counts and adding subtitles mainly benefit data perception and temporal reasoning, with limited gains in narrative understanding. Further analysis of model responses identifies typical failure modes in chart reading, evidence judgment, and instruction understanding. The benchmark is available at https://github.com/HKUSTDial/DataVista.

[82] Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning cs.CVPDF

Yicheng Xue, Han Wu, Jufeng Yang, Minjing Dong, Xinghao Chen

TL;DR: 提出 ViMoD，以紧凑粗粒度视觉上下文保存可恢复细节，并依据解码历史动态选择细粒度 token，支持多阶段推理按需回看证据。

Details

Motivation: 高分辨率图像会产生长视觉 token 序列，多步推理成本高。一次性剪枝或聚合在解码前固定压缩上下文，但不同推理阶段可能需要不同视觉证据，固定表示难以保留全部细节。

Result: DART 学习内容自适应区域分组与聚合容量，构造链接至可恢复原始细节 token 的紧凑粗表示；TRACE 读取解码历史预测后续证据需求，选择、保留或替换激活的细粒度 token。细节 token 补充至冻结骨干的持久粗上下文，避免持续关注全部视觉 token。Qwen3-VL-4B 在目标视觉 token 预算 20% 时，八个推理基准全部优于所测基线；平均归一化分数比最强一次性压缩基线高 39.0%。相对冻结骨干新增可训练参数仅 0.0546%。

Insight: 把视觉压缩从固定预处理改为推理过程中按需回看，使证据选择跟随问题推理阶段变化，同时通过粗上下文维持全局信息。

Abstract: Processing long visual token sequences from high-resolution images makes multi-step reasoning computationally expensive for multimodal Large Language Models (MLLMs). Existing one-shot pruning and aggregation methods compress visual tokens into a fixed context before decoding. However, visual evidence needs can shift as reasoning unfolds, making it difficult for a fixed compressed context to retain all the details needed across stages. To address this challenge, we propose ViMoD, a lightweight framework that maintains a compact visual context while preserving access to original fine-grained evidence as reasoning needs evolve. Deformable Aggregation of Region-wise Tokens (DART) learns content-adaptive groups and aggregation capacities, constructing compact Coarse representations linked to recoverable original Fine tokens. Temporal Routing for Adaptive Contextual Evidence (TRACE) integrates decoding history to anticipate upcoming evidence needs and select, retain, or replace active Fine-token groups. Selected Fine tokens augment the persistent Coarse context in the frozen backbone, enabling stage-specific evidence access without continuously attending to all visual tokens. On Qwen3-VL-4B, ViMoD outperforms all evaluated baselines on all eight reasoning benchmarks at a 20% target visual token budget, improving the mean normalized score by 39.0% over the strongest evaluated one-shot baseline. These gains are achieved with only 0.0546% additional trainable parameters relative to the frozen backbone.

[83] Perception Test 2026: Challenge Summary and Extension to City-scale Audio-Visual Reasoning cs.CV | cs.LGPDF

Fedor Kitashov, João Carreira, Shiry Ginosar, Dima Damen, Andrew Zisserman

TL;DR: 总结 2026 年 Perception Test 挑战赛及其城市尺度视听推理扩展，介绍新步行导览视频基准和各赛道获胜方案。

Details

Motivation: 第四届挑战赛关注空间智能，将原 Perception Test 的统一多选与定位视频问答扩展到城市尺度步行视频，以研究长距离环境中的空间和多模态推理。

Result: ECCV 2026 马尔默研讨会设置四个赛道：原 Perception Test 的统一多选视频问答和 grounding 视频问答，以及新建城市尺度 KilometerAudio、KilometerVision。报告描述新赛道基准并总结所有赛道获胜方案，包括一个在四赛道参赛且表现合格的通用模型。城市赛道获胜系统显示，复杂空间和多模态推理可由高成本代理流程解决，但单独使用多模态模型仍困难。

Insight: 结果提示城市尺度空间推理可通过复杂代理系统实现，但独立 MLLM 尚不能稳定完成；该文是挑战赛总结与基准扩展报告。

Abstract: Continuing the Perception Test challenge series, we organised the fourth edition as a workshop at the European Conference on Computer Vision (ECCV) 2026 in Malmö, Sweden. This edition focused on spatial intelligence and featured four different tracks: unified multiple-choice videoQA and grounded videoQA from the original Perception Test benchmark, alongside two new tracks based on city-scale walking-tour videos (KilometerAudio and KilometerVision). In this report, we describe the new benchmarks used for the city-scale tracks and summarise the winning solutions across all tracks, including a generalist model that competed across all tracks with satisfactory performance. The winning solutions in the newly added city-scale tracks demonstrated that complex spatial and multimodal reasoning can be solved by expensive agentic pipelines, but remains difficult for multimodal models used standalone.

[84] DVLA-RL++: Dual-Level Vision-Language Alignment with Reinforcement Learning Gating for Few-Shot Learning cs.CV | cs.LGPDF

Wenhao Li, Xianjing Meng, Qiangchang Wang, Zhongyi Han, Yilong Yin

TL;DR: 提出 DVLA-RL++，通过语义净化和反事实强化学习门控，在少样本分类中区分对象内在属性与偶然背景线索，并按层学习融合强度。

Details

Motivation: 少样本分类可借文本语义弥补视觉样本不足，但图文相似性可能同时反映物体内在属性和偶然上下文，使支持集原型被背景污染。

Result: DVLA-RL++ 使用 CSP 为标注支持图生成内在属性和干扰属性描述，并比较两者与每个支持 token 的一致性；基于歧义的拒绝边际稀疏分配证据，剩余质量由内在语义锚点补足。CRG 用奖励平衡识别性能与干扰暴露，学习逐层语义融合强度；同一 episode 上独立执行的参考轨迹提供配对学习信号。理论分析关联保留证据与锚点质量和原型稳定性，并给出无偏在线策略梯度条件。标准、细粒度和跨域基准上达到 SOTA，平均较 DVLA-RL 提高 1.4%。

Insight: 少样本原型应抑制偶然背景证据，语义锚点在视觉证据不可靠时提供回退；提升依赖有歧义感知的门控而非无差别图文融合。

Abstract: Few-shot learning aims to recognize novel categories from limited labeled examples. Recent studies incorporate textual semantics to compensate for limited visual observations and improve class representations. However, high image-text agreement may reflect both intrinsic object properties and incidental context, making support prototypes susceptible to contextual contamination. To address this problem, we propose DVLA-RL++, which extends DVLA-RL with complementary semantic purification (CSP) and counterfactual reinforcement-learning gating (CRG). Specifically, CSP generates intrinsic and nuisance descriptions from labeled supports and compares their agreement with each support token. An ambiguity-dependent rejection margin guides sparse evidence allocation, while an intrinsic semantic anchor fills the unassigned mass to provide a fallback when visual evidence is unreliable. CRG learns layer-wise semantic fusion strengths using a reward that balances recognition performance and nuisance exposure. An independently executed reference trajectory on the same episode provides a paired learning signal. Theoretical analysis relates retained evidence and anchor quality to prototype stability and establishes conditions for unbiased on-policy gradient estimation. Experiments on standard, fine-grained, and cross-domain benchmarks show state-of-the-art accuracy, with an average gain of 1.4% over DVLA-RL. The project page is available at https://peacelwh.github.io/TPAMI27-DVLA-RLpp/.

[85] VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling cs.CV | cs.AI | cs.MMPDF

Leigang Qu, Feng Cheng, Ziyan Yang, Bangbang Yang, Zhaoyang Huang

TL;DR: 提出 VINCIE-NExT，以图像域的视觉示例迁移编辑能力到视频：沿视频—图像—图像—视频子任务链生成编辑，并用共享空间位置编码传播外观改动。

Details

Motivation: 强视频编辑器难训练，因为需要昂贵的源视频—指令—编辑结果三元组；相比之下，图像编辑已有大量成对数据，作者探索通过上下文示例复用图像编辑能力。

Result: 框架用图像作为编辑意图中介，将视频编辑分解为可组合的 Video→Image→Image→Video 链，可用模型生成或用户给定的图像编辑对作为前缀视觉示例，充当每帧共享的空间外观蓝图。新位置编码将示例图和视频帧映射到共同空间坐标，以像素级传播外观改动。Chain-of-Editing 将链条作为逐阶段扩散执行，测试时可增加计算提升质量而无需重训。在 OpenVE-Bench 多种编辑类别上达到 SOTA；消融支持各组件作用。

Insight: 通过图像域演示复用大规模图像编辑数据，可减少对成对视频编辑数据的需求；测试时计算扩展沿编辑链渐进执行。

Abstract: Building a capable video editor remains significantly harder than a video generator: editing requires (source, instruction, edited) triplets that are prohibitively expensive to annotate and difficult to synthesize at scale, whereas image editing has already reached maturity with millions of such pairs readily available. In this work, we introduce VINCIE-NExT, a unified framework that transfers editing capability from images to videos through in-context visual demonstrations, alleviating the need for large-scale paired video editing data. VINCIE-NExT decomposes video editing into a structured chain of composable sub-tasks (Video -> Image -> Image -> Video), routing editing intent through the image domain and enabling scalable joint training from heterogeneous image and video corpora under a unified diffusion objective. An image editing pair, synthesized by the model or supplied by the user, is prepended as an in-context visual demonstration that serves as a spatial appearance blueprint for every output frame. To ground appearance edits across the interleaved context, we introduce a novel position encoding that links image demonstrations and video frames in a shared spatial coordinate system, enabling pixel-faithful propagation of appearance changes to every output frame. Chain-of-Editing further provides principled test-time scaling: by executing the sub-task chain as progressive diffusion stages, editing quality can be improved by investing additional compute without retraining. Comprehensive experiments on OpenVE-Bench demonstrate the state-of-the-art performance across diverse editing categories, with ablations confirming the effectiveness of each component.

[86] ContourVLA: A Closed-Loop Perception-Action Contour Policy for Generalized Referring Expression Segmentation cs.CVPDF

Ruicheng Zhang, Kaiwen Shen, Jiaqi Hou, Shuhan Yang, Junchao Huang

TL;DR: 提出 ContourVLA，将指代表达分割建模为闭环视觉动作策略，以可编辑轮廓为状态，通过几何动作分块迭代修正边界与语义 grounding。

Details

Motivation: 广义指代表达分割需同时识别语言指定的可变数量目标并精确描绘边界；传统级联 VLM 使用固定特征接口和单次掩码预测，难以自适应获取证据或校正几何。

Result: EASS 将轮廓引导的双向边界采样与状态条件多层多模态特征路由结合，按轮廓状态调整感知。监督初始化后，DECT-GRPO 联合优化离散 grounding 和连续轮廓动作，利用考虑误检与漏检的软预测—目标对应产生实例级奖励和信用。在 gRefCOCO val/testA/testB 上，gIoU 比最强所测基线分别高 8.7、2.8、2.7 分；RefCOCO、RefCOCO+、RefCOCOg 八个划分上 mIoU 均最高。

Insight: 将分割边界作为策略状态并反复观察/动作校正，使语义定位与边界几何共同优化；实例级信用同时考虑误检和漏检。

Abstract: Generalized referring expression segmentation (GRES) requires dynamically balancing high-level semantics for identifying a variable number of language-specified referents with fine-grained visual evidence for precise boundary delineation. This requirement challenges existing cascaded vision-language architectures, which typically rely on static feature interfaces and single-pass mask prediction, limiting adaptive perception and geometric correction. We introduce ContourVLA, a vision-language-action policy that recasts GRES as a closed-loop visuomotor process, in which editable contours serve as explicit policy states that condition multimodal perception and are updated by geometric action chunks. Evolution-Aware Semantic Scheduling (EASS) couples contour-guided bidirectional boundary sampling with state-conditioned routing of multilevel multimodal features, adapting perception to each contour state. Following supervised initialization, Dustbin-Augmented Entropic Credit Transport GRPO (DECT-GRPO) jointly optimizes discrete grounding and continuous contour actions with instance-level credits. Its rollout rewards and credits are derived from soft prediction-target correspondences that account for false positives and missed targets. ContourVLA improves gIoU over the strongest evaluated baselines by 8.7, 2.8, and 2.7 points on gRefCOCO val, testA, and testB, respectively, and achieves the highest mIoU across all eight RefCOCO, RefCOCO+, and RefCOCOg splits.

[87] Healthy Counterfactual Generation via Diffusion Inpainting for Mammography Classification cs.CVPDF

Inês Cruchinho Garcia, Mariana Mourão, Francisco Maria Calisto, Carlos Santiago, Jacinto Nascimento

TL;DR: 提出健康反事实乳腺摄影增强：在健康 BI-RADS 1 影像上训练扩散模型，用 RePaint 在病灶框内补绘正常组织并保留患者个体解剖结构。

Details

Motivation: 乳腺癌筛查 CAD 的假阴性会延误发现和治疗；以异常影像去除病灶生成健康反事实样本，可扩展分类器的正常分布训练数据。

Result: 作者训练去噪扩散概率模型学习健康乳腺影像，再用 RePaint 在标注病灶框内生成逼真的正常组织，替换病灶区域并保留患者解剖结构。相似度指标支持生成图像与真实图像的个体结构相似；放射科医生评估其逼真度与原数据集质量一致。VinDr-Mammo 上在 ConvNeXt、ViT、Mammo-CLIP 和 FPN-MIL 四种分类器中测试反事实增强，所有架构敏感度均提升，固定特异度 80% 时尤为明显；摘要未列具体提升幅度。

Insight: 基于患者自身影像的局部反事实补绘可保留个体解剖，而不只是合成通用健康图；人类阅片评价用于补充图像相似度证据。

Abstract: False negatives remain a critical limitation of computer-aided diagnosis (CAD) systems for breast cancer screening due to delayed detection and treatment. To address this issue, we propose a counterfactual data augmentation strategy that generates healthy mammograms by “erasing” lesions from anomalous images, thereby enriching the training distribution. We train a Denoising Diffusion Probabilistic Model on BI-RADS 1 (healthy) mammograms and use a RePaint-based sampling strategy to inpaint realistic normal tissue within annotated lesion bounding boxes. The resulting healthy counterfactuals replace annotated lesion regions with realistic healthy tissue while preserving patient-specific anatomical structure, as supported by similarity metrics between real and generated images. Image realism was further assessed by radiologists and found to be consistent with the original dataset quality. We evaluate counterfactual augmentation across four representative classifier architectures: a convolutional neural network (ConvNeXt), a vision transformer (ViT), a vision-language model pre-trained on mammogram-report pairs (Mammo-CLIP) and a multi-scale attention-based multiple-instance learning framework (FPN-MIL). Experiments conducted on the VinDr-Mammo dataset show improvements in sensitivity across all architectures, particularly at 80% fixed specificity, contributing towards more reliable CAD systems for breast cancer. Code is available at: https://github.com/ines03garcia/diffusion-based-counterfactual-generation.

[88] Connected Self Forcing: Beyond Local Learning in Video Autoregression cs.CVPDF

Dongbin Zhang, Chaoda Zheng, Kangjie Chen, Xiangyu Li, Shijia Chen

TL;DR: 提出 Connected Self Forcing，在自回归分块视频训练中重新连接跨 chunk 梯度，让后续预测对历史生成过程提供学习反馈。

Details

Motivation: Self Forcing 通过自身生成历史缓解曝光偏差，适合流式长视频训练；为节省显存通常 detach 历史 KV cache，虽保留前向依赖，却切断后续 chunk 对早期上下文生成的反向梯度。

Result: Connected Self Forcing 让梯度穿过生成潜变量回到早期上下文生成计算，不只连接历史 KV 写入；为减少显存，用 shortcut gradient replay 恢复跨 chunk 梯度而不保留完整 rollout 计算图。与分布匹配蒸馏结合后，历史 chunk 同时依据直接监督和其对后续视频生成的贡献进行训练。自回归视频生成实验显示，长时视觉质量和时间一致性提升，推理流程保持不变；摘要未列具体数值。

Insight: 跨 chunk 信用分配使早期历史表征也能根据其对远期结果的影响更新，可在不保存完整自回归图的情况下训练。

Abstract: To stream long videos while maintaining visual quality and temporal consistency, Self Forcing mitigates exposure bias through self-rollout training on self-generated histories with key-value (KV) caching. To keep memory manageable, it detaches historical caches, preserving forward dependencies between chunks but severing the backward gradient paths. We introduce Connected Self Forcing, a training framework that reconnects gradient paths across autoregressive chunks, allowing feedback from later predictions to guide how earlier context is generated. These connections go beyond historical KV-writing: gradients pass through generated latents into the computations that produced them, linking the generation of earlier context to its use in later predictions. To make this connected training memory-efficient, we develop shortcut gradient replay, which recovers cross-chunk gradients without retaining the full rollout computation graph. Integrated with distribution matching distillation, Connected Self Forcing trains historical chunks according to both their direct supervision and their contribution to subsequent generation. Experiments on autoregressive video generation show improvements in long-horizon visual quality and temporal consistency, without changing the inference procedure.

[89] VibeEdit: Image Editing with Canvas Instructions cs.CV | cs.AIPDF

Jinjing Zhao, Fangyun Wei, Yitong Wang, Xiuyu Wu, Yunuo Chen

TL;DR: 提出 VibeEdit 图像编辑界面，允许用户在画布上标注目标位置和编辑内容，并通过区域加权微调及强化学习完成局部编辑和背景保真。

Details

Motivation: 文本描述编辑意图通常容易，但指出具体对象或区域较麻烦，尤其是多个外观相似物体时。

Result: VibeEdit 用空间标记和可选短备注构成 canvas instruction，支持添加、删除、替换、属性修改和移动对象，无需另写文本提示。训练数据含 155 万组源图—目标图编辑对、对象掩码和结构化描述，训练时渲染画布指令。模型以 layer-decoupled conditioning 分别编码原图和指令，先做区域加权 SFT，再以 rubric 指导 RL 改善完成度、局部质量和未编辑区域保护。在 419 个独立人工整理、强调相似对象选择的基准案例上，VibeEdit VLM 量表分 79.9、画外 PSNR 32.8 dB；评测中最高分文本指令基线 FireRed 分别为 67.4 和 24.0 dB。

Insight: 画布标注将空间定位从文字表达中分离，可直接指出要改哪个对象，并用画外 PSNR量化未编辑区域保真。

Abstract: In text-guided image editing, describing the desired change is often straightforward, but identifying the intended object or region can be cumbersome, especially when several objects look alike. We introduce a new image editing interface that lets users place spatial marks and optional short notes directly on the image. Together, these annotations form a canvas instruction that specifies where to edit and what to change. Our editor, VibeEdit, follows these instructions to perform object addition, removal, replacement, attribute modification, and movement without a separate text prompt. We construct 1.55 million source-target edit pairs with object masks and structured edit descriptions, from which we render canvas instructions during training. We adapt Qwen-Image-Edit with layer-decoupled conditioning that separately encodes source images and canvas instructions for image editing. We train the model with region-weighted supervised fine-tuning, followed by rubric-guided reinforcement learning to improve edit completion, local edit quality, and preservation of unedited regions. We evaluate VibeEdit on an independently constructed, human-curated benchmark of 419 cases emphasizing target selection among similar objects. VibeEdit achieves a VLM rubric score of 79.9 and an outside-region PSNR of 32.8 dB, compared with 67.4 and 24.0 dB for FireRed, the highest-scoring text-instructed baseline in our evaluation.

[90] From Prompting to Composing: A Spatial Canvas Interface for Poster Generation cs.CV | cs.AIPDF

Yitong Wang, Fangyun Wei, Jinjing Zhao, Sirui Zhang, Hongyang Zhang

TL;DR: 提出 Spatial Canvas Interface 和 Compo 海报生成模型，让用户通过语义、身份、文字、像素四类空间绑定直接布局生成意图，而非仅用文本描述二维构图。

Details

Motivation: 海报构图本质上是二维空间关系，但文本提示要求用户把布局意图线性化成词语，控制对象位置和属性不直观。

Result: 空间画布支持四类互补绑定及单元素和全局外观的文本规格。Compo 基于预训练图像编辑模型适配，可直接使用用户画布，也可在代理模式下将高级请求转成计划画布。自动数据流水线为各绑定及其组合生成监督，使模型无需从零训练专用海报生成器。作者另构建评测基准，分别测量各绑定及联合构图遵循。实验显示，Compo 相较通用图像生成模型和专门海报系统有更强构图可控性，同时保持较高视觉质量；摘要没有给具体分数。

Insight: 将意图规格与视觉合成分离，让用户直接在空间中表达布局，并可由代理将高层要求转译为画布。

Abstract: Text prompting is an indirect interface for poster generation, requiring users to encode inherently two-dimensional composition intent into a one-dimensional sequence of words. We introduce a Spatial Canvas Interface that enables users to directly compose generation intent in space through four complementary binding types: semantic, identity, text, and pixel, together with Text Specifications for individual elements and global appearance. Based on this interface, we develop Compo, a poster generation model adapted from a pretrained image editing model to understand Spatial Canvas inputs and Text Specifications. Compo supports both direct inference, where users explicitly construct the canvas, and agentic mode, where a high-level request is automatically translated into a planned Spatial Canvas. To train Compo, we develop a scalable pipeline that automatically constructs supervision data for different binding types and their combinations, enabling efficient adaptation without training a specialized poster generator from scratch. We further introduce a benchmark that evaluates adherence to individual binding types and their joint composition. Experiments show that Compo achieves stronger compositional controllability than both general-purpose image generation models and dedicated poster generation systems while maintaining high visual quality. By decoupling intent specification from visual generation, our work shifts poster generation from prompting toward composing.

[91] Syn-Omni: Structured Specialization and Progressive Collaboration for Omnimodal Embeddings cs.CV | cs.AI | cs.IRPDF

Youngtaek Oh, Qiyu Wu, Hiromi Wakaki, Junmo Kim, Yuki Mitsufuji

TL;DR: 提出 Syn-Omni，以共享 LoRA 表达跨模态通用语义、模态专家 LoRA 学习专属信息，并通过渐进路由逐步建立跨模态协作的统一嵌入框架。

Details

Motivation: 全模态嵌入既需共享表示，也需保留不同模态特有特征；单一共享参数空间处理混合模态会限制通用与专属信息的结构分离。

Result: OME-LoRA 将适配分解为共享语义 LoRA 路径和按模态专门化的 LoRA 专家路径。PSR 先让专家建立模态特有先验，再逐步与其他专家交互以产生跨模态协同。在覆盖图像、视频、音频和视听的 81 项任务上，Syn-Omni 持续优于全模态嵌入基线。

Insight: 先专门化、后协作可降低过早混合带来的模态信息干扰；摘要未提供 81 项任务的具体分数或计算成本。

Abstract: Omnimodal embeddings naturally involve both shared representations and modality-specific features across heterogeneous inputs. However, existing omnimodal embedding methods often rely on a single shared parameter space over mixed-modality data, limiting structural separation between universal and modality-specific representations. To address this, we propose Syn-Omni, a unified framework for structured omnimodal adaptation with modality specialization and controlled cross-modal collaboration. Specifically, we introduce Orthogonal Modality-Expert LoRA (OME-LoRA), which decomposes adaptation into a shared LoRA path for universal semantics and modality-expert LoRA paths for modality-aware specialization. Furthermore, Progressive Synergy Routing (PSR) enables experts to first establish modality-specific priors, then gradually interact with other modality-experts for cross-modal synergy. Evaluated across 81 diverse tasks spanning image, video, audio, and audiovisual modalities, Syn-Omni consistently outperforms omnimodal baselines, demonstrating the effectiveness of structured specialization and cross-modal progressive collaboration.

[92] SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models cs.CV | cs.CLPDF

Hongxing Li, Jinyue Su, Dingming Li, Wenqi Zhang, Weiming Lu

TL;DR: 提出 SpaceCast-Bench，评估 VLM 从场景观测、空间干预到未见结果推断的预测性空间推理，并发现桥接视图和显式 3D 证据很重要。

Details

Motivation: 既有空间基准主要考察读取输入中已可见的关系，而真实空间智能还需构建场景、预测干预后变化并推理尚未观察的结果。

Result: 基准以 observe-transform-infer 为框架，含 182 个真实场景的 3,862 个问题、16 类任务，分静态感知、局部预测和全局预测三级，逐步要求场景理解、状态更新和未观测结果关系推理。21 个模型评估中，最强模型准确率仅 58.0%，人类为 87.2%；空间专用模型接近随机水平。控制分析显示桥接视图对整合分散观察关键，显式 3D 证据比生成结果图像/视频更可靠地帮助模型。在程序化生成数据上微调后，Qwen3-VL-4B 从 34.0% 升至 65.7%，且六个分布外基准宏平均均有提升。

Insight: 预测性空间推理需要把多视图证据连接起来并更新场景状态，单纯增加生成结果视觉材料并不如可靠 3D 证据稳定；当前模型仍显著落后人类。

Abstract: Existing spatial reasoning benchmarks mainly test spatial perception: reading off relations already visible in the input. Yet real-world spatial intelligence demands predictive spatial reasoning: constructing a scene from observations, anticipating how an intervention changes it, and reasoning about the unseen outcome. We introduce SpaceCast-Bench, the first benchmark to directly and diagnostically evaluate this capability. Built around an observe-transform-infer framework, its 3,862 questions from 182 real-world scenes span 16 task types at three levels: static perception, local prediction, and global prediction, progressively requiring scene understanding, spatial state updating, and relational inference over unobserved outcomes. Evaluating 21 models exposes a stark gap: the strongest model reaches only 58.0% against 87.2% human performance, while spatially specialized models remain near random chance. Controlled analyses further reveal that bridge views are critical for integrating distributed observations, and that explicit 3D evidence benefits models more reliably than generated outcome images or videos. Fine-tuning on our programmatically generated data lifts Qwen3-VL-4B from 34.0% to 65.7% with macro-average gains across six out-of-domain benchmarks.

[93] ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills cs.CV | cs.CLPDF

Hongxing Li, Dingming Li, Yixin Li, Yong Du, Wenqi Zhang

TL;DR: 提出 ViSkill，以组合式视觉技能卡保存成功交互，并让检索技能同时辅助策略推理和奖励塑形，形成技能库与 VLM 策略互相增强的闭环。

Details

Motivation: 技能增强代理可复用经验、提高样本效率，但多数方法以文本线性化空间布局和动作—状态对应，丢失关键几何结构；已有视觉技能方法又将技能库更新与策略优化分开。

Result: ViSkill 将成功交互编码为 VLM 可直接访问的视觉技能卡，检索结果用于推理与奖励塑形，新的成功轨迹再蒸馏回库中；可选冷启动机制加速早期学习。在 Sokoban、FrozenLake、PrimitiveSkill 上，整体成功率 0.89，冷启动后 0.91，超过所评闭源和开源基线，且收敛快于标准 PPO。

Insight: 技能的视觉结构可保留空间和动作状态关系，闭环回写让技能积累与策略改进互相强化；冷启动额外提高早期效率。

Abstract: Skill-augmented agents improve sample efficiency by distilling successful trajectories into reusable strategies. Yet most existing approaches remain text-centric, linearizing spatial layouts and action-state correspondences into language that loses critical geometric structure. Recent efforts have begun incorporating visual evidence, but construct and update skills separately from policy optimization, leaving their mutual improvement underexplored. We propose ViSkill, a visual-native skill learning framework that encodes successful interactions as composite visual skill cards directly accessible to VLM agents. Retrieved skills guide both inference and reward shaping, while successful trajectories are distilled back into the library, forming a closed feedback loop in which skill accumulation and policy improvement reinforce each other. An optional cold-start mechanism further accelerates early-stage learning. Evaluated on Sokoban, FrozenLake, and PrimitiveSkill, ViSkill achieves an overall success rate of 0.89, rising to 0.91 with cold-start initialization, outperforming all evaluated proprietary and open-source baselines while converging faster than standard PPO. Our code is available at https://github.com/ZJU-REAL/ViSkill.

[94] DVD: Dynamic Vector Decoding for Efficient MLLM-based Perception cs.CV | cs.AIPDF

Jinghua Hou, Zhe Liu, Hengshuang Zhao

TL;DR: 提出动态向量解码 DVD，把 2D/3D 检测表示转换为紧凑离散 token，并经轻量反 token 化器还原框、掩码或 3D 框。

Details

Motivation: MLLM 感知常用文本坐标编码，token 开销大；固定范围量化则受范围和精度限制，尤其难应对空间范围无界且定位精度要求高的 3D 任务。

Result: DVD 将 2D 边界框、2D 掩码和 3D 边界框等多种输出表示统一转为一维向量序列，再映射为高维空间中的紧凑离散 token；轻量 de-tokenizer 可把生成 token 无缝还原为原始 2D/3D 感知格式。在 RefCOCO 系列、SUN-RGBD、KITTI、Hypersim、nuScenes 等 2D/3D 基准上，作者报告表现优于基线，同时显著减少 token 开销和推理延迟；摘要未给具体数值。

Insight: 向量序列编码避免大量逐坐标文本 token，同时不受固定量化范围约束，为 MLLM 统一输出不同维度的几何感知结果提供接口。

Abstract: Multimodal large language models have made remarkable progress in bridging vision and language, facilitating various perception tasks essential for human-machine interaction, robotics, and autonomous driving. However, existing MLLM-based perception methods predominantly rely on text-based coordinate representation, which suffers from excessive token overhead, or fixed-range quantization, which suffers from range and precision constraints, especially for 3D domains with unbounded spatial range and high localization accuracy requirements. To address these challenges, we propose a dynamic vector decoding method named DVD, which unifies the representation of 2D and 3D perception tasks. Specifically, we first transform diverse perceptual representation (i.e., 2D bounding boxes, 2D masks, and 3D bounding boxes) into 1D vector sequences, which are then mapped to compact discrete tokens in the high-dimensional space. Then, a lightweight de-tokenizer enables seamless integration with MLLMs by decoding output tokens back to original 2D and 3D perceptual representations. Extensive experiments on 2D and 3D perception benchmarks including RefCOCO series, SUN-RGBD, KITTI, Hypersim, nuScenes demonstrate that DVD achieves superior performance in 2D and 3D tasks and reduces significantly the token overhead and inference latency. DVD provides an efficient and general framework for integrating perception capabilities into MLLMs, overcoming the inherent limitations of existing methods.

[95] Slot3R: Set-Associative Spatial Memory for Streaming 3D Reconstruction cs.CVPDF

Xiyuan Zhang, Yanming Yang, Kaiyuan Xu, Ruibo Li, Chi Zhang

TL;DR: 提出 Slot3R，一个免训练 Point3R 改造方案，让同一空间地址容纳多个状态记忆槽，避免把邻近但不同表面的证据过早平均融合。

Details

Motivation: 流式 3D 重建需在线保存逐帧证据，空间记忆可按重建位置组织历史；但 Point3R 同时用空间邻近决定寻址和是否融合，可能把不同表面、视角或可见性条件的 patch 指针错误合并。

Result: Slot3R 认为位置只应决定地址，而不意味着观测状态必须融合；作为免训练、集合相联改造，可在冻结 Point3R 主干时让多个状态共存于同一地址。稀疏有界读出进一步分离持久存储与逐帧解码器访问。在采样 300–500 帧时，7Scenes 点云准确性误差 Acc 降低 57.1%–63.1%，NeuralRGBD 降低 64.0%–72.0%，三个位姿基准的 Sim(3) 对齐绝对轨迹误差均降低，视频深度估计仍有竞争力。采样 600–1,000 帧时均能以约 19 FPS 完成，而 Point3R 和 InfiniteVGGT 从 800 帧起耗尽内存。

Insight: 同一位置可能包含需等待未来帧消歧的互补状态，多槽保留证据可避免早期平均破坏结构；优势在长序列显存限制场景尤明显。

Abstract: Streaming 3D reconstruction must preserve evidence from each frame while processing an expanding scene online. Spatial memory is a natural fit because it organizes history by reconstructed 3D location. Yet Point3R uses spatial proximity both to associate a new observation with an existing memory entry and to decide whether to fuse it, conflating co-location with state identity. Because pointers summarize image patches, nearby pointers may encode distinct surfaces, viewpoints, or visibility conditions; averaging them can destroy complementary evidence before later frames disambiguate it. We argue that location should determine address, not whether observations must merge. Slot3R realizes this principle as a training-free, set-associative retrofit that lets multiple states coexist at a shared address while keeping the pretrained Point3R backbone frozen. A bounded sparse readout further decouples persistent storage from per-frame decoder access. At 300-500 sampled frames, Slot3R reduces Point3R’s point-cloud accuracy error (Acc) by 57.1%-63.1% on 7Scenes and 64.0%-72.0% on NeuralRGBD, lowers Sim(3)-aligned absolute trajectory error (ATE) on all three pose benchmarks, and remains competitive on video-depth estimation. It completes all evaluated settings from 600 to 1000 sampled frames at about 19 FPS under the same protocol, whereas Point3R and InfiniteVGGT run out of memory at 800 frames and beyond.

[96] WOVEN: Weaving Visual World Modeling into Multimodal LLMs cs.CV | cs.CL | cs.LGPDF

Zheyu Fan, Yue Zhang, Mingkai Deng, Kangrui Wang, Qineng Wang

TL;DR: 提出 WOVEN，将视觉状态转移推理作为 MLLM 空间、具身、物理和时间推理的共享训练原语，并提供按场景—动作—推理类型组织的数据与基准。

Details

Motivation: 多模态模型在空间和物理推理上的多类失败可能源于共同的视觉转移推理不足；既有基准将这些能力分别评测，难以控制场景、动作和推理操作进行横向比较。

Result: WOVEN 含 36,076 个由视频预训练生成模型产生的多样真实感 rollout，覆盖 20 类场景、5 类动作和 8 类推理。对 38 个前沿 MLLM 的初测发现系统性能力缺口，最强模型仍明显落后人类，且跨模型家族、规模持续存在。多尺度训练显示可学得可迁移的共享能力：若干各约 2,000 样本的训练子集联合使 26 个外部基准中的 22 个提升，最高达 27.3 个百分点；WOVEN 数据可替代任务自身 30%–50% 训练数据而保持相近准确率。控制实验及留出基准前瞻验证建议按教授的推理操作而非所示动作/场景/领域选择监督，并偏好视觉状态变化较大的样本。

Insight: 视觉世界模型训练可从共享转移推理能力入手；数据选择应关注学到何种推理操作，而非仅匹配目标任务表面内容。

Abstract: Multimodal large language models (MLLMs) struggle with spatial, embodied, physical, and temporal reasoning. We hypothesize that these failures reflect a shared deficit in visual transition reasoning, and test whether this capability can serve as a shared training primitive, one that different models can learn from different supervision sources and reuse across different tasks, with a systematic training recipe. Existing benchmarks document these deficits separately but do not support controlled comparisons across scenes, actions, and reasoning operations. We therefore introduce WOVEN, a training source and benchmark for visual transition reasoning that organizes transition supervision by scene, action, and reasoning type, using diverse, realistic rollouts from video-pretrained generative models: 36,076 examples across 20 scene types, 5 action types, and 8 reasoning types. We first evaluate 38 frontier MLLMs (e.g., GPT-5.4 and Qwen3-VL-235B-A22B) and find a substantial and systematic deficit: even the strongest models fall far below humans, and the failures recur across model families and persist with scale. We then train MLLMs at multiple scales on WOVEN and find that they learn a shared capability that transfers broadly: training subsets of only about 2,000 items each collectively improve 22 of 26 external benchmarks by up to 27.3 percentage points, and WOVEN data can replace 30-50% of a task’s own training data with comparable accuracy. Controlled comparisons further yield a training recipe for visual world modeling, validated prospectively on held-out benchmarks: select supervision by the reasoning operation it teaches rather than by the actions, scenes, or domains it shows, and prefer larger changes to the visual state for robustness. Our work establishes visual transition reasoning as a reusable foundation for systematic visual world-model training in MLLMs.

[97] FastBench: Can Streaming VLMs Perceive High-Dynamic Real-World Streams? cs.CV | cs.CLPDF

Yuxuan Hu, Weikang Shi, Yang Bo, Xudong Lu, Xintong Guo

TL;DR: 提出 FastBench 高动态真实流式视频基准，并提出免训练 ProactiveFrame 动态调节帧率，研究上下文预算下的快速事件感知。

Details

Motivation: 流式视频 VLM 基准多是低动态场景；在有界上下文中，模型需平衡历史长度、空间分辨率和时间采样粒度，1–2 FPS 稀疏采样会错过快速事件。

Result: FastBench 以高帧率片段构造 QA，筛除 2 FPS 即可回答的问题，用 SAM3/CoTracker3 轨迹验证答案并经三轮人工检查；含 8 领域、6 种能力和前向/即时/回溯三种时间范围的 306 对 QA，并标注证据区间。ProactiveFrame 用文本 token 控制输入帧率，双层滑窗保留近期高帧率观察、将旧信息下采样。最强 Gemini-3.5-Flash 仅得 50.7%。Qwen3-VL-8B 从 2 FPS 的 32.9% 提至 24 FPS 的 44.6%，但历史压缩后增益趋于饱和。ProactiveFrame 比均匀稀疏采样高 5.4 和 1.5 个百分点，仍远低于 oracle 聚焦；当前 VLM 尚难仅凭流本身判断何时需要更细时间感知。

Insight: 高动态理解需要按需提高帧率，但模型自己尚难学会在何时加密采样；增加帧数有收益却受上下文压缩限制。

Abstract: Streaming Video Large Language Models (VLMs) enable continuous video understanding, yet existing benchmarks focus on low-dynamic scenarios. Under bounded context budgets, models must balance temporal history, spatial resolution, and temporal granularity; sparse sampling at 1–2 FPS misses fast events. We introduce FastBench to evaluate high-dynamic perception in real-world video streams. Its trajectory-grounded pipeline combines QA generation from high-FPS clips, filtering of questions answerable at 2 FPS, answer verification using SAM3 and CoTracker3 trajectories, and three rounds of human inspection. FastBench contains 306 QA pairs across eight domains, six capabilities, and forward, instant, and backward temporal scopes, with human-annotated evidence intervals. We also present ProactiveFrame, a training-free baseline that adjusts incoming frame rates through text tokens. A dual-tier sliding window retains recent high-FPS observations while downsampling older ones into sparse history. Experiments reveal substantial limitations: the strongest model, Gemini-3.5-Flash, scores only 50.7%. Denser sampling improves Qwen3-VL-8B from 32.9% at 2 FPS to 44.6% at 24 FPS, but gains saturate as history is compressed. ProactiveFrame outperforms sparse uniform sampling by 5.4 and 1.5 percentage points, yet remains well below oracle-guided focusing, showing that current VLMs struggle to determine from the stream alone when finer temporal perception is needed. FastBench provides a testbed for high-dynamic streaming video understanding. Code and data: https://github.com/Ashone3/FastBench.

[98] Multi-Agent Egocentric World Model with Fine-Grained Embodied Interaction cs.CV | cs.AIPDF

Dahyun Chung, Siyoon Jin, Hyunwook Choi, Honggyu An, Junyoung Seo

TL;DR: 提出 ME-World，将多智能体第一人称世界建模为共享 token 序列中的同步视流生成，建模细粒度动作及交互造成的共同环境状态更新。

Details

Motivation: 多数第一人称世界模型只预测单个智能体视角，而真实具身环境有多智能体共享世界和相互作用；既有多智能体方法多用移动、转相机等粗动作，细粒度交互尚未充分研究。

Result: ME-World 联合去噪多个第一人称视流，将每个流条件于所有智能体目标视角位姿，并以共享环境记忆 grounding 生成。作者使用真实及合成多智能体数据训练评估，并提出环境一致性、更新一致性和身份一致性指标。实验显示相较现有方法，共享世界一致性、动作控制、身份保持和视频质量均改善；摘要未提供数据集名称或定量数值。

Insight: 多个智能体的视频不能独立生成后再简单拼合，必须同步状态更新并维持跨视角动作与身份一致性。

Abstract: Egocentric world models predict first-person observations conditioned on an agent’s actions, but most focus on a single agent. Real embodied settings often involve multiple agents that act and interact within a shared environment. Existing multi-agent world models rely on coarse actions like locomotion, camera control, or discrete commands, leaving fine-grained embodied interactions underexplored. We formulate multi-agent egocentric world modeling as synchronized ego-stream generation for multiple agents interacting through fine-grained actions in a shared world. This requires cross-view action consistency, shared-environment consistency, and consistent propagation of interaction-induced state updates. We propose Multi-agent Egocentric World Model (ME-World), which jointly denoises multiple ego streams in a shared token sequence, conditions each stream on all agents’ target-view poses, and grounds generation with shared environment memory. We train and evaluate on real and synthetic multi-agent data and introduce shared-world consistency metrics for environment, update, and identity consistency. Experiments show ME-World improves shared-world consistency, action control, identity preservation, and video quality over existing methods.

[99] From What to Which: Decoding Modifier Grounding in Frozen MLLMs cs.CVPDF

Barbara Toniella Corradini, Caterina Gallegati, Ludovica Genovese, Vittorio Murino

TL;DR: 提出 OTTER，对冻结 MLLM 各生成 token 的实例 grounding 进行轻量探针评估，特别研究颜色、位置、关系/动作等修饰词是否携带目标实例信息。

Details

Motivation: 传统 token 级 grounding 主要定位名词描述的内容（如香蕉），忽略“黄色”“左边”等帮助区分具体实例的修饰词。

Result: OTTER 在冻结 MLLM 表示上训练监督探针，以最优传输将生成 token 与图像区域对齐，生成紧凑 grounding 图。结果显示，可从修饰词 token 解码区分实例的视觉信息，空间词最明显；但信息也存在于带上下文的对象名词中。grounding 在上下文扰动下仍有信息量，选中区域也与模型生成相关；学得的 OT grounding 可迁移到自由生成和跨数据集设定。

Insight: 指代 grounding 不只依赖对象名词，属性和空间关系词也可表达“究竟是哪一个实例”；该工作是在冻结模型表示上做解码探测，不改变 MLLM 本体。

Abstract: As Multimodal Large Language Models (MLLMs) can describe increasingly complex visual scenes, token-level grounding becomes crucial. Yet, when an MLLM generates “the yellow banana on the left”, established grounding approaches focus on what is in the image (“banana”), overlooking tokens that help describe which instance is meant (“yellow”, “left”). In this work, we ask whether frozen MLLM representations contain decodable grounding information about the referred instance across generated tokens, extending to modifiers such as attributes, spatial expressions, and relational/action terms. To address this question, we introduce OTTER, a lightweight supervised probe over frozen MLLM representations that uses Optimal Transport (OT) to align generated tokens with visual regions and produce compact grounding maps. Our results show that (i) instance-discriminative visual information can be decoded from modifier tokens, with the clearest evidence for spatial terms, but (ii) is not confined to them, as contextualized object nouns also carry referential information; (iii) the recovered grounding remains informative under context perturbations, while selected regions remain relevant to generation; and (iv) the learned OT-based grounding extends beyond the controlled setting to free generation and cross-dataset transfer.

[100] BudgetPix: Compute-Adaptive Tokenization for Pixel-Space Image Diffusion cs.CVPDF

Ozgur Kara, Yujia Chen, Daniel Watson, David Forsyth, James Matthew Rehg

TL;DR: 提出 BudgetPix，以熵引导四叉树和多尺度 token 动态分配像素扩散计算，使单一模型检查点可按推理预算调节图像 token 数。

Details

Motivation: 图像生成通常对同尺寸 patch 使用统一 token 数，无法按推理资源约束调整，也会对纯背景和复杂细节分配相同计算。

Result: BudgetPix 的自适应编码器根据视觉复杂度将固定尺寸图像映射为可变长度 token；尺度感知解码器从多尺度 token 重建固定分辨率图像，训练/采样流程支持不同 token 数。文生图中只用原计算量 25%，在 512² 匹配 MiniT2I-L 保真度，在 1024² 匹配 PixelDiT。MeanFlow 类条件生成中仅需 60% 计算，质量几乎无退化，FID 仅增加 0.8 分。人类和 VLM 评审均确认较既有预算自适应基线有更好的质量—效率折中。

Insight: 空间复杂度自适应 tokenization 可将算力从简单区域转移到细节区域，并在同一权重检查点上提供可调推理预算。

Abstract: Most image generation models rely on uniform tokenization, allocating the exact same computational budget to equally-sized image patches. This static paradigm cannot adapt to different resource constraints at inference time, and yields suboptimal quality-cost tradeoff by devoting the same effort to both plain backgrounds and intricate details. We propose BudgetPix, an adaptive tokenization framework that dynamically allocates compute based on visual complexity and spatial layout, enabling flexible computational budgeting at inference time. BudgetPix comprises three key components: (1) an adaptive encoder that maps a fixed-size image to a variable-length token sequence using an entropy-guided quadtree alongside a multi-scale patch embedder; (2) a scale-aware decoder reconstructs fixed-resolution images from multi-scale token sets; and (3) a flexible training and sampling schedule that enables pixel-space denoisers to operate across variable token counts. BudgetPix seamlessly integrates with existing pixel-space diffusion architectures, enabling a single checkpoint to be operated at a wide range of compute budgets. Evaluated on text-to-image generation, BudgetPix matches the fidelity of MiniT2I-L at $512^2$ and PixelDiT at $1024^2$ using just 25% of the original compute budget. In class-conditional generation using a MeanFlow backbone, BudgetPix requires merely 60% of the full compute budget to produce images with near-zero quality degradation, observing a marginal 0.8-point increase in FID. Comprehensive assessments by human and VLM judges confirm that BudgetPix establishes a significantly improved quality-efficiency tradeoff over prior budget-adaptive baselines. More details are available at our project page: https://karaozgur.com/BudgetPix

[101] RiCo: Neural Simulation of Rigid-Body Interactions via Local Contact Reasoning cs.CV | cs.AI | cs.GR | cs.LGPDF

Ruixiang Ouyang, Guanren Qiao, Fansen Meng, Yueci Deng, Ruixing Jin

TL;DR: 提出 RiCo，以稀疏邻近表面接触点建模刚体交互，将局部接触推理与刚体内部信息传播结合，以高效预测物体运动。

Details

Motivation: 准确刚体模拟要求理解表面局部接触如何塑造物体运动；端到端世界模型通常在整个场景或物体范围预测交互，未显式利用接触力只在相邻表面间传递这一局部性质。

Result: RiCo 为物体交互构造稀疏接触表面点邻域，融合每点状态、相邻表面的相对几何/运动/物理性质，在物体各点间推理局部接触对整体运动的作用。MOVi 基准上，100 帧位置误差降低 31%–35%，方向误差约降低 38%。接触保真也较高：相对真值的穿透时间差为 11.0%，平均穿透深度差为 2.22 mm；可零样本从小规模训练泛化至 270 个物体的场景。真实多球碰撞实验提供初步仿真到现实迁移证据。

Insight: 限制跨物体推理在接触邻域，同时在刚体内部传播接触信息，可保留细节而避免场景点对全连接的计算。实机迁移结果为初步证据。

Abstract: Accurate simulation of rigid-body interactions is essential for predictive physical world models. Despite recent progress in modeling object dynamics, capturing how local contacts between surfaces shape object motion remains challenging. While end-to-end world models predict interactions across entire scenes or objects, in practice, rigid-body contact is inherently local, and only nearby surfaces can directly exchange contact forces. Motivated by this observation, we introduce Rigid-body Contact Reasoning (RiCo), which represents interactions between objects through sparse neighborhoods of contact surface points. RiCo combines each point’s state with the relative geometry, motion, and physical properties of nearby surfaces, then reasons across the object’s points to determine how these local contacts jointly affect its motion. By confining cross-object reasoning to nearby surfaces while propagating contact information within each rigid body, RiCo retains fine-grained interaction details without the cost of modeling every pair of scene points. Such properties enable RiCo a higher accuracy and contact fidelity. Experiments on MOVi-benchmark demonstrate that RiCo reduces 100-frame position and orientation errors by 31-35% and approximately 38%, respectively, compared with baselines. Moreover, RiCo achieves high contact fidelity, with ground-truth-relative penetration-time and mean-depth differences of 11.0% and 2.22 mm, respectively. RiCo further generalizes zero-shot from small-scale training scenarios to scenes containing 270 objects. Our real-world multi-ball collision experiments further provide preliminary evidence of sim-to-real transfer.

[102] Is In-Domain Training Enough for Fine-Grained Industrial Anomaly Understanding? cs.CVPDF

Xingwu Zhang, Duanyang Du, Huiling Zhu, Jiayue Dai, Yixiao Liu

TL;DR: 提出 SiGMA，以多智能体分工和专用视觉缺陷专家改善细粒度工业异常理解，补足单个 MLLM 难以兼顾检测、定位、描述与推理的问题。

Details

Motivation: MM-IAU 需要模型同时定位和解释缺陷，但单个多模态模型难在这些能力上都表现优异；仅组合不同 MLLM 也难弥补共享的细粒度感知弱点。

Result: MMAD 基准上，域内训练的专门模型缺陷定位准确率最高 75.5%，人类专家为 92.3%，有些模型甚至比未训练基础模型检测更差。SiGMA 将工业知识/正常参考搜索交给多模态代理，缺陷专家依据查询—参考比较生成校准异常证据，再用无标签可靠性控制器按任务能力和查询证据质量融合各来源。MMAD 平均准确率达 85.2%，比最强训练专门模型和 Gemini-2.5-Pro 高 4.0%，距人类专家仅 1.5%；即使三个代理均不超过 9B 参数也达 84.4%，新 MLLM 可无需重训加入。

Insight: 细粒度缺陷感知需要专用视觉专家，而不是只靠扩大/集成通用 MLLM；按来源可靠度动态融合可兼顾知识与局部视觉证据。

Abstract: A single multimodal large language model (MLLM) struggles to excel simultaneously at detection, localization, description, and reasoning in multimodal industrial anomaly understanding (MM-IAU). We show that in-domain training does not close this gap. On MMAD, a widely adopted MM-IAU benchmark, trained specialists reach at most 75.5% accuracy in defect localization, against 92.3% for human experts, and even detect anomalies less accurately than their untrained base model. Meanwhile, different MLLMs offer complementary strengths but share this weakness in fine-grained perception, so combining them alone cannot remove it. We therefore propose SiGMA, a spatially grounded multi-agent framework that divides labor between heterogeneous MLLM agents and a dedicated visual defect expert. A multimodal searcher supplies industrial knowledge and normal references, the defect expert turns query-reference comparison into calibrated anomaly evidence, and a label-free reliability controller weighs each source by task-wise competence and query-level evidence quality. SiGMA reaches 85.2% average accuracy on MMAD, 4.0% above the strongest trained specialist and Gemini-2.5-Pro and within 1.5% of human experts. Even with three agents of at most 9B parameters, it reaches 84.4%, and new MLLMs join without retraining.

[103] Reasoning-Informed Visual Editing cs.CVPDF

Xue Yang, Peiyuan Zhang, Yilun Zhu, Qihao Yang, Mingxin Liu

TL;DR: 提出 RISEBench++ 和免训练代理 RISE-Agent，系统评估需多步推理的图像编辑能力，覆盖时间、因果、空间、逻辑、反事实及多轮混合推理。

Details

Motivation: 视觉编辑模型在复杂指令、外观一致性和多种输入格式上仍有挑战，缺少能细粒度测量推理驱动编辑的统一基准。

Result: RISEBench++ 将任务分为六类推理维度，细分 12 个子类、65 种任务，扩展至多图条件和 1,000 个英/中文人工标注测试案例。评估指令推理、外观一致性和视觉合理性，结合人类判断与 LMM-as-judge。RISE-Agent 无需训练，整合推理规划、工具执行和验证器细化，在多种 RISE 任务上超过多数强方法。58 种编辑方案（34 开源、19 闭源、5 代理）评估中，最强 GPT-Image-2.5 Sunburst 准确率仅 56.6%，揭示该任务仍具挑战。

Insight: 评估将推理编辑分解为多个能力和输入形式，结果显示当前编辑模型即使最强也未解决复杂推理任务；基准与代理各自提供测量和解决路线。

Abstract: Large Multi-modality Models (LMMs) have made significant progress in visual understanding and generation, but still face challenges in visual editing, particularly in following complex instructions, preserving appearance consistency, and supporting flexible input formats. To study this gap, we introduce RISEBench, the first benchmark for evaluating Reasoning-Informed viSual Editing (RISE), and extend it to RISEBench++, a more comprehensive and fine-grained benchmark for this emerging task. RISEBench++ extends the taxonomy into a hierarchical scheme spanning six reasoning dimensions: Temporal, Causal, Spatial, Logical, and Counterfactual Reasoning, together with Hybrid Reasoning integrating multiple reasoning types across multi-turn edits. These dimensions are further decomposed into 12 subcategories and 65 fine-grained task types. We expand input formats to include multi-image conditioning and scale the benchmark to 1000 human-annotated test cases, released in English and Chinese. We also improve our evaluation framework, assessing Instruction Reasoning, Appearance Consistency, and Visual Plausibility with human judges and an LMM-as-a-judge approach for more reliable and calibrated judgements. Beyond benchmarking, we introduce RISE-Agent, a training-free agentic framework integrating reasoning-driven planning, tool-augmented execution, and verifier-guided refinement, outperforming most strong existing approaches across diverse RISE tasks. We evaluate 58 visual editing approaches, including 34 open-source models, 19 closed-source models, and 5 agentic methods. The results reveal substantial challenges in reasoning-based visual editing, with even the strongest evaluated approach, GPT-Image-2.5 Sunburst, achieving only 56.6% accuracy. RISEBench++ highlights the limitations of contemporary editing models, provides insights, and indicates future directions for reasoning-aware visual editing.

[104] Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models cs.CV | cs.AIPDF

Hongxing Li, Yixin Li, Dingming Li, Zixuan Wang, Yuchen Yan

TL;DR: 提出 GPD 几何特权蒸馏，在 on-policy 自蒸馏中按问题路由深度、语义和俯视图证据，仅对错误轨迹施加特权 KL，部署模型仍只用 RGB。

Details

Motivation: VLM 空间推理弱点源于 RGB 缺少直接几何证据；推理错误常来自深度或方向感知，单用最终答案奖励无法纠正感知，而推理时直接加入 3D 会增加架构和延迟成本。

Result: GPD 将深度、语义和 BEV 线索渲染为紧凑文本，按问题路由给教师及参考答案，作为几何特权信号。只对错误轨迹增加特权 KL 以增强 GRPO，同时训练出的部署模型保持 RGB 输入。4B 骨干在 VSI-Bench 得 57.1，在 MindCube、SPARBench、MMSI-Bench、ViewSpatial 上平均 37.6，超过 GRPO 和答案特权 OPSD。消融显示几何与答案特权互补，问题条件路由优于全上下文注入，且只蒸馏错误轨迹更有效。

Insight: 可将训练时不可用的 3D 几何作为教师特权蒸馏到 RGB-only 学生，避免推理架构成本；问题条件路由减少无关特权信息。

Abstract: Spatial reasoning remains a persistent weakness of vision-language models (VLMs), because RGB inputs do not directly provide geometric evidence. Existing remedies either inject 3D into the model at inference, paying architecture and latency costs, or train with outcome rewards that supervise only the final answer. Spatial errors originate in perception: a misjudged depth or direction can be corrected only by the scene’s true geometry, which the 3D-scanned sources of spatial training corpora already provide. We propose GPD (Geometry-Privileged Distillation), which makes geometric evidence the privilege in on-policy self-distillation (OPSD). For each question, depth, semantic, and bird’s-eye-view (BEV) cues are rendered as compact text and routed to the teacher alongside the reference answer; a privileged KL, applied only to incorrect trajectories, augments GRPO, and the deployed model remains RGB-only. On the 4B backbone, GPD achieves 57.1 on VSI-Bench and 37.6 average across MindCube, SPARBench, MMSI-Bench, and ViewSpatial, outperforming both GRPO and answer-privileged OPSD across spatial reasoning benchmarks. Ablations confirm the complementarity of 3D and answer privilege, the advantage of question-conditioned routing over full-context injection, and the benefit of restricting distillation to incorrect trajectories.

[105] AgentGarten: Code Worlds for Evolving Agents cs.CVPDF

Jiawei Chi, Shangchen Miao, Zhiyuan Shi, Kailu Wu, Hanyang Wang

TL;DR: 提出 AgentGarten，将带持久状态的代码模拟器/游戏引擎与共享神经渲染器结合，构建可实时互动、可扩展的视觉代理学习环境。

Details

Motivation: 代理能从交互中学习的上限受训练环境制约；环境既要规则和状态一致，又需观测接近真实视觉分布。跨大量世界同时达到两者仍是瓶颈。

Result: AgentGarten 后端维护持久世界状态并执行代码定义交互规则，统一接口将结构化条件提供给神经渲染器以合成观测。渲染器将预训练视频模型适配几何条件，用 Adversarial Forcing 蒸馏并优化实时推理；该训练方法通过精确回放让历史预填充可微，使后续预测损失回传更新历史编码，并加入真实数据对抗监督提升视觉质量。代理在视觉环境中实时互动，每轮经验蒸馏成后续代理继承并细化的 playbook。经验研究显示仅需 4 轮即可学习，传统 RL 对照需要数百万轮；新世界可写为代码并经同接口渲染。

Insight: 将可执行代码世界和视觉渲染统一，可同时提供规则稳定性及逼真观察；playbook 经验继承显著缩短代理学习轮次。

Abstract: Interactive virtual worlds allow agents to learn through exploration and interaction. What agents can learn is bounded by the environments they practice in, which must be faithful, with consistent state, rules, and dynamics, and realistic, with observations that follow the real-world visual distributions. Achieving both across diverse worlds remains a bottleneck. We introduce AgentGarten, a framework that couples simulators and game engines with a shared neural renderer to build real-time interactive environments. Its simulation backends maintain persistent world state and execute program-defined interaction rules, while the renderer generates visual observations from structured conditions exported through a common interface. To build the neural renderer, we adapt a pretrained video model to geometry conditions, distill it with our proposed Adversarial Forcing, and optimize inference for real-time interaction. Adversarial Forcing makes history prefilling differentiable through exact replay, so that losses on later predictions update how the renderer encodes prior observations, and adds real-data adversarial supervision to improve its visual quality. In AgentGarten, agents perceive the world through visual observations, interact with it in real time, and improve by distilling each round of experience into playbooks that subsequent agents inherit and refine. Our empirical study demonstrates a substantial gain in learning efficiency, with agents learning from just 4 rounds compared with millions for a conventional reinforcement learning counterpart. As new worlds can be written as code and rendered through the same interface, environments can scale in both number and difficulty alongside their agents, a step toward agents that keep evolving through interactive experience.

[106] WorldAlign: Decoupled 4D Reward for World-Consistent Video Generation cs.CVPDF

Jing He, Kaixin Ding, Xingye Tian, Guibao Shen, Wenhang Ge

TL;DR: 提出 WorldAlign，将静态区域与动态主体分开，以几何世界先验评估静态结构、以 VLM 检查表观和物理动态，构造解耦 4D 奖励进行视频生成后训练。

Details

Motivation: 世界一致视频需同时保持不同视角下静态场景的 3D 结构，以及主体运动和跨时外观稳定。既有几何后训练常假设静态场景，难可靠评价动态场景中的静态部分，也常弱化动态一致性。

Result: WorldAlign 对静态区域使用语义引导掩码重投影，将几何与几何先验对齐，并加入相机运动奖励避免退化为几乎静止视频。对动态主体使用强 VLM 作为世界先验，基于样本特定清单评估动态性、物理合理性、形状和纹理一致性。该解耦奖励无需人类偏好标注即可支持在线后训练。在 Wan2.1、Wan2.2 两种预训练图生视频生成器上，静态与动态一致性均改善，且没有压制整体或主体运动；摘要未列数值。

Insight: 静态几何和动态主体适用不同世界先验，应分开提供反馈；额外相机运动奖励可避免模型靠停止运动获得虚假静态一致性。

Abstract: Faithful visual world simulation requires generated videos to maintain 4D world consistency, encompassing both static and dynamic consistency. Static consistency requires coherent 3D structure in static environments across viewpoints, while dynamic consistency requires plausible subject motion and consistent appearance over time. Geometry-aware post-training offers a promising way to improve world consistency. However, existing methods often rely on a static-scene assumption. Even those that accommodate dynamic scenes struggle to provide reliable static-consistency feedback, while dynamic consistency is often overlooked or inadequately assessed. To address these limitations, we introduce WorldAlign, a decoupled 4D reward framework that semantically separates static regions and dynamic subjects and provides feedback by aligning each with a world prior suited to its assumptions. For static regions, WorldAlign aligns static geometry with a geometric world prior through semantically guided masked reprojection, enabling more reliable static-consistency evaluation; an auxiliary camera-motion reward discourages nearly static solutions. For dynamic subjects, WorldAlign uses a strong vision-language model (VLM) as a dynamic world prior and constructs a VLM-as-a-judge reward based on sample-specific checklists that assess dynamicity, physical plausibility, shape, and texture consistency. This decoupled design enables more effective online post-training without requiring human preference annotations. Across two pretrained image-to-video generators, Wan2.1 and Wan2.2, WorldAlign jointly improves static and dynamic consistency over existing methods without suppressing overall or subject motion. These results support decoupled world-prior alignment for more faithful visual world simulation. Project page: https://worldalign.github.io/.

[107] GenIA: Generative Reconstruction with Test-Time Input Alignment cs.CVPDF

Stefano Esposito, Naama Pearl, Polina Karpikova, Samuel Rota Bulò, Lorenzo Porzi

TL;DR: 提出 GenIA，在测试时用几何和光度观测对 SAM3D 生成先验进行输入对齐，无需重训基础模型即可改善单目、多视图和动态物体 3D 重建。

Details

Motivation: 生成式 3D 基础模型能补全观测外几何，但预测可能不忠实于输入所见几何、外观或姿态。需要将生成先验与真实观测对齐，而不重新训练大模型。

Result: GenIA 从几何恢复平移和尺度、保留学习到的旋转先验；在去噪期间以可见性偏置注意力、跨观测融合和可微渲染引导对齐外观。可选后处理进一步适配外观潜变量、轻量解码器适配器和物体位置。框架还支持外部输入几何；给定动态物体时间形状时，可恢复共享且对齐输入的规范外观及稳定世界位置。合成与真实基准上，作者报告对单目、多视图和动态输入的位姿与重建均优于近期优化、逐帧图像转 3D 和视频转 4D 方法；摘要未给定量分数。

Insight: 测试时对齐将几何/光度观测用于校正生成先验，同时不改动基础模型参数；对动态对象还可统一规范外观和世界坐标。

Abstract: Reconstructing complete 3D object assets from monocular or sparse multi-view observations remains challenging. Generative 3D foundation models can complete object geometry beyond the observed views, but their predictions may not faithfully reproduce the observed geometry, appearance, or pose. We introduce GenIA, a framework for test-time input-aligned generation that grounds SAM3D’s generative prior in geometric and photometric observations without retraining the foundation model. We improve object pose by deriving translation and scale from geometry while retaining the learned rotation prior, and align appearance through visibility-biased attention, cross-observation fusion, and differentiable rendering guidance during denoising. An optional post-denoising refinement further adapts the appearance latent, lightweight decoder adapters, and object placement to the observations. Our framework also supports externally supplied geometry; when given temporal shapes of dynamic objects, it recovers a shared, input-aligned canonical appearance and stable world-space placement. Across synthetic and real benchmarks, GenIA improves pose prediction and object reconstruction from monocular, multi-view, and dynamic inputs, outperforming recent optimization-based, per-frame image-to-3D, and video-to-4D methods. Our project page is available at https://facebookresearch.github.io/GenIA.

[108] WorldCast: Distributed Multiplayer World Models cs.CVPDF

Ziyang Ye, Junchao Huang, Evelyn Zhang, Zhihao Xie, Ruicheng Zhang

TL;DR: 提出 WorldCast 分布式多人世界模型，每位玩家运行本地视频生成器和状态模型，通过交换玩家/场景状态而非集中联合生成维护多视角一致。

Details

Motivation: 多人世界模型需生成多个独立控制视角，同时让玩家和共享环境表征一致。联合多视角生成的计算成本随玩家数量增长。

Result: 训练时用记录的位置和地图几何监督状态模型，使其根据生成视频和控制输入估计玩家位置；客户端交换玩家状态并投影为相机对齐状态场，引导其他客户端渲染玩家。共享场景状态让客户端复用彼此生成观察，保持各视角外观一致。Counter-Strike 2 实验显示系统实时、分布式可扩展；相机对齐状态场使玩家渲染率较联合生成方法提高一个数量级以上，共享场景状态提升整局视觉一致性。各客户端实时运行且仅交换玩家/场景状态，无中心计算瓶颈；小时级 rollout 中画质稳定。

Insight: 将多人同步分解为独立本地生成加紧凑状态通信，可避免联合多视图生成随人数扩张的计算成本。

Abstract: Multiplayer world models must generate independently controlled views with consistent representations of both players and their shared environment. Most existing approaches coordinate multiple players through joint multi-view generation, whose cost grows with each additional player. We present WorldCast, a distributed multiplayer world model in which each player runs a local client comprising a video generator and a state model. Using recorded player positions and map geometry during training, the state model estimates the player’s position from generated video and control inputs. Clients exchange player states and project them into camera-aligned player state fields that guide where and how other players are rendered. Shared scene state enables clients to reuse one another’s generated observations to maintain consistent scene appearance across views. Experiments on Counter-Strike 2 demonstrate WorldCast’s consistency, real-time performance, and distributed scalability. The camera-aligned player state field improves player rendering rates by over an order of magnitude over joint-generation methods, while shared scene state improves visual consistency over whole rounds. Each client runs in real time and exchanges only player and scene states, enabling scalable multiplayer generation without a centralized computational bottleneck. Image quality remains stable over hour-long rollouts.

[109] MAMHOI: Factorizing Scene-Aware Human-Object Interaction through Affordances cs.CV | cs.AIPDF

Mingyuan Lei, Yoonchang Sung, Tat-Jen Cham

TL;DR: 提出 MAMHOI，以显式运动可供性作为场景理解与人体—物体动作生成之间的接口，分别从互补数据源学习而无需成对人—物—场景监督。

Details

Motivation: 复杂 3D 场景中的人—物交互生成需要同时判断交互在环境中是否可行并合成自然动作，但人—场景数据擅长场景适配运动，人—物数据更擅长交互细节，二者成对数据稀缺。

Result: MAMHOI 先以场景条件模型预测交互可行的位置和方式，再以可供性条件 HOI 模型生成对应人—物运动，通过显式 motion-affordance 分解连接场景理解与动作合成。复杂室内环境实验显示，相较基线减少物体—场景穿透，同时更好保留人—物交互质量，生成结果更真实且物理可行；摘要未提供数据集、数值或训练设置。

Insight: 可供性作为中间表示允许场景适配信息和交互动态从互补监督源中学习，缓解成对场景化 HOI 数据不足。

Abstract: Generating realistic human-object interactions (HOI) in complex 3D scenes requires two complementary capabilities: reasoning about interaction feasibility in the environment and synthesizing realistic human-object motion. However, supervision for these capabilities is rarely available jointly at scale. Human-scene datasets provide rich information about environment-aware motion, while human-object datasets capture detailed interaction dynamics, yet paired human-object-scene data remain scarce. We present MAMHOI, an affordance-mediated factorization for scene-aware human-object interaction generation. MAMHOI factorizes scene-aware HOI generation through an explicit motion-affordance interface between scene understanding and motion synthesis: a scene-conditioned model first predicts where and how an interaction can be feasibly executed, and an affordance-conditioned HOI model then generates the corresponding human-object motion. This factorization allows scene understanding and interaction dynamics to be learned from complementary sources of supervision without requiring paired human-object-scene data. Experiments in complex indoor environments show that MAMHOI reduces object–scene penetration while better preserving human–object interaction quality, yielding more realistic and physically feasible scene-aware interactions. Project page: https://leimingyuan.github.io/MAMHOI-project-page/

[110] OneSearch-VL: Unified Multimodal Deep Research Agent for Image and Video cs.CVPDF

Hongyu Li, Manyuan Zhang, Kaituo Feng, Shu Chen, Dian Zheng

TL;DR: 提出 OneSearch-VL，以视觉 grounding 证据图统一单图、多图和视频深度研究代理的数据构造、过程监督和操作级评估。

Details

Motivation: 不同图像/视频研究任务都需视觉定位、外部检索和事实组合，但关键难点是保持局部视觉锚点、实体关系、来源支持事实与答案操作间的依赖。

Result: Visually Grounded Evidence Graph（VGEG）将这些依赖编码为任务级共享引用。数据引擎用其构造并验证多图/视频问题、筛选专家轨迹，并生成 OneSearch-VL-SFT-110K 和 OneSearch-VL-RL-10K。Evidence-aware Visual-Grounded Rubric（EVGR）奖励从 VGEG 标注推导，在 RL 中监督证据可追溯性和视觉 grounding。作者另构建 OneSearch-MI-Bench 与 OneSearch-Video-Bench，按研究操作组织问题。OneSearch-VL-8B 在新多图和视频基准上较可用工具的 Qwen3-VL-8B 分别提高 20.2 和 17.6 个百分点，在七个图像基准及 VideoDR 上也有较大收益。

Insight: 将证据依赖作为显式图结构，能把视觉 grounding 和来源支持监督贯穿数据、训练与评估，避免只优化最终答案。

Abstract: Single-image, multi-image, and video deep research require different visual operations but share a workflow of visual grounding, external retrieval, and fact composition. A key challenge is to preserve the dependencies linking localized visual anchors, entity relations, source-supported facts, and answer-producing operations. We introduce OneSearch-VL, a unified agent centered on the Visually Grounded Evidence Graph (VGEG), which encodes these dependencies as a shared task-level reference for data construction, process supervision, and operation-level evaluation. Our VGEG-based data engine constructs and verifies multi-image and video questions and filters expert trajectories. Using these data, we assemble OneSearch-VL-SFT-110K and OneSearch-VL-RL-10K for SFT and RL, respectively. We further derive the Evidence-aware Visual-Grounded Rubric reward (EVGR) from VGEG annotations to supervise evidence traceability and visual grounding during RL. For fine-grained evaluation, we construct OneSearch-MI-Bench and OneSearch-Video-Bench, organizing questions by the research operations encoded in their VGEGs. Experiments show that OneSearch-VL-8B improves over Qwen3-VL-8B with tool access by 20.2 and 17.6 percentage points on the two new benchmarks, respectively, while also achieving substantial gains across 7 image benchmarks and VideoDR. Project repository: https://github.com/appletea233/OneSearch-VL

[111] Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matching cs.CV | cs.LGPDF

Luping Liu, Bingyi Kang, Yifan Wang, Dong Xu

TL;DR: 提出 FreeMatching，融合生成与语义基础表示及多来源异构监督，学习在图像编辑/参考生成等打破物理连续性的变换下仍保持身份的稠密对应。

Details

Motivation: 传统稠密匹配依赖平滑运动和刚性几何等时空先验；图像编辑与参考生成中的变换可能保持视觉身份却破坏物理连续性，旧假设失效。

Result: FreeMatching 使用经典数据集、跟踪视频和合成场景的异构监督，教师引导迭代细化可在无稠密对应标注下改善 IEG 匹配。单一模型在困难 IEG 图像对上明显提升对应质量，同时保留经典匹配基准的竞争力。作者还展示可将匹配作为身份保持定量指标，分数与人类判断相关；摘要未报告具体指标和相关系数。

Insight: 身份对应可由语义/生成表征而不仅是几何连续性支持，适合评估外观编辑或生成前后是否仍为同一物体。

Abstract: Dense correspondence matching has historically been bounded by simplifying spatio-temporal priors, such as smooth motion and rigid geometry. While effective for classical tasks, these assumptions break down in image editing and reference-guided generation (IEG), where transformations can preserve visual identity while breaking physical continuity. To establish identity-preserving correspondence across such transformations, we introduce FreeMatching, a generalizable framework combining generative and semantic foundation representations with heterogeneous supervision from classical datasets, tracked videos, and synthetic scenes. Teacher-guided iterative refinement further improves correspondence in IEG without dense correspondence annotations. Experimentally, a single FreeMatching model substantially improves correspondence quality on challenging IEG image pairs while retaining competitive performance on classical benchmarks. Furthermore, we demonstrate its utility as a quantitative metric for evaluating identity preservation, with scores that correlate with human judgment. The code is available at https://github.com/luping-liu/FreeMatching.

[112] Pumpire: Unified Benchmark for Metric Distance Estimation cs.CVPDF

Siyu Chen, Zehan Wang, Jiayang Xu, Yihan Wu, Jialei Wang

TL;DR: 提出 Pumpire 统一基准，直接测量图像或视频 3D 基础模型重建几何中的点对物理距离，评估模型对真实尺度的感知。

Details

Motivation: 既有方法常单独评估深度、相机内参或点云几何相似度，无法直接反映模型是否能准确估计两个 3D 点之间的物理距离。

Result: pumpire-6k 含 100 个真实场景，每场景标注物理测量的点对距离，并采集 64 帧，共 6,400 帧。评估协议覆盖有/无深度先验的图像级和视频级 3D 基础模型，可直接测量点对距离误差并跨设定比较。作者对代表性模型的 29 种基线配置进行实验并提供综合分析；摘要未给分数或模型排行。

Insight: 直接点对距离评估补足传统几何相似度指标未测量的尺度感知能力，可用于比较不同输入层级和深度先验设置。

Abstract: We present Pumpire, a unified benchmark for evaluating metric point-pair distance estimation capability of both image- and video-level 3D foundation models, with or without depth priors. In contrast to previous approaches that normally evaluate depth and camera intrinsics separately or evaluate point-clouds with geometric similarity metrics, which cannot directly reflect models’ point-to-point distance estimation capability, Pumpire directly assesses point-to-point distances from the reconstructed geometry. To this end, we collect a large-scale and diverse dataset (pumpire-6k) comprising 100 real-world scenes, each annotated with physically measured point-pair distances and containing 64 frames, for a total of 6,400 frames. Building on this dataset, we establish a holistic evaluation protocol that covers both image- and video-level 3D foundation models and enables direct assessment of point-pair distance errors and cross-setting comparison. We conduct extensive experiments across 29 baseline configurations of representative 3D foundation models and provide a comprehensive analysis of the results. By offering this benchmark, we target the more fundamental ability to perceive and estimate physical scale in the reconstructed 3D space, which prior evaluation protocols have largely overlooked. The project page can be found at https://pumpire.github.io/

[113] One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts cs.CV | cs.LGPDF

Adrian Bulat, Yassine Ouali, Georgios Tzimiropoulos

TL;DR: 提出 reViT，以共享 Transformer 块循环处理多个深度，并用小型专家库的凸组合按归一化深度编程 FFN 变换，在保持推理 FLOPs 时恢复深度专属能力。

Details

Motivation: 循环应用单一 Transformer 块可节省参数，但不同深度需不同变换；常见方法依赖中间特征蒸馏。论文探索通过深度条件专家组合使循环视觉编码器匹配完整深度模型。

Result: reViT 将每个循环深度的 FFN 表示为小型共享专家库的凸组合，连续归一化深度坐标控制专家混合，可重采样。ImageNet-1k 监督训练与 DINOv2 教师蒸馏两种设置的受控实验中，在相同单 FFN 预算下，权重空间混合是所测最强 MoE 方案，优于 token 路由和输出混合。从头训练的 reViT-B/16 以约少 70% 存储参数达到 DeiT III 准确率；只用教师输出特征蒸馏的 8 专家模型保留几乎全部 DINOv2 线性探针准确率，并迁移至分类、分割、深度预测。弹性深度训练使单检查点通过重采样深度坐标适配多个深度。固定深度部署时可物化成稠密图，移除在线路由/合并但存储会增大。

Insight: 连续深度坐标使 FFN 专家组合可适配不同循环深度；参数节省和推理预算需区分，因为部署物化会以存储换取免路由执行。

Abstract: In this work, we show that a single Transformer block, applied recurrently, can match the accuracy of a full-depth vision encoder at comparable inference FLOPs without intermediate feature distillation. reViT restores depth-specific transformations by representing the FFN at each recurrent depth as a convex combination of a small shared expert bank. A continuous normalized-depth coordinate programs this mixture, defining a resampleable trajectory through FFN parameter space. We evaluate this design in two regimes: supervised ImageNet-1k training and distillation from a DINOv2 teacher. Across both regimes, controlled adaptations identify weight-space merging as the strongest tested MoE family at a matching one-FFN budget, ahead of the token-dispatch and output-mixture alternatives. Trained from scratch, reViT-B/16 attains DeiT III accuracy with about 70% fewer stored parameters. An 8-experts model distilled using only the teacher’s output features retains nearly all of its DINOv2 teacher’s linear-probe accuracy and transfers across classification, segmentation, and depth prediction. Elastic-depth training allows one checkpoint (trained model) to operate at multiple tested depths by resampling the same normalized coordinate interval. For fixed-depth deployment, the recurrent block can be materialized as a conventional dense graph, removing online routing and merging without changing the one-FFN-per-depth compute but expanding deployment storage.

[114] LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generation cs.CVPDF

Suhwan Cho, Yonwoo Choi, Soongjin Kim, Jicheol Park, Taegyu Lim

TL;DR: 提出 LEGO，一种无需深度估计、点云提升或重投影的外景到第一人称视图合成方法，以学习式视角合成器提供结构，再让视频扩散模型补足细节。

Details

Motivation: 从单段外部摄像机视频生成第一人称视频时，两机位重叠少且目标视角许多内容不可见。传统流程估深、提升点云、重渲染后条件化视频扩散，深度误差会把像素投到错误位置。

Result: LEGO 微调 LVSM 风格 Transformer，直接从外景视频渲染第一人称视图，不显式使用深度、点云或重投影，内部学习跨视角对应。概率式映射将目标区域按学习对应分布对多个源位置求平均，可能损失细纹理但保留结构；作者认为这适合作为扩散模型条件，因为去噪擅长恢复细节。分布集中度还提供区域置信度，用于屏蔽低置信区并在早期布局去噪中引导生成器关注高置信内容。该方法持续优于显式重建基线，并可零样本迁移到其他数据集；摘要未列具体分数。

Insight: 扩散生成器的条件应优先给出结构对齐而非过锐但可能错位的纹理，细节可由扩散补全。置信度支持低可信区域遮蔽和早期布局引导。

Abstract: Generating an egocentric video from a single exocentric recording is a challenging case of novel view synthesis, as the two cameras share little overlap and much of the target view is unobserved. Current state-of-the-art methods reconstruct the scene explicitly by estimating depth, lifting the video into a point cloud, and re-rendering it from the egocentric camera to condition a video diffusion model. This deterministic mapping assigns each pixel to a single reprojected location, which preserves texture but translates depth errors into misplaced content. We ask what a video diffusion model should receive as its condition and propose a lifting-free answer: a learned view synthesizer, an LVSM-style transformer fine-tuned to render the egocentric view directly without depth, point clouds, or reprojection, resolving cross-view correspondence internally. In contrast, its probabilistic mapping averages each region over candidate source locations according to a learned correspondence distribution, preserving structure while fine texture is averaged away. We argue that this trade-off suits a diffusion generator, whose denoising training excels at restoring detail, so an effective condition should prioritize structural alignment over sharpness. This distribution’s concentration also yields a per-region confidence, used both to mask low-confidence regions and to guide the generator toward high-confidence areas during early layout-forming denoising steps. Our approach consistently outperforms the state-of-the-art explicit pipeline and generalizes to other datasets without retraining. The synthesizer thus supplies view structure, and the diffusion model its detail.

[115] VersaCamVLA: Camera-Configurable VLA Policies for Robotic Manipulation cs.CV | cs.ROPDF

Boyao Han, Chen Shi, Jingjing Qian, ZhuoTan Tian, Li Jiang

TL;DR: 提出 VersaCamVLA，以统一场景 token 接口将任意数量、任意位姿 RGB 视图映射成固定大小潜表示，提升机器人 VLA 对相机配置变化的适应性。

Details

Motivation: VLA 通常在固定相机数量和位姿下训练，部署时相机数量或位置变化会导致脆弱。

Result: VersaCamVLA 将相机集合表示与动作学习解耦，以多信号目标视图预测和腕部增强位姿采样 WAPS 学习任意变长、已知位姿 RGB 视图的固定大小场景 token；WAPS 利用自然腕相机运动获取免费位姿多样性。部署时轻量空间编码器把场景 token 作为补充视觉条件注入预训练 VLA，无需显式 3D 传感或新视角渲染。RoboTwin、LIBERO 和真实机器人平台实验显示，相较既有 VLA 与直接多视图基线表现更好，并在不同相机数量及未见相机位姿下保持鲁棒。

Insight: 腕相机自然运动可充当多视角位姿数据源，固定维场景 token 则允许 VLA 接收可变相机集合。摘要没有列出各平台的成功率。

Abstract: Vision-Language-Action (VLA) models have emerged as powerful foundations for robotic manipulation, but their reliance on fixed camera configurations during training makes them brittle to changes in camera count or pose during deployment. To overcome these limitations, we propose VersaCamVLA, a camera-configurable framework that decouples camera-set representation from action learning. VersaCamVLA learns a unified scene-token interface that maps an arbitrary, variable set of posed RGB views into fixed-size latent scene tokens. This is achieved via multi-signal target-view prediction and Wrist-Augmented Pose Sampling (WAPS), which leverages natural wrist-camera motion for free pose diversity. At deployment, a lightweight spatial encoder injects these compact scene tokens into a pretrained base VLA as a supplementary visual condition, requiring no explicit 3D sensing or novel-view rendering. Experiments on RoboTwin, LIBERO, and a real-robot platform demonstrate that VersaCamVLA consistently outperforms prior VLA methods and direct multi-view baselines, maintaining robust performance across varying camera counts and unseen camera poses.

[116] OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning cs.CVPDF

Zhongyu Yang, Jiale Tao, Ruitao Chen, Zuhao Yang, Yingfang Yuan

TL;DR: 提出 OmniCapBench，以密集标注的原子可验证单元评测细粒度视听视频字幕，兼顾实体指代、视觉镜头与音频事件的覆盖和局部定位。

Details

Motivation: 全字幕分数覆盖全面但缺少局部错误定位，局部 probe 有定位但缺整体覆盖，开放式 LLM 评审又不稳定；需要深层结构诊断多模态模型的时间和跨模态错误。

Result: OmniCapBench 将预测目标设为三条 track 的原子评估单元：实体指代、视觉镜头和音频事件，以确定性约束检查与局部 LLM 语义比较评分。基准含 786 个密集标注视频，可区分时间 grounding 失败、身份漂移、跨模态错位及幻觉描述。前沿 MLLM 评测显示局部感知较强，但长时视听推理较弱，尤其在身份漂移和跨模态错位方面；摘要未提供具体分数。

Insight: 原子化评估单元保留整体覆盖并给出局部定位，减少自由格式裁判的评分不稳定；结果指出长时身份和跨模态一致性仍是关键短板。

Abstract: Multimodal large language models (MLLMs) are rapidly evolving toward continuous audio–visual reasoning, creating an urgent need for evaluations that expose their capability limits. Audio–visual captioning is an ideal diagnostic task, yet current benchmarks face a coupled trade-off: whole-caption scores provide coverage without localization, local probes provide localization without coverage, and unconstrained LLM judges introduce instability. We introduce OmniCapBench (Omni-Video Caption Benchmark), a benchmark that reframes audio–visual caption evaluation as a deep-structured diagnostic framework. OmniCapBench shifts the prediction target from free-form text to sets of atomic, verifiable evaluation units across three tracks: entity references, visual shots, and audio events, enabling reliable scoring with deterministic constraint checks and localized LLM-based semantic comparisons. With 786 densely annotated videos, OmniCapBench effectively distinguishes MLLM perception errors, including temporal grounding failures, identity drift, cross-modal misalignment, and hallucinated descriptions. Evaluating frontier MLLMs reveals strong local perception but weak long-horizon audio–visual reasoning, particularly in identity drift and cross-modal misalignment, providing a fine-grained roadmap for omnimodal development.

[117] OuroWorld: Bringing Any 3D World Alive as Diverse, Endlessly Looping 3D Cinemagraphs cs.CV | cs.GRPDF

You-Zhe Xie, Ting-Wei Chou, Yu-Hsuan Li, Kaipeng Zhang, Zhixiang Wang

TL;DR: 提出 OuroWorld，将静态 3D Gaussian Splatting 场景转换成可从任意视角无缝循环的动态 3D cinemagraph，并用鲁棒周期 4DGS 吸收跨视角监督不一致。

Details

Motivation: 近期 3D 世界模型能生成逼真可探索场景，但场景往往静止。为静态场景加入多样动态需从单参考视频恢复跨视角运动，同时处理视图不一致和无缝循环。

Result: VLM 推断合理动态并指导视频模型生成参考视频，再提升并补全为多视图视频。Inconsistency-Robust Periodic 4DGS 用傅里叶形变场从构造上保证循环，参考视角锚定的 Grounded Drift Field 吸收跨视角不一致。方法可表达一般形变、物体运动和光照变化，而非只适用类流体运动。作者提出无真值评估，覆盖生动度、自然度、循环接缝连贯性和场景质量。39 个重建/生成场景上优于所有基线，用户研究偏好率为 70.8%–99.0%。

Insight: 周期形变场保证循环属性，参考视角锚定漂移场则处理跨视图错误监督；用户偏好范围因对比基线而变化。

Abstract: Recent 3D world models generate photorealistic, explorable scenes that remain frozen in time. OuroWorld is a mask-free framework that turns any static 3D Gaussian Splatting scene into a 3D cinemagraph: a dynamic scene with vivid, diverse motion looping seamlessly from any viewpoint. A vision-language model infers plausible dynamics and guides a video model to synthesize a reference video, which we lift and complete into multi-view videos. To learn from this imperfect supervision, we propose Inconsistency-Robust Periodic 4DGS: a Fourier-series deformation field guarantees looping by construction, while a Grounded Drift Field anchored at the reference view absorbs cross-view inconsistency. Unlike prior Eulerian methods limited to fluid-like motion, we capture general deformation, object motion, and illumination change. We introduce a ground-truth-free evaluation covering vividness, naturalness, loop seam coherence, and scene quality. On 39 reconstructed and generated scenes, OuroWorld outperforms all baselines and wins 70.8%-99.0% of user-study comparisons. Project page: https://ouroworld.userwei.com

[118] WorldGuide: Goal-Directed Video World Model for Procedural Task Execution cs.CVPDF

Ankan Deria, Komal Kumar, Hisham Cholakkal, Fahad Shahbaz Khan, Salman Khan

TL;DR: 提出 WorldGuide，将程序化视频生成建模为闭环视觉世界任务执行：从初始图像和目标预测原子动作、生成动作视频，再依据生成进展决定下一步或结束。

Details

Motivation: 长程程序任务需根据实际执行结果自适应生成，而开环视频生成不能响应执行状态，现有闭环系统又常依赖预训练执行器或间接验证，决策动作与成功实现之间仍有缺口。

Result: Planner 和 Executor 使用相同的逐步程序示范训练：规划器根据视觉进展预测下一原子动作或任务完成，执行器直接学习实现预测动作；层级视觉记忆以有界历史 token 成本维持长程状态。因缺少联合训练所需的逐步动作—视频监督，作者构建约 5.9 万段有步骤标注视频，覆盖 245 个任务、27 类程序。WorldGuide-Bench 任务成功率 33.33%，高于强视频模型 MiniMax-H3 的 29.90%，尽管后者获得参考动作计划；goal-only 条件下，VideoCraft-Bench 上为 47.69%，MiniMax-H3 为 32.73%。

Insight: 规划与学习到的动作执行必须耦合，才能做目标导向程序视频生成；其训练数据对动作步骤进行显式标注。

Abstract: Video generators and video-based world models can synthesize plausible visual trajectories, but long-horizon procedural tasks require generation to adapt to what has actually been produced. A model must determine the next action from its generated state, execute that action, and recognize when the task is complete. Open-loop generation cannot adapt to execution outcomes, while existing closed-loop systems often rely on pretrained executors or indirect verification. This leaves a gap between deciding an action and successfully realizing it. We formulate procedural video generation as \emph{closed-loop task execution in visual world space} and introduce \textbf{WorldGuide}. Given only an initial image and a task goal, WorldGuide predicts an atomic action, generates its corresponding video clip, and uses the generated result to select the next action or terminate. The Planner and Executor are trained on the same step-level procedural demonstrations: the Planner learns to predict the next atomic action or task completion from visual progress, while the Executor is directly trained to realize the predicted actions. Hierarchical visual memory maintains state across long-horizon execution with bounded history token cost. Due to the lack of step-level action-video supervision for joint planner-executor training, we introduce \textbf{WorldGuide Bench}: approximately 59K step-annotated videos across 245 tasks and 27 procedural categories. WorldGuide achieves a 33.33% Task Success on \textbf{WorldGuide-Bench}, compared with 29.90% for the strong recent video model MiniMax-H3, even though MiniMax-H3 receives reference action plans, and achieves 47.69% on \textbf{VideoCraft-Bench} compared with 32.73% for MiniMax-H3 under goal-only conditioning. These results demonstrate the importance of coupling planning with learned execution for goal-directed procedural video generation.

[119] What 30,000 Hours of Ego-centric Video Does Not Teach cs.CVPDF

Jiahua Dong, Anurag Bagchi, Yash Jangir, Muhammad Zubair Irshad, Sergey Zakharov

TL;DR: 研究 3 万小时第一人称视频能教会世界模型什么，直接测量分布外场景中的代理动作与物体交互保真，发现代理建模进步远快于物体物理行为。

Details

Motivation: 世界模型可替代物理仿真器，但离实用仍远。作者检验扩展大量第一人称人类视频对动作和物体交互建模的作用，而不只依靠不透明下游指标。

Result: 数据集含 30,000 小时视频，覆盖 1,000 多种场景、14,000 名贡献者。在困难分布外基准上直接评估动作与物体交互保真。训练数据扩展 100 倍后，两类表现均提升但不均衡：代理动作建模较好，物体保真仍低且提升缓慢。作者还显示代理收益不必由数据量导致，精心设计视觉条件可用更少数据使代理保真饱和，从而单独测量物体保真并发现其饱和点。随后通过把模型容量从场景外观转向物体动态的监督方案改善物体保真，但差距仍大；结论迁移至下游人形建模。

Insight: 大量第一人称数据足以让模型接近代理动作建模上限，却未让它同步学会物体动力学；缩小差距需要改进监督方式，而不只是继续堆数据。

Abstract: World models offer a promising alternative to physics-based simulators, yet remain far from practical deployment. We ask how far scaling ego-centric human video takes them, using a dataset of 30,000 hours spanning over 1,000 scene types and 14,000 contributors. Rather than relying on opaque downstream metrics, we directly evaluate agent and object-interaction fidelity on a challenging out-of-distribution benchmark. Increasing training data by 100x improves both, but unevenly: the agent is modeled well, while object fidelity remains far lower and improves slowly. We show that the agent gains need not come from data, and a careful visual conditioning design saturates fidelity with a fraction of it, which lets us measure object fidelity on its own and discover its saturation point. We then introduce a supervision scheme that shifts capacity from scene appearance toward object dynamics, improving object fidelity though a substantial gap remains. Finally, our conclusions transfer to downstream humanoid modeling. Overall, our results suggest that scaling ego-centric data brings agent modeling close to its limit while leaving its effects on the world far behind, and that closing this gap will depend on how models are trained, not only on how much data they see.

[120] Rubric-CEPR: Self-Evolving Image Editing via Reward-Verified Self-Distillation cs.CVPDF

Ritesh Thawkar, Shubham Patle, Shravan Venkatraman, Rao Muhammad Anwer

TL;DR: 提出 Rubric-CEPR，让图像编辑器用自身生成样本和内部表示自我改进，通过拆解式编辑—保留评分筛选后再蒸馏，无需人工编辑目标或外部训练时奖励模型。

Details

Motivation: 编辑模型继续提升通常依赖人工编辑配对或外部奖励模型，获取昂贵；现实感强的编辑也可能没有完成指令，或改动本应保留内容。

Result: 规划器从无标签图像生成结构化编辑指令，编辑器采样多个候选，冻结 Critic 使用编辑器已有特征分解检查修改实现、旧状态去除和内容保留。不可补偿门控拒绝不可行候选，最佳验证样本再经轻量 adapter 训练回编辑器。在 Qwen-Image-Edit 上，ImgEdit 评分从 4.36 升至 4.60（+5.5%），物体隔离提升 24.9%，并迁移到 GEdit-Bench、Complex-Edit；同一流程使 Step1X-Edit 的 ImgEdit 提升 7.8%。

Insight: 用内部表示加分项门控验证自生成样本，可避免外部奖励模型和人类配对编辑成本，也减少只因逼真而奖励未完成修改的错误。

Abstract: Instruction-guided image editors have become highly capable, yet improving them further still depends on human-edited training pairs or external reward models. Such supervision is costly to obtain and can reward plausible failures: a realistic output may leave the requested change undone or alter content that should be preserved. In this work, we strive to improve a pretrained image editor using only its own generations, without human-edited targets or an external training-time reward model. To this end, we propose a self-evolving framework, named Rubric-CEPR, that verifies the editor’s own samples with its internal representations through a rubric-augmented Contrastive Edit-Preservation Reward (CEPR). A Planner proposes structured edit instructions from unlabeled images, the Editor samples multiple candidate edits, and a frozen Critic scores each candidate with decomposed rubric checks for edit realization, removal of the old state, and content preservation, using features already exposed by the editor. Non-compensatory gates reject infeasible candidates, and the best verified candidate is distilled into the editor through lightweight adapter training. On Qwen-Image-Edit, Rubric-CEPR improves ImgEdit from 4.36 to 4.60 (+5.5%), with a +24.9% gain on object isolation, and transfers to GEdit-Bench and Complex-Edit. The same procedure also improves Step1X-Edit by +7.8% on ImgEdit. We hope our approach will serve as a solid baseline for image editors that improve themselves from their own verified samples. Our code is publicly available at $\href{https://riteshthawkar.github.io/Rubric-CEPR/}{\text{this URL}}$

cs.CL [Back]

[121] Large Language Model-Assisted Preparation of Transportation Management Plans: A Case Study with WisDOT WisTMP System cs.CLPDF

Zihao Sheng, Pei Li, Zilin Huang, Yen-Jung Chen, Yuhao Luo

TL;DR: 提出用本地微调开源 LLM 辅助 WisDOT WisTMP 交通管理计划编写，并基于历史文件构建问答数据集分析模型能力与不足。

Details

Motivation: 施工区交通管理计划对于安全和通行至关重要，但编写仍耗费大量人工并高度依赖专业经验。作者探索 LLM 是否能提高计划内容生成效率，同时在本地部署以保障数据安全。

Result: 研究从历史 WisTMP PDF 构建结构化 JSON 问答数据，并在多种规模开源模型上微调后本地部署。微调显著改善标准文本生成指标。分章节和策略层分析发现，模型总体表现较强，但倾向于过量生成策略，且难以给出项目特定论证和准确成本估算；模型从 7B/8B 扩至 14B 的增益有限。摘要没有提供具体指标分数或生成时间数据。

Insight: 该案例展示领域适配可提升计划文本指标，但对项目事实、成本和策略数量的控制仍是关键弱点。结果针对 WisDOT WisTMP 场景，尚不能直接推广到其他交通部门流程。

Abstract: Work zones are critical yet hazardous components of transportation infrastructure, requiring carefully designed Transportation Management Plans (TMPs) to ensure safety and mobility. However, TMP preparation remains labor-intensive and heavily dependent on practitioner expertise. This paper proposes a Large Language Model (LLM)-assisted framework to automate TMP content generation, leveraging the WisDOT WisTMP system as the application context. The framework fine-tunes multiple open-source LLMs across different model scales and deploys them locally to ensure data security. To support model training, we construct a domain-specific dataset from historical WisTMP documents by converting PDF files into structured question-answer pairs in JSON format. Experimental results show that fine-tuning significantly improves performance across standard text generation metrics. Further section-wise and strategy-level analyses reveal that, while LLMs achieve strong overall performance, they tend to over-generate strategies and struggle to produce project-specific justifications and accurate cost estimates. In addition, scaling from 7B/8B to 14B yields limited gains. These findings demonstrate the potential of LLMs to improve TMP preparation efficiency while highlighting remaining challenges in LLM-assisted TMP development. The source code and demo videos will be publicly available at https://zihaosheng.github.io/TMP-LLM/.

[122] Stochastic Teacher Intervention for Agentic On-Policy Distillation cs.CL | cs.AIPDF

Junnan Liu, Linhao Luo, Zhijun Chen, Qianren Mao, Thuy-Trang Vu

TL;DR: 提出 STI-OPD，在多轮工具智能体蒸馏中按师生策略差异概率性接管学生动作，并通过重要性加权反向 KL 学习混合策略轨迹。

Details

Motivation: 在线策略蒸馏（OPD）用学生自身 rollout 提供逐 token 教师监督，但多轮学生决策会改变后续观察，早期错误不断累积，使轨迹偏离教师分布并降低监督可靠性。

Result: STI-OPD 用 KL 散度估计师生策略差异，并映射为教师接管动作的概率；按概率随机决定是否以教师动作替换学生动作，在教师监督和学生探索之间自适应平衡。为从混合策略生成的轨迹学习，作者引入重要性加权反向 KL，校正教师响应相对学生策略的采样偏差。工具增强推理和长时交互实验中，STI-OPD 在每个评测基准和学生规模上均超过最强已有 OPD 基线；消融显示差异引导接管和重要性加权都带来收益。

Insight: 教师接管由当前策略差距决定，比固定阈值或固定计划更能针对轨迹偏移；混合轨迹的采样校正用于保持原在线蒸馏目标。摘要没有给出具体分数。

Abstract: On-policy distillation (OPD) efficiently transfers capabilities from a stronger teacher to a student language model through dense token-level supervision on student-generated rollouts and has shown promise on complex tasks such as mathematical reasoning. However, in multi-turn agentic tasks, student decisions shape subsequent observations, causing early errors to accumulate across turns. The resulting trajectories can drift away from the teacher’s rollout distribution, making the teacher’s token-level supervision less reliable or even counterproductive for OPD training. To address this issue, we introduce STI-OPD, a stochastic teacher intervention framework for multi-turn agentic OPD. During multi-turn interaction, STI-OPD uses teacher intervention guided by teacher-student policy discrepancy to replace the student’s proposed action with a teacher-generated one to maximize the acquisition of reliable supervision. We further develop a stochastic intervention strategy, addressing the limitations of previous threshold-based or fixed-schedule approaches, that estimates policy discrepancy using KL divergence and maps it to an intervention probability. By sampling whether to intervene from this probability, STI-OPD adaptively balances teacher control with student exploration. To learn from the resulting mixed-policy trajectories, we introduce an Importance-Weighted Reverse KL objective that corrects the token sampling mismatch between teacher-generated responses and the student policy to preserve the original OPD objective. Across tool-integrated reasoning and long-horizon interaction, STI-OPD outperforms the strongest prior OPD baseline on every evaluated benchmark and student size. Ablations further show that both discrepancy-guided intervention and importance weighting contribute to these gains.

[123] Disentangling Linguistic and Paralinguistic Information with Routed Sparse Autoencoders cs.CL | eess.ASPDF

Beimnet Bekele Guta, Xiaoyu Yang, Guangzhi Sun, Philip C. Woodland

TL;DR: 提出带路由监督与跨因素对抗的 TopK 稀疏自编码器，将自监督语音编码器中的语言与副语言信息分配到不同特征路由。

Details

Motivation: 自监督语音表示常把语言内容与说话人、情绪、韵律等副语言因素纠缠在同一空间，难以有选择地保留或抑制其中一类因素。

Result: 方法对冻结 SPEAR 和 WavLM 表示训练 TopK 稀疏自编码器，并加入路由专属监督与跨因素对抗。独立探针显示因素选择性保留/抑制：语言路由保留更强语言信息；副语言路由保留说话人身份、情绪、韵律等因素，这些因素在语言路由中显著降低。在 LibriSpeech 学得的路由组织可迁移到 MSP-Podcast，无需重训表示侧路由。特征空间路由干预还能迁移被交换因素，同时大体保留未变路由承载的信息。

Insight: 结果在不同编码器、语料、独立探针和表示干预中支持路由选择性分离。摘要未提供探针准确率、泄漏量或稀疏度等具体数值。

Abstract: Self-supervised speech encoders contain linguistic and paralinguistic information in a shared, entangled representation space. We combine a TopK sparse autoencoder with route-specific supervision and cross-factor adversaries. Across frozen SPEAR and WavLM encoders, independent probes show factor-specific retention and suppression: linguistic information remains stronger in the linguistic route, while paralinguistic factors, including speaker identity, emotion, and prosody, are retained in the paralinguistic route and substantially reduced in the linguistic route. The route organisation learned on LibriSpeech persists on MSP-Podcast without representation-side retraining. Feature-space route interventions further transfer the swapped factor while largely preserving the information carried by the unchanged route. These results show consistent route-selective separation across encoders, corpora, independent probes, and representation-level interventions.

[124] Grammar Concept Annotation at Scale: Deployed Fine-Tuned Small Language Models Outperform Prompted Frontier Models cs.CL | cs.AIPDF

Marjan Celikik, Ana Peleteiro Ramallo, Javier Morales

TL;DR: 将教师生成的语法概念标注监督蒸馏到 Qwen3.5 小模型，并部署 0.8B 语法掌握跟踪器，在两个人工基准上超过提示式前沿模型。

Details

Motivation: 课堂即时纠错能促进第二语言学习，但零散反馈很难累计成可操作的语法掌握情况。前沿模型可以从师生对话标注语法概念，但大规模使用成本高。

Result: 作者过滤并重平衡教师标注，微调 Qwen3.5 小语言模型；把标注规范内化到 adapter 后，部署的 0.8B 模型可搭配简洁匹配提示。两个人工整理基准采用逐渐严格的嵌套匹配标准（概念、证据片段、正确性）评估，部署 0.8B 模型和 4B 对照在精确率、召回率上均超过提示式 GPT-5.4 和 GPT-5.6 Sol。0.8B 服务成本约降低 16 倍。在线功能实验中，学习者参与度显著提升 15.8%，预约课时提升 2.1%，新课 GMV 提升 13.2%。

Insight: 把标注契约微调进小模型可减少冗长提示与推理成本；线上实验还报告参与度及业务指标变化。摘要未给出两个人工基准的样本规模或效果置信区间。

Abstract: Corrective feedback is among the best-evidenced drivers of second-language acquisition, yet corrections delivered during lessons rarely accumulate into an actionable view of grammar mastery. Prompted frontier models can provide such a view from learner–tutor lesson transcripts, but they are costly at scale. We close this gap by fine-tuning Qwen3.5 small language models (SLMs) on filtered and rebalanced teacher-generated supervision, then deploying an efficient 0.8B model in an end-to-end grammar mastery tracker for all English learners on our platform. Internalizing the annotation contract into adapter weights enables pairing the 0.8B model with a compact matched prompt rather than verbose instructions. On two human-curated benchmarks, both the deployed 0.8B model and a 4B reference comparator outperform prompted GPT-5.4 and GPT-5.6 Sol in precision and recall under nested matching criteria of increasing strictness: concept, evidence span, and correctness. The deployed 0.8B SLM reduces serving cost by approximately 16$\times$. A feature-level online experiment shows significant gains in learner engagement ($+15.8%$) and key business metrics, including scheduled hours ($+2.1%$) and GMV from new lessons ($+13.2%$).

[125] Lapras: Latent Reasoning for Time Series Language Models cs.CL | cs.LGPDF

Yuliang Chen, Yu Yvonne Wu, Patrick Langer, Arvind Pillai, Sudarshan Regmi

TL;DR: 提出 Lapras，让时间序列语言模型在连续潜空间进行中间推理、仅输出最终文本答案，并以教师—学生自蒸馏降低解释错误和生成成本。

Details

Motivation: CoT 要把高维连续时间信号转写成离散文字，模型可能遗漏关键模式或生成与输入不符的描述；后续推理依赖这些描述，错误会逐步传播。

Result: Lapras 以在参考 CoT 上训练的教师显式推理，再让学生在答案阶段与教师隐状态对齐，将推理能力转为时间序列—语言联合潜空间计算。四种 TSLM 骨干、五个时间序列问答基准上，相比显式 CoT 平均 F1 最高提升 10.79%，生成 token 少 23.9 倍。连续思维还能用标准语言解码转成可读推理文本。

Insight: 连续潜在推理减少中间信号离散化与显式生成开销，同时保留可解码解释。摘要报告最高增益和整体 token 倍数，没有给出不同基准逐项结果。

Abstract: Time Series Language Models (TSLMs) offer a promising path toward time series understanding by reasoning over temporal signals and producing natural language answers and explanations. A common approach is Chain-of-Thought (CoT), which generates step-by-step rationales linking relevant signal patterns to final answers. Although these models learn from reference CoT traces during post-training, generating faithful descriptions of input time series at inference remains challenging. Expressing high-dimensional, continuous temporal representations in discrete language tokens may cause the model to neglect task-relevant patterns or describe them inaccurately. Because later reasoning steps build on these descriptions, early errors propagate, leading to incorrect answers with plausible explanations that are inconsistent with the input signal. We propose Lapras (Latent Post-trained Reasoning Across Series), a post-training framework that equips TSLMs with latent reasoning. A model trained with Lapras reasons through a sequence of continuous thoughts in the joint time series-language space, producing text only for the final answer. It learns this through teacher-student self-distillation, where a teacher trained on CoT reference traces reasons explicitly through text. The student aligns its hidden states with the teacher’s at the answer stage, transferring the teacher’s reasoning ability into its latent computation. We evaluate Lapras across four TSLM backbones on five time series question answering benchmarks. Lapras improves average F1 by up to 10.79% over explicit CoT while generating 23.9x fewer tokens. Lapras’s continuous thoughts can also be decoded into readable reasoning traces via standard language decoding, preserving textual explanations. Together, these results highlight Lapras as a promising post-training paradigm for efficient, effective, and interpretable TSLM reasoning.

[126] RAG-Stress: Probing the Limits of Evidence Reliance in Retrieval-Augmented Generation cs.CL | cs.AIPDF

Shunyuan Zhou, Hao Chen, Tianyu Wang, Goose Lin, Zaiyuan Wang

TL;DR: 提出 RAG-Stress 受控诊断协议，检验误导性检索证据何时让原本答对的问题被模型改成错误答案，并区分证据遵循与事实可靠性。

Details

Motivation: 模型跟随检索证据不必然保证事实正确：误导内容可能使原先答对的问题被改错。标准准确率将改答错误和原有错误混在一起，遮蔽这一行为。

Result: 协议固定问题和标准答案，仅改写一条断言使其支持指定错误答案，并组合两种来源优先级政策与答案片段在证据中的开头/中部/结尾三种位置。对每模型无检索时答对的问题子集计算误导率，同时报告全评测集干净准确率。评估 15 个 API、开源和 RL 搜索代理系统，涵盖 TriviaQA-RC、HotpotQA、SearchQA，另测英语和中文 MedQA。要求优先信任文档的指令比允许使用先验知识的指令造成更高误导率；三个 QA 基准上跨模型和位置的差距均值为 10.9–13.5 个百分点。两种政策下平均误导率顺序为末尾 > 开头 > 中部，但个别模型并非都如此。另对 500 个问题和两个检查点做配对审计，支持有害覆盖增加，但未证明有益纠错也相应提高。

Insight: RAG 评估需记录证据让模型保留、替换还是纠正原答案，而不能把“遵循证据”当作事实准确。证据位置会影响覆盖行为，且不同模型并不完全遵循平均趋势。

Abstract: Following retrieved evidence does not guarantee factual correctness: misleading evidence can induce a model to replace an answer it previously gave correctly. Standard accuracy measures obscure this behavior by combining answer replacement with preexisting errors. We introduce RAG-Stress, a controlled diagnostic protocol for examining the limits of evidence reliance in retrieval-augmented generation. The protocol holds the question and reference answer fixed, edits one assertion to support a designated incorrect answer, and crosses two source priority policies with three positions of the answer span within the evidence text. We measure misleading rate (MR) on each model’s subset of questions answered correctly without retrieval, alongside clean accuracy on the full evaluation set. We evaluate fifteen systems spanning API models, open models, and search agents trained with reinforcement learning on TriviaQA-RC, HotpotQA, and SearchQA, with additional English and Chinese MedQA evaluations. Instructions that prioritize documents consistently produce higher MR than those permitting reliance on prior knowledge. Averaged over models and positions, the gap ranges from 10.9 to 13.5 percentage points across the three QA datasets. Mean MR follows End $>$ Beginning $>$ Middle under both policies, although individual models do not uniformly follow this ordering. A separate paired audit of 500 questions and two checkpoints supports increased harmful override without establishing a corresponding improvement in beneficial correction. These findings distinguish evidence adherence from factual reliability and motivate evaluating whether retrieved evidence preserves, replaces, or corrects a model’s answers.

[127] BeliefScope: Diagnosing Evidence-Driven Revision and Pressure-Induced Shifts in Large Language Models cs.CLPDF

Shuai Guo, Yidong Cui

TL;DR: 提出 BeliefScope，一个黑盒受控框架，区分模型因相关证据而修正观点和因无新事实的用户方向性施压而改变回答。

Details

Motivation: 模型对同一命题的回答变化可能由真实相关证据或仅施加方向压力的用户提示导致；仅观察回答改变无法判断影响来源。

Result: 框架围绕固定目标命题，交叉操作证据与压力，并分别设置局部控制；在概率报告、类别判断和行动建议等适配通道中测量回答变化。作者先用可控合成条件检验观察设计的可归因性，再用已知真值恢复、针对性消融和半合成压力测试分析噪声与异质性影响。覆盖 36 个 Qwen/Llama 系列组合，并对包含 Gemma3-12B 的 12 个模型族做专项检查。响应画像高度依赖评估上下文：匹配控制、解码或接口变化可改变广义模型差异，但部分较窄的模型内模式较稳定。降低对目标方向压力的服从，可能表示稳定抵抗，也可能是回答转向相反方向。

Insight: 研究强调对模型回答变化的来源归因必须绑定具体测试条件，并为结论设有效边界；“更少随压力改答”本身不能区分稳健与反向偏移。摘要未给出跨模型效应大小。

Abstract: A language model may revise the same proposition after receiving genuinely relevant evidence or after receiving directional user pressure that adds no relevant fact. The observable response shift alone therefore does not reveal which source drove the change. We introduce BeliefScope, a controlled black-box framework for separating these two sources of influence around a fixed target proposition. BeliefScope crosses Evidence and Pressure with factor-specific local controls and measures response changes through probability reports, categorical judgments, and action recommendations on channel-appropriate scales. To determine when these observable contrasts support reliable attribution, we evaluate the observation design under controlled synthetic conditions. Known-truth recovery and targeted ablations establish where Evidence- and Pressure-related effects can be separated, while semi-synthetic stress tests map how that recoverability changes as the observation process becomes noisier and more heterogeneous. Across a 36-family Qwen/Llama study, with targeted 12-family checks that also include Gemma3-12B, the resulting profiles show substantial evaluation-context dependence: broad model-level differences can change under matched controls, decoding, or response interfaces, while some narrower within-model patterns remain stable. Instruction interventions further show that reduced target-aligned Pressure following can reflect either stable resistance or movement in the opposite direction. BeliefScope summarizes these measurements as a conditional belief-response profile that keeps diagnostic effects tied to the evaluation conditions under which they are observed, together with explicit validity boundaries for that diagnosis.

[128] From Retrieval to Reconstruction: Constructing Evolvable Cognitive Memory for Long-Term Dialogue cs.CLPDF

Zirui Liao, Zhengxian Wu, Zhuohong Chen, Yunyao Yu, Xiaoyu Liu

TL;DR: 提出 CogMem，以 PEC²F 图结构保存来源明确的主张、事实和事件，并通过确定性图算子重建长期对话中的查询相关记忆。

Details

Motivation: 长期对话代理需要可靠地跨多轮推理，但传统 RAG 把记忆当作被动存储，难区分带来源的主观信念与无来源事实/事件，也难连接分散在不同会话中的证据。

Result: CogMem 采用 Person-Event-Concept-Claim-Fact 图模式；Claim 节点保留主观陈述的来源和对象，Fact 与 Event 节点分别表示语义和情景知识。对话轮次增量转成带来源记录、合并成高层事实，并在同一来源相互矛盾时整合为有时间范围的 Claim 视图。检索端由 LLM 意图解析驱动规则控制器，组合锚定、遍历、交集和证据 grounding 四种确定性图操作。LoCoMo 和 LongMemEval 实验表现强，尤其在多跳、时间和知识更新任务上；消融及语义坍缩探针支持认知区分、信息合并和代理式检索的互补贡献。

Insight: 显式保留说话来源、命题对象与时间范围，有助于长期记忆处理相互冲突的说法，而非把对话压成无来源事实。摘要未提供各基准的具体分数。

Abstract: Large Language Models (LLMs) serving as long-term dialogue agents require memory systems that support reliable reasoning over extended interactions. However, existing Retrieval-Augmented Generation (RAG) frameworks typically treat memory as passive storage, making it difficult to distinguish source-attributed beliefs from unattributed event/fact records and to connect evidence dispersed across sessions. We introduce CogMem, a cognitive memory architecture based on the PEC$^2$F (Person-Event-Concept-Claim-Fact) graph schema. Dedicated Claim nodes preserve the source and target of subjective statements, while Fact and Event nodes represent semantic and episodic knowledge. Dialogue turns are incrementally converted into provenance-aware graph records, consolidated into higher-level facts, and reconciled into temporally scoped Claim views when the same source provides conflicting updates. For retrieval, a rule-based controller driven by LLM intent parsing composes four deterministic graph operators—anchoring, traversal, intersection, and evidence grounding—to reconstruct query-relevant context. Experiments on LoCoMo and LongMemEval show strong performance, especially on multi-hop, temporal, and knowledge-update tasks. Ablations and a semantic-collapse probe support complementary contributions from epistemic separation, consolidation, and agentic retrieval. Code: https://github.com/Silent-Rain02/CogMem.

[129] ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery cs.CLPDF

Houcheng Jiang, Mao Zheng, Mingyang Song, Qiyong Zhong, Jie Sun

TL;DR: 提出 RECAL，在结构化剪枝前依据未剪枝教师与剪枝探针的前向 KL 调整校准统计，使模型保留有利于后续在线策略蒸馏恢复的预测。

Details

Motivation: 结构化剪枝降低推理模型部署成本，却可能损害能力，使随后依赖学生自生成轨迹的在线策略蒸馏（OPD）难以恢复；离线蒸馏后仍残留的剪枝损伤会限制恢复上限。

Result: RECAl 是可插拔的恢复感知校准方法：用未剪枝教师与剪枝探针间前向 KL 识别剪枝破坏的教师支持预测，再重加权校准统计，引导已有剪枝准则保留这些预测。在多模型和多种剪枝法上，OPD 后数学推理表现持续改善，AIME 最高提升 16.7 个百分点，代码生成多数比较也提升。进一步分析显示，RECAL 降低严重受影响 token 上的残留损伤，且经过恢复训练仍保有性能优势。

Insight: 剪枝校准目标应考虑后续学生在线恢复过程，而不只看剪枝后即时性能；提前保留教师认为重要的预测有利于 OPD。AIME 提升为最高值，摘要没有列模型和剪枝配置对应表。

Abstract: Structured pruning reduces the deployment cost of reasoning language models, but the resulting capability degradation can hinder subsequent on-policy distillation (OPD) recovery. Because OPD relies on student-generated trajectories, pruning damage that persists after offline distillation can limit its effectiveness. We propose RECAL, Recovery-Aware Calibration, a simple plug-and-play approach that improves OPD recovery by adjusting calibration before pruning. RECAL uses forward KL between an unpruned teacher and a pruned probe to identify teacher-supported predictions disrupted by pruning, then reweights calibration statistics to guide existing pruning criteria toward preserving these predictions. Across multiple models and pruning methods, RECAL consistently improves mathematical reasoning after OPD, achieving gains of up to 16.7 percentage points on AIME, alongside improvements in most code-generation comparisons. Further analysis shows that RECAL reduces residual damage at heavily affected tokens and establishes performance advantages that persist through recovery. These results demonstrate the value of recovery-aware calibration for improving on-policy distillation recovery of pruned reasoning models.

[130] MetaEncoder: Exploring the Limit of Bi-Encoders for Multimodal System One Decision Making with Natural Language Interface cs.CL | cs.AIPDF

Jianpeng Cheng, Guangyu Sun, Aashu Singh, Benyu Zhang, Haixing Dai

TL;DR: 提出 MetaEncoder，把预训练 Muse-Glimmer 30B 解码器微调成双编码器，以自然语言统一表达请求和候选项并扩展到多模态封闭/开放候选决策。

Details

Motivation: System One 模型输出受约束决策或概率分布而不是自由文本，常用结构化 schema 编码状态、意图和候选。论文重新探索纯自然语言接口，让用户请求和候选选择都以自然语言表示，同时接入图像/视频辅助信息。

Result: MetaEncoder 通过单向对比学习训练请求—候选对齐的双编码器，面向小型封闭候选集（<256）及百万级开放候选检索。评估覆盖 11 个基准套件、190 项任务，包括多模态决策、理解和检索，展示其在哪些任务超过 SOTA 多模态编码器，也识别其在推理密集任务上的限制。摘要未列具体指标或胜出任务比例。

Insight: 将决策接口统一为自然语言可能兼顾可扩展候选检索和图像/视频上下文，但双编码器的匹配范式在复杂推理任务仍有局限。

Abstract: System One models output constrained decisions and probability distributions rather than free-form text generation. While prevailing paradigms rely on structured schema objects to encode state, intent, and candidate choices, we revisit a fully natural language-based System One interface. In this framework, both the user request and each candidate option are expressed in natural language, supported by multimodal (image and video) auxiliary inputs. We introduce MetaEncoder, which fine-tunes a pre-trained Muse-Glimmer 30B decoder into an instruction-following decision-making encoder. To scale effectively across both small closed-set (< 256) and massive open-set (millions) candidate spaces, MetaEncoder employs a bi-encoder architecture trained via unidirectional contrastive learning for request-candidate alignment. We conduct extensive evaluations across 11 benchmark suites and 190 tasks spanning multimodal decision-making, understanding (closed-set) and retrieval (open-set), highlighting where MetaEncoder beats SOTA multimodal encoders, as well as its current limits on reasoning-intensive tasks.

[131] SignRAG: Unified Retrieval-Augmented Gloss-Free Sign Language Translation cs.CL | cs.CV | cs.MMPDF

Zhi Rao, Yucheng Zhou, Qianran Sun, Yiqing Huang, Longcan Yuan

TL;DR: 提出 SignRAG，将分层视觉语言预训练、目标域检索库和检索效用强化微调结合，用于无 gloss 手语翻译并兼容 decoder-only LLM。

Details

Motivation: 无 gloss 手语翻译既有预训练主要针对 encoder-decoder 语言模型，难以直接迁移至当前 decoder-only LLM。下游翻译也可能从目标域相似实例获得提示，但检索上下文既可能有益也可能误导。

Result: SignRAG 先学习语言学 grounding 的手语表示，再联合对齐手语编码器与 LLM，以减轻跨模态优化不平衡。下游适配除参数微调外，还构建目标域检索图库，提供实例级翻译线索。检索效用引导强化微调 RUG-RFT 将翻译质量奖励与检索效用奖励结合，鼓励有益检索并抑制有害依赖。多个手语翻译基准上达到新 SOTA；作者称其是首个在 CSL-Daily 所有报告指标上超过有 gloss 监督方法的无 gloss 方法。

Insight: 手语翻译可结合目标域样例检索，但需显式学习何时采用检索结果；分层预训练使框架适配 decoder-only LLM。摘要未提供各基准分数。

Abstract: Contemporary decoder-only large language models (LLMs) have demonstrated strong capabilities across a wide range of domains. However, existing pretraining paradigms for gloss-free sign language translation (SLT) are largely designed around conventional encoder-decoder pretrained language models, which limits their direct applicability to decoder-only LLMs. To address this limitation, we propose SignRAG, a unified framework combining hierarchical pretraining, target-domain retrieval augmentation, and retrieval-aware reinforcement fine-tuning. Hierarchical pretraining first learns linguistically grounded sign representations and then jointly aligns the sign encoder with an LLM, mitigating cross-modal optimization imbalance. For downstream adaptation, SignRAG complements parameter-based fine-tuning with a target-domain retrieval gallery that provides instance-specific translation cues. To ensure that retrieved contexts are used appropriately, we further introduce Retrieval Utility-Guided Reinforcement Fine-Tuning (RUG-RFT), which combines translation-quality and retrieval-utility rewards to encourage beneficial retrieval use while suppressing harmful reliance. Experiments on multiple SLT benchmarks establish new state-of-the-art performance. In particular, to the best of our knowledge, SignRAG is the first gloss-free approach to outperform gloss-supervised methods across all reported metrics on CSL-Daily. Our code has been released at \href{https://github.com/shahelaojieraozhi/SignRAG}{GitHub}, together with models of different sizes to support future academic research.

[132] AdaptEvo: Adaptive Agent Learning with Evolving Supervision cs.CLPDF

Shijun Wan, Jiancong Xie, Hang Xu, Jin Duan, Qixiong Wang

TL;DR: 提出 AdaptEvo，通过按参考置信度调节过程与结果奖励，并从训练失败中演化决策知识和评估规则，应对规则不完备及监督判断质量不一的任务。

Details

Motivation: 规则约束的上下文决策需要把书面规则应用到个案证据，但规则可能遗漏流程指导，参考判断也可能与规则/证据支持程度不同。

Result: 训练模块 CA-GRPO 按参考置信度平衡结果奖励与过程奖励；演化模块从反复出现的错误中提炼可复用决策知识，并更新流程评分规则以发现遗漏错误。作者构建工业多模态内容审核数据集，包括训练集、规则不变的 In-Period 和规则变化后的 Out-of-Period 测试集。Qwen3.6-35B-A3B 在 In-Period exact-label 准确率 61.9%、二元决策准确率 72.2%，比 GRPO 分别高 7.5 和 3.7 个百分点。Out-of-Period 中，未注入演化决策知识的 CA-GRPO 策略在所测检查点均保持相对基础模型 exact-label 优势，而 GRPO 随训练继续下降；CA-GRPO 两项指标均优于所测固定奖励混合。

Insight: 按参考判断可信度调配奖励可改善规则变化下的稳健性；该结果与失败知识注入区分开，说明 CA-GRPO 本身在外期测试仍有增益。摘要未给 Out-of-Period 精确分数。

Abstract: Rule-governed contextual decision tasks require models to apply specified rules to case-specific context and evidence. Written rules can leave gaps in decision guidance and process evaluation, while reference judgments vary in their support from the rules and evidence. To address these challenges, we introduce AdaptEvo, a framework for learning under imperfect supervision that couples confidence-adaptive policy optimization with evolving decision knowledge and evaluation rubrics. Its Training module uses Confidence-Adaptive GRPO (CA-GRPO) to balance outcome and process rewards according to reference confidence. Its Evolution module synthesizes reusable decision knowledge from recurring failures across training cases and refines process rubrics to detect overlooked errors. To support empirical evaluation, we construct an industrial multimodal content moderation dataset comprising a training set and In-Period and Out-of-Period test sets, with the latter collected under changed rules. Using Qwen3.6-35B-A3B, AdaptEvo achieves 61.9% exact-label accuracy and 72.2% binary decision accuracy on In-Period, exceeding GRPO by 7.5 and 3.7 percentage points, respectively. On Out-of-Period, the policy trained with CA-GRPO retains exact-label accuracy gains over the base model across evaluated checkpoints without injected decision knowledge, while GRPO declines with continued training. CA-GRPO also outperforms the tested fixed reward mixtures on both Out-of-Period metrics.

[133] Beyond Sequences: Distilling Structured Decision Memory for LLM Recommendation cs.CLPDF

Leikun Liang, Guoshuai Wang, Xingsheng He, Yushan Han, Yunyi Xuan

TL;DR: 提出 MARI，以结构化决策记忆保存用户的目标、约束和权衡，检索历史理由辅助 LLM 推荐，重点改善相似物品间的困难选择。

Details

Motivation: 推荐系统常将异质用户行为展平成同类 token 序列，忽略观看、购买等行为的不同决策角色，也难表示价格—质量权衡等语境，导致相似商品的困难选择效果下降。

Result: MARI 建立决策记忆库，归档离线后验蒸馏得到的结构化决策记忆，简洁记录用户目标、约束和权衡，这些记忆由多种行为和用户生成内容提取。在线时检索相关记忆增强 LLM 推理，避免处理长原始序列的高成本。广泛实验显示，MARI 显著优于标准下一物品预测和新提出的困难选择预测任务中的 SOTA 基线，同时在线延迟开销较低；定性分析产生可读、可操作的用户决策洞察。摘要未列具体指标。

Insight: 把历史行为整理为可读决策理由，比直接拼接长序列更易解释复杂取舍，并将昂贵记忆构建放到离线完成。摘要未描述数据来源和用户隐私处理。

Abstract: Despite the adoption of large language models (LLMs) in recommendation systems, prevailing approaches mostly model single-type behaviors (e.g., views or purchases). Even when incorporating multiple behaviors, existing methods flatten heterogeneous actions into homogeneous token sequences, ignoring their distinct decision-making roles. This flattening fails to capture semantic hierarchies and contextual nuances in complex decision-making, such as trade-offs between price and quality. Consequently, performance degrades in critical ``difficult-choice’’ scenarios involving highly similar items. To bridge this gap, we propose MARI (Memory-Augmented Recommendation with Interpretability), which grounds predictions in explicit, structured decision evidence. MARI maintains a Decision Memory Bank (DMB) that archives users’ past rationales as Structured Decision Memories (SDMs): concise records of goals, constraints, and trade-offs. These SDMs are generated offline via Post-Hoc Decision Distillation from heterogeneous behaviors and user-generated content. By retrieving relevant SDMs to augment LLM reasoning, MARI achieves interpretability and scalability without the prohibitive cost of processing long raw sequences. Extensive experiments show MARI significantly outperforms state-of-the-art baselines on standard next-item prediction and a newly introduced Difficult Choice Prediction task, incurring low latency overhead by decoupling memory construction from online inference. Qualitative analyses reveal actionable, human-readable insights into user decision-making, marking a concrete step toward reasoning-aware recommendation systems.

[134] SAIL: Scientific Agentic Intelligence via a Science-Aware Loop cs.CL | cs.DL | cs.IRPDF

SAIL Model Team, Boyuan Sun, Bryan Dai, Che Liu, Chi Liu

TL;DR: 介绍 SAIL，35B 总参数、3B 激活参数的科学研究开放模型，并通过科学感知改进循环从失败诊断中生成训练任务、逐轮提升研究能力。

Details

Motivation: 文献调研、科学编程和多步研究流程需要多类能力，单纯增加模型规模或泛化训练未必弥补证据筛选、科学假设、计划与修订上的具体短板。

Result: 由前沿模型代理分析 SAIL 的任务失败并构造针对性训练任务：检查文献检索和证据选择、科学代码中的假设与推理，以及长程研究的规划和修订。代理利用论文库和科学代码仓库构建问题、交互轨迹和可执行任务环境。多轮迭代后，SAIL 通过监督微调、专家训练、多教师在线策略蒸馏和代理强化学习训练。作者报告其在多项科学研究任务上达到有竞争力的表现，参数量显著少于领先开放权重模型；摘要未给出基准和具体分数。

Insight: 改进循环将代理用于能力诊断和训练环境构建，并反复依据失败补齐短板；摘要强调开放模型效率，但不足以判断不同科学任务的具体强弱。

Abstract: We introduce SAIL, an open model with 35B total and 3B active parameters for literature research, scientific coding, and multi-step research workflows. SAIL is developed through a science-aware improvement loop: agents built on frontier AI models analyze its task failures and construct training tasks that address the underlying capability gaps. The diagnosis examines search and evidence selection in literature tasks, scientific assumptions and reasoning in coding, and planning and revision in longer investigations. The agents draw on paper collections and scientific code repositories to build problems, interaction trajectories, and executable tasks with the required environments and tools. We repeat this loop over multiple development cycles and train SAIL through supervised fine-tuning, specialist training, multi-teacher on-policy distillation, and agentic reinforcement learning. SAIL achieves competitive performance across scientific research tasks with substantially fewer parameters than leading open-weight models.

[135] Residual Advantage: Student-Relative Teacher Guidance for RL with Verifiable Rewards cs.CLPDF

Xiaobing Chen, Zhiqi Pang

TL;DR: 提出 Residual Advantage（RA），将有界师生概率残差作为逐步奖励并按每条回答中心化，仅在回复内部重分配验证器优势；CoRA 进一步在线适配教师。

Details

Motivation: RLVR 只给整条回答一个结果标签，缺少步骤级信用分配；OPD 虽在学生访问的前缀上提供 token 信号，但逐点监督不直接刻画整个词表中的师生分歧。无界对数比监督还可能让教师过度主导，而教师未必适合指导偏离其路径的学生。

Result: RA 将师生概率残差转成有界单步奖励，以学生策略下状态价值为基线形成优势，并在每条回答内中心化后加到验证器优势上。因此引导项每条回答均值为零，保留回复平均标签，只在步骤间重新分配信用。CoRA 用同一批带验证分数的学生样本及验证器优势更新教师 LoRA，并将更新教师用于下一轮残差引导。在 Qwen3-1.7B/4B 学生与 Qwen3-8B 教师的三项数学基准上，RA 与 GRPO 或 REINFORCE++ 组合在全部 24 个比较中改善原算法；macro Avg@8 提升 1.7–3.6 分、Pass@8 提升 3.9–6.3 分，均超过仅教师 OPD；CoRA 再额外提升 Avg@8 1.0–1.5 分。

Insight: 零均值残差优势只调整同一回复内的步骤信用，不改变验证器对回答整体质量的平均判断；CoRA 让教师适配学生实际尝试。结果覆盖三个数学基准和两种序列优势算法。

Abstract: Reinforcement learning with verifiable rewards (RLVR) and on-policy distillation (OPD) have become two main paradigms for post-training reasoning models. RLVR gives each response a single outcome label, leaving the steps inside it without separate credit. OPD provides token-level guidance at student-visited prefixes, but its pointwise signal does not directly reflect the pattern of teacher–student disagreement across the vocabulary. Dense, unbounded log-ratio supervision can amplify the teacher’s influence, yet a strong solver is not necessarily a suitable guide when the student’s solution paths depart from the teacher’s. We propose Residual Advantage (\RA{}), which treats the teacher–student probability residual as a bounded one-step reward, subtracts the corresponding state value under the student policy to form a standard advantage, and centers the result within each response before adding it to the verifier advantage. The guidance term has zero mean within each response, so the verifier advantage remains the response’s mean label and the teacher only redistributes credit among the steps within it. \CoRA{} further updates a teacher LoRA with verifier advantages on the same scored student batch and uses the updated teacher in the next iteration’s residual, adapting guidance to the student’s attempts. With Qwen3-1.7B-Base and Qwen3-4B-Base students and a Qwen3-8B teacher, \RA{} combined with GRPO or REINFORCE++ improves the underlying sequence-advantage algorithm in all 24 comparisons on three mathematical benchmarks, raising macro Avg@8 by 1.7–3.6 points and Pass@8 by 3.9–6.3 points. Both combinations surpass teacher-only OPD, and \CoRA{} adds a further 1.0–1.5 Avg@8 points.

[136] Learning the Loop, Not Just the Page: Execution-Grounded Loop Learning for Web Generation cs.CLPDF

Yuxin Meng, Ruixu Zhang, Junjie Wang, Yuhan Suo, Yuhan Sun

TL;DR: 提出 WebLoop，在共享策略中联合学习网页生成、评论和修订，通过需求区分能力及下游帮助度间接训练无可执行结果的 Critic。

Details

Motivation: 可执行奖励的网页生成研究主要优化最终页面，较少研究如何诊断和修复不完善实现。Generator 和 Refiner 有直接环境回报，但 Critic 只能影响后续行为，没有可直接执行的结果作为奖励。

Result: WebLoop 用需求级可判别性和下游有帮助程度作为互补信号训练不执行代码的 Critic，先建立可靠诊断，再引入后果感知信用分配；生成、评论、修订三个角色通过组相对策略学习联合优化。Qwen3.5-9B 在 WebRise 获得 Overall 41.5，在 WebGen-Bench 获得 38.9% 准确率，较基础模型分别高 11.3 和 15.4 分。收益迁移到首次生成，在 27B 规模仍存在，并能从纯文本训练泛化到多模态输入。控制实验排除了仅多做一次修订即可解释全部增益的可能。

Insight: 评论器没有直接执行结果，因此其训练需要将诊断质量与下游修订收益联系起来；联合学习闭环本身对效果有贡献。摘要未提供各基准的其他分项指标。

Abstract: Functional Web generation is increasingly optimized with executable rewards, yet existing methods largely focus on the quality of the final page and leave the process of diagnosing and repairing imperfect implementations underexplored. We identify a central challenge in this setting: the Generator and Refiner produce executable artifacts with direct environment rewards, whereas the intermediate Critic influences downstream behavior without a directly executable outcome. We introduce WebLoop, an execution-grounded framework that jointly learns generation, critique, and refinement within a shared policy. WebLoop trains an execution-free Critic with complementary signals for requirement-level discriminability and downstream helpfulness, first establishing reliable diagnosis and then introducing consequence-aware credit, while all three roles are jointly optimized with group-relative policy learning. With Qwen3.5-9B, WebLoop reaches 41.5 Overall on WebRise and 38.9% accuracy on WebGen-Bench, improving the base model by 11.3 and 15.4 points, respectively. The gains transfer to first-pass generation, persist at 27B scale, and generalize from text-only training to multimodal inputs. Controlled analyses further show that the improvement cannot be explained by an additional refinement pass alone, highlighting the importance of learning the Critic and the loop itself.

[137] Smoothing the Top-k Exposure Boundary for Sparse Mixture-of-Experts cs.CL | cs.LGPDF

Yunkai Chai, Tong Zhu, Xiaoye Qu, Xuyang Hu, Guanjie Chen

TL;DR: 提出弹性专家路由，在固定 top-k 附近随机采样激活专家数，将专家监督边界从硬阈值变为局部平滑分布，同时保持期望计算量与推理预算。

Details

Motivation: 稀疏 MoE 传统训练固定选择 top-k 专家，使连续路由分数突然变成全监督或零反馈。分数略微波动就可能把竞争力接近的专家放在边界两侧，形成脆弱的曝光差异。

Result: Elastic Expert Routing 从以 k 为中心的局部离散分布随机采样活跃专家预算；对称采样邻域使期望计算成本匹配确定性路由，推理预算保持不变。监督微调中，OLMoE-1B-7B 与 Qwen3-30B-A3B 的下游宏平均分别提升 0.84 和 2.02 分；从头预训练时，下游任务平均较固定 top-k 基线高 1.6 分。

Insight: 训练期随机化使边界附近专家逐步获得反馈，不改变平均训练算力或推理时专家数。摘要没有报告具体数据集、评估方差或路由分布宽度。

Abstract: Sparse Mixture-of-Experts models scale parameter capacity efficiently while maintaining a fixed compute budget per token. However, traditional training paradigms enforce a static choice of top-$k$ experts, which converts a continuous routing distribution into a rigid step function. This constraint introduces a brittle boundary where highly competitive experts are arbitrarily separated into full-supervision and zero-feedback zones based on minor score fluctuations. To address this issue, we propose Elastic Expert Routing, which stochastically samples the active expert budget from a localized discrete distribution centered at $k$. Over multiple training iterations, this mechanism softens the sharp threshold into a gradual probability distribution. Because the sampling neighborhood remains symmetric, this approach matches the expected computational cost of deterministic training, while preserving the inference budget. Extensive experiments demonstrate the efficacy of our method on both supervised fine-tuning and from-scratch pretraining settings. During supervised fine-tuning, elastic routing improves downstream macro-averages on OLMoE-1B-7B and Qwen3-30B-A3B by $+0.84$ and $+2.02$ points, respectively. In addition, in from-scratch pretraining, it outperforms the static top-$k$ baseline by $1.6$ points on average across downstream tasks.

[138] Measuring Cultural Alignment Beyond the Average: A Framework for Evaluating Maternal-Health LLM Interactions in Indian Contexts cs.CLPDF

Umaira Izhar, Gunjan Arora, Pushpendra Singh

TL;DR: 提出 MH-INDIC，用十维文化行为框架评估印度北部城市及半城市孕产健康 LLM 交互，区分总体分布匹配和人口/家庭画像条件下的敏感性。

Details

Motivation: 医疗 LLM 评估多关注事实、安全与流畅度，较少检查生成交互是否反映文化情境中的健康推理；孕产健康决策尤其受到社会关系和规范影响。

Result: MH-INDIC 将文化行为操作化为十个孕产健康推理维度，并用北印度城市与半城市 102 名孕期及产后女性的 26 项调查评估十个 LLM。若干模型能近似人群总体分布，但所有模型在不同人口及家庭画像上的行为变化都明显小于人类样本，显示总体对齐不代表画像条件敏感性。作者再用对齐度最高的闭源和开源模型，在零样本、自我条件化和人类画像 grounding 提示下生成文化条件化对话；人类 grounding 提升了画像对齐和对话质量评分。摘要未给出具体评分数值。

Insight: 评估文化对齐不能只比较总体平均，还要测量不同人口和家庭情境下的行为变化；人类画像提示可改善个体化文化 grounding。样本来自北印度特定城市/半城市人群，适用范围有限。

Abstract: Existing evaluation methods for healthcare LLMs primarily assess factual correctness,safety, and fluency, while providing limited insight into whether generated interactions reflect culturally situated healthcare reasoning. This limitation is particularly important in maternal health, where care decisions are shaped by social and relational norms. We introduce MH-INDIC, a culturally grounded evaluation framework for maternal-health interactions in urban and semi-urban North Indian contexts that operationalises cultural behaviour through ten dimensions of maternal-health reasoning. Using a 26-item survey administered to 102 pregnant and postpartum women from urban and semi-urban North India, we evaluate ten LLMs. We distinguish population level cultural alignment from profile-level behavioural variation. Although several models approximate the human population-level distribution, all evaluated systems exhibit substantially lower variation across demographic and household profiles than the human cohort, revealing a gap between aggregate alignment and profile-conditioned sensitivity. As a downstream application of MH-INDIC, we use the strongest-aligned proprietary and open-source models to generate culturally conditioned maternal-health dialogues under zero-shot, self-conditioned, and human-grounded prompting. Human-grounded conditioning produces stronger profile alignment and dialogue quality ratings, suggesting that measured cultural profiles can improve the cultural grounding of generated interactions

[139] Probing for Long-Horizon Deductive Reasoning Capabilities in Language Models with Prolog cs.CLPDF

Hadeel Al-Negheimish, Jasna Ilieva, Yoon Kim

TL;DR: 提出 ProloNg 合成长程 Prolog 演绎基准，测试语言模型在递增推理深度和长上下文下是否能超越简单检索。

Details

Motivation: 前沿 LLM 理论上可处理百万 token 上下文，但长上下文是否能支持深层推理而不只是信息检索仍待验证。

Result: ProloNg 系统改变 Prolog 问题的演绎深度，最难案例深度为 22、上下文约 62K token。对来自五个前沿模型家族的八个推理模型评测后，随着推理深度增加，性能显著下降；多数模型在深度超过 10 后接近随机猜测。摘要未提供各模型的逐深度准确率。

Insight: 长上下文窗口本身并不保证长程演绎能力；该基准通过可控逻辑深度将上下文长度和推理链难度联系起来。

Abstract: Current frontier LLMs can theoretically process long contexts with 1M tokens or more. But to what extent can they go beyond simple retrieval and perform deeper reasoning over such long contexts? We empirically investigate long-horizon reasoning capabilities of LLMs, focusing on deductive logic expressed in Prolog. We construct ProloNg, a synthetic testbed to probe Prolog Long Reasoning, which systematically varies the complexity (reasoning depth) of problems, where the hardest case has a reasoning depth of 22 and 62k context length. We study 8 reasoning models across 5 families of frontier LLMs, and find that performance degrades substantially as reasoning depth grows, with the majority of models approaching chance beyond depth 10.

[140] Large Language Model Turnover Undermines Screening for Artificial Intelligence-Assisted Scientific Writing cs.CL | cs.AI | cs.CY | cs.DL | cs.SIPDF

Kazuki Nakajima, Takayuki Mizuno

TL;DR: 量化 LLM 版本更替对科学论文 AI 写作筛查器的影响，发现检测器的校准与召回会在模型代际边界急剧失效。

Details

Motivation: 期刊和会议开始筛查 LLM 撰写文本，筛查可靠性常通过固定版本基准评估；实际使用的 LLM 不断更新，旧版本上测得的准确度可能不适用于新版本。

Result: 作者取 4,000 篇 ChatGPT 前的 PNAS 摘要，并收集三个供应商于 2023 年 6 月至 2026 年 8 月发布的 23 个 LLM 改写版本；比较从每次更新都重训到完全不更新等检测器维护策略。只用供应商旧版本训练的检测器可在代际边界失效：在最明显边界前，假阳性率校准至 1% 时可检出 99% 以上改写；刚跨过边界后仅检出 3.8%。用后续版本训练的检测器也可能漏检早期版本改写。两个模拟筛查策略要么误标约八分之一人类摘要，要么漏掉最新版本约三分之一改写。商业检测器也在边界后版本上漏掉大多数改写，同时几乎不误报人类文本。

Insight: 检测器准确率应被视为随模型版本变化的临时结果，每有新版本发布就需重新验证，也应回测旧版本。词汇差异大体追踪了检测迁移成功或失效的位置。

Abstract: Journals and conferences have begun to screen submitted manuscripts for text written using large language models (LLMs). The reliability of this screening rests on benchmark evaluations against a fixed set of LLM versions, while the versions in actual use keep changing. Here we quantify how this LLM turnover affects the screening of scientific manuscripts. We paired 4,000 pre-ChatGPT abstracts from the Proceedings of the National Academy of Sciences with their rewrites by 23 LLM versions from three vendors, released between June 2023 and August 2026. We then trained detectors under maintenance scenarios ranging from a detector retrained on every new version to one trained once and never updated. Detectors trained only on a vendor’s past versions can collapse at the boundaries between model generations: calibrated to falsely flag 1% of human-written abstracts, they catch above 99% of rewrites just before the sharpest boundary and 3.8% just after it. Detectors trained on later versions can also miss rewrites of earlier ones. Vocabulary differences between versions largely track where detection transfers and where it fails. In the two screening scenarios we simulated, screens covering all 23 versions either flagged one in eight human-written abstracts or missed one in three rewrites of the newest version. Indeed, a commercial detector missed most rewrites of the version just after the sharpest boundary while flagging almost no human-written abstracts. Research-integrity policy should therefore treat the benchmark accuracy of a detector as provisional, to be re-verified with every LLM release, including earlier versions.

[141] DIAL-OPD: Learning More from Fewer Tokens in On-Policy Distillation cs.CL | cs.AIPDF

Anhao Zhao, Haoran Xin, Junlong Tong, Yingqi Fan, Xuan Lu

TL;DR: 提出 DIAL-OPD，根据师生概率尺度加权 token 奖励来选择有学习价值的监督位置，发现仅保留 40% token 也可超过全 token 在线蒸馏。

Details

Motivation: 在线策略蒸馏的采样 token 版本成本较低，但减少 token 监督未必降低效果，说明关键是选择学习价值高的 token。单纯分歧指标忽略概率尺度，师生都几乎不认为可能的 low-low token 仍可能因对数比产生高奖励并干扰学习。

Result: DIAL-OPD 用师生概率的对数平均数缩放奖励幅度，连接对数概率和概率空间；参数 β 控制权重，仅保留评分最高的 token。四组师生和七个数学推理基准上，与九种基线比较时仅保留 40% token 仍优于 Vanilla OPD 及其全 token 变体，平均准确率最高提升 5.25 个百分点，AIME25 Pass@16 从 13.33% 翻倍至 26.67%。在相同保留率下，较最强选择基线的平均准确率相对提高最高 18%。使用 4B 教师时，DIAL-OPD 在两个学生规模下都超过使用 8B 教师的最强全 token 基线。中等 β 可在抑制 low-low token 和保留有用分歧之间平衡。

Insight: 更大的监督量或教师规模未必更有效；根据绝对概率尺度筛除高奖励但缺乏推理价值的 token，可将有限监督集中在关键步骤。

Abstract: On-policy distillation (OPD) supervises student-generated trajectories with token-level teacher signals. Its sampled-token variant avoids the cost of full-vocabulary probabilities. Yet we find that training on fewer tokens can outperform full-token OPD, challenging the intuition that more supervision improves learning. This motivates selecting tokens by learning value. Existing disagreement-based criteria ignore probability scale: tokens assigned negligible probability by both models, termed low-low tokens, can receive large log-ratio rewards and hinder learning. We propose DIAL-OPD, a token-selection method that bridges log-probability and probability spaces by weighting reward magnitude with the logarithmic mean of teacher and student probabilities. A parameter beta controls this weighting, and the highest-scoring tokens are retained. Across 4 teacher-student pairs and 7 mathematical reasoning benchmarks, we compare DIAL-OPD with 9 baselines. Retaining only 40% of tokens, it outperforms Vanilla OPD and its full-token variants, with mean accuracy gains reaching 5.25 percentage points over Vanilla OPD, and doubles AIME25 Pass@16 from 13.33% to 26.67%. It also achieves up to an 18% relative improvement in mean accuracy over the strongest token-selection baseline at matched retention ratios. With a 4B teacher, DIAL-OPD surpasses the strongest full-token baseline using an 8B teacher at both student scales, showing that effective supervision allocation can outweigh teacher scaling. Further analysis shows that moderate beta balances suppressing low-low tokens against preserving useful disagreements. Token-level evidence reveals that DIAL-OPD filters high-reward tokens with limited reasoning value while preserving supervision critical to reasoning correctness.

[142] Thinking Inertia: LLMs Keep Thinking When Told Not To cs.CLPDF

Dianqiao Lei, Kevin Qinghong Lin, Pan Lu, Philip Torr, James Zou

TL;DR: 定义并评估 LLM 的“无思考”能力，提出三层指标区分仅答案、相关但非推理文字和显式推理，并发现显式禁思提示仍难消除推理痕迹。

Details

Motivation: 模型越来越多提供显式思考模式，但关闭该模式或没有长轨迹并不能可靠定义无思考：关闭时仍可能输出推理，长文本也可能只是填充。

Result: 作者将回答拆成最终答案前文本和答案，分别评估严格仅答案遵循率、指令感知的问题—前文本相关度、以及 LLM 评审的显式推理率。对六个模型、布尔题/多选题/开放题评测六种提示干预。显式无思考控制无法可靠消除可见推理，即“思考惯性”；候选答案空间越开放，显式推理越常见。布尔和多选题准确率稳定；开放题中仅答案遵循与任务准确率存在权衡。对同一问题改写答案空间发现提供候选答案更易得到仅答案响应。

Insight: 无思考不是配置开关能保证的能力，应与推理能力并列评测；评估需区分空前缀、相关非推理文字和显式推断。

Abstract: Large Language Models (LLMs) increasingly ship with explicit “thinking modes”, yet their counterpart, “no-thinking”, has received far less attention. We study LLMs’ no-thinking behavior along two axes. a. How to measure no-thinking? Prior work typically defines no-thinking through proxies such as a disabled thinking mode or the absence of long traces. These proxies are unreliable: disabled thinking modes may still emit reasoning, while long traces may contain filler rather than genuine inference. We instead normalize each response into a pre-answer trace and final answer, and evaluate it at three levels: (i) Empty-Thinking Rate for strict answer-only compliance; (ii) instruction-aware Question-Pre-answer Relevance for similarity between the question and pre-answer trace; and (iii) LLM-as-judge Explicit Inference Rate for visible explicit inference. Together, these metrics distinguish answer-only output, relevant but non-inferential text, and explicit inference. b. How does no-thinking vary across tasks and models? We evaluate six prompting interventions on six LLMs across Boolean, multiple-choice, and open-ended questions. We find that explicit no-think controls cannot reliably eliminate visible inference. Models instead exhibit “Thinking Inertia”: explicit inference persists even under strict controls and becomes more prevalent as the answer space opens. Accuracy remains stable on Boolean and multiple-choice tasks, whereas open-ended tasks reveal a trade-off between answer-only compliance and task accuracy. Rewriting the same questions across answer spaces shows that supplying candidate answers makes answer-only responses easier to produce. These findings establish no-thinking as a non-trivial capability: stopping explicit reasoning cannot be assumed from model settings or instructions alone and deserves systematic evaluation alongside reasoning ability.

[143] DPPM: Dual-Path Parametric Memory for Personalized Language Models cs.CLPDF

Yuhao Chen, Shuochen Liu, Jiayao Shi, Jian Hong, Chen Cheng

TL;DR: 提出 DPPM 双路径参数记忆，将历史证据直接聚合与按序更新变化状态结合，生成融合证据积累和偏好修订的个性化 LoRA。

Details

Motivation: 长程个性化要求模型跨会话跟踪用户偏好。参数记忆虽可减少推理时拼接历史，但独立编译上下文不明确如何跨会话整合，递归更新又可能逐渐削弱早期证据。

Result: DPPM 的 Evidence 路径直接池化交互历史表示以保留旧证据，Delta 路径逐步更新关联状态以捕捉偏好变化；两路融合后产生条件于历史的 LoRA 适配器。多个骨干上优于作者评估的基线，在 PersonaMem-v2 上达到 54.22%，PrefEval 上达到 86.79%。摘要未列其余骨干和对比方法分数。

Insight: 个性化记忆需同时保留累积证据和表达时间变化：只做池化会忽略更新顺序，只做递归可能遗忘早期偏好。双路径提供了简洁折中。

Abstract: Long-term personalization requires language models to use interaction history to track users’ preferences across sessions. Parametric memory encodes this interaction history into model parameters or adapters, reducing the need to include it in the inference context. However, independent context compilation leaves cross-session integration unspecified, while recurrent updates can attenuate earlier evidence. To address these challenges, we propose Dual-Path Parametric Memory (DPPM). Its Evidence path directly pools representations of the interaction history to preserve earlier evidence, while its Delta path sequentially updates an associative state to capture changes. Fusing both outputs produces history-conditioned LoRA adapters that combine evidence accumulation with ordered revision. Across multiple backbones, DPPM outperforms the evaluated baselines, achieving 54.22% on PersonaMem-v2 and 86.79% on PrefEval. These results suggest that DPPM provides a simple and effective design choice for cross-session personalized parametric memory.

[144] From Sparse Representations to Behavioral Insights for Multimodal Depression Assessment cs.CLPDF

Guimin Hu, Zihao Song, Jiachen Luo, Jiayuan Xie, Ruichu Cai

TL;DR: 提出 BehavDep，以稀疏潜在因子和语义桥接解释多模态抑郁评估，并把不同视频行为观测聚合为用户级预测。

Details

Motivation: 多模态抑郁评估可利用行为模式，但现有方法常使用稠密、难解释的联合表示，难以看清模型预测依据和不同观察之间的行为差异。

Result: BehavDep 将多模态行为表示分解为稀疏潜在因子，再通过语义桥关联行为概念。针对用户级标签与视频级异质行为之间的不匹配，方法在弱监督下学习视频级抑郁倾向分数，并聚合多次观察进行用户级评估。作者报告其整体评估表现最佳，并揭示互补模态贡献、跨观察行为异质性，以及概念编辑对预测的响应；摘要未给数据集和具体分数。

Insight: 稀疏因子加行为语义映射为多模态预测提供可检查结构，也允许观察概念变化时预测如何响应。摘要没有给临床诊断验证或模型校准结果。

Abstract: Multimodal depression assessment offers a promising approach to analyzing behavioral patterns associated with depression. However, existing methods often rely on dense and opaque multimodal representations, making it difficult to interpret the behavioral patterns underlying their predictions. In this work, we introduce BehavDep, a sparse factor-based framework that decomposes multimodal behavioral representations into sparse latent factors and associates them with behaviorally meaningful concepts through a semantic bridge. To address the mismatch between user-level annotations and heterogeneous video-level behaviors, BehavDep further learns video-level depression tendency scores under weak supervision and aggregates information across multiple observations for user-level assessment. Extensive experiments demonstrate that BehavDep achieves the best overall assessment performance while revealing complementary modality contributions, heterogeneous behavioral patterns across observations, and prediction responses to concept-level editing. These results show that BehavDep provides a structured and interpretable approach to analyzing multimodal behavioral representations for depression assessment.

[145] Event-Centric Memory with Query-Aware Graph Augmentation for Long-Term Conversational Agents cs.CLPDF

Yichen Liu, Chunfeng Yuan, Haowei Liu, Wenjuan Li, Zefeng Lin

TL;DR: 提出 QGMem，以事件为基本单位构建长期对话记忆，并在查询到来时将相关事件转成局部图作为工作记忆，支持多跳、冲突与超长对话推理。

Details

Motivation: 平面记忆轻量但事件关系和状态变化隐式；图记忆虽显式表示结构，却增加构建成本，长历史还会引入无关关系。

Result: QGMem 将长对话转成带事件索引的原子记忆，保留单次经历，再将关联单元合并为动态记忆轨迹，记录状态变化和当前状态。查询时混合检索互补候选，并通过查询感知重排激活紧凑工作记忆；随后组织为局部图，以图 token 和文本工作记忆共同输入 LLM，暴露关系依赖并支持冲突感知推理。六个基准实验显示，检索、多跳证据组合、冲突解决和超长对话推理均稳定改善，且上下文紧凑、推理成本适中；摘要未列具体分数。

Insight: 把图结构延迟到查询时，仅为相关事件构图，可兼顾平面记忆的轻量性与图推理的显式关系表达。

Abstract: For persistent and personalized conversational agents, memory systems can enable them to remember, update, and reason over long histories by storing past interactions and retrieving relevant information. Existing memory systems typically follow two paradigms: flat-structured memory and graph-based memory. The former is lightweight but leaves event relations and state updates implicit, while the latter explicitly models memory structure but incurs additional construction cost and introduces irrelevant relations over long histories. To address these limitations, we propose QGMem, a novel memory construction and activation framework motivated by human memory, in which experience is organized into events and query-relevant events are modeled by graph as working memory. QGMem converts long dialogue histories into event-indexed atomic memory units that preserve individual experiences and consolidates related units into dynamic memory traces that retain state trajectories and current states. When a query arrives, hybrid memory retrieval gathers complementary candidate memories, and query-aware reranking activates the most relevant units as a compact working memory. To expose relational dependencies in the working memory and support conflict-aware reasoning, QGMem organizes the working memory as a local graph, which is then encoded as a graph token and provided to the LLM together with the textual working memory to improve evidence utilization during answer generation. Experiments across six benchmarks validate the framework and show consistent gains in retrieval, multi-hop evidence composition, conflict resolution, and ultra-long dialogue reasoning with compact contexts and moderate inference cost.

[146] MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement cs.CLPDF

Xiaomi LLM-Core Team, :, Zongming Qiao, Ziyue Hua, Zirui Ou

TL;DR: 报告 MiMo-V2.6 全模态模型系列，通过扩展强化学习算力、任务环境多样性和奖励评估计算，探索大规模 RL 自我提升训练。

Details

Motivation: 强化学习是推动基础模型自我改进的重要范式；要扩展 RL 训练，需要足够的预训练探索空间、训练基础设施、更多环境和可靠的长时程奖励。

Result: RL 前在广泛多模态语料上中期训练，并在混合 SWA 架构上搭建扩展基础设施。RL 从三个维度扩展：异步大批训练，每步处理 1,568 条样本和 2.7–3.7B token，最长上下文 1M；覆盖代码、通用、视觉和网络安全等多类 agent 环境；以组式代理评分增加长程任务 grader 计算，改善奖励精度并引导更短、更省 token 的解答。为稳定训练，冻结 MoE 路由器并加入多层防 reward hacking 策略；系统支持统一轨迹表示、高并发多框架 rollout、控制/数据平面解耦和训练—推理一致性。作者将训练动态、RL 环境及框架开源；摘要未给出模型基准分数。

Insight: 该报告重点描述扩大 RL 训练的三个资源轴和系统工程，而非提供性能基准对比。冻结路由和反 reward hacking 防护用于维持大规模训练稳定。

Abstract: Reinforcement learning (RL) is the central training paradigm for advancing large foundation models towards self-improvement. This report introduces the MiMo-V2.6 series, an omni-modal family that pushes the frontier of model intelligence by scaling RL compute. Prior to RL, we conduct mid-training on a broad multimodal corpus to provide ample exploration space, and build a solid infrastructure on the pretrained hybrid-SWA architecture to support subsequent scale-up. We scale RL compute along three dimensions: (1) larger batches and higher throughput, with an asynchronous training that consumes 1,568 samples and 2.7-3.7B tokens per step at context lengths of up to 1M; (2) more diverse and complex environments, spanning code, general, visual, and cyber domains under a mixture of agent harnesses; and (3) more grader compute, via groupwise agentic grading that yields more accurate reward signals for long-horizon tasks and steers the model towards shorter, more token-efficient solutions. To keep training stable at scale, we freeze the MoE router and establish a multi-layer defense against reward hacking. We further build infrastructure for mixed-task agentic RL, including a unified trajectory representation, high-concurrency multi-framework rollout, decoupled control and data planes, and training-inference consistency. We open-source the training dynamics, RL environments, and RL framework to facilitate reproduction and further research on scaled RL and model self-improvement.

[147] When History Helps and Hurts: Selective History Use across Multimodal Turns cs.CLPDF

Shuoyang Sun, Kerui Gu, Hao Fang, Shaoli Huang, Bin Chen

TL;DR: 提出 ReTurn 多模态多轮基准，控制问题、媒体和答案不变，分别检验历史问题迁移到当前媒体与历史媒体支撑当前问题的选择性历史使用。

Details

Motivation: 多轮交互中，历史问题可能仍相关但旧答案已过时；当前问题也可能依赖历史证据，即使新媒体与历史信息冲突。既有评测很少把历史使用需求与问题本身难度分开。

Result: ReTurn 含 7,000 个视觉和音频基础任务。Reconfirm/Reground 要求将历史任务应用于当前媒体，并改变历史信息是否一致；Retrieve/Rebind 要求用历史媒体回答当前问题，并改变当前媒体竞争性。每组配对保持目标问题、媒体和答案不变，支持开放式与多选评测，并有匹配单轮版本作可答性参照。13 个全模态、视觉语言和音频语言模型的开放回答中，模型级中位准确率由直接输入的 93.7% 降至对话中的 72.3%。行为探针显示，记住问题不等于正确应用任务，竞争媒体会把回答从历史目标引走；监督适配仅部分改善。

Insight: 多轮可靠性取决于选择并正确应用所需历史信息，而非单纯是否记住历史；配对设计控制目标内容以隔离历史使用机制。

Abstract: Reliable multimodal interaction depends on selective use of conversational history: an earlier question may remain relevant while its previous answer is outdated, whereas a current request may depend on historical evidence despite conflicting new observations. Existing multi-turn evaluations rarely separate these history-use demands from underlying question difficulty. To address this gap, we introduce ReTurn, a benchmark of 7,000 base tasks spanning visual and audio evidence for evaluating selective history use. For task-carrying history, Reconfirm/Reground require applying a historical question to current media while varying historical agreement; for evidence-carrying history, Retrieve/Rebind require answering a current question using historical media while varying current-media competition. Each pair preserves the target question, media, and answer. Tasks support open-ended and multiple-choice evaluation, with matched single-turn counterparts serving as answerability references. Across 13 omni-modal, vision-language, and audio-language models, median model-level open-ended accuracy falls from 93.7% with direct input to 72.3% in conversation. Behavioral probes show that high question recall can coexist with weaker task application, while competing media can redirect answers away from historical targets. Supervised adaptation yields only partial gains. ReTurn provides a controlled framework for assessing whether multimodal models select and use the historical information required by each request.

[148] Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks cs.CLPDF

Xing Li, Qingcheng Chang, Jinzhong Ning, Changfeng Xu, Shenlong Zhang

TL;DR: 评测输出选项而非自由文本的专用 System One 决策模型 Jev，并与 19 个不同规模通用 LLM 比较知识、推理和多语言多选任务。

Details

Motivation: 专用决策模型直接从给定候选中选答案，与自由文本生成式 LLM 的能力边界不同，需要通过覆盖知识和复杂推理的基准比较其竞争力。

Result: Jev 在 13 个多选基准上评测，覆盖知识、推理和多语言理解，并与前沿、代表性和小型三档共 19 个 LLM 比较。知识和常识基准上可与前沿模型竞争，在 MMLU-Redux 和 ARC-Challenge 上得分最高；除数学外，也超过大多数代表性模型和所有小模型。但在数学文字题 MathQA 上，分数比前沿模型中位数低 17.7 分，且低于全部 19 个对照 LLM。

Insight: 专用选项决策模型适合主要依赖知识的选择任务，但需要多步计算的数学题仍明显较弱；能力结论取决于任务类型。

Abstract: Jev is a “System One” model that returns a choice among given options instead of generating text. We study how such a specialized decision model compares with general-purpose large language models (LLMs). We evaluate Jev on 13 multiple-choice benchmarks covering knowledge, reasoning, and multilingual understanding, and compare it with 19 LLMs in three tiers: frontier, representative, and small. Jev is competitive with frontier LLMs on knowledge and commonsense benchmarks and obtains the best score on MMLU-Redux and ARC-Challenge. Outside mathematics, it also outperforms most representative LLMs and all small LLMs. However, it falls behind on mathematical word problems: on MathQA, it is 17.7 points below the frontier median and scores lower than all 19 LLMs. These results indicate that a specialized decision model can match general-purpose LLMs on decisions that rely mainly on knowledge, but not on decisions that require multi-step calculation.

Zhaoxin Yu, Qingchao Kong, Dajun Zeng, Wenji Mao

TL;DR: 提出基于归因理论的 LLM 社会责任与责备判断基准及隐藏表示探测方法，分析模型判断与人类规律的一致程度及其内部维度。

Details

Motivation: 社会归因指把外部事件解释为行为者社会行为的原因与理由，涉及责任、责备或赞扬；该过程在社会心理学已有理论基础，但 LLM 社会推理中的系统性研究不足。

Result: 作者围绕责任和责备构建 7,639 个判断问题，包括归因理论经典情境 Vignette 子集与真实社会叙事 Reality 子集。评估 32 个代表性 LLM 和 5 个非 LLM 基线，并探测五个关键归因维度在隐藏空间中的表示及其对最终判断的影响与人类理论是否一致。结果显示当前 LLM 与人类责任/责备判断存在可测但不完整的一致性，且一致性与模型规模正相关；某些归因维度可从特定隐藏状态位置稳定解码，其影响方向与人类归因理论一致。

Insight: 模型的社会判断并非纯粹不可解释：部分理论维度可从内部表示读出，且影响最终输出的方式与人类理论相符，但总体人类一致性仍不完整。

Abstract: Large language models (LLMs) are increasingly deployed in sociotechnical systems where social attribution, the reasoning process attributing external events to the causes and reasons of agents’ social behaviors, plays a critical role. These processes involve judgments of social cause, responsibility, and blame/credit to agents. Although attributional models are well-studied in social psychology and cognition through Attribution Theory, social attribution remains underexplored in AI, particularly LLM social reasoning. This paper provides the first systematic exploration of LLM social attribution. Our work focuses on responsibility and blame attributions, examining current LLMs’ judgments and their underlying internal mechanisms. Guided by attribution theory, we construct a social attribution benchmark consisting of a Vignette subset based on classic scenarios from attribution theory research and a Reality subset based on real-world social narratives, yielding 7,639 responsibility/blame judgment questions. On this basis, we evaluate 32 representative LLMs and 5 basic non-LLM baselines. To further explore the internal mechanisms underlying the LLM judgment process, we develop a probing-based methodology to investigate the latent-space representations of 5 key attribution dimensions and the consistency of their influences on LLM judgments compared to those in human social attribution. Our research findings reveal that current LLMs exhibit measurable but incomplete agreement with human responsibility and blame judgments, and meanwhile, this agreement is positively correlated with model size. Some attribution dimensions are systematically decodable from specific positions in LLM hidden states, and their influences on the final judgment are consistent with those indicated by human Attribution Theory. The dataset and associated code are available at https://github.com/Yuzhaoxin946/SAB-Bench.

[150] EgoVoice: Proactive Spoken Assistance from Egocentric Multimodal Streams cs.CL | cs.CV | cs.SDPDF

Heeseung Kim

TL;DR: 提出 EgoVoice，训练和评估从第一人称连续视听流主动决定何时开口、说什么的可穿戴 AR 助手。

Details

Motivation: 主动视频助手、语音对话和第一人称任务理解分别进步，但现有系统未解决连续视听流中主动干预时机与内容的联合决策。

Result: 作者从 HoloAssist 真实讲师视频制作训练数据，先做声源分离和语音重合成得到干净音频，再把每段视频变为逐时刻决定保持沉默或口头指导的任务。用数据微调全模态 LLM，并以直接偏好优化进一步训练主动干预行为。闭源与开源模型实验显示，既有系统很少能给出时机恰当且有意义的主动干预；EgoVoice 相较零样本骨干在干预时机、内容相关性和人类偏好上均有明显改善。

Insight: 主动助理不仅要回答用户问题，还要根据连续场景判断是否以及何时介入；语音重合成用于从原始讲师数据构造更清洁的多模态训练信号。摘要未报告各维度具体分数。

Abstract: Wearable augmented reality (AR) assistants are moving toward continuous real-world interaction, where they perceive the user’s activity through first-person video and audio and provide timely spoken guidance without being explicitly asked. While proactive video assistants, spoken dialog systems, and egocentric task understanding have each advanced rapidly, existing systems do not address the joint problem of deciding when to speak and what to say from continuous first-person streams. We introduce EgoVoice, a framework for training and evaluating proactive egocentric spoken assistants. From HoloAssist video recordings of real human instructors, we construct clean audio streams through source separation and speech resynthesis, and convert each video session into a format where the model must decide at each moment whether to remain silent or provide spoken guidance. We fine-tune an omni-modal LLM with our data, and further improve its proactive intervention behavior with direct preference optimization. Experiments across closed and open-source models show that existing systems rarely produce well-timed, meaningful proactive interventions, while EgoVoice yields clear improvements in intervention timing, content relevance, and human preference over the zero-shot backbone.

[151] Language Models as AI Research World Models cs.CLPDF

Zijun Wang, Zewen Liu, Minhua Lin, Zhaotian Weng, Zhan Shi

TL;DR: 研究语言模型作为 AI 研究世界模型，使用历史实验知识预测新干预的结果，帮助研究代理在有限实验预算下选择更可能成功的实验。

Details

Motivation: AI 研究代理能提出实验，但在真实环境中执行能力有限；预算有限时，提前预测候选干预结果是实现持续自我改进的关键。

Result: 评估收集九个研究环境、超过 2,600 条预训练、后训练与推理实验记录，累计超过 171,000 H100 GPU 小时。由真实实验经验获得的研究知识，使世界模型预测同一环境未见干预的 Spearman 相关提高 0.27，且可跨环境复用。例如仅用 OLMo3、Marin、Nanochat 的预训练经验，在 Qwen3 环境中选择实验的 regret 比零经验减少 78%。固定选择预算的多轮 Autoresearch 中，使用环境内和跨环境研究知识的世界模型分别使最佳增益提高 15.8% 和 11.6%。对 13 个世界模型 LLM 的消融发现，加入研究知识对干预排序的提升可能超过单独换模型或增加推理力度。

Insight: 跨模型迁移的实验记录可成为预测研究干预效果的知识，改进有限预算下的实验选择。研究知识有效性通过相关性、选择 regret 和多轮研究收益等不同指标评估。

Abstract: AI research agents automate the cycle of proposing, implementing, and evaluating experiments, opening a path toward recursive self-improvement. Yet their ability to propose experiments outpaces their capacity to execute them in real environments, making outcome prediction a key capability for sustained self-improvement under limited experimental budgets. We investigate language models as Research World Models (RWMs), which predict the outcomes of candidate interventions across research environments. Our evaluation draws on over 2,600 experimental records from nine research environments spanning pretraining, post-training, and inference, representing more than 171,000 H100 GPU-hours of experimentation. Research knowledge acquired from real experimental experience improves RWM predictions of unseen interventions within the same environment (Spearman +0.27), and can be reused across environments. For example, using only pretraining experience from OLMo3, Marin, and Nanochat, an RWM reduces selection regret in the Qwen3 environment by 78% compared with zero-experience setting. These benefits extend to multi-round Autoresearch under a fixed selection budget: RWMs with in-env and cross-env research knowledge increase the best gain achieved by 15.8% and 11.6%, respectively. Ablations across 13 language models used as RWMs show that adding research knowledge can improve intervention ranking more than changing models or increasing reasoning effort alone. These findings support language models as RWMs and motivate accumulating experimental data for future RWM training.

[152] HarnessSQL: Harness-Native Training for SQL Agents in Realistic Database Environments cs.CLPDF

Haolin Yang, Jipeng Zhang, Jian Xie, Shuaishuai Gong, Sirui Han

TL;DR: 提出 HarnessSQL，让 Text-to-SQL 模型在真实数据库执行框架中完成监督微调和强化学习，保留检查 schema、试探查询、诊断错误和修订假设的交互结构。

Details

Motivation: 传统 Text-to-SQL 训练将问题直接映射到静态 SQL，但真实数据库代理需在有状态多轮交互中探索 schema、运行查询和修复错误；若执行 harness 只在推理时出现，会造成明显训练—部署不匹配。

Result: HarnessSQL 构建隔离、可执行数据库环境和隐藏执行 oracle，在目标 SQL harness 内 rollout 教师，只保留验证轨迹进行全序列 SFT，再做执行奖励 RL。在 Spider 2.0-SQLite 上，Qwen3-8B 执行准确率由 15.5% 升至 45.2%，Qwen3-14B 由 22.2% 升至 54.8%；并能迁移到 BIRD-Interact 和 LiveSQLBench 等分布外交互基准。

Insight: 把训练轨迹放在与部署相同的数据库交互循环中，让模型学习长时程执行与修复，而非仅记住静态查询映射。

Abstract: Text-to-SQL models are commonly trained to map questions directly to static queries, whereas real-world database agents operate through stateful, multi-turn interaction with live databases – inspecting schemas, executing probe queries, diagnosing errors, and revising hypotheses. This creates a critical train-deploy mismatch, as the execution harness that mediates this interaction is introduced only at inference time. To bridge this gap, we propose HarnessSQL, a harness-native post-training framework that preserves the full interaction structure throughout both supervised fine-tuning and reinforcement learning. HarnessSQL builds isolated, executable database environments paired with hidden execution oracles, rolls out teachers directly inside the target SQL harness, and retains only verified trajectories for full-sequence SFT, followed by execution-reward RL. Across Spider 2.0-SQLite, HarnessSQL dramatically boosts the execution accuracy of compact models, raising Qwen3-8B from 15.5% to 45.2% and Qwen3-14B from 22.2% to 54.8%, while transferring effectively to out-of-distribution interactive benchmarks such as BIRD-Interact and LiveSQLBench. Our findings demonstrate that training database agents directly within their execution harness is essential for mastering complex, long-horizon database workflows.

cs.AI [Back]

[153] Plan-and-Patch: Diffusion Language Models for Agentic Planning cs.AI | cs.CL | cs.LGPDF

Syamantak Kumar, Jiang Guo, Hassan Hamad, Hideo Kobayashi, Yi Xiang

TL;DR: 提出 Plan-and-Patch，用扩散语言模型并行生成结构化计划，并只重生成失效局部、保留前后步骤，以提升长程智能体计划修复速度。

Details

Motivation: 长程智能体执行中，环境假设可能失效、工具返回意外结果或动作失败，需修订计划；失败常只影响局部，整计划重生成可能无谓改动原本有效的前后结构。

Result: Plan-and-Patch 通过并行 unmasking 生成程序式计划，并以保留前缀和后缀、填补选中区域的方式修复。比较 DreamReasoner-8B 与 Qwen3-8B 作为扩散/自回归规划器：在不做任务专用训练的 Natural Plan 上，扩散模型计划修复成功率 53.7%，接近自回归模型 27.0% 的两倍。在 ALFWorld、TextCraft 任务专用训练后，两类规划器计划生成成功表现相近，但扩散规划器平均生成延迟低 39%–46%。

Insight: 扩散生成允许在保留计划未受影响部分的前提下并行填补局部空缺，适合计划结构化修复；未经训练的 Natural Plan 结果与任务专用训练结果应分开理解。

Abstract: Planning is increasingly important for long-horizon agents, where successful execution requires coordinating subgoals, tool use, and intermediate outcomes over many steps. Yet assumptions made during planning may be invalidated by the environment, tools may return unexpected results, or actions may fail. Effective agents must therefore not only generate plans, but also revise them. Such revisions often affect only part of a plan, leaving the preceding and subsequent structure intact. Rather than regenerate the entire plan and risk unnecessary changes, repair can regenerate the affected region conditioned on the preserved prefix and suffix. We introduce Plan-and-Patch, a plan-and-act framework in which a diffusion language model (dLLM) generates a structured, program-like plan through parallel unmasking and repairs it by filling in selected regions while keeping the surrounding steps fixed. We compare DreamReasoner-8B and Qwen3-8B as diffusion and autoregressive (AR) planners. On Natural Plan without task-specific training, diffusion (53.7%) achieves nearly twice the plan repair success rate of AR (27.0%). After task-specific training on agentic benchmarks, ALFWorld and TextCraft, the planners achieve similar observed success in plan generation, while diffusion reduces mean plan-generation latency by 39-46% relative to AR. Our results show that Plan-and-Patch provides a framework for faster plan generation and effective plan repair in long-horizon agents.

[154] StoreBench: A Live-Commerce Environment for Evaluating and Training Autonomous Operator Agents cs.AI | cs.CL | cs.LGPDF

Daksh Raghuvanshi, Ved Vedere, Yifan Wang

TL;DR: 提出 StoreBench，一个基于生产级电商后端的动态直播电商环境，用于评估和训练代理在不确定经营条件下的长程规划与经济决策。

Details

Motivation: 多数代理强化学习基准是静态环境：世界只在代理动作时变化、奖励只是终局判定且通过门槛任意设定，难评估真实运营中的持续变化和风险。

Result: StoreBench 让代理经营中型线上服饰店：顾客全天下单、供应商调价或失效、市场冲击不定时发生；代理使用人与运营者相同的 29 种工具，窗口操作预算让模拟时间由动作数决定、与模型延迟无关。通过阈值按脚本锚点策略校准，奖励防护针对一组 reward hack，每条动作序列可确定性重放。七个前沿 LLM 在 11 个 30–45 天及全年场景、三个世界种子、匹配推理预算下评估；最佳 DeepSeek-V4-Pro 仅通过 49% 任务—种子单元，低于智能分诊启发式策略的 97%。人类专家平均综合分 0.708，略高于模型 0.700。Claude Code harness 下，多数模型全年表现大幅改善；Qwen3.5-27B 仅在五个不相交任务上 GRPO 训练后，留出任务均分从 0.136 升至 0.373。

Insight: 基准以实时变化、动作预算、可复现回放和防奖励作弊构造经营压力测试；当前前沿模型仍未达到脚本策略水准。为防污染，只公开部分训练任务和示例轨迹，不开放完整评测套件。

Abstract: Reinforcement learning environments are now a primary lever for improving large language model (LLM) capabilities in post-training, yet most agentic benchmarks remain static: the world moves only when the agent acts, the reward is a terminal verdict, and the pass bar is set arbitrarily. We introduce StoreBench, a live-commerce environment in which an agent runs a mid-size online apparel store on a production-grade commerce backend, testing long-horizon planning and economic judgment under uncertainty. Customers order around the clock, suppliers reprice and fail, and market shocks arrive with partial or no warning. The agent acts through the same 29 merchant tools a human operator would use, under a windowed operation budget that makes simulated time a function of actions taken, so model latency cannot influence simulated time. Pass thresholds are calibrated against scripted anchor policies, the reward is hardened against a catalogue of reward hacks, and every episode replays identically given a sequence of actions. We evaluate seven frontier LLMs on 11 scenarios of 30 to 45 days and a full simulated year, over three world seeds at matched reasoning effort. No model matches the scripted smart-triage policy on average: the best, DeepSeek-V4-Pro, passes 49% of task-seed cells against the heuristic’s 97%. Human experts working through the same tools and budgets outscore every model (mean composite 0.708 vs. 0.700). Over a full simulated year under the Claude Code harness, most models show dramatic performance improvement. In a GRPO post-training run, Qwen3.5-27B trained on only five disjoint tasks raises its mean composite on the held-out evaluation tasks from 0.136 to 0.373. We release five example training-split tasks, ten sample trajectories, and the scoring and verification tooling; the full environment and evaluation suite are withheld to keep the benchmark uncontaminated.

[155] GameCommBench: A Unified Benchmark and Type-Aware Evaluation for AI-Generated Game Commentary cs.AI | cs.CLPDF

Qirui Zheng, Zhengteng Lin, Yunyi Xiao, Junhao Li, Keyuan Cheng

TL;DR: 提出 GameCommBench 和 TACE，统一评估棋盘、体育和电竞的 AI 游戏解说，并按解说功能类型验证指标与人类一致性。

Details

Motivation: 游戏解说需结合多模态感知、策略推理和上下文知识；现有研究分散于不同游戏、模态与协议，文本重叠或整体评分不能体现不同类型解说的功能差异。

Result: GameCommBench 覆盖棋盘游戏、体育和电竞，解说与异质游戏上下文对齐并标注解说类型。TACE 按类型结构化评价，并由作者验证可靠性及与人类判断的一致性，再评测代表性 AI 解说系统。结果显示模型能力画像并不均衡，实时观察和策略分析是主要瓶颈。摘要未提供具体游戏、指标或模型分数。

Insight: 解说质量需按观察描述、策略分析等功能拆分，统一基准与类型感知评估可改善跨游戏比较的可解释性。

Abstract: Game commentary is an open-ended generation task requiring multimodal perception, strategic reasoning, and contextual knowledge. Existing AI-Generated Game Commentary (AI-GGC) studies remain fragmented across games, modalities, and evaluation protocols, while overlap-based or holistic evaluators fail to capture the functional heterogeneity of commentary. We introduce \textsc{GameCommBench}, a unified benchmark spanning board games, sports, and esports, with commentary aligned to heterogeneous game contexts and annotated by commentary type. We further propose Type-Aware Commentary Evaluation (TACE), a structured framework for evaluating different types of commentary. We then validate TACE for reliability and human agreement, and use it to benchmark representative AI commentators. Results reveal non-uniform capability profiles, with live observation and strategic analysis emerging as major bottlenecks. Together, \textsc{GameCommBench} and TACE provide a diagnostic foundation for comparable and interpretable AI-GGC evaluation.

[156] RL-ARC: Calibrating Large Reasoning Models via Reasoning-guided Uncertainty cs.AI | cs.CL | cs.LGPDF

Gukhyeon Lee, SangKeun Lee

TL;DR: 提出 RL-ARC，以推理过程置信度辅助校准答案置信度：正确答案用推理置信度正则，错误答案则增加过度自信惩罚。

Details

Motivation: RLVR 强化推理能力，却不显式优化置信度，可能导致校准退化和过度自信。已有校准训练仍可能在分布偏移下过度自信，并牺牲推理表现。

Result: RL-ARC 联合利用 reasoning confidence 与 answer confidence：正确案例中前者作为答案置信校准的引导正则，错误案例中将其用于过度自信惩罚。在分布内和分布外设置的综合实验中，作者报告校准改善，推理模型能依具体问题自适应估计置信度，且推理性能没有明显牺牲。摘要未提供数据集、校准误差或准确率数值。

Insight: 过程置信度可为最终答案置信度提供额外信号，特别有望减轻分布外过度自信；效果声明需结合完整实验指标衡量。

Abstract: Language models (LMs) are commonly trained with Reinforcement Learning with Verifiable Rewards (RLVR) to enhance their reasoning capabilities. However, since RLVR does not explicitly account for calibration during training, it can lead to severe calibration degradation, including overconfidence. Recent calibration-aware training methods for LMs, which incorporate objectives for uncertainty estimation into training, improve calibration but still exhibit overconfidence under distribution shift, while sacrificing reasoning performance. To this end, we propose RL-ARC, a calibration-aware training framework that jointly leverages reasoning confidence and answer confidence. Specifically, RL-ARC leverages reasoning confidence as an auxiliary signal for calibrating answer confidence, applying it as reasoning-guided regularization for correct cases and as an overconfidence penalty for incorrect cases. Comprehensive results across ID and OOD settings show that, beyond improving calibration, RL-ARC enables reasoning models to adaptively estimate confidence based on the given question without substantially sacrificing reasoning performance, thereby highlighting the importance of reasoning confidence for training reliable reasoning models.

[157] UniData: Universal Multimodal Instruction Generation Pipeline cs.AI | cs.CLPDF

Jiaqi Tang, Yi-Feng Wu, Yuting Zhang, Hao Lu, Bowen Fu

TL;DR: 提出 UniData 通用多模态指令生成流程，将简单需求扩展为多事件、多轮任务，再由 any-to-any 大模型生成多模态指令并清理轮次间冗余推理。

Details

Motivation: 高质量多模态指令数据制作昂贵，已有自动生成方法常支持模态有限，也难处理多轮指令。

Result: UniData 先把用户需求扩展为多样事件，再交给 any-to-any 大模型生成多模态指令，最后利用轮次关联纠正无关或冗余推理以提升数据质量。作者另构建 UniDataset，含 20,000 条、覆盖九种模态的数据。实验报告 UniData 数据质量达到 SOTA，且可提升其他多模态模型的理解与生成能力；摘要未提供具体分数或模型细节。

Insight: 多轮数据构造需显式利用轮次依赖清除冗余，不能只把多条单轮样本拼接起来；九种模态数据集用于训练和评估该流水线。

Abstract: Multimodal Large Language Models (MLLMs) are increasingly being applied in a wider range of real-world scenarios. However, due to the substantial labor cost, creating high-quality multimodal instruction datasets for MLLMs remains a significant challenge. Although some methods propose to generate instruction data, they often face limitations in modality support and struggle with generating multi-round instructions. To address these problems, we introduce UniData, a universal instruction generation pipeline, to transform simple user requirements into multi-round, multimodal instructions. Specifically, UniData first expands user requirements into multiple diverse events. Using these events, UniData then integrates an any-to-any large model for multimodal instruction generation. Finally, UniData enhances data quality by correcting irrelevant and redundant inference flow, leveraging correlations between instruction rounds. To train this pipeline, we also build UniDataset, a dataset comprising 20,000 entries across nine modalities for improved multimodal generation. Our experiments demonstrate that UniData achieves SOTA performance in data quality and can also enhance the understanding and generation capabilities of other multimodal models.

[158] Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks cs.AI | cs.CL | cs.CV | cs.LGPDF

Haoyu Zhao, Zhengxu Yu, Zhiyuan He, Meng Fang, Rasul Tutunov

TL;DR: 提出 Memento 3，以外部自然语言规则手册表示可修订世界模型，并通过编译、执行回放和验证让冻结 LLM 智能体在交互中递归自我改进。

Details

Motivation: 在未知环境行动需要从有限观察推断世界规律并随新证据修订；多种世界模型可能都解释已观察交互，却对未见状态做出不同预测。

Result: 冻结 LLM 维护带可修订假设的语义规则手册，对未知规律保持未指定，并编译为执行代码做预测和规划。智能体循环执行观察、反思、修规则、编译和验证，以预测误差改进手册和代码；只有当 LLM 判断代码忠实于手册且单元级精确回放重现观测转移时，才接受更新。群体扩展并行维护多个世界模型、共享交互证据并用预测指导探索。在 ARC-AGI-3 上，单模型智能体通关全部 25 个公开游戏，平均 RHAE 为 100.0，动作数为人类的 44%。Atari Pong 案例中，学习的反馈控制器在三个不同开局评估回合中均以 21:0 获胜，期间不再调用 LLM。

Insight: 该系统将 LLM 参数冻结，把可验证世界模型更新放到外部记忆和代码中；游戏控制结果限于公开 ARC-AGI-3 和 Pong 案例。

Abstract: Learning to act in unfamiliar environments requires agents to infer how the world works and revise that understanding as new evidence arrives. Yet limited observations can support multiple world models that explain past interactions but predict different outcomes in unseen states. We introduce Memento 3, building on the Memento series to enable frozen LLM agents to continually learn explicit world models through external memory. The agent maintains a natural-language rulebook as persistent semantic memory, recording revisable hypotheses about environment dynamics while leaving unknown aspects underspecified. It compiles this rulebook into executable code for prediction and planning. Through a continual loop of observation, reflection, rule revision, compilation, and verification, the agent uses prediction errors to refine both the rulebook and its code. Updated code is accepted only when the LLM judges it faithful to the rulebook and cell-exact replay reproduces the observed transitions. We investigate this process as a model-based route to recursive self-improvement (RSI): the agent autonomously explores the environment, revises its world model, and uses verified updates to guide subsequent interaction and learning, while the underlying LLM remains fixed. A population extension maintains multiple world models in parallel, sharing interaction evidence and using their predictions to guide exploration. On ARC-AGI-3, the single-model agent clears every level of all 25 public games, achieves a mean Relative Human Action Efficiency (RHAE) of 100.0, and uses 44% of the human action count. In an Atari Pong case study, a learned feedback controller wins 21:0 in each of three evaluated episodes with different openings, without further LLM calls.

[159] When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimation cs.AI | cs.CLPDF

Yiruo Cheng, Shen Huang, Xiaoshuai Song, Jiejun Tan, Guanting Dong

TL;DR: 提出 RACE 跨轮推理自适应训练，根据移除额外推理后参考动作似然的下降，识别哪些推理轮对当前和后续动作不可或缺。

Details

Motivation: LLM 智能体通常每轮动作前都推理，但先前推理可能持续支持后续动作，并非每轮都需重新生成。难点在于不依赖高成本生成式验证判断既有推理是否足够。

Result: 作者发现，移除额外推理后后续参考动作似然下降幅度可近似衡量这些动作能否由早先推理支持。基于此提出 LoGiC，渐进识别对当前及后续参考动作影响有限的推理轮，并将删除信号用于监督微调和代理强化学习，使策略学会何时推理、何时直接行动。在四个代表性智能体基准上，RACE 显著降低推理成本，同时维持或提高任务表现；摘要未给具体数字。

Insight: 跨轮动作支持提供一种轻量信号，避免对每次是否需新推理都额外生成并验证候选链；效果覆盖四种代理任务。

Abstract: Large language model (LLM)-based agents have demonstrated strong capabilities on complex tasks. They typically perform reasoning before each action throughout an interaction trajectory. However, reasoning may not be necessary at every turn, as reasoning produced earlier can continue to support subsequent actions. A key challenge is therefore to determine when existing reasoning remains sufficient and when a new reasoning step is needed, without relying on costly generation-based verification. We find that decreases in the likelihood of subsequent reference actions after removing additional reasoning closely track whether those actions remain recoverable given earlier reasoning, providing an effective and lightweight signal for estimating cross-turn action support. Based on this observation, we propose Reasoning Adaptation through Cross-Turn Estimation (RACE), a training approach for adaptive agent reasoning. RACE introduces a Likelihood-Guided Progressive Reasoning Cover Detection (LoGiC) procedure that progressively identifies reasoning turns whose removal has limited impact on the current and subsequent reference actions. The resulting removal signals are incorporated into both supervised fine-tuning and agentic reinforcement learning, enabling the policy to learn when to reason and when to act directly. Extensive experiments on four representative agent benchmarks show that RACE substantially reduces reasoning cost while maintaining or improving task performance.

[160] Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competition cs.AI | cs.CLPDF

Kaisen Yang, Qingle Liu, Kejin Wang, Yicheng Zhao, Jieming Li

TL;DR: 提出对抗启发式学习 AHL 与 AAArena 基准，使用 AI 代理在固定比赛/评估预算下修改可执行游戏策略与配套软件，测试有限经验下的长期学习。

Details

Motivation: 对抗游戏推动了启发式搜索和强化学习发展，但从少量样本学习并适应策略仍困难；代理可将经验转化为代码策略更新，同时保持模型权重固定。

Result: AAArena 收集 12 种真实对抗游戏和 1,920 个历史人类程序，按真实比赛流程评估。代理需理解规则、选择对手、分析回放并修改游戏智能体，在固定对局与评估预算内争取最高排名。评测多种模型与 harness 配置，Opus5.5 配 Claude Code 获得六枚金牌，但没有任何配置赢得其余六个人类天梯。规则说明更复杂的游戏整体更难。对手选择和密集反馈有助策略改善，代理既能从自身 on-policy 回放学习，也能从其他玩家的 off-policy 回放学习。

Insight: 代码策略修订可作为权重固定条件下的启发式学习机制，但当前代理仍难理解复杂规则、实现策略和长期迭代；人类在六个游戏天梯上保持领先。

Abstract: Adversarial games have driven advances from heuristic search to reinforcement learning, yet learning and adapting strategies from limited samples remain challenging. AI agents offer an alternative by turning game experience into revisions of executable policies. Building on heuristic learning (HL), we formalize Adversarial Heuristic Learning (AHL), a paradigm that uses AI agents as learning engines to refine game policies and supporting software while keeping model weights fixed. We introduce AAArena, a benchmark comprising 12 authentic adversarial games and 1,920 archived human programs, with an evaluation protocol modeled on real-world game competitions. Agents interpret rules, choose opponents, analyze replays, and revise game agents to achieve their highest ranking within fixed match and evaluation budgets. We evaluate \val{completedmodels} model and harness configurations: Opus5.5 with Claude Code earns 6 gold medals, while no evaluated configuration tops the remaining 6 human ladders. Performance is generally weaker in games with more complex rule specifications. Further experiments show that opponent selection and dense feedback support policy improvement, and that agents learn from both on-policy replays of their own matches and off-policy replays of other players’ matches. These results highlight HL’s potential in adversarial games and identify persistent challenges in game understanding, strategy implementation, and long-horizon policy development.

[161] Overcoming Prior Barriers: Supervised Fine-Tuning under Long-Tail Distribution cs.AI | cs.CL | cs.LGPDF

Haohui Wang, Jiahao Xu, Wangzhi Zhan, Tong Zeng, Dongqi Fu

TL;DR: 提出先验障碍 prior barrier 概念及 PASS 指令选择方法，根据预训练模型对目标概念的支持程度自适应分配有限 SFT 监督预算。

Details

Motivation: 监督微调所需概念在预训练模型中的支持度不同：常见概念通常学得较好，稀有概念较弱。长尾先验意味着头部和尾部概念从不同起点开始，固定均匀指令选择可能分配低效。

Result: 作者定义 prior barrier 量化预训练模型在竞争概念与目标概念之间的支持差异，观察到其呈长尾分布。理论推导 SFT 长尾先验下预测风险上界，明确先验障碍和累积监督证据如何共同影响表现。PASS 从参考答案构造概念，估计每条指令提供的区分证据，并根据当前覆盖不足程度动态分配选择预算。在四种骨干—预算设置上，均持续超过七种 SOTA 指令选择方法；消融显示自适应预算分配稳定优于均匀分配。

Insight: 有限 SFT 预算应优先用于仍缺乏区分证据的尾部概念，且指令本身必须提供有效判别信息；摘要未给性能提升数值。

Abstract: Supervised fine-tuning (SFT) adapts pretrained large language models (LLMs) to downstream tasks, but the required concepts can receive substantially different levels of pretrained support. Frequent concepts are more likely to be well learned, whereas rare concepts may remain weakly represented. We introduce a novel notion named prior barrier to quantify how strongly the pretrained model supports competing concepts over the target concept. We observe that prior barriers follow a long-tail distribution, placing head and tail concepts at different starting points for SFT: head concepts face lower prior barriers, whereas tail concepts require additional instructions to overcome their higher prior barriers. Our theoretical analysis further derives a predictive risk bound for SFT under long-tail prior barriers, explicitly characterizing how the prior barrier and accumulated SFT evidence jointly determine predictive performance. Motivated by this prior barrier-dependent demand, we propose PASS, an adaptive SFT instruction selection method that constructs reference-derived concepts and estimates the distinguishing evidence provided by each instruction, and adaptively allocates the selection budget toward concepts that remain insufficiently covered under the current selection. In this way, PASS jointly considers which instructions can provide useful evidence and where additional supervision is needed under a limited budget. Experiments show that our method consistently outperforms seven state-of-the-art instruction selection methods on four backbone-budget settings. An ablation study further shows that PASS’s adaptive allocation consistently improves over uniform allocation.

[162] Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness cs.AI | cs.CL | cs.CYPDF

Saisab Sadhu, Shreeyans Arora, Pratinav Seth

TL;DR: 以反事实替换法审计法律 CoT 是否真正依赖其引用的法条/判例，并发现模型常能正确报出法律依据，但更换依据后判决改变并不稳定。

Details

Motivation: LLM 法律解释常通过点名法条或先例作为判决依据，但引用正确是否代表模型的判决实际依赖该权威，尚未直接检验。

Result: 作者固定案件事实，将生成答案引用的法律权威替换为无关权威，并从隐藏状态解码判决变化；评估 7 个 8B–70B 开源模型、四个司法与合同推理基准。要求引用适用权威时，模型生成中正确引用率为 66.7%–100%，但判决对权威替换的敏感度低得多：CaseHOLD 为 0%–21.7%，ECHR/SCOTUS 为 30.0%–76.7%，ContractNLI 为 43.3%–50.0%。模型规模和一个尽力复现的专用法律模型均未消除差距。五个核心模型红队评估中，隐藏于案件事实的对抗指令遵循率为 73.3%–96.4%，远高于判决替换敏感度。

Insight: 点名正确法源不是判决忠实依赖该法源的可靠证据；模型还可能被案件文本中的对抗指令操纵。作者报告这些差异在排除提示措辞噪声和采样混杂的检查中复现。

Abstract: Large language models increasingly justify legal decisions by naming the statute or precedent behind a verdict, treated as evidence that the decision follows from it. We test this directly: holding case facts fixed, we substitute the named legal authority for an unrelated one and decode a model’s evolving verdict from its hidden states. Across seven open-weight models (8B-70B) and four benchmarks spanning judicial and contractual reasoning, when explicitly required to justify a verdict by naming the governing authority, models name the correct one in 66.7%-100% of generations, while the verdict changing when the authority changes is far less consistent: 0.0%-21.7% on CaseHOLD, 30.0%-76.7% on ECHR and SCOTUS, and 43.3%-50.0% on ContractNLI. Neither scale nor a purpose-built legal-reasoning model (a best-effort LoRA reproduction; Section 6) closes this gap. A red-teaming evaluation on five core models finds compliance with an adversarial instruction hidden in the case facts (73.3%-96.4%) exceeds verdict-swap sensitivity by a wide margin, holding without exception across model rankings. Naming a legal authority is thus a poor proxy for a verdict’s dependence on it, while the same verdict remains separately vulnerable to adversarial manipulation. Both findings replicate across checks ruling out prompt-wording noise and confounded sampling, and bear directly on the use of generated legal explanations as compliance or audit artefacts.

[163] Harness Compilation: Which Decisions Should a Small Vision-Language Model Keep? cs.AI | cs.CVPDF

Minhao Fan, Yinyi Liu, Jiayu Zhao, Zihan Teng, Song Chen

TL;DR: 提出 Harness Compilation，离线改造小型 VLM 与外部工具 harness 的分工，以学生实际执行轨迹驱动复用知识、决策策略和部署配置迭代。

Details

Motivation: 小型 VLM 可能能读外部证据却难主动获取它。系统可通过改造 harness 补足，但哪些决策应由模型承担、哪些交给外部流程需要按模型行为实测决定。

Result: 冻结小型学生 VLM，由大型教师基于学生执行轨迹修订可复用内容和控制策略，再用独立验证集选择部署 harness；部署无需更新学生权重或调用教师。七种视觉问答设置、最多 9B 学生上，三次独立构建平均较裸模型提升 9.9–23.9 分。五类运行决策（调用、选择、参数生成、证据整合、弃答）干预分析发现，请求证据和生成开放查询较昂贵，受限选择及阅读已给文本仍适合交给学生。事实卡对十个学生都有效，但决策策略迁移不均；不适配时应为新模型重新编译。100 条练习数据时，HC 三项任务超过仅答案 LoRA；更大训练预算可匹配/超过固定 harness，与之结合又在 SlideVQA 超过各自单用。

Insight: 不要统一移除或保留学生的所有决策，而应依据行为证据分配任务；事实知识迁移较稳定，流程控制策略更依赖模型。

Abstract: Small vision-language models may be able to read external evidence yet struggle to obtain it. We introduce Harness Compilation (HC), an offline procedure that adapts the division of work between a frozen small VLM and its external harness. A large teacher uses student execution traces to revise reusable content and control, while a separate validation set selects the deployed harness. Deployment requires neither weight updates nor teacher calls. Across seven visual question-answering settings with students of at most 9B parameters, HC improves scores over bare students by 9.9-23.9 points, averaged over three independent builds per setting. Interventions on five runtime decision types (invocation, selection, argument generation, evidence integration and abstention) show why this allocation matters: requesting evidence and generating open queries can be costly, whereas bounded choices and reading supplied text can remain useful student work. Fact cards benefit all ten evaluated students, but decision policies transfer unevenly. Recompilation for a new student model helps when the transferred interface no longer fits the student. With 100 practice items, HC exceeds answer-only LoRA on three tasks. Larger training budgets can match or surpass a fixed harness, while combining the two improves SlideVQA beyond either alone. These findings support allocating work from measured student behavior rather than uniformly removing decisions.

[164] MultiWorldBench: Do Independently Controlled Views Describe One Shared World? cs.AI | cs.CVPDF

Zhangbo Xu, Ruoxi Zhang, Rui Hu, Yisong Wang

TL;DR: 提出 MultiWorldBench，以 495 种配置诊断多人世界模型的独立控制视角是否真正描述同一持久共享世界。

Details

Motivation: 多人世界模型需要让独立控制的视角在共享环境和状态下保持一致；单个视图看起来合理并不保证其与其他视图代表同一个持续变化的世界。

Result: MultiWorldBench 是 Minecraft 基准，含七个任务套件、十项能力，评估独立控制、跨视角运动、共享状态同步、持久性、结构推理、并发交互和延迟回访。对 Solaris、Gamma-World、MineWorld 评估，并以 Engine GT 为参考。生成系统十项能力平均分最高为 Gamma-World 21.39，其次 Solaris 20.88、MineWorld 1.89，Engine GT 为 91.69。Gamma-World 在多项控制、共享状态和回访能力上较好，Solaris 在跨视图运动和竞态一致性上领先。但生成系统的状态持久性最高仅 8.00、结构一致性最高仅 1.33，空间推理和建筑身份保持均未成功。人类偏好总体排序相同，与自动评估维度 Spearman 相关均值为 0.96。

Insight: 当前多人生成模型可以生成可信单视角，却仍难维护世界结构和状态持久性；基准将共享世界一致性与单视图画质区分开来。

Abstract: Multiplayer world models must ensure that independently controlled views remain consistent with one shared and persistent world. We introduce MultiWorldBench, a diagnostic Minecraft benchmark containing 495 case configurations across seven task suites and ten capabilities, including independent control, cross-view motion, shared-state synchronization, persistence, structural reasoning, concurrent interaction, and delayed revisit. We evaluate Solaris, Gamma-World, and MineWorld, using Engine GT as a reference. Gamma-World achieves the highest ten-capability average among the generated systems at 21.39, followed by Solaris at 20.88 and MineWorld at 1.89, while Engine GT reaches 91.69. Gamma-World performs better on several control, shared-state, and revisit capabilities, whereas Solaris leads in cross-view motion and race-condition consistency. Nevertheless, all generated systems score at most 8.00 on state persistence and 1.33 on structural consistency, and none succeeds in spatial reasoning or building-identity preservation. Human preferences produce the same overall ranking and show strong alignment with the automatic evaluation, with a mean dimension-level Spearman correlation of 0.96. These results show that plausible individual views do not yet constitute a coherent multiplayer world.

cs.CR [Back]

[165] False Claims, Credible Images: A Red-Teaming Benchmark for Commercial Image Generators cs.CR | cs.CVPDF

Zeyu Ye, Yanchun Li, Sibei He, Meng Xie, Hangtao Zhang

TL;DR: 提出 EpiReal-Bench 和 EpiReal-Attack，系统评测商业图像生成器生成逼真视觉错误信息的风险，并用技能引导黑盒优化绕过对齐防护。

Details

Motivation: 商业图像生成模型能生成逼真且含文本的新闻、教材等视觉材料，也能伪造现实错误信息。模型可能知道声明为假，却仍将该声明制作成可信视觉证据，现有安全评测较少覆盖这种视觉误导边界。

Result: EpiReal-Bench 包含 10,000 条虚假声明提示及 10,000 张对应生成图，覆盖 10 类现实声明与 10 种可信视觉格式。EpiReal-Attack 通过 Pareto 选择和多模态反馈，在保持真实感、文字可读和语义忠实的同时优化绕过命令。四个商业模型上，超过 70% 的虚假声明提示产生忠实描绘错误信息的图像；攻击框架将该比率提升至 95%。

Insight: 图像安全过滤不应只看画面是否含有传统有害内容，还需判断图像文本/叙事断言是否伪造现实证据；生成物易传播且可信，现有对齐存在明显缺口。

Abstract: Image-generation models can now produce text-rich, natural-looking visual artifacts that are hard to distinguish from real-world evidence, such as news reports and textbook pages. Yet, the same capability introduces a new risk: these models can just as easily fabricate visual misinformation. Even commercial models (e.g., GPT-Image-2) readily produce it. Curiously, we find that these models can recognize a claim as false when asked, yet still render that very claim as credible visual evidence. This discrepancy points to a blind spot in current alignment: safeguards judge what an image shows, not what it asserts; however, existing red-teaming benchmarks target conventional harmful content, such as violent or explicit imagery, and say little about where the alignment boundaries lie for visual misinformation, especially in commercial models. To fill this gap, we introduce EpiReal-Bench, the first systematic benchmark for evaluating visual misinformation risks in commercial image generators, comprising 10k false-claim prompts and 10k corresponding generated images that span 10 real-world claim categories and 10 credible visual formats. We further introduce EpiReal-Attack, a skill-guided black-box optimization framework that uses Pareto-based selection and multimodal feedback to identify commands that bypass alignment safeguards while preserving visual realism, textual legibility, and semantic fidelity. Experiments on four commercial models reveal that more than 70% of false-claim prompts elicit images that faithfully depict the corresponding misinformation, and EpiReal-Attack pushes this rate to 95%. Most worryingly, these models are only a click away, and their outputs are cheap to spread yet hard to disbelieve, leaving this dimension of alignment largely unguarded.

cs.GR [Back]

[166] WorldBench: Evaluating LLMs on Three.js Voxel World Generation cs.GR | cs.AI | cs.CL | cs.CV | cs.SEPDF

Krish Bakshi

TL;DR: 提出 WorldBench，动态探索并测量 LLM 生成的 Three.js 体素世界，再与源代码交叉核验，避免单视角或代码文本评分带来的误判。

Details

Motivation: LLM 可生成完整交互式 3D 世界，但自动评估不可靠：VLM 只看有限渲染视图，容易漏掉小物体；LLM 只读源代码，又可能把未运行的功能算作已实现。

Result: WorldBench 基于一条描述含十种生态、物理、昼夜和季节变化的漂浮体素岛提示构建评测。评估器运行世界、控制时间、环绕视角，并调用导航代理逐个呈现生态，同时读取代码；代码声明必须与实际内容吻合，视觉属性则在可测时对照像素。五个前沿模型生成世界中，纯视觉和纯代码视角在 32% 必需项目上意见不一致，常见原因包括代码有但画面不显示和固定视角遗漏小物体。删除功能的变异测试中，纯代码裁判对五个被移除功能中的四个仍给满分；WorldBench 将被移除功能保留的得分从 7.11 中的 5.44 降至 3.55，剩余误判主要来自代码存在但从未运行。

Insight: 评估代码生成世界必须结合运行时探索、像素测量和代码核验；只看任一通道都会把未执行功能或未观察内容误当作正确。

Abstract: Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges take one view of the output: a vision-language model scores a few rendered snapshots, or a language model reads the source. On worlds written by five frontier models we find that the two views disagree on 32% of required items, mostly code that no frame shows, and that fixed views miss small close-up contents. We present WorldBench, a benchmark and judge for open-ended, LLM-generated Three.js worlds. From one prompt describing a floating voxel island with ten biomes, physics, and day/night and seasonal cycles, the judge explores the running world, controlling its clock, orbiting it, and sending a navigator agent to frame each biome, and reads the code for what it sees. Neither channel is trusted on its own: a code quote counts only if it is text the source contains, and visual claims are checked against measured pixels where the property is measurable. A mutation test, in which we remove features by construction, shows that code-only judging gives full credit to four of five removed features, because their code remains in the file. Our judge cuts the points kept on removed features by a third (5.44 to 3.55 of 7.11), and what it still credits is mostly code that exists but never runs. We evaluate five frontier models: Claude Fable 5.1, GPT-6 Astra, Kimi K3, Grok 4.7 and Gemini 3.1 Pro. Code, prompt, tests and judge configuration are available at https://github.com/KrishBakshi/worldbench

cs.HC [Back]

[167] Hybrid Cinematography: Previsualizing and Managing Hallucination Risk in Generative Video Reshooting cs.HC | cs.CVPDF

Nhan, Tran, Neal Wadhwa, Abe Davis, Stefan Stojanov

TL;DR: 提出混合电影摄影工作流，在拍摄现场以可编辑 3D 镜头计划和代理视频实时预估生成式重拍可能需要的幻觉区域，帮助导演及时补拍或调整机位。

Details

Motivation: 传统拍摄在镜头完成后基本锁定机位；生成式视频重拍可以事后改机位，但可能需要生成未拍摄到的内容，且风险常在离开片场后才被发现。

Result: Hybrid Cinematography 用 3D 镜头计划和拍摄代理素材实时评估幻觉风险。发现素材支持不足后，创作者可迭代调整镜头计划、探索兼顾实拍与生成的机位、拍摄引导式补镜，或知情接受幻觉风险。作者实现移动 AR 片场规划/拍摄/审阅应用及处理现有视频的离线流程。与资深电影制作人研究显示，提前可视化风险会影响机位决策，并暴露创作意图与生成幻觉之间的张力。

Insight: 生成式重拍风险可在拍摄阶段通过预演反馈进行管理，降低后期发现无法可靠生成的内容缺口；研究侧重工作流与创作者决策，而非生成模型质量指标。

Abstract: On a film set, the camera move is committed during a take. Generative video reshooting lets filmmakers change it afterward, but may require hallucinating unrecorded content, a gap sometimes discovered only after leaving the set. We present Hybrid Cinematography, a workflow that bridges physical capture and generative reshooting to manage hallucination risk while filmmakers can still act on it. Using an editable 3D shot plan and a proxy of the take, our previsualization evaluates hallucination risk in real time. Seeing where the take lacks support, filmmakers can iteratively adjust the plan, explore moves that balance capture and generation, shoot guided pickups, or knowingly accept hallucination. We demonstrate the workflow through a mobile augmented reality application for on-set planning, capture, and review, and an offline pipeline for existing video. A study with experienced filmmakers reveals how previsualizing risk informs camera decisions and exposes tensions between creative intent and generative hallucination.

cs.IR [Back]

[168] Autoregressive Retriever: Improving Query Understanding from Item Feedback for Universal Multimodal Retrieval cs.IR | cs.CVPDF

Jianfei Zhao, Yifan Wang, Feng Zhang, Xin Sun, Chong Feng

TL;DR: 提出自回归检索器 ARR，在多模态检索时交替检索信息项并更新查询嵌入，让已检索内容逐步澄清后续搜索需求。

Details

Motivation: 通用多模态检索通常只编码查询一次，随后对独立索引项按相似度排序；已检索内容若能帮助理解需求，却无法反过来改写查询表示。

Result: ARR 交替选取信息项和更新查询嵌入，最后用更新表示排序整个集合。监督微调用逐步对比监督训练反馈使用；RL 将反馈项视为动作，以相关项最终 reciprocal rank 优化选择。查询侧 adapter 可在固定项目索引上学习该过程。作者在域内和零样本基准上均报告强性能，平均超过比较基线；进一步分析显示推理时反馈提升检索，训练时学习反馈还会改进初始查询嵌入，即使尚未观察任何项目。摘要未列具体分数。

Insight: 检索结果可以作为查询理解的反馈，而不仅是最终候选；通过固定索引与查询侧适配器实现无需重建 item embedding 的反馈学习。

Abstract: Universal multimodal retrieval typically encodes a query once and ranks independently indexed items by embedding similarity. This design supports efficient search, but leaves the query representation unchanged even when retrieved items could help clarify the information need. We introduce the AutoRegressive Retriever (ARR), a multimodal retrieval model that learns both to select informative items and to use their content to refine subsequent retrieval. ARR alternates between retrieving an item and updating the query embedding, then uses the final embedding to rank the collection. Supervised fine-tuning teaches the encoder to use feedback through stepwise contrastive supervision. Reinforcement learning treats feedback items as actions and optimizes their selection using the final reciprocal rank of a relevant item. A query-side adapter enables this optimization against a fixed item index. ARR demonstrates strong retrieval performance on both in-domain and zero-shot benchmarks, outperforming the compared baselines on average. Further analyses show that feedback improves retrieval at inference time and that training with feedback also improves the initial query embedding, before any item is observed.

cs.LG [Back]

[169] Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models cs.LG | cs.CLPDF

Yi Wang, Rui Qian, Yu Li, Haoyang Yao, Wenjie Wang

TL;DR: 提出 LoopOPD/D-LoopOPD，用 LoopLM 后层循环计算作为自身教师，对中间循环学生进行在线蒸馏，并随共享参数更新动态刷新教师。

Details

Motivation: 循环语言模型通过在多个计算步骤重复使用共享参数，以较少参数增加推理能力，但后训练困难：奖励监督稀疏且难跨循环扩展，外部教师又可能不可用或与学生上下文不匹配。

Result: LoopOPD 在学生生成 rollout 上使用冻结的末端循环策略作为计算更充分的教师，为中间循环提供密集监督，不需外部教师或特权信息。D-LoopOPD 随共享模型参数更新持续刷新末端循环教师，实现循环自改进。论文分析蒸馏更新如何沿循环深度传播，并推导单次更新能同时改善两个深度局部表现的充分条件。在 Ouro-Thinking 模型实验中，LoopOPD 提升数学推理，D-LoopOPD 动态更新后进一步改善；尽管只用数学数据训练，通用推理和代码基准也有收益。

Insight: 循环模型可以利用额外内部计算形成自蒸馏教师，避免外部教师的可用性与上下文不匹配问题；动态教师刷新是进一步改进来源。摘要没有列出具体分数。

Abstract: Looped Language Models (LoopLMs) offer a parameter efficient approach to scaling reasoning by reusing shared parameters across recurrent computation steps. Despite their promise, effective post-training of LoopLMs remains challenging. Existing approaches either provide reward based supervision that is sparse or costly to extend across loops, or rely on external teachers or privileged information, leading to limited teacher availability or teacher-student context mismatch. To address these limitations, we introduce LoopOPD, a cross-loop on-policy distillation framework that uses additional recurrent computation within a LoopLM as its own source of supervision. LoopOPD uses a frozen terminal loop policy as a compute privileged teacher for an intermediate loop student on student generated rollouts, providing dense supervision without an external teacher or privileged information. We further propose Dynamic LoopOPD (D-LoopOPD), which continually refreshes the terminal loop teacher as the shared model parameters are updated, enabling recurrent self-improvement. We characterize how distillation updates propagate across loop depths and derive sufficient conditions under which a single update yields simultaneous local improvement at both loop depths. Experiments on Ouro-Thinking models show that LoopOPD improves mathematical reasoning, while D-LoopOPD yields further gains through dynamic teacher updates. Despite being trained only on mathematical data, the resulting models also improve on general reasoning and code generation benchmarks, demonstrating that recurrent computation can serve as an effective source of supervision for LoopLMs. Our code and model checkpoints will be released upon acceptance.

[170] Conversational Task Disambiguation over Tabular Data: Leakage-Aware Formulation, Benchmark Suite, and Training cs.LG | cs.AI | cs.CLPDF

Nafiseh Ghoroghchian, Luis Scoccola, Tina Sedaghat, Omid Vaheb, Hannah Chen

TL;DR: 提出可泄漏感知的表格对话任务消歧框架，并构建 AmbiTab，将澄清提问策略与最终任务解题分开评测、训练。

Details

Motivation: 表格对话任务需先通过交流补足用户意图，再执行表格或数据库操作。传统任务成功率混合了澄清和解题能力，也可能被用户模拟器泄漏的额外信息抬高；已有数据还缺少统一歧义表示与访问边界。

Result: 框架定义含歧义的可验证任务，显式分解 agent 的提问策略/解题策略与环境的 oracle/verifier，提供分离评测指标、oracle 泄漏定义、无需评审器的泄漏诊断以及提问策略训练目标。其在 Text-to-SQL 上实例化为 AmbiTab，统一六个歧义数据集，并说明 agent、oracle、verifier 可访问的信息。作者评估澄清策略与泄漏后用强化学习训练提问者；训练后的提问策略在六个数据集上均改善消歧指标，在五个数据集上提高任务成功率，泄漏诊断可测量训练对 oracle 泄漏的影响。

Insight: 将交互式澄清与最终查询执行拆开，有助于判断成功究竟来自更好的提问还是来自模拟器泄漏；访问边界被写入基准表示。摘要未报告具体提升数值。

Abstract: Conversational task disambiguation over tabular data uses dialogue to resolve missing information about a user’s intended task before producing a solution over tables or databases. Existing evaluation and training lack a leakage-aware foundation. Task success mixes the agent’s disambiguation and solution-generation capabilities and can also reflect oracle leakage, that is, information that a user simulator reveals beyond what a real user would. Existing datasets also lack a shared representation of ambiguities and access boundaries. We introduce the notion of an ambiguous verifiable task, which formalizes ambiguities and resolutions, decomposing the agent into an asking policy and a solution policy, and the environment into an oracle and verifier. This framework provides baselines and metrics for evaluating task disambiguation separately from solution generation, formal definitions of oracle leakage, judge-free leakage diagnostics, and a training objective for the asking policy. We instantiate the framework in text-to-SQL with AmbiTab, a benchmark suite that unifies six ambiguous datasets under a common representation specifying what the agent, oracle, and verifier may access. We evaluate clarification strategies and oracle leakage, and train an asking policy with reinforcement learning. The trained asker improves our disambiguation metrics on all six datasets and task success on five, and our leakage diagnostics measure how training affects oracle leakage.

[171] Measuring and Mitigating Solution Mode Collapse in RLVR cs.LG | cs.CLPDF

Liv G. d’Aliberti, Marwa Abdulhai, Sofiia Druchyna, Peter Henderson, Manoel Horta Ribeiro

TL;DR: 提出 ModeBench 测量 RLVR 训练中正确解法模式多样性，并提出 Re:Max 把每个已验证解法模式等量放入回放缓冲区以缓解模式坍缩。

Details

Motivation: 可验证奖励通常只看答案对错，不区分正确答案是常见模式还是新解法；保留多种正确策略可给用户选择，也可能改善模型整体问题解决能力。

Result: ModeBench 为多解任务提供同时返回正确性和发现模式的验证器。作者观察到，RLVR 后训练中即使准确率不降甚至提升，概率也会集中到更少正确模式；前沿模型训练前本身也已有高度集中。Re:Max 为每个新发现的模式保存一个验证样例，并均匀训练这些记忆，使只发现过一次的解法与频繁出现的模式获得同等训练。三个模型规模、两种 RL 目标和更困难任务构造上的实验显示，回放同时提高策略成功率和成功解法多样性；摘要未列具体数值。

Insight: 若奖励只看正确性，RLVR 可能以多样性换取准确率或稳定性；按模式而非按出现频次均衡回放可保留多种有效解法。

Abstract: A language model (LM) can usually answer the same question in more than one way, but reinforcement learning with verifiable rewards (RLVR) is indifferent to which correct answer a model produces. A solution will earn the same reward whether it is the thousandth copy of a familiar answer or one the model has never produced before. Yet, there is potential value in having the model retain multiple correct solutions as it is trained. For instance, multiple modes may give users a choice and provide problem-solving strategies that improve overall model performance. Here, we introduce ModeBench, a benchmark of multi-solution tasks in which the verifier returns both correctness and mode discovered. We then use ModeBench to measure how solution diversity changes under RLVR post-training. We find that RLVR post-training concentrates probability onto fewer correct modes even as accuracy holds or improves, and moreover, that frontier models are already highly concentrated. We then introduce our solution, Re:Max, which stores one verified example per discovered mode in a replay buffer and trains on those stored modes uniformly. A solution found once is, therefore, practiced as often as one found repeatedly. Across three model scales, two RL objectives, and harder task constructions, replay improves both how often a policy succeeds and how many different ways it can succeed.

[172] ActiveMedAgent: Cost-Aware Trajectory Learning for Multimodal Medical Diagnosis cs.LG | cs.AI | cs.CL | cs.CV | cs.MMPDF

Weiwei Ma, Xiaobing Yu, Peijie Qiu, Jin Yang, Zhaoqi An

TL;DR: 提出 ActiveMedAgent，通过离线学习的轻量控制器为每步新增医学证据估计诊断效用减成本，决定继续获取模态还是提交诊断。

Details

Motivation: 临床诊断是顺序获取证据的过程，医生通常只在额外检查有望消除不确定性时才增加成本。多模态 AI 也需学会何时请求更多证据、何时停止。

Result: 框架给定冻结且通过 API 调用的 VLM，维护候选诊断概率分布，估计每次信息获取的逐步诊断效用减去成本，并在这些轨迹上离线训练轻量 MLP 控制器。三个常用基准上，轨迹策略学习持续优于无引导的信息获取及全模态基线。作者识别出信息过载效应：在 175 个案例中，智能体能用更少通道做出正确诊断，而全模态基线失败，表明学习省略什么与学习获取什么同样重要。摘要没有提供各基准具体分数。

Insight: 主动控制模态获取可避免不必要输入造成的信息过载，并节约成本；摘要报告了更少通道成功、全模态失败的 175 个案例。

Abstract: Clinical diagnosis is inherently sequential: clinicians escalate from cheap to costly tests only when additional evidence is expected to resolve diagnostic uncertainty. We present ActiveMedAgent, a framework that brings this cost-aware sequential logic to multimodal medical AI. Given a frozen, API-accessed vision-language model, ActiveMedAgent tracks probability distributions over candidate diagnoses and scores each acquisition by its per-step diagnostic utility minus cost. A lightweight MLP controller is then trained offline on these scored trajectories, learning when to request additional evidence and when to commit. Across three commonly used benchmarks, trajectory-based policy learning consistently outperforms both unguided acquisition and full-modality baselines. Notably, we identify an information overload effect. In 175 cases, the agent produces a correct diagnosis with fewer channels while the full-modality baseline fails, showing that learning what to omit can be as important as learning what to acquire.

[173] Do LLMs Learn from Rewards in Context? : Rethinking the role of reward in In-Context Reinforcement Learning cs.LG | cs.CLPDF

Minchan Kwon, Seunghee Koh, Sunghyun Baek, Minsung Bae, Junmo Kim

TL;DR: 通过受控实验检验直接上下文强化学习 ICRL 是否真正从奖励中学习，发现轨迹带来改进但奖励改变几乎不影响结果，更像上下文学习而非推理时 RL。

Details

Motivation: LLM 代理可在推理时积累上下文经验而不更新参数，常被称为 in-context RL；但直接输入轨迹—奖励对的上下文学习是否让奖励发挥 RL 学习信号作用尚未验证。

Result: 作者在四个基准、六个模型上测试直接 ICRL，比较奖励翻转、随机化、移除，并加入明确鼓励探索、利用或考虑奖励的元提示。奖励确实被读取，但影响很小：不同奖励处理下性能改进曲线几乎不变。轨迹能带来改进，但语义内容并非关键，打乱或破坏的轨迹与真实轨迹效果相当。这些现象与 ICL 的已知行为相符，因此作者认为直接 ICRL 更适合视为 ICL 的一种形式，而非推理时 RL。

Insight: 依赖上下文示例提升不等于从奖励学习；输入分布和示范内容等 ICL 因素可能比奖励塑形和探索更重要，对代理记忆设计有影响。

Abstract: LLM agents increasingly improve at inference time by accumulating experience in context rather than by updating parameters. This process is often described as in-context reinforcement learning (ICRL). Whether in-context learning (ICL) can actually play the role of RL, however, has not been tested. We study this question in its simplest form, direct ICRL, where the model conditions directly on raw trajectory-reward pairs, and ask whether the reward acts as a learning signal. Through controlled experiments on four benchmarks across six models, we find that the reward is read, but its effect is small: flipping, randomizing, or removing the reward leaves the improvement curve almost unchanged, and this holds even under meta-prompts that explicitly instruct the model to explore, exploit, or reason over rewards. Trajectories drive improvement, but not through their semantic content: shuffled or corrupted trajectories work as well as real ones. These patterns closely mirror those known in ICL, suggesting that direct ICRL is better understood as a special case of ICL than as inference-time RL. This reframing has implications for agent memory design: ICL factors such as input distribution and demonstrations may matter more than RL elements such as reward shaping and exploration.

[174] The Lattice of Transition Laws cs.LG | cs.CLPDF

T. Y. Tsui, Jiatao Gu, Lingjie Liu

TL;DR: 提出转移律格点统一描述自回归、扩散及二者之间的解码计划，并以依赖代价预测固定步数预算下不同计划的排序。

Details

Motivation: 扩散和自回归通常视为不同生成范式，混合模型也多手工固定解码计划。论文询问是否能在实际解码前预测给定步数预算下各计划的性能优劣。

Result: 作者将这些模型表示为同一 corruption lattice 上的路径，把计划代价定义为并行步骤丢弃的依赖关系。零代价计划的最少步骤数由数据几何决定，对离散 token 和连续场相同。若数据在图上满足马尔可夫性并沿路径依赖，最少步数等于图的 tree-depth：序列为对数级，网格则与边长线性相关。少于该阈值时任何计划都有正代价，可用预训练权重估计的两两依赖核在解码前排序。文本、图像和视频生成实验验证了大部分计划排序预测，覆盖多个指标和基准。

Insight: 同一依赖几何框架可比较自回归与扩散解码，并给出并行步数的结构下限；论文报告多数而非全部预测获实验支持。

Abstract: Diffusion and autoregression (AR) have long been seen as different categories of generative models, with diffusion specialising in continuous fields and AR specialising in discrete tokens. Recent work seeks to combine the advantages of the two models, and each hybrid fixes its decoding schedule by design. In this paper, we ask whether the performance of decoding schedules of one model can be predicted before decoding at a fixed number of steps. We describe diffusion, AR, and models in between as paths on one corruption lattice, and define the cost of a schedule as the dependence its parallel steps discard. The cost shows that the fewest steps of a zero-cost schedule are set by the geometry of the data, in the same way for tokens and for continuous fields. In particular, for data that are Markov on a graph and dependent along its paths, the fewest steps equal the graph’s treedepth, which is logarithmic in the length of a sequence and linear in the side length of a grid. With fewer steps than the treedepth, every schedule pays a positive cost, whose ranking we predict before decoding with a kernel of pairwise dependence estimated from pretrained weights. Across text generation, image generation, and video generation, we verify most of the predictions about the rankings of different schedules under different metrics and benchmarks. This work therefore provides a design principle for decoding for future AR models, diffusion models, and anything in between. Our code is available at https://github.com/TSUITUENYUE/The-Lattice-of-Transition-Laws.

[175] Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals cs.LG | cs.CLPDF

Lei Zhao, Qichao Zhao, Bowen Zuo, Qishi Zhan

TL;DR: 分析在线策略蒸馏 OPD 的学习信号消失问题，比较大规模教师与自强化教师的训练平台期，并提出与学生初始策略接近时的局部恢复理论保证。

Details

Motivation: OPD 可把能力从模型间迁移，但失效机制仍不清楚。作者发现在代码与数学推理任务中，大规模教师可能让学生较早进入损失平台，需厘清这与自强化教师效果差异的关系。

Result: 200 次更新后，大教师 OPD 平均最终损失降低 25.1%，而先对初始学生进一步 RL 训练得到的自强化教师对应降低 96.2%。在小学习率连续时间理想化系统中分析，并用训练日志发现平台期与梯度学习信号代理早期下降、但损失仍较高相关；作者明确指出这并未证明梯度变弱的根因。理论上，对共享参数化下足够接近初始学生的教师，证明了满足正则条件时的局部恢复保证。平台期与非平台期运行中，相对参数变化仅 0.025%–0.098%，OPD 前后表示线性 CKA 各层均大于 0.98。作者据此提出表示适应不足可能导致信号坍缩，但将其标为待验证假设。

Insight: 仅比较教师强弱不足以解释 OPD 效果，学生表示变化极小和梯度信号衰减可能有关；论文区分了经验相关性、条件理论保证和仍待检验的机制假设。

Abstract: On-policy distillation (OPD) enables effective capability transfer between language models, yet the mechanisms underlying its failures are not fully understood. Across code generation and mathematical reasoning, OPD with larger-scale teachers exhibits early loss plateaus, with an average final loss reduction of 25.1% after 200 updates, compared with 96.2% for self-RL teachers, obtained by further reinforcement learning (RL) training of the initial student. To understand this difference, we analyze OPD as an idealized continuous-time dynamical system in the small-learning-rate limit. Our training-log diagnostics associate these plateaus with an early decline in a gradient-based learning-signal proxy while substantial loss remains; these measurements do not establish why the underlying gradient weakens. We further prove a local recovery guarantee for teachers sufficiently close to the initial student in a shared parameterization under regularity conditions, offering a conditional explanation for the success of self-RL teachers in our experiments. Across runs with and without loss plateaus, we observe small relative parameter changes (0.025-0.098%) and high similarity between the student’s representations before and after OPD (linear CKA $>0.98$ across layers). These observations suggest that limited representation adaptation may contribute to learning-signal collapse, a hypothesis that remains to be tested. Code is available at https://github.com/leizhao7/opd-learning-signals.

[176] From a Prompt to Repertoires: Evolving Functional REpertoires Enable LLM Continual Learning cs.LG | cs.CLPDF

Fengyuan Liu, Yue Wang, Hangxi Guo, Fengyuan Liu, Chenxu Wu

TL;DR: 提出 EFRE，以可进化的功能集合取代单一提示进行大模型持续学习：兼容新任务时细化已有功能，发生冲突时创建新功能以缓解灾难性遗忘。

Details

Motivation: 持续学习要求学习新技能且不损害已有能力。提示优化虽比参数更新高效，在单项任务可有竞争力，但连续适配会灾难性遗忘，规则还会过拟合局部任务分布。

Result: EFRE 按新任务与已有功能的兼容性演化功能库：兼容更新细化现有功能，冲突更新则新增功能。在三任务持续学习序列中，最终平均表现比 GRPO 高 7.50 个百分点；适配 Bio 任务后，FinQA 性能仅下降 1.56 个百分点，而基础提示优化方法下降 25.10 个百分点。在最小代理系统和不同骨干上也观察到持续改善。

Insight: 持续学习可通过管理可组合功能提示的演化来进行，而非总是更新单一提示；冲突触发新功能有助于避免把新旧规则硬合并造成遗忘。

Abstract: Continual learning remains challenging for large language models, which must enable models to acquire new skills and knowledge without degrading existing capabilities. Existing approaches typically address this challenge by carefully designing how model parameters are updated. In contrast, prompt optimization avoids costly parameter updates while achieving competitive or even superior performance to reinforcement learning methods such as GRPO on individual knowledge-intensive and reasoning tasks. This raises a natural question: \textit{Can prompt optimization, as an efficient adaptation approach, be directly applied to continual learning?} Our analysis shows that, under sequential task adaptation, it suffers from catastrophic forgetting, while optimized prompts accumulate rules that overfit to local task distributions. To address these limitations, we propose \emph{Evolving Functional REpertoires} (EFRE), which replaces a single prompt with a repertoire of functions that evolves as new tasks arrive: compatible updates refine existing functions, while conflicting updates trigger the emergence of new ones. On a three-task continual-learning stream, EFRE achieves a final average performance 7.50 percentage points higher than GRPO. Moreover, after adaptation to the Bio task, its performance on FinQA decreases by only 1.56 percentage points, compared with 25.10 percentage points for the base prompt optimization method. We further instantiate EFRE in a minimal agent system and observe consistent improvements across different backbone models. Overall, these results demonstrate EFRE’s strong performance in continual learning for large language models and highlight its substantial potential for continual learning in advanced agent systems.

[177] 4-Tensor Attention Model for Semantic Physical Reality cs.LG | cs.CLPDF

Jongwook Kim, Sangheon Yun

TL;DR: 提出四阶张量注意力更新，用单一 softmax 在位置—时间窗口和语义/时间上下文双纤维间预测场景下一语义状态，并在故事续写任务上与普通 Transformer 对比。

Details

Motivation: 作者探索一种面向视频生成与机器人规划的语义物理现实表示：将场景状态按位置、时间及两类语义维度共同组织，避免仅在一维序列上进行注意力更新。

Result: 模型窗口由位置和时间构成，另含语义纤维及时间上下文纤维，注意力在窗口上联合归一化；编码器、渲染器和规划器不属于该更新模块。为单独测试更新本身，作者在 ROCStories 上训练相同的下一句预测任务。验证集每种设置仅用一个随机种子；三个匹配设置下，末句交叉熵比自由运行一维 Transformer 低：H=2,L=2 时 5.3%，H=4,L=2 时 2.6%，H=4,L=3 时 2.4%。H=4,L=2 两模型参数量相近，172.5M 对 175.9M；同两块 GPU 上四阶模型训练用时 2.4 小时，基线 45.2 小时，基线按目标 token 自由运行逐次前向。

Insight: 初步结果显示联合张量位置结构可在 ROCStories 下一句预测上改善交叉熵及训练时间，但单种子、单一文本任务不足以验证视频生成或机器人规划用途。

Abstract: We describe a 4-tensor attention model that predicts the next semantic state of a scene, for video generation and robot planning. A window of states has positions (x, t) and two fibers, a semantic fiber and a temporal-context fiber, and one softmax normalizes attention jointly over the window. Frames and an agent’s situation are written as those states; the encoder, the renderer, and the planner remain outside the update. To test the update on its own, we train on ROCStories, where each window poses the same next-sentence task at the semantic layer. On the validation split, with one seed per setting, the last-sentence cross-entropy on the three matched settings is lower for the 4-tensor model than for a free-running one-dimensional transformer by 5.3% at H=2, L=2, by 2.6% at H=4, L=2, and by 2.4% at H=4, L=3. At H=4, L=2 the parameter counts are nearly the same, 172.5M and 175.9M. On the same two GPUs that 4-tensor run finished in 2.4 hours and the baseline run in 45.2 hours; the baseline is trained by free-running decoding, one sequential forward pass per target token.

[178] GRPODropout: Less is More for Online Reinforcement Learning Rollouts cs.LG | cs.AI | cs.CLPDF

Hexuan Deng, Zihao Yan, Xuebo Liu, Shuo Nie, Yue Wang

TL;DR: 提出 GRPODropout，在 GRPO 更新前选择性丢弃少量高概率正优势 rollout 并重中心化其余优势，以较少样本减轻策略熵塌缩。

Details

Motivation: GRPO 等 RL 提升语言模型推理，但策略熵塌缩会降低采样多样性、限制探索。除调整奖励/熵正则或 token 权重外，作者探索改变哪些完整 rollout 参与更新。

Result: GRPODropout 在标准策略更新前，按简单策略移除少量高概率且优势为正的 rollout，并对保留优势重新中心化；方法仅改变 rollout 使用方式，计算开销可忽略。作者提供 rollout 层面理论分析以指导设计和阈值选择。实验报告相较原始 GRPO，准确率更高、actor 熵更高，同时策略更新使用更少 rollout 样本；摘要未列具体基准和提升数值。

Insight: 并非每条被采样的正向轨迹都对更新有益；有选择地排除部分高概率正优势样本可能避免过度收缩策略分布。

Abstract: Reinforcement learning (RL) methods such as GRPO substantially improve large language model reasoning but often suffer from policy entropy collapse: the loss of sampling diversity weakens exploration and limits further improvement. Existing methods address this issue either through algorithm-level interventions, such as reward modification and entropy/KL regularization, or through token-level reweighting. We investigate a complementary perspective: entropy collapse can also be mitigated by changing which generated rollouts contribute to policy updates. Under the same sampling budget, not all rollouts contribute positively to an update, and selectively excluding some can improve learning. To address this, we propose GRPODropout: before the standard update, we use a simple strategy that selectively removes a small number of high-probability positive-advantage rollouts and recenters the retained advantages. To motivate this design, we develop a rollout-level theoretical analysis that guides method design and threshold selection. The method changes only rollout usage, and adds negligible computational overhead. Experiments show higher accuracy than original GRPO and higher actor entropy while using fewer rollout samples for updates, illustrating “less is more.” This work provides insight into RL rollout usage: removing some rollouts can improve performance. Code is available at https://github.com/hexuandeng/GRPODropout/.

[179] When KL Regularization Misfires in Group Policy Optimization cs.LG | cs.CLPDF

Fei Ding

TL;DR: 分析 GPO 中 KL 正则与组内奖励交互的七类失效，并提出 ZCPO，用条件 KL 测得的相对漂移校准组内奖励系数。

Details

Motivation: 有时去掉参考策略 KL 正则反而改善组策略优化，说明参考策略信息如何进入组相对更新需重新审视。

Result: 作者分析七类 KL—奖励交互问题：奖励裁剪后残余 KL 更新、梯度抵消后的 KL 更新、同奖励组中的 KL 更新；KL 随响应长度增长且相对贡献失衡；KL 集中于少量 token；以及将 k1 纳入奖励时的采样噪声。ZCPO 用条件 KL 衡量相对漂移，以之校准组内奖励系数并纳入基础代理目标。数学推理实验和消融在作者设置中支持有效性；摘要未列具体数字。

Insight: KL 正则的影响取决于组内奖励结构及 token/长度分布，不宜仅视为统一强度约束。ZCPO 试图让参考策略信号随组内相对偏移调节。

Abstract: Why does removing reference-policy KL regularization sometimes improve group policy optimization? This motivates studying how reference-policy information should enter group-relative updates. We analyze seven potential failure modes in the interactions between KL and rewards: residual KL updates after reward clipping, after gradient cancellation, and in groups with identical rewards; KL growth with response length and an imbalance in its relative contribution; KL concentration on a small number of tokens; and sampling noise when k1 is incorporated into rewards. We propose Zero-Sum Calibrated Policy Optimization (ZCPO), which uses relative drift measured by conditional KL to calibrate within-group reward coefficients and integrates them into the base surrogate. Mathematical reasoning experiments and ablations support this design’s effectiveness in our settings.

[180] Deflating the Hessian: Rank-4 W4A4 Quantization for Multimodal Diffusion Transformers cs.LG | cs.CV | math.OCPDF

Shiwen Wang, Pengxiang Zhao, Xiaoming Yuan

TL;DR: 提出低秩 W4A4 PTQ 框架，通过联合校准目标导出 deflated Hessian，扣除低秩分支已补偿误差并显式建模激活量化噪声。

Details

Motivation: 扩散 Transformer 的低秩支路可缓解 4-bit 权重—激活量化损失，但既有方案常把低秩补偿与残差量化分开优化、需较高秩，或做二阶权重更新却忽略显著的激活量化误差。

Result: 作者将低秩辅助 W4A4 量化表述为耦合校准问题，从联合目标推导优化求解器；消去输出侧低秩因子得到 deflated Hessian，同时引入激活噪声代理抑制激活量化误差。五种扩散骨干上，rank-4 方法的 PSNR、LPIPS 持续优于 rank-4 SVDQuant；在 SANA-1.6B、FLUX.1-schnell、FLUX.1-dev 上超过 rank-32 SVDQuant，秩小 8 倍，量化最高快 6.25 倍。Qwen3-8B 上 MMLU 从 rank-32 SVDQuant 的 61.50% 提升到 68.17%。

Insight: 量化误差、低秩补偿和激活噪声需联合优化；秩更低同时提高图像质量和推理/校准效率。

Abstract: In diffusion transformers, low-rank branches can mitigate 4-bit weight–activation (W4A4) post-training quantization (PTQ) loss by decomposing each weight into a low-bit residual and a high-precision low-rank component. Existing low-rank PTQ approaches, however, either optimize low-rank compensation and residual quantization separately, often requiring higher ranks, or rely on second-order weight updates without explicitly modeling activation quantization error, which becomes particularly pronounced under 4-bit quantization. To address these limitations, we present \method{}, a unified framework modeling low-rank-assisted W4A4 PTQ as a coupled calibration problem and deriving optimization-based solvers from the joint objective. Eliminating the output-side low-rank factor yields a \emph{deflated Hessian} that discounts residual errors already captured by the low-rank component, while an activation-noise surrogate is incorporated to suppress activation quantization error. Across five diffusion backbones, rank-4 \method{} consistently outperforms rank-4 SVDQuant in PSNR and LPIPS. It further surpasses rank-32 SVDQuant on SANA-1.6B, FLUX.1-schnell, and FLUX.1-dev with an $8\times$ smaller rank and up to $6.25\times$ faster quantization. Furthermore, on the Qwen3-8B LLM, rank-4 \method{} improves MMLU accuracy from 61.50% to 68.17% over rank-32 SVDQuant. Overall, \method{} achieves better W4A4 performance with substantially lower rank and quantization cost.

[181] CoPoE: Multimodal Fusion via Decomposable Disease-Coordinate Product-of-Experts for Missing-Modality Alzheimer’s Diagnosis cs.LG | cs.CVPDF

Chihun An, Ikbeom Jang

TL;DR: 提出 CoPoE，用风险、病理、神经退行、临床阶段四轴疾病坐标和掩码专家乘积融合，适应阿尔茨海默病诊断中的任意模态缺失。

Details

Motivation: AD 诊断需融合临床、影像、基因和生物标志物，但临床队列常不规则缺模态；合成缺失输入可能引入人工伪信息，直接池化可用模态又让潜空间难解释。

Result: CoPoE 将多模态证据映射到 RPNS（Risk、Pathology、Neurodegeneration、Stage）结构潜空间，每个观测模态为完整 RPNS 向量建模对角高斯专家，掩码 Product-of-Experts 仅融合实际可用模态；缺失模态不进入融合路径，因此无需在 RPNS 路径进行插补。ADNI 缺模态实验中，在共享非 PET 嵌入基准下，全部模态时性能最佳，15 种观测子集的平均 AUROC 最高，且原始概率 ECE、Brier、NLL 明显改善。PET 监督探针显示全模态下病理 P 轴信息更丰富；缺少直接体液样本输入时仍保留 tau 相关信号。

Insight: 结构化疾病坐标有助于解释模态证据，掩码 PoE 可直接处理不同缺失组合而不生成替代输入；结论限于 ADNI 非 PET 基准。

Abstract: Multimodal Alzheimer’s disease (AD) diagnosis benefits from integrating heterogeneous clinical, imaging, genomic, and biomarker evidence, but clinical cohorts frequently suffer from irregular modality missingness. Existing fusion methods often synthesize absent inputs, risking the introduction of artificial surrogates, or pool available signals into uninterpretable latent spaces. We present CoPoE (Disease-Coordinate Product-of-Experts), a disease-coordinate framework that maps multimodal evidence into a structured latent space partitioned into four distinct biological and clinical axes: genetic Risk, molecular Pathology, Neurodegeneration, and clinical Stage (R/P/N/S). Each observed modality parameterizes a diagonal Gaussian expert over the full RPNS vector, and a masked Product-of-Experts architecture fuses only the available modalities. Consequently, absent modalities add no factor to the fusion path, allowing the network to preserve a robust, decomposable posterior for any non-empty modality subset without synthetic imputation in the RPNS path. Through extensive missing-modality experiments on the ADNI dataset, CoPoE achieves the best all-modality performance and the highest mean AUROC across all 15 observed-subset evaluations among standardized missing-modality fusion baselines under a shared non-PET ADNI embedding benchmark, while substantially improving raw-probability ECE, Brier score, and NLL. Furthermore, PET-supervised probing shows evidence enrichment within the pathology (P) block under full modalities, with tau-related signal retained even when direct fluid biospecimen inputs are withheld. Our code is available at https://github.com/labhai/CoPoE.

cs.RO [Back]

Gengze Zhou, Yicong Hong, Jiazhao Zhang, Xunyi Zhao, Jian Zhou

TL;DR: 提出 NavGPT-3 分层导航运行时，将长程推理模型和低延迟 VLA 动作策略作为可中断线程协同控制机器人，并通过场景变化按需分配视觉 token。

Details

Motivation: 长程 agentic RL 语言模型能推理并制定多步目标，但实时物理交互仍需要低延迟动作策略。需要一个运行时连接两者，能依据突发事件打断当前线程并切换机器人控制权。

Result: 运行时让推理、行动和监控各自作为拥有独立上下文、工具和权限的线程，由调度器决定哪个线程控制运动。底层 NavGPT VLA 用 1,928 万样本训练，依场景变化按比例分配视觉 token；单独 8B 模型在 R2R-CE 得到 74.51 SR，在 RxR-CE 得到 78.19 SR。完整 harness 在 R2R-CE 达 SOTA 81.51 SR；RxR-CE 首次达到人类水平：成功率 90.43 vs 90.4 SR，路径保真 78.47 vs 77.7 nDTW，每 episode 1 分 22 秒，人类约 3 分钟。工具与动作策略共同作用，使推理循环缩短，系统最短响应时间从每次语言模型决策 3–19 秒降至动作策略 0.5–1 秒（1–2 Hz）。

Insight: 高层语言模型和低层 VLA 的分层线程运行时可在慢速规划与快速反应之间切换；评估显示 harness 整体表现优于单独动作策略。

Abstract: Language models trained with long-horizon agentic reinforcement learning can generalize knowledge through reasoning, express precise actions, and pursue goals over many steps, raising the ceiling on what an embodied agent can understand and decide. Physical interaction, however, remains the domain of action policies, which provide dense, low-latency control. We present NavGPT-3, a harness that connects the two models, with an OS-like runtime built above it: reasoning, acting, and monitoring run as threads with their own context, tools, and permissions, while the runtime schedules them and decides which thread controls the robot’s motion, so that the robot can react to sudden real-world events through interruption and thread switching. Beneath it, our action policy NavGPT VLA, trained on 19.28M examples, allocates visual tokens using codec allocation, in proportion to scene change; its 8B model alone reaches 74.51 SR on R2R-CE and leads RxR-CE with 78.19 SR. With the complete harness, NavGPT-3 sets the state of the art on R2R-CE (81.51 SR) and, for the first time, brings an autonomous agent to human level: on RxR-CE it matches human followers in success (90.43 vs. 90.4 SR) and path fidelity (78.47 vs. 77.7 nDTW) at 1 min 22 s per episode, versus roughly 3 min for a human. We comprehensively ablate the harness design and the interaction between the two models, showing how tools and the action policy shape the path from language-model reasoning to physical control: when NavGPT VLA executes the route, the reasoning loop shortens and the system’s minimum reaction time falls from 3-19 s per language-model decision to 0.5-1 s per action-policy step (1-2 Hz). These results show that designing this embodied interface is central to connecting frontier language-model intelligence with low-level physical control. We will release all models, code, and evaluation records.

[183] SafeInferCom: Safe Inference-Time Compute via Verifier-Guided Mid-Generation Intervention for Robotic Task Planning cs.RO | cs.AI | cs.CL | cs.LOPDF

Weizhe Xu, Jialiang Fan, Mengyu Liu, Fanxin Kong

TL;DR: 提出 SafeInferCom，在推理生成过程中监控并形式化验证机器人计划中间状态，保留有效计划、针对违规部分修正，以减少无效持续推理。

Details

Motivation: 大型推理模型继续生成可能覆盖有效中间计划，或未能修复约束违规，降低规划可靠性并浪费推理计算。

Result: 框架在不打断原始解码轨迹的情况下暴露并验证中间计划；SafeInferCom 用形式验证结果引导生成时保留有效部分、修复错误。多种 LRLM 和规划域实验显示，一次推理常出现推理—响应不一致且自纠正有限；SafeInferCom 相较单次推理提升规划成功率并加速纠错。与迭代细化结合后，相较单纯细化可进一步提升成功率并减少 token。还在 VirtualHome 评估，并提供真实机械臂演示。摘要未给量化成绩。

Insight: 推理时计算并非越多越好，若没有中间计划验证，后续 token 可能破坏正确步骤；约束验证可把算力用于局部修正。

Abstract: Large Reasoning Language Models (LRLMs) enable multi-step reasoning for robotic task planning, but continued reasoning can overwrite valid intermediate plans or leave constraint violations unresolved, reducing planning reliability and wasting inference-time computation. We develop an inference-time monitor that exposes and verifies intermediate plans without disrupting the original decoding trajectory. Building on this monitor, we propose SafeInferCom, a formal verifier-guided framework that preserves valid intermediate plans and directs error correction during generation. Experiments across multiple LRLMs and planning domains reveal reasoning-response inconsistency and limited self-correction under one-shot inference. SafeInferCom improves planning success and accelerates error correction relative to one-shot inference. When combined with iterative refinement, it further improves success while reducing token usage compared with refinement alone. We additionally evaluate SafeInferCom in VirtualHome and provide a real-world robotic-arm demonstration.

[184] When Listening Becomes Easier: Scrubbing Visual Cues for Shortcut-Free VLAs cs.RO | cs.CV | cs.LGPDF

Jasper Gerigk, Kenzo Aspuru-Takata, Chin-Hsuan Wu, Mohammad Mohammadi, Shuhong Zheng

TL;DR: 提出 task scrubbing，通过对抗训练降低 VLA 对视角、背景等视觉捷径的依赖，并使用 action margin 这一无需策略 rollout 的表示指标评估模型易受捷径影响程度。

Details

Motivation: 机器人示范多样性有限，策略可能利用视角或背景等与任务偶然相关的特征。收集丰富示范成本高，因此需要算法方式提高视觉线索变化时的泛化。

Result: 作者发现不同 VLM 骨干对视觉捷径学习的敏感性差异较大，且其行为与 action margin 指标相关。捷径特征常在早期层进入动作表示，不同模型在后层用语言信息修正捷径的程度不同。Task scrubbing 是一种域对抗训练方法，鼓励模型关注语言任务而非视觉伪相关。在多个 VLA、多个视觉线索的仿真和真实世界实验中，该方法提升分布外鲁棒性，并常能消除视觉捷径学习；摘要未列成功率等具体数字。

Insight: 语言条件可用于压制由有限示范诱发的视觉伪相关；不同骨干的内部修正能力不同，action margin 可在不跑策略 rollout 时诊断风险。

Abstract: Shortcut learning is a prevalent issue in robot learning. The limited diversity of robot demonstration datasets can mislead policies into exploiting spurious correlations between tasks and irrelevant features, such as viewpoint or background. Collecting sufficiently diverse robot demonstrations is costly and inefficient, motivating algorithmic alternatives. We focus on vision-language-action (VLA) models and discover that different vision-language model backbones exhibit substantially different levels of susceptibility to visual shortcut learning. We find that model behavior correlates with our proposed representation-level metric, action margin, which requires no policy rollouts. Visual shortcuts consistently enter action representations in early layers, with models differing in the extent to which later layers correct them by incorporating language information. To boost models’ attention to language, we introduce task scrubbing, a new domain-adversarial training method that decreases models’ likelihood of using visual shortcuts and improves VLAs’ generalization. Experiments in both simulation and the real world across multiple VLAs and visual cues show that task scrubbing improves out-of-distribution robustness and often eliminates visual shortcut learning.

[185] OmniDex: Scaling Dexterous Hand Grasping to Diverse Cluttered Scenes cs.RO | cs.CVPDF

Naiyu Fang, Zhongjin Luo, Yuxin Mo, Siyuan Huang, Jianbo Liu

TL;DR: 提出 OmniDex 大规模灵巧抓取数据集与模型，通过高质量物体/支撑面组合生成 260 万场景和 4 亿场景专属抓取真值，并用软赢家通吃与物理约束处理多模态抓取。

Details

Motivation: 灵巧抓取是具身 AI 基础能力，需要大量数据；仿真虽主流，但杂乱场景更接近现实，却受大规模抓取标注稀缺限制。

Result: 作者整理高质量 3D 物体与支撑基座，以 seed-and-filter 方式绕开缓慢的逐场景优化，构建超过 260 万场景及 4 亿条场景专属抓取真值，包含多样真实布局、语义和几何观察。OmniDex 用 Soft Winner-Takes-All 学习和仿人物理约束处理抓取多模态性及末毫米精度错误，并用物理排序避免后优化延迟。实验报告达到 SOTA，并能泛化到多样场景、视角和未见物体；摘要未给具体指标。

Insight: 大规模场景构造与场景级真值扩展了杂乱抓取训练；模型训练时显式施加物理约束，试图在不做昂贵测试时优化的情况下获得精确抓取。

Abstract: Dexterous grasping is the foundational primitive in embodied AI, demanding massive data to train robust models. As real-world data collection is expensive, simulation has become the mainstream paradigm. Yet, while cluttered scenes best reflect real-world applications, learning to grasp within them is bottlenecked by a critical scarcity of large-scale data. To resolve this, we curate high-quality 3D objects and supporting bases, proposing a scalable seed-and-filter strategy that bypasses sluggish scene-level optimization. This yields an unprecedented benchmark comprising over 2.6 million scenes and 0.4B scene-specific grasp ground truths, featuring diverse realistic layouts paired with rich semantic and geometric observations. Furthermore, we introduce the OmniDex model to overcome the grasp multimodality and last-millimeter precision errors plaguing current generative models. By coupling Soft Winner-Takes-All learning with human-inspired physical constraints during training, and utilizing physics-driven ranking, our approach achieves robust dexterous grasping without the latency of post-optimization. Experimental results show that OmniDex model achieves state-of-the-art performance and strong generalization across diverse scenes, views, and unseen objects.

[186] Being-M0.7: A Latent World-Action Model for Humanoid Robots cs.RO | cs.CV | cs.LGPDF

Junpeng Yue, Boyuan Li, Yuxuan Wang, Zepeng Wang, Yuhui Fu

TL;DR: 提出 Being-M0.7，以视频和人体动作混合数据学习潜在世界—动作先验，再经机器人中期训练和动作后训练迁移到人形机器人全身控制。

Details

Motivation: 人形机器人的行走—操作需要结合未来场景变化和全身运动，但机器人示范稀缺。公开视频与人体动作数据更丰富，却常缺少视频—动作成对监督，且人体动作不能直接作为可执行机器人动作。

Result: 作者整理超过 10,000 小时人类中心原始数据，融合纯视频、纯动作和配对视频—动作流，学习互补视觉动力学与全身运动结构。联合预测未来潜视觉状态和动作，促使视觉表示编码未来运动学；机器人中期训练适配机器人视角和身体动力学。动作后训练中，动作专家通过门控交叉注意力融合冻结适配先验的视觉预测表示、当前图像和本体感知，生成可执行全身命令。Being-M0.7 在 SIMPLE 上所有比较基线中总成功率最高，在 Unitree G1 真实行走—操作任务上匹配最强基线；摘要未给分数。

Insight: 混合模态人类数据可先学可迁移的视觉—运动先验，再用机器人阶段适配执行动力学，从而减少直接机器人示范依赖。

Abstract: Humanoid loco-manipulation requires coordinated locomotion and manipulation informed by future scene evolution and whole-body motion, yet learning these capabilities is constrained by scarce robot demonstrations. Human video and motion datasets offer scalable supervision, but many contain only video or motion rather than paired video-motion data. Moreover, human motion does not directly specify executable robot actions. We present Being-M0.7, a latent world-action model that transfers visual-motion priors learned from mixed-modality human data to humanoid control through pre-training, robot mid-training, and action post-training. We curate a corpus from more than 10,000 hours of raw human-centric data, integrating video-only, motion-only, and paired video-motion streams to learn complementary visual dynamics and whole-body kinematic structure. Joint prediction of future latent visual states and motion encourages visual representations to encode future kinematics. Robot mid-training adapts this coarse-grained prior to robot viewpoints and body dynamics. During action post-training, an action expert combines visual predictive representations from the frozen, adapted prior with current images and proprioception through gated cross-attention, grounding predictive context in executable whole-body commands. Being-M0.7 achieves the highest aggregate success rate among the compared baselines on SIMPLE and matches the strongest baseline on real-world Unitree G1 loco-manipulation tasks.

[187] WAM-Cache: Staleness-Bounded KV Reuse for Efficient World Action Models cs.RO | cs.CV | cs.LGPDF

Kai Ding, Yang He, Ruijie Quan, Yi Yang

TL;DR: 提出 WAM-Cache，跨动作块保留视频 DiT 的层级 KV，只稀疏刷新 token，并依动作专家注意力、视觉潜变量惊讶度和严格缓存年龄上限选择刷新位置。

Details

Motivation: 闭环世界动作模型每个动作块都运行视频 DiT 编码当前观察，prefill 是主要计算开销，但多数免训练加速仍完整计算。按视觉变化刷新 token 的直觉策略即使有真值 KV 漂移 oracle 也远弱于密集基线。

Result: 作者发现动作准确性取决于动作专家注意哪里，而非哪个视觉区域发生变化。WAM-Cache 结合动作专家交叉注意力与视觉潜变量 surprise 选刷新集合，并以缓存年龄上限避免误差累积。在 Fast-WAM 上，RoboTwin 2.0、LIBERO 和真实机器人实验中视频 DiT prefill FLOPs 降低 32%–42%；仿真策略成功表现距密集基线 0.7–1.8 个百分点，实机差 2.5 个百分点。

Insight: KV 缓存刷新应面向下游动作使用价值，而非只追踪视觉变化；年龄上限是控制陈旧状态误差的重要约束。

Abstract: World Action Models (WAMs) enable generalist robot manipulation by conditioning an action expert on representations from a pretrained video Diffusion Transformer (DiT). In closed-loop control, the video DiT runs at every chunk to encode the current observation into layerwise key-value (KV) pairs that the action expert queries. This prefill dominates the per-chunk computational cost, yet existing training-free accelerations leave it fully dense. We present WAM-Cache, a training-free framework that retains layerwise key-value representations across chunks and recomputes only a sparse refresh set of tokens. Crucially, we find that the intuitive heuristic of refreshing visually drifted tokens plateaus far below the dense baseline, even with an oracle predicting ground-truth KV drift. Downstream action accuracy is instead governed by where the action expert attends, not by what moved. WAM-Cache therefore selects the refresh set by uniting the action expert’s cross-attention with visual latent surprise, complemented by a strict age bound that suppresses compounding error. On Fast-WAM, WAM-Cache cuts video DiT prefill FLOPs by 32-42% across RoboTwin 2.0, LIBERO, and real-world experiments, while staying within 0.7-1.8 percentage points of the dense policy in simulation and 2.5 points on a real robot.

[188] WARP-VLA: Wrist-Camera Adaptation for View-Robust Policy Execution in Vision-Language-Action Models cs.RO | cs.CVPDF

Junmyeong Lee, Dongmin Shin, Min-Gyu Park, Wooseok Jeon, Inho Chang

TL;DR: 提出 WARP-VLA，以 MoE 专家学习不同腕部相机视角的特征变换，由路由器隐式估计视角并适配，无需相机外参作为额外输入。

Details

Motivation: VLA 通常在固定相机配置训练，跨装置部署很难复现相同相机位姿；腕部相机随机器人运动，安装微小差异也会改变细粒度几何线索。

Result: WARP-VLA 为不同腕部视角训练专属特征专家，并根据隐式视角信息进行混合。LIBERO 腕部视图扰动实验中，π-0.5 平均成功率从 39.2% 提升至 78.3%。真实机器人实验显示，仿真中学到的特征级适配可迁移到多种部署设置。作者公开腕部视角鲁棒性基准与即插即用实现。

Insight: 相机视角鲁棒性可通过特征变换专家和隐式路由获得，而不依赖精确外参；真实机器人迁移验证补充了 LIBERO 结果。

Abstract: Despite recent advances in Vision-Language-Action models (VLAs) for robotic manipulation, their performance remains sensitive to changes in camera configuration. The problem becomes more evident in cross-setup deployment, as reproducing the exact camera pose used for training is nearly impossible. Unlike fixed external views, wrist views are more challenging because the camera moves with the robot, causing even small mounting variations to alter fine-grained geometric cues. To address this, we propose WARP-VLA, a camera-view robust VLA for diverse wrist camera configurations. WARP-VLA adopts a Mixture-of-Experts (MoE) architecture where individual experts learn view-specific feature transformations, and a router combines them based on implicit view information. This allows the policy to be deployed without requiring camera extrinsic parameters as additional input. Through experiments on the LIBERO benchmark, WARP-VLA improves the average success rate of pi-0.5 from 39.2% to 78.3% under wrist-view perturbations. The real-robot experiments further show that the feature-level adaptation learned in simulation successfully transfers to diverse deployment settings. To facilitate reproducibility and future research, we release our wrist viewpoint robustness benchmark and a plug-and-play implementation.

[189] CoCam4D: Geometry-Aware Cooperative 4D Perception for Camera-Only Autonomous Driving cs.RO | cs.CVPDF

Soham Pahari, Sudip Das, Arindam Das, Ujjwal Bhattacharya

TL;DR: 提出 CoCam4D，以不确定性感知 3D 高斯表示进行仅摄像头多车协同感知，并用 35 字节动态物体原语满足 C-V2X 通信约束。

Details

Motivation: 车辆受遮挡、盲区、传感距离和复杂场景限制；协同感知可共享多车观测，但纯相机的单目深度随距离不确定，限制几何融合。

Result: CoCam4D 用 VGGT 前馈网络生成带不确定性的 3D 高斯场景表示，使多车辆可高效合并观测；一车的可靠证据可降低另一车深度不确定性，无需 LiDAR。Dynamic Object Primitives 以 35 字节紧凑编码适配 C-V2X。实验较近期纯视觉方法在 OPV2V+ 提升 11.48%，在 DAIR-V2X-C 提升 10.62%。

Insight: 显式表达几何不确定性有助于协同融合而非无条件合并深度估计；紧凑动态物体表示面向实际车联网传输。

Abstract: Autonomous vehicles often suffer from limited perception due to occlusions, blind spots, limited sensor range, and the complex nature of surrounding environments. Multi-agent collaborative perception (CP) addresses these challenges by allowing vehicles to share sensory information and reconstruct the scene cooperatively. However, camera-only perception remains fundamentally limited by the uncertainty of distance-dependent monocular depth estimation. We propose CoCam4D, a Bayesian framework for collaborative perception that explicitly models geometric uncertainty. It uses a VGGT-based feedforward network to generate 3D Gaussian scene representations with associated uncertainty estimates, enabling multiple vehicles or agents to efficiently combine their observations. By sharing compact Gaussian primitives, reliable observations from one agent can reduce the depth uncertainty of another without requiring LiDAR sensors. To support real-world deployment, we introduce Dynamic Object Primitives (DOPs), a compact 35-byte representation designed for efficient C-V2X communication. Extensive experiments show that our proposed method consistently outperforms recent vision-only methods, achieving improvements of 11.48% on OPV2V+ and 10.62% on DAIR-V2X-C, demonstrating the potential of geometrically grounded collaborative perception for LiDAR-free autonomous driving.

[190] A Minimal Optical-Flow Representation for Vision-Based Tactile Rotation Classification in Robotic Manipulation Across Gravity Domains cs.RO | cs.CVPDF

Oscar Martinez-Bernal, Mario Cavero-Vidal, Francesco Grella, Carol Martinez

TL;DR: 研究利用 126 维稠密光流特征进行跨重力条件的触觉旋转分类，并发现地球训练的高精度不能保证太空环境迁移。

Details

Motivation: 视觉触觉传感器提供丰富接触信息，但高分辨率图像处理对空间机器人等资源受限平台代价高。作者测试是否可用紧凑触觉运动表示跨重力分类物体旋转。

Result: 模拟 GelSight Mini 的稠密光流按 7×9 网格聚合成 126 个特征，分类地球、火星、月球和轨道重力下负载引发的旋转方向。重力对特征造成小但显著偏移，占方差 1.6%（R²=0.016）。仅地球数据训练时，XGBoost 准确率从地球 94.4% 降至轨道 75.9%；在四种重力数据混合训练的单一模型达到总体 96.3%，各域为 95.1%–97.0%，无需将重力作为输入。特征还能压缩到 40 维并保持 95.7% 准确率，XGBoost 单次推理 0.14ms。

Insight: 即使重力造成的特征偏移占比小，也会明显影响域外表现；触觉感知的太空迁移需在训练和验证中纳入重力域变化。

Abstract: Vision-based tactile sensors provide rich contact information, but processing high-resolution images can be costly for resource-constrained platforms such as space robots. This work investigates whether a compact representation of tactile motion can classify object rotation across different gravity conditions. Dense optical flow from a simulated GelSight Mini is aggregated over a 7x9 grid into 126 features and used to classify the direction of load-induced rotation under Earth, Mars, Moon, and orbital gravity. Gravity causes a small but significant shift in these features, accounting for 1.6% of their variance (R2 = 0.016). Despite its small magnitude, this shift affects models trained only on Earth data: XGBoost accuracy decreases from 94.4% on Earth to 75.9% in orbit. In contrast, a single model trained across all four gravity domains achieves 96.3% overall accuracy and 95.1%-97.0% across individual domains, without using gravity as an input. The representation can also be reduced to 40 features while retaining 95.7% accuracy, with XGBoost requiring only 0.14 ms per inference. These findings show that Earth-gravity performance alone is insufficient to establish the transferability of tactile perception for space robotic manipulation, highlighting the need to account for gravity-induced domain shifts during training and validation.

Jinkai Zhang, Jingyi Xu, Yuanhong Yu, Jiarui Guo, Ruizhen Hu

TL;DR: 提出 SuperNav，以专用 agent harness 扩展预训练 MLLM 的语义导航能力，并把运动执行交给导航技能和工具，而不对 MLLM 做导航专用微调。

Details

Motivation: 通用服务机器人需在陌生环境中理解多样请求并导航；直接微调 MLLM 预测导航动作依赖训练数据覆盖，可能限制新请求和新环境泛化。

Result: SuperNav 保留 MLLM 的通用能力用于理解请求、场景和决策，harness 提供导航技能、物理交互工具及任务进度/上下文管理。统一视觉点接口让模型直接在图像指定目标点，并依据执行反馈修正决策。实例级、多物体和需求驱动任务上超过四个评估基线；HM3D 类别级评估及真实四足机器人部署也展示了跨环境可用性。摘要未提供具体分数。

Insight: 让 MLLM 负责语义决策、工具负责低层移动，可减少导航专用微调与训练覆盖对泛化的限制；统一图像点接口连接语言规划与机器人动作。

Abstract: General-purpose service robots need navigation systems that can handle diverse human requests in unfamiliar environments, combining task generality with scene generality. Some existing methods fine-tune multimodal large language models (MLLMs) to predict navigation actions, making their behavior dependent on the coverage of navigation training data and potentially limiting generalization to new requests and environments. Our key insight is to let the MLLM focus on interpreting requests, understanding scenes, and making decisions while preserving its general-purpose capabilities and delegating motion execution to navigation tools. To realize this idea, we introduce SuperNav, which equips a pretrained MLLM with a specialized agent harness without navigation-specific fine-tuning of the MLLM. Our harness supports these decisions with Navigation Skills, agent-oriented Tools for physical interaction, and task-progress and context management. A unified visual-point interface connects decision-making to motion by allowing the model to specify destinations directly in images and revise its decisions from execution feedback. Together, these components support sustained navigation across different task requirements and environments. SuperNav outperforms four evaluated baselines on instance-level, multi-object, and demand-driven tasks. Category-level evaluation on HM3D and deployment on a real quadruped robot further demonstrate its applicability across environments. Project Page: https://zju3dv.github.io/SuperNav/

[192] Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement cs.RO | cs.CVPDF

Kairui Hu, Siyuan Hu, Fangzhou Hong, Zhaoxi Chen, Ziwei Liu

TL;DR: 提出 Code-Only-as-Policy（COAP），用可跨 episode 运行的显式代码库跟踪机器人/环境状态并直接做动作决策，支持通过代码代理递归改进。

Details

Motivation: 现有机器人控制常在回路中保留模型：VLA 从观测映射动作，或 Agent Harness 运行时调用 VLM。作者提出若状态能被准确表示，决策逻辑可完全写入代码而无需推理模型实时参与。

Result: COAP 以摄像头图像和本体感觉用代码测量并跟踪机器人、环境和任务状态，每次决策均基于该状态；同一代码跨 episode 复用，不同任务共享库。作者分析明确可存储状态、代码可控/快速运行并能灵活恢复失败、能力可继承扩展三项优势。闭环编码代理可修改共享库实现递归自改进。在 RoboDojo 42 项双手任务上，测试时无模型达到 70.24% 成功率。作者指出上限取决于状态表示准确性和代码逻辑鲁棒性；代码库也可成为 VLA/Agent Harness 的高效数据引擎。

Insight: 显式代码策略减少实时模型调用、提升可检查性，但高度依赖状态表征和规则代码可靠性。

Abstract: Most robot policies keep a model in the control loop: a VLA maps observations to actions, and an Agent Harness, such as Agent-as-Policy or Harness VLA queries a VLM for decision making at run time. We propose a different view: the embodied world is an Embodied Turing Machine, whose tape is the robot and environment state and rules are the policy. If this state can be represented accurately, the decision making can be written entirely in code. We therefore propose Code-Only-as-Policy (COAP): code measures and tracks the robot, environment, and task state from camera images and proprioception, and makes every decision from it. The same code applies across episodes, and different tasks share one library without a VLM or VLA in the loop. Compared with VLAs and Agent Harnesses, we analyze three advantages of COAP: (i) Explicit State: the state can be stored in code; (ii) Execution: code makes decision making controllable, recovers from failures flexibly, and runs fast and cheaply online; (iii) Extensibility: new tasks reuse, inherit, or extend the shared library, so capabilities can accumulate over tasks. These advantages make COAP a suitable medium for recursive self-improvement (RSI): coding agents develop the library in a closed loop, and each change is explicit and controllable. On RoboDojo’s 42 bimanual tasks, the resulting library reaches a success rate of 70.24% without a model at test time. The upper bound of COAP lies in how accurately the state is represented for decision making and how robust the code logic is. We thus propose COAP as a new paradigm for embodied tasks; since it applies across episodes, it can also serve as an efficient data engine for VLAs and Agent Harnesses.

[193] DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training cs.RO | cs.CVPDF

Junyan Li, Ruizhi Li, Yu Liu, Xiangshuo Liu, Mingchao Sun

TL;DR: 提出 DreamTrue，多视角跨具身机器人世界模型，以几何校准、反事实动作后训练和人工标注缺陷奖励提升动作忠实度及物理合理性。

Details

Motivation: 机器人数据训练世界模型会遇到两类问题：不精确标定损害动作跟随，失败交互覆盖不足会让模型预测偏向成功情形。

Result: DreamTrue 将动作轨迹渲染为空间图像条件，并进行离线几何校准以对齐目标视频；再修改记录动作轨迹，合成更广动作及接触配置的未来视频进行反事实后训练。为在没有配对未来真值时评价预测，作者构建人工标注视频数据，覆盖机器人、物体和交互缺陷，训练具身视频奖励模型并用于 RL 后训练。在 AgiBot 上动作跟随达 SOTA，人工评估交互缺陷率从 48.12% 降至 6.25%；在 2026 AgiBot World Challenge 世界模型赛道排名第一。

Insight: 反事实动作扩大交互覆盖，缺陷奖励则在无未来配对真值时提供物理合理性信号；文中实机结论针对 AgiBot 评测及挑战赛。

Abstract: We present DreamTrue, a multi-view, cross-embodiment robot world model for action-faithful and physically plausible video prediction. Training such a model on existing robot datasets faces two obstacles: imprecise calibration can impair action following, while limited coverage of unsuccessful interactions can bias predictions toward successful outcomes. To improve action following across embodiments, we render action trajectories into image-space conditions and introduce offline geometric calibration to align these conditions with the target videos. To broaden interaction coverage, we introduce counterfactual post-training, modifying recorded action trajectories and generating future videos under a wider range of actions and contact configurations. To provide feedback on these predictions without paired ground-truth futures, we construct a human-annotated video dataset covering robot, object, and interaction defects and use it to train an embodied video reward model. Its scores guide reinforcement-learning post-training toward more physically plausible interaction outcomes. On AgiBot, DreamTrue attains state-of-the-art action following, while reducing the human-assessed interaction defect rate from from 48.12% to 6.25%. Notably, our model ranks first in the world model track of the AgiBot World Challenge 2026. The project page can be found at https://brave-eai.github.io/DreamTrue.

[194] Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demonstration cs.RO | cs.CVPDF

Jusuk Lee, Sungha Kim, Yeonsoo Park, Jonguk Cheon, Yoonkyo Jung

TL;DR: 提出 Dex-One2Many，从单个人类操作视频提取序列场景图作为仿真 RL 的关系约束，引导灵巧手探索并泛化到示范未见物体姿态和抓取。

Details

Motivation: 从单个人类视频学习灵巧操作可降低机器人示范成本，但严格模仿演示动作限制对新物体初始姿态、目标和抓取的泛化；无先验直接 RL 又难探索多阶段高维任务。

Result: Dex-One2Many 将视频抽象为序列场景图，作为 RL 生成多样 reset 状态的关系约束和各阶段密集奖励。场景图约束相对关系而非精确姿态，使 reset 覆盖演示外物体位姿与抓取；配合阶段初始化和密集奖励缩短探索。完全在仿真训练后，零样本迁移至真实多指手。五项工具使用和操作任务上，已见配置比基线高 6.5%；未见场景下泛化优势扩大至 71%。

Insight: 从单示范提取关系结构而非复制绝对动作轨迹，可在保留任务先验的同时扩大训练状态分布。

Abstract: While learning dexterous manipulation from a single human video offers a promising alternative to costly robot demonstrations, many recent methods predominantly imitate demonstrated motions. Such strict motion matching often limits generalization to initial object poses, goal poses, and grasps not shown in the video. Alternatively, discovering a policy via reinforcement learning (RL) allows for broad generalization, but without prior guidance, it struggles with high-dimensional exploration in complex, multi-stage tasks. To address these coupled generalization and exploration challenges, we present Dex-One2Many, a real-to-sim-to-real framework that learns a generalizable dexterous manipulation policy from a single human video. Our key insight is to abstract the video into sequential scene graphs that guide RL, enabling efficient exploration while preserving broad generalizability. The graphs serve as generative constraints for sampling diverse reset states and provide dense rewards for each stage. Because the graphs constrain relations rather than exact poses, these reset states cover object poses and grasps beyond the video, while initializing each stage from them with dense rewards keeps exploration short and guided. Trained entirely in simulation, Dex-One2Many transfers zero-shot to a real multi-fingered hand. Across five tool-use and manipulation tasks, Dex-One2Many exceeds baselines by 6.5% in seen configurations, while its robust generalization widens this gap to 71% in unseen scenarios.

cs.SD [Back]

[195] Selective Listening: Mechanism-Guided Control of Audio Influence in Large Audio-Language Models cs.SD | cs.CL | cs.LGPDF

Yulin Sun, Kele Xu, Yong Dou

TL;DR: 提出 ICAP-Gate，按任务条件控制音频语言模型中特定音频路径的影响，针对不需要听音时无关音频改变文本推理答案的问题。

Details

Motivation: 多模态 LLM 的总体准确率可能掩盖音频造成的成对答案漂移，因为音频带来的修正和破坏会互相抵消。作者通过成对漂移分析定位可操作的架构特定晚期音频路径。

Result: 在四个 LALM、两个推理基准、环境声和自然语音干扰下，ICAP-Gate 在全部 16 个全划分模型—条件评估中，Influence Rate 和 Answer Flip 的点估计都低于不门控推理。固定抑制会使四个模型的 ASR 都退化，而 ICAP-Gate 在明确要求音频的指令下保留通路，ASR 表现与不门控相当。四个对照设置中，成对漂移点估计均低于缓解提示；相较八样本 Self-Consistency，ICAP-Gate 每查询只生成一次并能达到有竞争力的稳定效果。受控 ARC 测量中，Self-Consistency 延迟为未门控的 7.0–9.2 倍。

Insight: 选择性调节模态影响优于一概屏蔽音频，可在不需要听音时降低干扰、同时保留 ASR 路径。作者报告的是点估计比较，摘要未给误差区间。

Abstract: Large audio-language models (LALMs) exploit multimodal evidence, yet task-irrelevant audio can alter text-reasoning decisions when listening is unnecessary. Aggregate Accuracy can hide this paired drift because audio-induced repairs and damages may cancel. Paired drift analysis and targeted interventions identify architecture-specific, intervention-sensitive late audio pathways as actionable control points. We introduce ICAP-Gate, which applies mechanism-guided, task-conditioned control to each model’s pathway. Across four LALMs, two reasoning benchmarks, and environmental-sound and natural-speech interference, ICAP-Gate has lower point estimates for Influence Rate and Answer Flip than ungated inference in all 16 full-split model–condition evaluations. Fixed suppression degrades automatic speech recognition (ASR) across all four models, whereas ICAP-Gate matches ungated ASR performance by preserving the pathway for explicit audio-demand instructions. ICAP-Gate has lower paired-drift point estimates than mitigation prompting in all four evaluated settings and provides competitive stabilization relative to eight-sample Self-Consistency while using one generation per query; in controlled ARC measurements, Self-Consistency incurs $7.0$–$9.2\times$ ungated latency. These results establish selective modality influence control as a design principle for robust multimodal reasoning.

[196] SepGen: Multi-Stem Audio-Video Separation and Generation in a Single Model cs.SD | cs.CVPDF

Aviad Dahan, Rajaei Khatib, Yonatan Bitton, Idan Szpektor, Lior Wolf

TL;DR: 提出 SepGen，让预训练音视频生成器单次联合采样生成视频、混合音轨和每个带字幕声源的独立 waveform stem，也可按自由文本从已有混音中分离声源。

Details

Motivation: 新视角 4D 音视频场景渲染需要逐个定位声源并传播至观察者，联合音视频生成器却通常只输出混合音频，无法取得单独声源波形。

Result: SepGen 扩展预训练音视频生成器，支持生成和分离两种模式：生成时每个声源字幕规定对应 stem 内容，双人对话可按原始轮次输出每位说话者；分离时混合音轨保持不变，字幕描述待提取内容。作者在文本合成场景及其他生成器生成和真实语音、音乐、音效录音上评测。给定混音及含对话台词的字幕时，SepGen 超过语言条件分离器，语音场景优势最明显；即使从字幕中移除台词仍保持领先。

Insight: 联合生成单独声源轨可支持后续空间定位与新视角传播；同一模型还可用自然语言描述分离真实录音。

Abstract: A 4D audio-visual scene comprises a video, the dynamic geometry it depicts, and the sound sources that populate it, each with its own position and trajectory. Rendering such a scene from a novel viewpoint requires that every source be available as an individual waveform, so that it can be localized in the scene and propagated to the observer before the signals are mixed. Joint audio-video generators can synthesize the video and its soundtrack, but the soundtrack is emitted as a single audio-mix in which the sources are not individually accessible. We present SepGen, which extends a pretrained audio-video generator to emit the video, the mixed soundtrack, and one waveform per captioned source in one joint sampling run. SepGen supports two complementary modes: generation and separation. In generation mode, each source caption specifies what its stem contains. A two-speaker dialogue, for example, comes out as one stem per speaker in the original turn order. In separation mode, the input audio-mix remains clean while the captions specify what to extract, so the model can decompose a recording from a free-text description. We evaluate generation on scenes synthesized from text, and separation on scenes rendered by other generators and on real recordings of speech, music, and sound effects. Given an audio-mix and captions that carry the spoken lines, SepGen outperforms language-conditioned separators, most clearly on speech, and it keeps the lead when the lines are removed from the captions. Code, checkpoints, and datasets are available at https://sepgen.github.io/

eess.AS [Back]

[197] Conversational Voice Aesthetic Model with Reinforcement Learning from Human Listeners eess.AS | cs.CL | cs.LG | cs.MM | cs.SDPDF

Xilin Jiang, Shun Zhang, Tejas Jayashankar, Yinghao Aaron Li, Osama Hanna

TL;DR: 提出 Conversational Voice Aesthetic Model（CVAM），从语境和回复语音描述声音审美特征并预测九类属性，使用听众偏好强化学习对齐主观感知。

Details

Motivation: 语音情绪和表达方式等感知字段具有主观性，没有唯一正确标签；评估自然对话中的真实或合成语音审美，需要直接以人类听者判断作为监督。

Result: CVAM 接收对话语境和回复语音，描述显著声音特征并预测覆盖性别、音高、节奏、情绪、表达方式等九种类别属性。作者从 CANDOR 语料的 3,000 条真实与合成语音回复中，每条收集约 10 个人工标注；先对合成描述和标签监督微调，再用人类判断进行 GRPO 优化。实验显示 CVAM 与人类听者的一致性优于 Gemini 3.1 Pro 和开源语音 LLM，也超过单个人类与其余人类多数意见的一致率。

Insight: 在人类主观分歧较大的语音审美任务中，用多听者判断训练模型能够达到超过单个人类标注者的群体一致性。摘要未给各属性的分数。

Abstract: We introduce Conversational Voice Aesthetic Model, a speech large language model for describing the voice aesthetics of real or synthetic speech responses in natural conversational contexts. Given a context and a response speech, CVAM describes salient moments that characterize the voice and predicts nine categorical attributes spanning gender, pitch, pacing, emotion, and delivery. The key challenge lies in perceptual fields such as emotion and delivery, which are inherently subjective and lack definitive ground truth. Therefore, we collect ~10 human annotations for each of 3k real and synthetic responses derived from the CANDOR corpus. CVAM is supervised finetuned on synthesized aesthetic descriptions and labels, then optimized with Group Relative Policy Optimization on human judgments. Experiments show that CVAM better agrees with human listeners than Gemini 3.1 Pro and open-source speech LLMs, and outperforms single-human-vs.-rest agreement. Together, we demonstrate the importance of grounding voice aesthetics in human perception and propose a principled framework for human alignment.

Table of Contents

cs.CV [Back]

[1] SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows cs.CV | cs.AIPDF

[2] A Camera-Native Stereo VR180 Dataset cs.CV | cs.MMPDF

[3] Seeing Through the Glare: A Multi-Source Benchmark and Ocular-Adaptive Pixel MeanFlow for Eyeglass Reflection Removal cs.CVPDF

[4] VICO: Visual Environments Co-Evolving for Vision-Language Model Reasoning cs.CV | cs.AIPDF

[5] SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages cs.CVPDF

[6] Fluid-Gen-Zero: Grounding Pretrained Video Generators in Physics without Training cs.CV | cs.GRPDF

[7] Less from More: Reinforcing Sparse Video Reasoning from Dense References cs.CVPDF

[8] LVSPM: Long Sequence View Synthesis and Pose Estimation Model cs.CVPDF

[9] Mid-Training Language Models on Raw Video cs.CV | cs.AI | cs.LGPDF

[10] Region-Aware CLS Token Augmentation for Fine-Grained Image Retrieval cs.CV | cs.LGPDF

[11] Omni-Diffusion-Distill: Few-Step Distillation of Unified Multimodal Diffusion Large Language Models cs.CV | cs.LGPDF

[12] Rendering-Free Lookahead for Question-Guided Active Vision cs.CV | cs.ROPDF

[13] Transforming Image Editors into Video Editors cs.CVPDF

[14] Expression-Diverse References for Identity-Preserving Video Generation cs.CVPDF

[15] Learning What to Trust in Multimodal Learning under Noisy Supervision cs.CV | cs.LGPDF

[16] AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understanding cs.CV | cs.AIPDF

[17] Refine Connections, Close the Gap: A Reliable Enhancement Framework for Driving Scene Topology cs.CVPDF

[18] Continuous Ground-Truth Construction and a Recovery Policy for Air–Water Robotic Tracking cs.CV | cs.AIPDF

[19] TKCAM: Text and Keyframe to Camera Trajectory Generation cs.CVPDF

[20] DiscoVL: Unveiling Disentangled C ross-Modal Representation Learning via Orthogonal Adversarial Regularization for V ision-Language Models cs.CVPDF

[21] Improving Image-Based Nutrition Estimation Through Multimodal Food-Item Verification and Recovery cs.CV | cs.AI | eess.IVPDF

[22] SP-DocReader: Difference-Aware Self-Play for Precise Document OCR cs.CV | cs.AIPDF

[23] VAMR: Multi-Question Agentic Reasoning for Efficient Long-Form Video Understanding cs.CV | cs.AIPDF

[24] A Unified Score Matching Paradigm for Video Anomaly Detection and Anticipation cs.CV | eess.IVPDF

[25] Multimodal Remote Sensing Image Registration: A Comprehensive Review, Challenges and Prospects cs.CVPDF

[26] MATE4D: Matrix-Guided Editable 4D Generation from a Single Image cs.CVPDF

[27] IntactWorld: Joint World Modeling with Intact Features cs.CVPDF

[28] 3DTexMOR: 3D Gaussian Multi-Object Removal via Texture-Space Inpainting cs.CVPDF

[29] Dissecting Representation Structure in Vision Transformers: A Rigorous Architectural Study cs.CV | cs.LGPDF

[30] GATOR: Generative and Agentic 3D Object Reconstruction From Casual Images cs.CV | cs.ROPDF

[31] MiniVer-V: Identifying Minimal Sufficient Evidence for Short Video Verification cs.CV | cs.CL | cs.MMPDF

[32] V-CoLA: Vision Token Compression with Linear Attention cs.CV | cs.LGPDF

[33] Spatial-Frequency-Aware Implicit Neural Representation of Multidimensional Signals via MLP-KAN Fusion cs.CVPDF

[34] iCATS: Fast Video Generation via Interaction-Aware Sparse Attention and Timestep-Adaptive Sparsity cs.CVPDF

[35] Efficient Multi-Granularity Knowledge Transfer for Radiology Report Generation cs.CVPDF

[36] Collaboratively Guided Adversarial Robust Distillation with Teacher-Favorable Examples cs.CVPDF

[37] PathLang: A Language-Centered Benchmark for Vision-Language Models in Computational Pathology cs.CVPDF

[38] FloorSAV: Elucidating Spatial Audio-Visual Context with 2D Floormap for AV-LLMs cs.CV | cs.LGPDF

[39] Point-Focused Attention Meets Context-Scan State Space: Robust Biological Visual Perception for Point Cloud Representation cs.CVPDF

[40] EgoPhys: Estimating Peak Contact Force and Mechanical Work from Egocentric Manipulation Video cs.CVPDF

[41] CRISP: Fixing Flying Pixels in Latent LiDAR Generation via Diffusion Decoding cs.CV | cs.AIPDF

[42] Rethinking Contrastive Loss in CLIP Post-training: A Complementary Framework with Frozen Text Encoder cs.CV | cs.AIPDF

[43] FastJEV: Understanding Redundancy for Compact JEV Inference cs.CVPDF

[44] GroundSight at GroundLM 2026 Shared Tasks: GoldenViewVQA cs.CVPDF

[45] Missing Modality-Aware Calibration for Trustworthy Brain Tumor Segmentation cs.CV | cs.AIPDF

[46] Fresco++: Frequency-Guided and Canonical-Consistent Optimization for Fine-Grained Head Avatar Modeling cs.CVPDF

[47] EchoDiST: Self-distillation-based joint learning for diffusion-conditioned echocardiographic myocardial motion estimation cs.CVPDF

[48] SAGE: Sink-Aware Guided Emphasis for Visual Grounding in Vision-Language Decoders cs.CV | cs.CLPDF

[49] Learning to Retrieve: Internalizing Memory Retrieval for Video World Models cs.CVPDF

[50] Conditional Residual Prediction: Improving Autoregressive Video Diffusion without a Bidirectional Teacher cs.CV | cs.AI | cs.LGPDF

[51] Stop My Dancing! Understanding, Detecting and Attributing Motion-Aware Deepfake Videos cs.CV | cs.CRPDF

[52] Parametric Trajectory Distillation for Few-Step Video Generation cs.CVPDF

[53] MSGAT: Multi-Head Spiking Graph Attention with Similarity-Space Fusion for Image-Text Retrieval cs.CV | cs.AIPDF

[54] Hankel Subspace Self-Supervised Learning for Parallel MRI Reconstruction cs.CVPDF

[55] SV-TAD: Native Sparse Convs for Efficient Temporal Action Detection cs.CVPDF

[56] Beyond Report Imitation: Clinically Aware Multi-Image Ultrasound Report Generation from Visible Evidence cs.CV | cs.AIPDF

[57] TAM: Task-Aware Memory Distillation for Efficient Spatiotemporal Prediction cs.CV | cs.AIPDF

[58] DisFace3DNet: Explainable Facial Attractiveness Prediction via 3D Component Disentanglement cs.CVPDF

[59] Perceptually Grounded and Semantics-Aware Evaluation for Holistic Co-Speech Gesture Generation cs.CVPDF

[60] VESSI - VLM-Enhanced Support for Surveillance and Investigations cs.CVPDF

[61] Towards Unified Evaluation of Prompt Enhancers for Video Generation cs.CVPDF

[62] Streaming-Aware Diffusion for Real-Time Video Super-Resolution via Cross-Step Attention cs.CVPDF

[63] From Video Clips to Creation Trajectory: Sora100K for AI-Native Video Creation cs.CVPDF

[64] Memory Forcing: Attendable Mid-Horizon History for Streaming Video Generation cs.CVPDF

[65] Skill-V: Verifiable Self-Evolving Skill Library for Interactive Agents cs.CVPDF

[66] Phase-aware video generation for physics-grounded dynamics and interactions cs.CVPDF

[67] Seek-and-View Reasoning for Multi-View Spatial Understanding cs.CVPDF

[68] From Pixels to Structure: Lightweight Vision-Language Models for Document OCR and Structured JSON Extraction cs.CV | cs.AIPDF

[69] From Suppression to Repair: Mitigating Object Hallucination in Large Vision-Language Models via Localized Distribution Alignment cs.CV | cs.AIPDF

[70] Open-Vocabulary Audio-Visual Event Localization via Complex-Valued Fusion cs.CV | cs.LG | cs.MM | cs.SD | eess.IVPDF

[71] VEDJE: Video-Efficient Discriminative Joint Encoder for Scalable Video-Text Retrieval cs.CVPDF

[72] Relative Patch Response Learning for Generalizable AI-Generated Image Detection cs.CVPDF

[73] Pose-Free Feed-Forward 3D Inpainting via Learnable Mask Attention and Support Token Refinement cs.CVPDF

[74] Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations cs.CV | cs.AIPDF

[75] Revisiting Identity and Spectra Dispersion in Media-Bridged Time Series Forecasting: Linking Multivariate Signals and Narrative Flows cs.CV | cs.LGPDF

[76] Does Target Alignment Mean Target Recovery? An Evidence-Ladder Study of Adversarial Claims on Contrastive Encoders cs.CVPDF

[77] Look Where You Can: Active View Selection for CAD Reconstruction under Occlusion cs.CVPDF

[78] Right Screen, Wrong Transition: World Models as Verifiers for GUI Agents cs.CVPDF